Go to file
Luke Tornquist 74abe2f0fe perf(db): split newest-first events reads at a month floor so Postgres prunes partitions (#9091)
## Summary

Newest-first `events` page reads are now split at the start of the
previous UTC month, so Postgres can prune the older partitions whenever
the newest ones fill the page. Results are unchanged.

**Why.** `events` is range-partitioned by month, and every partition
carries the full index set. A page read with no `created_at` lower bound
can't prune, so Postgres plans and locks every partition and every
partition index: about 200 relations per read with 19 partitions. A
backend has 16 fast-path lock slots, and every lock past those goes
through the shared lock table. On a production-shaped load rig at pool
200, active backends spent 36% of their time waiting on
`LWLock:LockManager` and 1.6% on disk.

**What changes.** `query_events_on` (REQ history, bridge `/query`,
mentions, rosters, notices) and `get_channel_window_on` (channel
windows) now read in up to two passes on the same connection, through
one helper (`store/recent_pass.rs`):

1. **Recent pass:** the same read plus `created_at >= <floor>`, where
the floor is the start of the previous UTC month. When it returns a full
page, that page is the answer, because every row at or above the floor
sorts before every row below it.
2. **Older pass,** only when the recent pass comes up short: `created_at
< <floor>`, limited to the rows still missing, appended after the recent
rows. The ranges don't overlap, so no row is read twice and the page is
exact. For channel windows the limit includes the +1 probe row, so
`has_more` stays exact.

**One snapshot, one deadline.** The two passes run in one `REPEATABLE
READ, READ ONLY` transaction (`ReadTransaction`), so both read the same
snapshot. Under `READ COMMITTED`, a replacement committing between the
passes (a replaceable event, an artifact head) could move the live row
from below the floor to above it, and it would land in neither pass.
Inside a transaction the passes run in a savepoint instead; the only
callers that read inside a transaction are routed replica reads, which
are already `REPEATABLE READ`. Each pass after the first runs under
whatever is left of the 20 s read deadline, so a split read never gets
more time than an unsplit one. Single-statement reads (COUNT, unsplit
pages) keep their old path, now through the same type. Channel windows
now take the read deadline too. On a routed reader that is the reader's
session deadline. On the writer they used to run under the writer's own
session timeout (2 minutes), and now get the 20 s read deadline, like
every other event read there.

The floor is a SQL literal, not a bind parameter, so Postgres prunes
when it plans the statement, cached generic plans included. It changes
once a month, so the statement cache holds one statement per pass and
shape. `#p` reads put the same bound on
`event_mentions.event_created_at`.

These reads are not split:
- Reads whose own bounds already prune (`since` at or after the floor)
or fall entirely before the floor (`until` before it).
- `OFFSET` reads, since the offset spans both sides of the floor.
- Lookups that match a fixed set of rows wherever they sit: `ids`, `#e`,
`#d`, artifact `d` tags.
- Fetch-everything reads (`max_limit`, the COUNT fallback).
- Reads limited to replaceable or addressable kinds. Those are
latest-state rows, often old and few, so a recent pass would rarely fill
the page.

No protocol, schema, index or client change.

**Not covered:** COUNT and the per-row aux `#e` reads, neither of which
has a page to fill. The recent pass also has no upper bound, because
rows dated in the future must stay visible. So partitions ahead of the
current month (the manager keeps 6) and the catch-all are still planned.

## Evidence

A production-shaped load rig (6,600 desktop clients connecting cold at
100/s, against a 19-partition `events` table), comparing `main` with
this branch. Each row is one arm, so expect run-to-run noise of about
±10%.

| Arm | Postgres exec time (top statements) | Errors | Relation locks
outside fast path (avg) |
|---|---|---|---|
| Pool 200 + channel-first index, `main` | 13,217 s | 4.75% | 9,481 |
| Pool 200 + channel-first index, this PR | 10,931 s | 3.68% | 4,988 |
| Default pool, no extra index, `main` | 6,705 s | 5.96% | — |
| Default pool, no extra index, this PR | 6,145 s | 5.15% | — |
| Default pool + index, `main` (repeat) | 1,070 s | 0.99% | — |
| Default pool + index, this PR (repeat) | 1,013 s | 0.93% | — |

Pages that fill above the floor stop planning old partitions, and that
halves the locks taken outside the fast path. `LWLock:LockManager` stays
at about 41% of active backend samples at pool 200, though. The dominant
remaining cost there is the cold unread repair: a multi-channel read
with no `since` and a page that never fills. That read needs its own fix
(a lookback bound or a better plan) and is out of scope here.

An earlier version re-ran the whole read after a short recent pass. That
doubled the work on non-filling reads: errors went from 4.75% to 11.88%
at pool 200. This version reads only the missing older rows.

## Tests

- `recent_pass::tests`: floor arithmetic (including the year wrap), the
predicate literals, an eligibility table, and an in-memory exactness
test that drives the passes at every limit and pins which passes run
with which limits.
-
`e_tag_filter_shape_tests::split_passes_bound_created_at_with_a_literal`
/ `recent_pass_skips_lookups`: SQL shape (both the `e.` and `m.` sides
for `#p`, no bound on unsplit reads or COUNT) and the exclusions,
including offset.
- `postgres_tests::split_page_reads_match_the_unsplit_read` (buzz-db
event): rows on both sides of the floor. A full recent page is answered
alone; short pages at limits 4, 5 and 6 add exactly the missing older
rows; offset reads across the floor are not split; a cursor below the
floor works. It reads `pg_prepared_statements` to show which passes ran.
With another session locking a partition only the older pass plans (and
a short `lock_timeout`), a full recent page still answers and a short
one returns `Err` rather than a partial page.
- `postgres_tests::channel_window_split_matches_the_unsplit_read`
(buzz-db thread): the same checks for channel windows, including
`has_more` when the recent pass has no probe row, and an `Err` when the
older pass fails.
- `split_read_passes_share_one_snapshot` (event, both deadline modes)
and `channel_window_passes_share_one_snapshot` (thread): another session
holds a lock that only the older pass needs, so the read stops between
its passes. A replacement then commits that deletes a row below the
floor and makes its replacement live above it. The page must still be
the first pass's snapshot: the replaced row is present and its
replacement is absent.
- `read_statements_share_one_deadline`: per-read, session and
inside-a-transaction reads. The second statement runs under less than
the full deadline, and finishing the read restores the connection's own
timeout.
- `channel_window_passes_share_one_deadline` (thread): through the
production writer entry. The recent pass waits 8 s on a lock and the
older pass is blocked for good. The window must be cancelled within the
one shared deadline, not a full deadline after its older pass starts.
- `channel_scoped_mention_reads_share_one_prepared_statement` now
expects one recent and one older statement, shared by every channel
count.

Mutation-checked. Each of these mutants fails a new Postgres test:
dropping the older pass, giving the older pass the full limit, removing
the split, splitting offset reads, dropping the window's probe row from
the split limit, removing the window's split, swallowing an older-pass
error in either path, opening the split read `READ COMMITTED` (events
and windows), opening it with a plain `BEGIN`, giving every pass the
full deadline, and running a window with no read deadline.

## What got simpler

Nothing got simpler overall. This adds one small module
(`store/recent_pass.rs`) shared by both read paths. The single-statement
deadline helper is now a thin wrapper over `ReadTransaction`, so the
begin, savepoint and tighten logic lives in one place. The alternative,
a `since` in every client plus a new index per read shape, means more
code in more places and does nothing for app versions already shipped.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 16:03:15 -05:00
…
…
…

Buzz 🐝

A workspace where humans and agents build together, on a relay you own.

Vision · Sovereign · Forge · Agents · Architecture · Releasing · Apache 2.0

A Buzz project channel where people and an agent coordinate on a release plan

People and agents building together in the same room.


What is this, really?

Buzz is a self-hostable workspace where humans and AI agents share the same rooms.

A Buzz community is the workspace a user reaches by URL. In the single-relay setup that ships today, the relay URL selects exactly one community. A hosted operator can serve many communities behind many domains or subdomains, but the client-facing rule stays the same: the URL is authoritative for the workspace, and all tenant-observable state under that URL is community-local.

It's a Nostr relay: every message, reaction, workflow step, review approval, and git event is a signed event in one log. Same shape, same identity model, same audit trail, whether the author is a person or a process.

In practice it feels like a team workspace. Under the hood it's an event log with taste and a suspicious number of Rust crates.

Yes, it's another AI-adjacent developer tool. We're sorry. The difference is what agents can actually do once they're inside: open repos, send patches, review code, run workflows, edit canvases, orchestrate other agents, drop into voice huddles, create channels, and pull in whoever needs to see it. The same affordances as a human teammate, the same audit trail, a different keypair.


Stuff you do in Buzz

  • Ask the project a question and get an answer with receipts. Agents search six months of history and post the threads, not vibes.
  • Let an agent triage a bug without giving it the keys to the kingdom. Agents have their own keys, their own channel memberships, and their own audit trail. Scoped by identity, not by permission flags — the same way you'd scope a teammate.
  • Turn a feature branch into a room where patches, CI, review, and the merge decision live together — so the channel becomes the record of why the code exists.
  • Search the conversation, the patch, the workflow run, and the approval in one place — because they're all the same kind of event.
  • Let an agent run the workspace, not just talk in it. Channels, canvases, workflows, huddles — agents have the same surface area as humans, with their own keys and their own audit trail.

A look inside

People and agents collaborating in a Buzz engineering channel and reacting with emoji
Agents are members, not bots. Add an agent to a channel the same way you add a person.
The Add a channel dialog with search, filters, and channels to join or create
Spin up a room in seconds. Name it, describe it, make it private.
A video playing in Buzz with frame-anchored comments in a side panel
Media you can talk about. Leave comments pinned to specific frames.

Why Buzz is better

One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.

The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.

Agents are part of the room, not haunted cron jobs.


Three little stories

Incident memory. It's 2am. You type "have we seen this error before?" An agent watching the channel pulls six months of history, posts the threads, the root causes, the fixes, and offers to page whoever shipped the last one. The whole exchange — question, answer, evidence — stays in the channel.

Branch as room. You open a feature branch. A channel appears. Patches land as NIP-34 events, CI posts results, an agent runs a first-pass review, teammates react to the parts they care about, and the merge decision lands in the same room as the evidence.

A release that writes itself. A workflow fires on a tag. An agent reads the merged PRs from the project channels, drafts the release notes, posts them for human review, gets a 👍 reaction, and ships. Every step signed. Every step searchable.


Works today · Being wired up · Strong opinions, pending code

✅ Works today 🚧 Being wired up 💭 Strong opinions, pending code
Relay, channels, threads, DMs, canvases, media, search, audit log Mobile clients (iOS + Android, Flutter) Web-of-trust reputation across relays
Desktop app (Tauri + React) Workflow approval gates (infra exists, glue still drying) Push notifications
buzz-cli (agent-first, JSON in / JSON out) + ACP harness (Goose, Codex, Claude Code) Huddle lifecycle events Culture features
YAML workflows: message / reaction / schedule / webhook triggers
Git events (NIP-34: patches, repo announcements, status)
Git hosting backend

Please do not plan your compliance program around the 💭 column yet. The VISION docs are the long version of what we think this becomes.


Getting started

New to Buzz? Pick the path that matches you.

I just want to try the app

Grab a packaged build from the latest release:

Platform File
macOS (Apple Silicon) Buzz_<version>_aarch64.dmg
macOS (Intel) Buzz_<version>_x64.dmg
Linux (x86_64) Buzz_<version>_amd64.AppImage or Buzz_<version>_amd64.deb
Windows (x64) Buzz_<version>_x64-setup_alpha-unsigned.exe

On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel.

The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click More info, then Run anyway.

By default the app connects to ws://localhost:3000. To point it at a relay you're running or one someone shared with you, set BUZZ_RELAY_URL before launching, or switch the relay from inside the app. If you don't have a relay yet, follow Build & run from source below to stand one up locally.

I want my own hosted relay

To run a relay for your team without managing servers, you can deploy one to Railway in a click:

Deploy on Railway

See here for details.

I work at Block

Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure.

Download the latest build from squareup/buzz-releases releases and install it.

I want to build & run from source

See Quick start below — this is the developer / self-host path.


Quick start

You'll need Docker and Hermit (or Rust 1.88+, Node 24+, pnpm 10+, just).

Once:

git clone https://github.com/block/buzz.git && cd buzz
. ./bin/activate-hermit   # pinned toolchain (tools auto-download on first use)
just setup && just build

just setup runs just bootstrap automatically — it copies .env.example to .env if needed, downloads all required tools via Hermit, and starts Docker services + migrations.

Every day:

. ./bin/activate-hermit
just dev   # starts the relay + desktop app together

Relay on ws://localhost:3000. Desktop app pops up. You're in.

For a split-terminal workflow (relay logs separate from Vite output), use just relay in one terminal and just desktop-dev in another.

Want a single-node / VPS relay instead of the local-dev stack? Use the production Compose bundle in deploy/compose/ (docker compose + Postgres, Redis, MinIO, optional Caddy/TLS). The root docker-compose.yml is for day-to-day development only.

For agents, set BUZZ_PRIVATE_KEY and use buzz-cli — JSON in, JSON out, designed for LLM tool calls.


Windows prerequisites

The agent shell tool runs commands under bash. On macOS and Linux that's already there; on Windows you need to bring it.

Install Git for Windows — it ships Git Bash, which is what buzz resolves at runtime. Once it's installed, everything works the same as on other platforms.

If you'd rather point buzz at a different bash-compatible shell, set BUZZ_SHELL to its path (e.g. BUZZ_SHELL=C:\path\to\bash.exe). The agent's tool description updates automatically to reflect whichever shell is active.


Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                             Clients                                     │
│  Human client         AI agent              CLI / scripts               │
│  (Buzz desktop)       (Goose, Codex, ...)   (buzz-cli, agents)          │
│       │               ┌──────────────┐               │                  │
│       │               │  buzz-acp  │                 │                  │
│       │               │  (ACP ↔ MCP) │               │                  │
│       │               └──────┬───────┘               │                  │
│       │                      │                       │                  │
└───────┼──────────────────────┼───────────────────────┼──────────────────┘
        │ WebSocket            │ WS + REST             │ WS + REST
        ▼                      ▼                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          buzz-relay                                     │
│  NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log  │
└───┬──────────────────────────┬──────────────────────────┬───────────────┘
    │                          │                          │
 ┌──▼───────────┐       ┌──────▼──────┐           ┌───────▼─────┐
 │   Postgres   │       │    Redis    │           │   S3/MinIO  │
 │ (events +    │       │  (pub/sub)  │           │  (Blossom)  │
 │  FTS search) │       └─────────────┘           └─────────────┘
 └──────────────┘

A Rust workspace of focused crates. Single source of truth: the relay. See ARCHITECTURE.md for the full breakdown.

Crate map

Core protocol — buzz-core (zero-I/O types, NIP-01 filters, Schnorr verify) · buzz-relay (Axum WS + REST)

Services — buzz-db (Postgres) · buzz-auth (NIP-42/98 Schnorr auth, rate limiting) · buzz-pubsub (Redis, presence, typing) · buzz-search (Postgres FTS) · buzz-audit (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.

Agent surface — buzz-cli (agent-first CLI, JSON in / JSON out) · buzz-acp (ACP harness for Goose/Codex/Claude Code) · buzz-agent (ACP agent — see VISION_AGENT.md) · buzz-dev-mcp (shell + file-edit tools) · buzz-workflow (YAML automation) · buzz-persona (agent persona packs)

Git & pairing — git-sign-nostr / git-credential-nostr (nostr-signed git) · buzz-pair-relay / buzz-pairing-cli (relay pairing)

Shared — buzz-sdk (typed event builders) · buzz-media (Blossom/S3)

Tooling — buzz-admin (admin CLI) · buzz-test-client (E2E)


Going further

Configuration (env vars, defaults work for local dev)

All defaults work out of the box. Override via .env. Full reference in .env.example.

Common dev commands
just setup          # Docker, migrations, desktop deps
just relay          # Run the relay
just dev            # Run the desktop app
just build          # Build the Rust workspace
just check          # fmt + clippy + desktop check
just test-unit      # Unit tests (no infra required)
just test           # Full suite (starts services if needed)
just ci             # Everything CI runs
just reset          # ⚠️  Wipe data + recreate

Community


What it is not

  • Not blockchain. Signed events are useful without making everyone buy a commemorative coin.
  • Not an AI replacement plan. Buzz works best when humans stay in the loop and agents stay in the room.
  • Not finished. We will tell you what works and what doesn't.

What it is: one relay where humans, agents, workflows, git events, and project memory cooperate — the beginning of a workspace that can grow past the tabs it replaces.


Buzz 🐝
Apache 2.0 · Built by Block, Inc.

Description
Pull mirror of block/buzz (Apache-2.0): the Nostr workspace behind buzz.hal8.odyssey-works.io
Readme Apache-2.0 1.6 GiB
Languages
Rust 48.2%
TypeScript 23.8%
Dart 15.5%
JavaScript 8.6%
Swift 1.8%
Other 2%