1f16f621d8
## Summary
A cold relay taking a burst of new connections spent most of its CPU on
two things that don't scale with useful work. This PR removes both.
**Log filtering held a contended lock.** `EnvFilter` (tracing-subscriber
0.3.23) takes a shared `RwLock` read on every span enter and exit, even
when no directive names a span. Every poll of an instrumented future
enters its span, and the relay installs two per-layer `EnvFilter`s (logs
and OTEL), so every worker contends on those locks. Plain `target=level`
directive lists now filter through
`tracing_subscriber::filter::Targets`, which keeps no per-span state.
Directives that name spans or fields still use `EnvFilter`, which is the
only filter that understands them.
**Each reader re-read the community's open channels.** A reader's
accessible channels are its member channels plus the community's listed
open channels, and the open half is the same for every reader. The
per-reader cache stored the union, so every miss fetched every open
channel row again. The lookup is now two cached halves:
- member channels, per reader, cached as before;
- open channels, once per community.
The shared half is used for a bounded time. Cross-pod invalidation is
best-effort, so a pod can miss the drop that follows a channel becoming
private, unlisted or deleted elsewhere. Each cached open set records
when its database read began, and readers use it only for
`OPEN_CHANNELS_MAX_AGE` (1 s) after that. After that the next reader
re-reads it. **Accepted trade-off:** if an invalidation is lost, any
reader on that pod can be granted a just-revoked open channel, history
included, for at most 1 s. That includes a reader with no cache entry,
which on `main` always read the database. This is a new exposure for
those readers. With no bound, the shared set could have served that
channel for its 10 s TTL. Readers with a cached entry already had a 10 s
window on `main`, so for them the bound is tighter than before. An exact
per-community version check would remove the window but needs a schema
change, so it's tracked as a follow-up.
Invalidations still drop the halves. A community-wide accessible-channel
or channel-deleted invalidation drops both halves, and a membership
invalidation drops only that reader's member half. The age bound counts
from the start of the read, so open-set fills need no fence: a fill that
raced a drop is still at most 1 s stale. Uncached callers of
`get_accessible_channel_ids` get the same set as before, now built from
the two halves. Open-set reads are counted by
`buzz_open_channels_cache_hits_total` and
`buzz_open_channels_cache_misses_total{reason="absent"|"expired"}`.
**Metric meaning change:** `buzz_accessible_channels_cache_hits_total`
and `buzz_accessible_channels_cache_misses_total` keep their names but
now count only the per-reader member half. Dashboards that read them as
whole-lookup hit rates should add the open-set counters above.
Empty levels (`target=`) stay on `EnvFilter`. `Targets` reads an empty
level as ERROR while `EnvFilter` reads it as TRACE, so only
`target=level` directives with a non-empty level take the `Targets`
path.
## Results
Measured with the relay load generator (#8636) on one Linux machine. The
relay ran as a release build on 8 pinned cores, with local Postgres,
Redis and MinIO; the generator ran on separate cores. The fixture is one
community with 2,000 open channels. Clients connect in a burst against a
cold relay: 48k WebSocket clients and 4.8k HTTP agents.
Each arm restored the same database snapshot and ran the same generator
binary and scenario. Arms were rotated (base/before/after, then
before/after/base, then after/base/before), and the table shows medians
of three runs per arm. Both columns after base also carry #8666, so the
comparison isolates this PR's design.
- **before** is this PR's first revision, which had an unbounded shared
open set.
- **after** is this head, where the open set is bounded to 1 s.
| | base | before | after |
|---|---|---|---|
| relay cores (mean) | 6.37 | 2.39 | 2.45 |
| connect p99 | 3,870 ms | 40 ms | 42 ms |
| subscribe EOSE p99 | 12,952 ms | 110 ms | 112 ms |
| history EOSE p99 | 1,252 ms | 68 ms | 72 ms |
| live delivery p99 | 26,421 ms | 54 ms | 55 ms |
| HTTP query p99 | 16,426 ms | 44 ms | 43 ms |
| DB tuples returned/s | 1.04M | 43k | 49k |
| DB transactions/s | 4,397 | 4,065 | 4,062 |
| relay RSS peak | 12.6 GB | 10.7 GB | 10.7 GB |
| errors | 16, plus 241k throttled | 0 | 0 |
The 1 s bound costs about 1,400 extra open-set reads per run, within
run-to-run noise of the unbounded version. One run in each of the two
right-hand arms had a slow tail (connect p99 0.8–1.3 s), which the
medians absorb.
An earlier revision bound the shared set to a per-reader digest instead.
That was exact, but in the same kind of A/B its per-miss digest query
returned about 28 times the tuples per second of the unbounded version.
It gave back the database savings, so this revision uses the time bound.
The open-channel gain scales with how many listed open channels a
community has. The filter gain doesn't depend on the fixture.
## Rollout
There's no flag, so the change takes effect on deploy. There's no schema
or data change, so reverting the commit is the rollback.
1. Deploy to staging and compare relay CPU and connect and EOSE p99
under a connection burst.
2. Watch `buzz_open_channels_cache_misses_total{reason="expired"}`.
Expect at most about one per second per active community per pod.
3. Confirm the deployed `RUST_LOG` and `BUZZ_OTEL_FILTER` are plain
`target=level` lists, so they take the `Targets` path. Other values fall
back to `EnvFilter` and keep today's cost.
## What got simpler
- `invalidate_channel_deleted_local` now reuses the community-scoped
accessible-channel drop instead of repeating it.
- Tests seed accessible channels through one `seed_accessible_channels`
helper instead of writing cache internals.
## Tests
- `telemetry::tests::only_plain_target_level_directives_use_targets`
checks which directive strings are plain. It covers span directives,
field directives, whitespace, empty segments, empty targets (`=debug`)
and bad levels.
- The rewritten `env_filter_tests` assert on captured log output rather
than `Display` strings or cached callsite interest, so they are stable
when run in parallel. `span_directives_keep_env_filter_semantics` proves
that `[conn]=debug` still enables events only inside the span.
-
`state::tests::postgres_tests::a_lost_open_set_invalidation_grants_a_revoked_channel_only_until_max_age`
runs two relay states on one database, with no pub/sub between them. For
each revocation (private, deleted, unlisted), it:
1. primes the reader pod;
2. revokes the channel in SQL and invalidates only the writer pod;
3. shows that a new reader is still granted the channel within the
window;
4. ages the cached set past the limit;
5. asserts that a new reader and an earlier cached reader are both
denied.
Dropping the age check fails it. Never caching the open set fails its
within-window control.
- `env_filter_tests::directive_filters_log_what_env_filter_logs` emits
events at every level for six targets, including prefix lookalikes. It
asserts that the filter built for each of 17 directive lists logs
exactly what `EnvFilter` logs for that list. Routing `buzz_relay=` to
`Targets` fails it.
- `state::tests::accessible_channels_join_member_and_open_halves` checks
the merge. The two existing scoped-invalidation tests now also assert
the open half.
- These state tests, and the two existing ones, were not in the `just
test-unit` allowlist. They are infra-free, so they are now selected by
exact name, and `scripts/run-tests.sh` mirrors that.
Identified debt, not fixed here: the per-reader member half keeps the
pre-existing unfenced read-then-insert (a raced fill only reaches that
reader, as before).
`handlers::req::tests::expiry_during_stalled_{history,search}_read_releases_permit_and_claim`
are not selected by any CI lane.
buzz-review-completed
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Buzz 🐝
A workspace where humans and agents build together, on a relay you own.
Vision · Sovereign · Forge · Agents · Architecture · Releasing · Apache 2.0
People and agents building together in the same room.
What is this, really?
Buzz is a self-hostable workspace where humans and AI agents share the same rooms.
A Buzz community is the workspace a user reaches by URL. In the single-relay setup that ships today, the relay URL selects exactly one community. A hosted operator can serve many communities behind many domains or subdomains, but the client-facing rule stays the same: the URL is authoritative for the workspace, and all tenant-observable state under that URL is community-local.
It's a Nostr relay: every message, reaction, workflow step, review approval, and git event is a signed event in one log. Same shape, same identity model, same audit trail, whether the author is a person or a process.
In practice it feels like a team workspace. Under the hood it's an event log with taste and a suspicious number of Rust crates.
Yes, it's another AI-adjacent developer tool. We're sorry. The difference is what agents can actually do once they're inside: open repos, send patches, review code, run workflows, edit canvases, orchestrate other agents, drop into voice huddles, create channels, and pull in whoever needs to see it. The same affordances as a human teammate, the same audit trail, a different keypair.
Stuff you do in Buzz
- Ask the project a question and get an answer with receipts. Agents search six months of history and post the threads, not vibes.
- Let an agent triage a bug without giving it the keys to the kingdom. Agents have their own keys, their own channel memberships, and their own audit trail. Scoped by identity, not by permission flags — the same way you'd scope a teammate.
- Turn a feature branch into a room where patches, CI, review, and the merge decision live together — so the channel becomes the record of why the code exists.
- Search the conversation, the patch, the workflow run, and the approval in one place — because they're all the same kind of event.
- Let an agent run the workspace, not just talk in it. Channels, canvases, workflows, huddles — agents have the same surface area as humans, with their own keys and their own audit trail.
A look inside
 Agents are members, not bots. Add an agent to a channel the same way you add a person. |
 Spin up a room in seconds. Name it, describe it, make it private. |
 Media you can talk about. Leave comments pinned to specific frames. |
|
Why Buzz is better
One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.
The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.
Agents are part of the room, not haunted cron jobs.
Three little stories
Incident memory. It's 2am. You type "have we seen this error before?" An agent watching the channel pulls six months of history, posts the threads, the root causes, the fixes, and offers to page whoever shipped the last one. The whole exchange — question, answer, evidence — stays in the channel.
Branch as room. You open a feature branch. A channel appears. Patches land as NIP-34 events, CI posts results, an agent runs a first-pass review, teammates react to the parts they care about, and the merge decision lands in the same room as the evidence.
A release that writes itself. A workflow fires on a tag. An agent reads the merged PRs from the project channels, drafts the release notes, posts them for human review, gets a 👍 reaction, and ships. Every step signed. Every step searchable.
Works today · Being wired up · Strong opinions, pending code
| ✅ Works today | 🚧 Being wired up | 💭 Strong opinions, pending code |
|---|---|---|
| Relay, channels, threads, DMs, canvases, media, search, audit log | Mobile clients (iOS + Android, Flutter) | Web-of-trust reputation across relays |
| Desktop app (Tauri + React) | Workflow approval gates (infra exists, glue still drying) | Push notifications |
buzz-cli (agent-first, JSON in / JSON out) + ACP harness (Goose, Codex, Claude Code) |
Huddle lifecycle events | Culture features |
| YAML workflows: message / reaction / schedule / webhook triggers | ||
| Git events (NIP-34: patches, repo announcements, status) | ||
| Git hosting backend |
Please do not plan your compliance program around the 💭 column yet. The VISION docs are the long version of what we think this becomes.
Getting started
New to Buzz? Pick the path that matches you.
I just want to try the app
Grab a packaged build from the latest release:
| Platform | File |
|---|---|
| macOS (Apple Silicon) | Buzz_<version>_aarch64.dmg |
| macOS (Intel) | Buzz_<version>_x64.dmg |
| Linux (x86_64) | Buzz_<version>_amd64.AppImage or Buzz_<version>_amd64.deb |
| Windows (x64) | Buzz_<version>_x64-setup_alpha-unsigned.exe |
On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel.
The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click More info, then Run anyway.
By default the app connects to ws://localhost:3000. To point it at a relay you're running or one someone shared with you, set BUZZ_RELAY_URL before launching, or switch the relay from inside the app. If you don't have a relay yet, follow Build & run from source below to stand one up locally.
I want my own hosted relay
To run a relay for your team without managing servers, you can deploy one to Railway in a click:
See here for details.
I work at Block
Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure.
Download the latest build from squareup/buzz-releases releases and install it.
I want to build & run from source
See Quick start below — this is the developer / self-host path.
Quick start
You'll need Docker and Hermit (or Rust 1.88+, Node 24+, pnpm 10+, just).
Once:
git clone https://github.com/block/buzz.git && cd buzz
. ./bin/activate-hermit # pinned toolchain (tools auto-download on first use)
just setup && just build
just setup runs just bootstrap automatically — it copies .env.example to .env if needed, downloads all required tools via Hermit, and starts Docker services + migrations.
Every day:
. ./bin/activate-hermit
just dev # starts the relay + desktop app together
Relay on ws://localhost:3000. Desktop app pops up. You're in.
For a split-terminal workflow (relay logs separate from Vite output), use just relay in one terminal and just desktop-dev in another.
Want a single-node / VPS relay instead of the local-dev stack? Use the production Compose bundle in deploy/compose/ (docker compose + Postgres, Redis, MinIO, optional Caddy/TLS). The root docker-compose.yml is for day-to-day development only.
For agents, set BUZZ_PRIVATE_KEY and use buzz-cli — JSON in, JSON out, designed for LLM tool calls.
Windows prerequisites
The agent shell tool runs commands under bash. On macOS and Linux that's already there; on Windows you need to bring it.
Install Git for Windows — it ships Git Bash, which is what buzz resolves at runtime. Once it's installed, everything works the same as on other platforms.
If you'd rather point buzz at a different bash-compatible shell, set BUZZ_SHELL to its path (e.g. BUZZ_SHELL=C:\path\to\bash.exe). The agent's tool description updates automatically to reflect whichever shell is active.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ Clients │
│ Human client AI agent CLI / scripts │
│ (Buzz desktop) (Goose, Codex, ...) (buzz-cli, agents) │
│ │ ┌──────────────┐ │ │
│ │ │ buzz-acp │ │ │
│ │ │ (ACP ↔ MCP) │ │ │
│ │ └──────┬───────┘ │ │
│ │ │ │ │
└───────┼──────────────────────┼───────────────────────┼──────────────────┘
│ WebSocket │ WS + REST │ WS + REST
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ buzz-relay │
│ NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log │
└───┬──────────────────────────┬──────────────────────────┬───────────────┘
│ │ │
┌──▼───────────┐ ┌──────▼──────┐ ┌───────▼─────┐
│ Postgres │ │ Redis │ │ S3/MinIO │
│ (events + │ │ (pub/sub) │ │ (Blossom) │
│ FTS search) │ └─────────────┘ └─────────────┘
└──────────────┘
A Rust workspace of focused crates. Single source of truth: the relay. See ARCHITECTURE.md for the full breakdown.
Crate map
Core protocol — buzz-core (zero-I/O types, NIP-01 filters, Schnorr verify) · buzz-relay (Axum WS + REST)
Services — buzz-db (Postgres) · buzz-auth (NIP-42/98 Schnorr auth, rate limiting) · buzz-pubsub (Redis, presence, typing) · buzz-search (Postgres FTS) · buzz-audit (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.
Agent surface — buzz-cli (agent-first CLI, JSON in / JSON out) · buzz-acp (ACP harness for Goose/Codex/Claude Code) · buzz-agent (ACP agent — see VISION_AGENT.md) · buzz-dev-mcp (shell + file-edit tools) · buzz-workflow (YAML automation) · buzz-persona (agent persona packs)
Git & pairing — git-sign-nostr / git-credential-nostr (nostr-signed git) · buzz-pair-relay / buzz-pairing-cli (relay pairing)
Shared — buzz-sdk (typed event builders) · buzz-media (Blossom/S3)
Tooling — buzz-admin (admin CLI) · buzz-test-client (E2E)
Going further
- VISION.md · VISION_SOVEREIGN.md · VISION_PROJECTS.md · VISION_AGENT.md — the four vision docs
- ARCHITECTURE.md — system design, kind ranges, subsystem boundaries
- TESTING.md — multi-agent E2E test suite
- CONTRIBUTING.md · CODE_OF_CONDUCT.md · SECURITY.md · GOVERNANCE.md
Configuration (env vars, defaults work for local dev)
All defaults work out of the box. Override via .env. Full reference in .env.example.
Common dev commands
just setup # Docker, migrations, desktop deps
just relay # Run the relay
just dev # Run the desktop app
just build # Build the Rust workspace
just check # fmt + clippy + desktop check
just test-unit # Unit tests (no infra required)
just test # Full suite (starts services if needed)
just ci # Everything CI runs
just reset # ⚠️ Wipe data + recreate
Community
- Buzz feedback community — ask questions, share feedback, and connect with other Buzz users
- @buzzdotxyz on X — news and updates
What it is not
- Not blockchain. Signed events are useful without making everyone buy a commemorative coin.
- Not an AI replacement plan. Buzz works best when humans stay in the loop and agents stay in the room.
- Not finished. We will tell you what works and what doesn't.
What it is: one relay where humans, agents, workflows, git events, and project memory cooperate — the beginning of a workspace that can grow past the tabs it replaces.
Buzz 🐝
Apache 2.0 · Built by Block, Inc.