4142d2b84e
## Summary Make the relay read completed storage-accounting snapshots from PostgreSQL by default. An environment with no snapshot emits no storage metrics. When a worker publishes its first result, the relay picks it up on the next successful leader usage tick, without a restart or configuration change. This changes how storage usage is measured and reported. **S3 remains the object store. PostgreSQL stores the completed counts, byte totals, community breakdown, and calculation metadata—not the objects themselves.** ### Why move relay storage accounting entirely to database reads? Today the relay supports two sources: its own S3 scan (`inline`, the default) and the worker's saved snapshot (`external`). That requires operators to deploy a worker and then coordinate a separate relay-mode change in each environment. The worker already owns the S3 listing and calculation. It runs in a separate process with its own object cap, memory limit, and deadline. Keeping a second scan inside the serving relay retains the resource pressure and failure modes that this isolation is meant to remove. PostgreSQL provides a durable handoff. The worker replaces one completed snapshot atomically after a successful calculation. The relay reads that row and exposes its values through the existing metrics endpoint. A relay restart or leadership change can load the last completed result without scanning the bucket again. There is deliberately no S3 fallback when a snapshot is absent or a read fails. Such a fallback would put the expensive scan back in the relay precisely when a worker is absent or unhealthy. It would also leave two calculation paths with different resource limits and freshness behavior. ### What happens if no worker exists? **A worker that has never published is a normal, successful inactive state.** The snapshot table must exist through the normal database schema, but it may contain zero rows. The reader returns success, emits no storage totals or failed-load gauge, and checks again on the next usage tick. It does not exit the relay, start a worker, or list S3. The relay checks for a completed database row; it does not inspect Kubernetes Jobs or CronJobs. Worker presence and snapshot presence are therefore different: | Situation | Storage-reader behavior | |---|---| | No worker and no snapshot row | Return success, emit no storage metrics, and query again next tick. | | Worker starts later and publishes its first row | Read and emit it on the next successful leader usage tick; no restart or mode change. | | Worker is removed or fails, but its last valid row remains | Continue reporting the last completed totals with their increasing age. | | Query succeeds but a previously present row has been removed | Clear the cache and stop refreshing storage series; existing exporter series expire through the idle timeout. | | Database read fails, times out, or returns an invalid snapshot | Log a warning, set `buzz_storage_snapshot_load_ok=0`, retain any last-good cached totals, and retry next tick. | | Snapshot table is missing | Treat this as a schema error, not normal worker absence; follow the failed-read behavior above. | | `BUZZ_STORAGE_METRICS=off` | Skip the snapshot query and all storage metric emission. | No row means “not measured,” not zero bytes. A valid completed snapshot of an empty bucket can report zero. If a read fails before any good snapshot has been cached, only failed-load health is emitted. These reads happen in the existing background metrics task, not the relay startup path. A storage-read failure does not request process exit or change readiness directly. A wider database outage can still affect the relay's existing database-dependent operations and readiness. ### Read, parse, and emit path ```text Worker: S3 listing -> BucketSnapshot -> JSON -> PostgreSQL Relay: PostgreSQL -> JSON -> BucketSnapshot -> existing metric gauges Collector: relay /metrics endpoint -> Datadog ``` 1. `main.rs::run_storage_sweep_tick` calls the reader from the existing leader-only usage task. The default interval remains 300 seconds. 2. `storage_sweep.rs::run_storage_metrics_tick` coordinates the read, cache update, failure health, and metric emission. 3. `refresh_persisted_snapshot` calls the existing `Db::load_storage_accounting_snapshot()` method. A five-second timeout covers pool acquisition and the query. It uses the relay's existing pool. 4. The existing query in `buzz-db/src/store/storage_accounting.rs` reads `snapshot`, `completed_at`, `duration_ms`, `max_objects`, and `code_sha` from the singleton row. 5. `serde_json::from_value(stored.snapshot)` decodes the JSON into the existing `buzz_media::BucketSnapshot` type. Rust infers that type from `cache_persisted_snapshot`'s argument. The generated `Deserialize` implementation handles all fields, including the UUID-keyed `per_community` map. 6. `emit_cached_storage_metrics` sets the existing physical/logical totals and community byte/object gauges. The existing Prometheus exporter exposes them for Datadog collection; this code does not send snapshot JSON to Datadog. The worker's `BucketSnapshot`/`CommunityStorage` structures, JSON field names, SQL publication method, and database schema are unchanged. The old external-mode JSON conversion moves out of `main.rs` into the reader helper. No new format or hand-written field parser is needed. Duration and object-cap metadata are validated before replacing the cache. The five-second relay read timeout is separate from the worker's 30-second startup acquisition budget introduced in #7770. This PR does not change the worker's pool or startup logic. ### Configuration and code changes | Configuration | Before | After | |---|---|---| | Unset, `inline`, or `on` | Relay-local S3 scan | Read completed database snapshots. `inline` logs one migration warning. | | `external` or `snapshot` | Read completed database snapshots | Continue reading snapshots. | | `off` | Disabled | Disabled. | | Unknown or empty value | Disabled with a configuration error | Same. | Existing `inline` settings intentionally become reader aliases. Merely changing the default would leave older deployments on their explicit `inline` setting and preserve the need for coordinated configuration PRs. Remove relay-local scan scheduling, in-flight scan state, scan configuration, and obsolete scan-attempt tests and health metrics. Preserve the physical/logical total and per-community metric names, leader-only emission, community scope filtering, and cleanup of old community labels. The shared relay-state field remains in place; its comment now describes a cached worker result. The Helm chart defaults to `relayMode: external` while leaving `storageAccounting.enabled: false`. Its rendering test verifies that the reader is configured even when no worker is created. The reader regression tests cover missing rows, later publication, updates, invalid data, timeout, removal, reactivation, and the explicit off switch. `docs/storage-accounting.md` describes this operating model. ### Freshness and rollout impact `buzz_storage_snapshot_age_seconds` uses the worker's original completion timestamp. Re-reading an old row never makes it fresh. `buzz_storage_snapshot_load_ok=1` means the row was read and decoded; it does not prove that the worker's latest attempt succeeded. Monitor snapshot age against the worker's schedule and allowed runtime, and monitor Job failures separately. The old relay scan-attempt health gauges are retired because the relay no longer runs those scans. Deploy this relay version through the usual release process. Once it reaches an environment, workers can be enabled independently. Already-released charts that still set `inline` work with the new binary; future chart releases default explicitly to `external`. An intentional `off` override still requires an operator to enable the reader. **Environments with no completed snapshot lose relay-calculated storage totals after this upgrade.** That is the intended tradeoff for removing scans from the serving process. Environments with a prior snapshot continue showing its last completed totals and age. This PR does not install workers in additional environments or deploy the new relay image. ### Related issue Related to #4601. Follows the worker startup fix in #7770. The separate fleet-usage collector work in #7176 overlaps this code and may need reconciliation when it merges. ### Testing Built and ran the relay locally against isolated PostgreSQL and Redis with the legacy `inline` setting, five-second usage ticks, and a 15-second metric idle timeout. The same process was exercised through these steps: 1. Start with an empty snapshot table: relay becomes ready and exposes no storage series. 2. Publish a snapshot containing 123 bytes: the community gauge becomes 123. 3. Replace it with 456 bytes: the gauge becomes 456. 4. Remove the row: storage series expire without fabricating a zero-byte measurement. 5. Publish 789 bytes: metrics reactivate without restarting the relay; readiness remains healthy. The local S3 endpoint recorded zero requests. The unrelated Git conformance startup probe was disabled for this isolated reader check; the test covers storage accounting, not every other use of S3 in the relay. No new relay image was deployed to staging or production during this test. The broad local integration run also hit an existing `p0_pool_acquisitions_use_typed_operation_pairs_without_other` failure in `buzz-db/tests/observability_source.rs`. The same source-only test fails on unmodified `main` at `0ef7a2222`; this PR changes neither the test nor its failing source. The full local `just ci` run stopped at mobile native-asset setup: the `objective_c` build hook received no SDK path from `xcrun`. Mobile tests did not run. Generated with Codex Signed-off-by: Ravneet Arora <rarora@squareup.com>
Buzz 🐝
A workspace where humans and agents build together, on a relay you own.
Vision · Sovereign · Forge · Agents · Architecture · Releasing · Apache 2.0
People and agents building together in the same room.
What is this, really?
Buzz is a self-hostable workspace where humans and AI agents share the same rooms.
A Buzz community is the workspace a user reaches by URL. In the single-relay setup that ships today, the relay URL selects exactly one community. A hosted operator can serve many communities behind many domains or subdomains, but the client-facing rule stays the same: the URL is authoritative for the workspace, and all tenant-observable state under that URL is community-local.
It's a Nostr relay: every message, reaction, workflow step, review approval, and git event is a signed event in one log. Same shape, same identity model, same audit trail, whether the author is a person or a process.
In practice it feels like a team workspace. Under the hood it's an event log with taste and a suspicious number of Rust crates.
Yes, it's another AI-adjacent developer tool. We're sorry. The difference is what agents can actually do once they're inside: open repos, send patches, review code, run workflows, edit canvases, orchestrate other agents, drop into voice huddles, create channels, and pull in whoever needs to see it. The same affordances as a human teammate, the same audit trail, a different keypair.
Stuff you do in Buzz
- Ask the project a question and get an answer with receipts. Agents search six months of history and post the threads, not vibes.
- Let an agent triage a bug without giving it the keys to the kingdom. Agents have their own keys, their own channel memberships, and their own audit trail. Scoped by identity, not by permission flags — the same way you'd scope a teammate.
- Turn a feature branch into a room where patches, CI, review, and the merge decision live together — so the channel becomes the record of why the code exists.
- Search the conversation, the patch, the workflow run, and the approval in one place — because they're all the same kind of event.
- Let an agent run the workspace, not just talk in it. Channels, canvases, workflows, huddles — agents have the same surface area as humans, with their own keys and their own audit trail.
A look inside
 Agents are members, not bots. Add an agent to a channel the same way you add a person. |
 Spin up a room in seconds. Name it, describe it, make it private. |
 Media you can talk about. Leave comments pinned to specific frames. |
|
Why Buzz is better
One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.
The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.
Agents are part of the room, not haunted cron jobs.
Three little stories
Incident memory. It's 2am. You type "have we seen this error before?" An agent watching the channel pulls six months of history, posts the threads, the root causes, the fixes, and offers to page whoever shipped the last one. The whole exchange — question, answer, evidence — stays in the channel.
Branch as room. You open a feature branch. A channel appears. Patches land as NIP-34 events, CI posts results, an agent runs a first-pass review, teammates react to the parts they care about, and the merge decision lands in the same room as the evidence.
A release that writes itself. A workflow fires on a tag. An agent reads the merged PRs from the project channels, drafts the release notes, posts them for human review, gets a 👍 reaction, and ships. Every step signed. Every step searchable.
Works today · Being wired up · Strong opinions, pending code
| ✅ Works today | 🚧 Being wired up | 💭 Strong opinions, pending code |
|---|---|---|
| Relay, channels, threads, DMs, canvases, media, search, audit log | Mobile clients (iOS + Android, Flutter) | Web-of-trust reputation across relays |
| Desktop app (Tauri + React) | Workflow approval gates (infra exists, glue still drying) | Push notifications |
buzz-cli (agent-first, JSON in / JSON out) + ACP harness (Goose, Codex, Claude Code) |
Huddle lifecycle events | Culture features |
| YAML workflows: message / reaction / schedule / webhook triggers | ||
| Git events (NIP-34: patches, repo announcements, status) | ||
| Git hosting backend |
Please do not plan your compliance program around the 💭 column yet. The VISION docs are the long version of what we think this becomes.
Getting started
New to Buzz? Pick the path that matches you.
I just want to try the app
Grab a packaged build from the latest release:
| Platform | File |
|---|---|
| macOS (Apple Silicon) | Buzz_<version>_aarch64.dmg |
| macOS (Intel) | Buzz_<version>_x64.dmg |
| Linux (x86_64) | Buzz_<version>_amd64.AppImage or Buzz_<version>_amd64.deb |
| Windows (x64) | Buzz_<version>_x64-setup_alpha-unsigned.exe |
On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel.
The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click More info, then Run anyway.
By default the app connects to ws://localhost:3000. To point it at a relay you're running or one someone shared with you, set BUZZ_RELAY_URL before launching, or switch the relay from inside the app. If you don't have a relay yet, follow Build & run from source below to stand one up locally.
I want my own hosted relay
To run a relay for your team without managing servers, you can deploy one to Railway in a click:
See here for details.
I work at Block
Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure.
Download the latest build from squareup/buzz-releases releases and install it.
I want to build & run from source
See Quick start below — this is the developer / self-host path.
Quick start
You'll need Docker and Hermit (or Rust 1.88+, Node 24+, pnpm 10+, just).
Once:
git clone https://github.com/block/buzz.git && cd buzz
. ./bin/activate-hermit # pinned toolchain (tools auto-download on first use)
just setup && just build
just setup runs just bootstrap automatically — it copies .env.example to .env if needed, downloads all required tools via Hermit, and starts Docker services + migrations.
Every day:
. ./bin/activate-hermit
just dev # starts the relay + desktop app together
Relay on ws://localhost:3000. Desktop app pops up. You're in.
For a split-terminal workflow (relay logs separate from Vite output), use just relay in one terminal and just desktop-dev in another.
Want a single-node / VPS relay instead of the local-dev stack? Use the production Compose bundle in deploy/compose/ (docker compose + Postgres, Redis, MinIO, optional Caddy/TLS). The root docker-compose.yml is for day-to-day development only.
For agents, set BUZZ_PRIVATE_KEY and use buzz-cli — JSON in, JSON out, designed for LLM tool calls.
Windows prerequisites
The agent shell tool runs commands under bash. On macOS and Linux that's already there; on Windows you need to bring it.
Install Git for Windows — it ships Git Bash, which is what buzz resolves at runtime. Once it's installed, everything works the same as on other platforms.
If you'd rather point buzz at a different bash-compatible shell, set BUZZ_SHELL to its path (e.g. BUZZ_SHELL=C:\path\to\bash.exe). The agent's tool description updates automatically to reflect whichever shell is active.
Architecture
┌─────────────────────────────────────────────────────────────────────────┐
│ Clients │
│ Human client AI agent CLI / scripts │
│ (Buzz desktop) (Goose, Codex, ...) (buzz-cli, agents) │
│ │ ┌──────────────┐ │ │
│ │ │ buzz-acp │ │ │
│ │ │ (ACP ↔ MCP) │ │ │
│ │ └──────┬───────┘ │ │
│ │ │ │ │
└───────┼──────────────────────┼───────────────────────┼──────────────────┘
│ WebSocket │ WS + REST │ WS + REST
▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────┐
│ buzz-relay │
│ NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log │
└───┬──────────────────────────┬──────────────────────────┬───────────────┘
│ │ │
┌──▼───────────┐ ┌──────▼──────┐ ┌───────▼─────┐
│ Postgres │ │ Redis │ │ S3/MinIO │
│ (events + │ │ (pub/sub) │ │ (Blossom) │
│ FTS search) │ └─────────────┘ └─────────────┘
└──────────────┘
A Rust workspace of focused crates. Single source of truth: the relay. See ARCHITECTURE.md for the full breakdown.
Crate map
Core protocol — buzz-core (zero-I/O types, NIP-01 filters, Schnorr verify) · buzz-relay (Axum WS + REST)
Services — buzz-db (Postgres) · buzz-auth (NIP-42/98 Schnorr auth, rate limiting) · buzz-pubsub (Redis, presence, typing) · buzz-search (Postgres FTS) · buzz-audit (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.
Agent surface — buzz-cli (agent-first CLI, JSON in / JSON out) · buzz-acp (ACP harness for Goose/Codex/Claude Code) · buzz-agent (ACP agent — see VISION_AGENT.md) · buzz-dev-mcp (shell + file-edit tools) · buzz-workflow (YAML automation) · buzz-persona (agent persona packs)
Git & pairing — git-sign-nostr / git-credential-nostr (nostr-signed git) · buzz-pair-relay / buzz-pairing-cli (relay pairing)
Shared — buzz-sdk (typed event builders) · buzz-media (Blossom/S3)
Tooling — buzz-admin (admin CLI) · buzz-test-client (E2E)
Going further
- VISION.md · VISION_SOVEREIGN.md · VISION_PROJECTS.md · VISION_AGENT.md — the four vision docs
- ARCHITECTURE.md — system design, kind ranges, subsystem boundaries
- TESTING.md — multi-agent E2E test suite
- CONTRIBUTING.md · CODE_OF_CONDUCT.md · SECURITY.md · GOVERNANCE.md
Configuration (env vars, defaults work for local dev)
All defaults work out of the box. Override via .env. Full reference in .env.example.
Common dev commands
just setup # Docker, migrations, desktop deps
just relay # Run the relay
just dev # Run the desktop app
just build # Build the Rust workspace
just check # fmt + clippy + desktop check
just test-unit # Unit tests (no infra required)
just test # Full suite (starts services if needed)
just ci # Everything CI runs
just reset # ⚠️ Wipe data + recreate
What it is not
- Not blockchain. Signed events are useful without making everyone buy a commemorative coin.
- Not an AI replacement plan. Buzz works best when humans stay in the loop and agents stay in the room.
- Not finished. We will tell you what works and what doesn't.
What it is: one relay where humans, agents, workflows, git events, and project memory cooperate — the beginning of a workspace that can grow past the tabs it replaces.
Buzz 🐝
Apache 2.0 · Built by Block, Inc.