Ian Oberst a8fe2b612c feat(db): advance an empty partition catch-all under bounded locks (#7993)
## Summary

Adds bounded, serialized advancement of an empty right-edge partition
catch-all for `events` and `delivery_log`. The repaired catalogs
currently end in an empty catch-all from 2027-01-01, and the merged
manager (#6515) only audits them. Without this change, January writes
land in the catch-all, after which only an operator can move them.

When `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED=true`, each relay keeps six
dedicated monthlies after the current month. It does this at startup and
on every audit interval by replacing the empty catch-all with canonical
monthlies and a new canonical `<table>_p_future` after the horizon. From
today's catalogs the first run creates January–March 2027 and moves the
catch-all to April 2027.

## Safety design

- **Refuse before locking.** The following return `operator_required`
without taking any table lock:
  - a populated catch-all, or one whose occupancy is unknown;
- a catch-all with child-only triggers, which the replacements would not
inherit;
  - a misaligned bound;
  - a `DEFAULT`, anomalous or pending-detach child;
  - a name collision.
- **One winner.** A schema-scoped `pg_try_advisory_xact_lock` is taken
before any DDL. Losers report `skipped_locked` and never issue DDL. The
winner re-plans from a fresh audit before taking any table lock and
checks for collisions. A runner that lost a race therefore sees the
committed layout as a no-op, not as colliding names.
- **Writer lock order.** First `ONLY` the parent is locked `ACCESS
EXCLUSIVE` under a 2 s `lock_timeout`. Then each foreign-key counterpart
(today only `communities`) is locked in OID order `SHARE ROW EXCLUSIVE
NOWAIT`, then the catch-all `NOWAIT`.
- The counterparts are read again once the parent is locked, and that
set is the one locked. Any FK DDL on or to the parent conflicts with the
parent lock, so the set can't change after that. An FK committed while
maintenance waited is therefore locked `NOWAIT` too, so attaching never
waits on it under the parent lock. The read before the lock remains as a
cheap early refusal.
- This is the order an insert takes locks (parent, then the FK check on
`communities`), so maintenance cannot deadlock with ingest. The internal
manual runbook used to lock `communities` first; it now uses this order
too.
- `SHARE ROW EXCLUSIVE` is the strongest lock the DDL takes on
`communities`; measured on PG 16 and 17, the catch-all `DROP` takes
none, as long as the catch-all's only foreign keys are those inherited
from the parent. A catch-all with its own foreign key, or one that
another table's foreign key references, is refused: once before any
lock, and again after the catch-all lock, where the check is
authoritative. Readers and other writers' `FOR KEY SHARE` checks
proceed. A concurrent `communities` writer makes the run fail fast with
`lock_timeout`.
  - A `40P01` deadlock victim is also reported as `lock_timeout`.
- Parents that are referenced by a foreign key, or that reference an
inherited table, are refused, including when such a key commits while
maintenance waits on the parent.
- **Re-plan under lock.** The catalog is re-audited under those locks.
The transaction proceeds only if the plan is unchanged.
- **Bounded.** `statement_timeout` 5 s,
`idle_in_transaction_session_timeout` 5 s, and a 10 s client deadline
per table. A deadline-abandoned connection is closed, never returned to
the pool, and the server rolls back anything uncommitted. If the
deadline fires after `COMMIT` was sent, the change may have landed; the
next audit reports the truth. The orphaned backend keeps its locks until
its in-flight statement ends, at most `statement_timeout`.
- **Verify before commit.** The changed catalog is re-audited inside the
transaction for bounds, canonical kinds, trigger parity and catch-all
emptiness, and serving safety must not regress. It also checks that
every new child has valid, ready clones of each parent index and
validated clones of each PK/unique/FK constraint.

- **Audits tolerate a concurrent advance.** Catalog rows come from the
statement snapshot, but `pg_get_expr` renders from current catalog
state. So an audit that overlaps another relay's `DROP` of a catch-all
can see a partition that no longer exists. The read-only audit detects
this (a NULL bound, or `42P01` from the emptiness probe) and retries up
to twice with a fresh snapshot.
- **Fail fast on audit probes.** If another relay's audit is probing the
catch-all's emptiness, the catch-all `NOWAIT` lock fails and the run
reports `lock_timeout`. The parent lock is then released immediately,
and the next runner or interval completes the advance.

Uncovered-month creation (`BUZZ_PARTITION_MANAGER_CREATE_ENABLED`,
default `true`) now runs through the same locked transaction, replacing
the unlocked per-month `CREATE`. That flag remains the kill switch for
all automatic partition DDL: with it off, advancement is disabled too.
With advancement disabled (the default) the periodic loop stays
read-only and startup behaves as before, apart from the locking.

A runner that planned DDL always re-audits before returning, even when a
peer did the work, so it never reports the pre-maintenance layout. A
serialized re-plan that finds only disabled work reports
`skipped_disabled`.

## Behavior changes

- The relay audit horizon and the `buzz-admin partition-audit` default
are now 6 months (`PARTITION_MANAGER_MONTHS_AHEAD`), up from 3. Until
advancement is enabled, months already covered by the catch-all make
`degraded` true. That correctly reports a runway under six months.
- New metric `buzz_partition_maintenance_runs_total{table,outcome}` with
outcomes `noop | skipped_disabled | created | advanced | skipped_locked
| operator_required | lock_timeout | deadline | error`.
`buzz_partition_create_attempts_total` keeps its per-month `created |
skipped_covered | error` labels. `error` there still means a DDL or
collision failure; `lock_timeout` and `deadline` runs add nothing to it,
and only `buzz_partition_maintenance_runs_total` records them.
- `ensure_future_partitions` (free function and `Db` method) is removed;
`maintain_partitions` is the one entry point.
- New advisory-lock label `lock_type="partition_maintenance"`.

## Rollout

Enable `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED` per environment in BPCI:
`bb-block-staging` first, then production. The staging canary proves a
real advance because the six-month horizon already extends past January.
Fallback: rerun the internal manual runbook, which uses the same
parent-first lock order as of 2026-10-05. It keeps `ACCESS EXCLUSIVE` on
the counterparts, because its detach/attach can drop FK triggers on
them.

## Testing

- `cargo test -p buzz-db --lib partition`: 17 unit tests. The 10 new
ones cover:
- plans: repaired layout, policy gating, no-op, refusal states including
child-only triggers, December→January;
  - the create kill switch also stopping advancement;
  - the month cap at exactly 120;
- per-month metric outcomes for transient failures versus DDL failures;
  - the mid-audit-drop retry classifier.
- `cargo test -p buzz-db --lib partition -- --ignored` (PostgreSQL 17):
48/48 on 9 consecutive runs, including 17 new tests:
  - advance to the canonical layout, with routing and idempotent rerun;
  - replacing a canonically named catch-all in one transaction;
  - populated catch-all refused in under 1 s while the parent is locked;
  - a `communities` writer making the run fail fast and roll back;
  - a `communities` reader not blocking the advance;
- an in-flight insert's FK check, and its write to `communities`, behind
queued maintenance completing without a deadlock;
- a foreign key committed while maintenance waits on the parent, with a
writer on its new counterpart, failing fast as `lock_timeout` on that
table instead of waiting under the parent lock;
- a foreign key referencing `events`, committed while maintenance waits
on the parent, being refused under the parent lock;
- a catch-all's own foreign key, or a foreign key referencing it, being
refused before the parent lock with the key intact, and a catch-all key
committed while maintenance waits on the parent being refused after the
catch-all lock;
  - a `40P01` deadlock victim being classified as `lock_timeout`;
  - parent contention timing out without blocking the other table;
  - held advisory lock skipping with no DDL;
- concurrent runners converging, with at most a transient
`lock_timeout`;
- a stale audit after a concurrent advance being a no-op, and the losing
runner returning the committed layout;
- a re-plan that finds only disabled work reporting `skipped_disabled`;
  - an audit retrying when a listed partition is dropped mid-audit;
  - a catch-all with a child-only trigger being refused;
- client deadline abandoning the attempt and closing the session (scoped
to the test's own blocked backend);
  - catch-all name collision;
  - incoming foreign key refused.
- Mutation checks:
- restoring the old counterpart-first `ACCESS EXCLUSIVE` order fails the
reader test (`lock_timeout`) and the writer test (a ~1 s deadlock wait);
- dropping `NOWAIT` from the counterpart lock, or allowing a populated
catch-all, fails the matching tests;
- removing the parent-locked counterpart re-read fails the committed-FK
tests (the parent is held 2.06 s waiting on the new counterpart), and
ignoring its refusal fails the referencing-FK test;
- locking the counterparts `SHARE ROW EXCLUSIVE` before the parent fails
the writer test;
  - removing either catch-all foreign-key check fails its test.
- Full PostgreSQL lane: 716/718. Both failures are timing-sensitive
tests outside partition code, and each passes 3/3 when run alone:
  - `hanging_redis_peer_times_out_before_the_health_listener_binds`;
-
`readiness_check_cancellation_balances_waiter_and_inflight_connection`.
- `cargo clippy -p buzz-db -p buzz-relay -p buzz-admin --all-targets -D
warnings` and `cargo fmt --check` are clean.

## Related

Refs #2396.

This PR conflicts textually with #7994 (partition audit observability
follow-ups) in `crates/buzz-db/src/store/partition.rs` and
`crates/buzz-relay/src/main.rs`. Whichever PR merges second will be
rebased onto the other.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 12:59:10 -07:00
2026-03-06 21:00:58 +00:00
2026-06-10 21:07:02 +00:00
2026-07-27 14:18:24 -04:00

Buzz 🐝

A workspace where humans and agents build together, on a relay you own.

Vision · Sovereign · Forge · Agents · Architecture · Releasing · Apache 2.0

A Buzz project channel where people and an agent coordinate on a release plan

People and agents building together in the same room.


What is this, really?

Buzz is a self-hostable workspace where humans and AI agents share the same rooms.

A Buzz community is the workspace a user reaches by URL. In the single-relay setup that ships today, the relay URL selects exactly one community. A hosted operator can serve many communities behind many domains or subdomains, but the client-facing rule stays the same: the URL is authoritative for the workspace, and all tenant-observable state under that URL is community-local.

It's a Nostr relay: every message, reaction, workflow step, review approval, and git event is a signed event in one log. Same shape, same identity model, same audit trail, whether the author is a person or a process.

In practice it feels like a team workspace. Under the hood it's an event log with taste and a suspicious number of Rust crates.

Yes, it's another AI-adjacent developer tool. We're sorry. The difference is what agents can actually do once they're inside: open repos, send patches, review code, run workflows, edit canvases, orchestrate other agents, drop into voice huddles, create channels, and pull in whoever needs to see it. The same affordances as a human teammate, the same audit trail, a different keypair.


Stuff you do in Buzz

  • Ask the project a question and get an answer with receipts. Agents search six months of history and post the threads, not vibes.
  • Let an agent triage a bug without giving it the keys to the kingdom. Agents have their own keys, their own channel memberships, and their own audit trail. Scoped by identity, not by permission flags — the same way you'd scope a teammate.
  • Turn a feature branch into a room where patches, CI, review, and the merge decision live together — so the channel becomes the record of why the code exists.
  • Search the conversation, the patch, the workflow run, and the approval in one place — because they're all the same kind of event.
  • Let an agent run the workspace, not just talk in it. Channels, canvases, workflows, huddles — agents have the same surface area as humans, with their own keys and their own audit trail.

A look inside

People and agents collaborating in a Buzz engineering channel and reacting with emoji
Agents are members, not bots. Add an agent to a channel the same way you add a person.
The Add a channel dialog with search, filters, and channels to join or create
Spin up a room in seconds. Name it, describe it, make it private.
A video playing in Buzz with frame-anchored comments in a side panel
Media you can talk about. Leave comments pinned to specific frames.

Why Buzz is better

One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.

The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.

Agents are part of the room, not haunted cron jobs.


Three little stories

Incident memory. It's 2am. You type "have we seen this error before?" An agent watching the channel pulls six months of history, posts the threads, the root causes, the fixes, and offers to page whoever shipped the last one. The whole exchange — question, answer, evidence — stays in the channel.

Branch as room. You open a feature branch. A channel appears. Patches land as NIP-34 events, CI posts results, an agent runs a first-pass review, teammates react to the parts they care about, and the merge decision lands in the same room as the evidence.

A release that writes itself. A workflow fires on a tag. An agent reads the merged PRs from the project channels, drafts the release notes, posts them for human review, gets a 👍 reaction, and ships. Every step signed. Every step searchable.


Works today · Being wired up · Strong opinions, pending code

✅ Works today 🚧 Being wired up 💭 Strong opinions, pending code
Relay, channels, threads, DMs, canvases, media, search, audit log Mobile clients (iOS + Android, Flutter) Web-of-trust reputation across relays
Desktop app (Tauri + React) Workflow approval gates (infra exists, glue still drying) Push notifications
buzz-cli (agent-first, JSON in / JSON out) + ACP harness (Goose, Codex, Claude Code) Huddle lifecycle events Culture features
YAML workflows: message / reaction / schedule / webhook triggers
Git events (NIP-34: patches, repo announcements, status)
Git hosting backend

Please do not plan your compliance program around the 💭 column yet. The VISION docs are the long version of what we think this becomes.


Getting started

New to Buzz? Pick the path that matches you.

I just want to try the app

Grab a packaged build from the latest release:

Platform File
macOS (Apple Silicon) Buzz_<version>_aarch64.dmg
macOS (Intel) Buzz_<version>_x64.dmg
Linux (x86_64) Buzz_<version>_amd64.AppImage or Buzz_<version>_amd64.deb
Windows (x64) Buzz_<version>_x64-setup_alpha-unsigned.exe

On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel.

The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click More info, then Run anyway.

By default the app connects to ws://localhost:3000. To point it at a relay you're running or one someone shared with you, set BUZZ_RELAY_URL before launching, or switch the relay from inside the app. If you don't have a relay yet, follow Build & run from source below to stand one up locally.

I want my own hosted relay

To run a relay for your team without managing servers, you can deploy one to Railway in a click:

Deploy on Railway

See here for details.

I work at Block

Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure.

Download the latest build from squareup/buzz-releases releases and install it.

I want to build & run from source

See Quick start below — this is the developer / self-host path.


Quick start

You'll need Docker and Hermit (or Rust 1.88+, Node 24+, pnpm 10+, just).

Once:

git clone https://github.com/block/buzz.git && cd buzz
. ./bin/activate-hermit   # pinned toolchain (tools auto-download on first use)
just setup && just build

just setup runs just bootstrap automatically — it copies .env.example to .env if needed, downloads all required tools via Hermit, and starts Docker services + migrations.

Every day:

. ./bin/activate-hermit
just dev   # starts the relay + desktop app together

Relay on ws://localhost:3000. Desktop app pops up. You're in.

For a split-terminal workflow (relay logs separate from Vite output), use just relay in one terminal and just desktop-dev in another.

Want a single-node / VPS relay instead of the local-dev stack? Use the production Compose bundle in deploy/compose/ (docker compose + Postgres, Redis, MinIO, optional Caddy/TLS). The root docker-compose.yml is for day-to-day development only.

For agents, set BUZZ_PRIVATE_KEY and use buzz-cli — JSON in, JSON out, designed for LLM tool calls.


Windows prerequisites

The agent shell tool runs commands under bash. On macOS and Linux that's already there; on Windows you need to bring it.

Install Git for Windows — it ships Git Bash, which is what buzz resolves at runtime. Once it's installed, everything works the same as on other platforms.

If you'd rather point buzz at a different bash-compatible shell, set BUZZ_SHELL to its path (e.g. BUZZ_SHELL=C:\path\to\bash.exe). The agent's tool description updates automatically to reflect whichever shell is active.


Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                             Clients                                     │
│  Human client         AI agent              CLI / scripts               │
│  (Buzz desktop)       (Goose, Codex, ...)   (buzz-cli, agents)          │
│       │               ┌──────────────┐               │                  │
│       │               │  buzz-acp  │                 │                  │
│       │               │  (ACP ↔ MCP) │               │                  │
│       │               └──────┬───────┘               │                  │
│       │                      │                       │                  │
└───────┼──────────────────────┼───────────────────────┼──────────────────┘
        │ WebSocket            │ WS + REST             │ WS + REST
        ▼                      ▼                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          buzz-relay                                     │
│  NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log  │
└───┬──────────────────────────┬──────────────────────────┬───────────────┘
    │                          │                          │
 ┌──▼───────────┐       ┌──────▼──────┐           ┌───────▼─────┐
 │   Postgres   │       │    Redis    │           │   S3/MinIO  │
 │ (events +    │       │  (pub/sub)  │           │  (Blossom)  │
 │  FTS search) │       └─────────────┘           └─────────────┘
 └──────────────┘

A Rust workspace of focused crates. Single source of truth: the relay. See ARCHITECTURE.md for the full breakdown.

Crate map

Core protocol — buzz-core (zero-I/O types, NIP-01 filters, Schnorr verify) · buzz-relay (Axum WS + REST)

Services — buzz-db (Postgres) · buzz-auth (NIP-42/98 Schnorr auth, rate limiting) · buzz-pubsub (Redis, presence, typing) · buzz-search (Postgres FTS) · buzz-audit (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.

Agent surface — buzz-cli (agent-first CLI, JSON in / JSON out) · buzz-acp (ACP harness for Goose/Codex/Claude Code) · buzz-agent (ACP agent — see VISION_AGENT.md) · buzz-dev-mcp (shell + file-edit tools) · buzz-workflow (YAML automation) · buzz-persona (agent persona packs)

Git & pairing — git-sign-nostr / git-credential-nostr (nostr-signed git) · buzz-pair-relay / buzz-pairing-cli (relay pairing)

Shared — buzz-sdk (typed event builders) · buzz-media (Blossom/S3)

Tooling — buzz-admin (admin CLI) · buzz-test-client (E2E)


Going further

Configuration (env vars, defaults work for local dev)

All defaults work out of the box. Override via .env. Full reference in .env.example.

Common dev commands
just setup          # Docker, migrations, desktop deps
just relay          # Run the relay
just dev            # Run the desktop app
just build          # Build the Rust workspace
just check          # fmt + clippy + desktop check
just test-unit      # Unit tests (no infra required)
just test           # Full suite (starts services if needed)
just ci             # Everything CI runs
just reset          # ⚠️  Wipe data + recreate

What it is not

  • Not blockchain. Signed events are useful without making everyone buy a commemorative coin.
  • Not an AI replacement plan. Buzz works best when humans stay in the loop and agents stay in the room.
  • Not finished. We will tell you what works and what doesn't.

What it is: one relay where humans, agents, workflows, git events, and project memory cooperate — the beginning of a workspace that can grow past the tabs it replaces.


Buzz 🐝
Apache 2.0 · Built by Block, Inc.

Description
Pull mirror of block/buzz (Apache-2.0): the Nostr workspace behind buzz.hal8.odyssey-works.io
Readme Apache-2.0 1.9 GiB
Languages
Rust 47.7%
TypeScript 25.6%
Dart 13.6%
JavaScript 9%
Swift 1.7%
Other 2.3%