Will Pfleger df3f28a2c4 feat(relay): add NIP-FI shadow mode for HTTP, WebSocket and admin (#8034)
Stack: this PR → #8062

🤖 Follows #8028 (merged).

Adds `BUZZ_NIP_FI_MODE=shadow`. The relay loads and validates the full
enforce configuration (issuers, communities, lifetimes, command
issuers), evaluates NIP-FI evidence wherever enforce would decide, and
records what enforce would have done, while every request, upgrade and
session behaves exactly as in `off`. The one approved exception is admin
disconnect: shadow verifies the command and returns `200`, where Off has
no verifier and returns `503`. It lets an operator measure the
would-deny rate, including Hosts missing from `BUZZ_NIP_FI_COMMUNITIES`,
before switching to `enforce`.

- **Mode predicates.** `NipFiMode` gains `is_off`, `restricts`,
`evaluates`, `enforces`, `denies_unconditionally` and `observes_only`,
built on two exhaustive `match` bases (`restricts`, `evaluates`);
`is_off` is a `matches!` and the rest derive from the bases, so a new
mode cannot silently fall into one side. Every call site uses them,
including the Git membership-lookup failure from #8005, which uses
`restricts()` so shadow keeps Off's body. A test scans each crate's
`src/` and fails on any `NipFiMode::<variant>` in production code
outside the predicate definitions and the env parser, which catches
`==`, `matches!` on any number of lines, `if let` and or-patterns.
- **Startup.** Shadow goes through enforce's path, so every enforce
startup refusal applies, including the required connection lifetime.
Missing-setting errors name the configured mode through one formatter
(`mode_requires`); the enforce text is byte-identical. Shadow builds the
assertion verifier and claims command replays under its own Redis
prefix, both through the production `AppState::new`.
- **HTTP and Blossom.** `admit_nip_fi_http` keeps Off's result in shadow
and additionally replays enforce in enforce's order: first the router
guard's steps (community, then assertion), then the handler's steps
(NIP-98, pairing, deny set) on the same NIP-98 proof, through
`evaluate_enforce_steps`, which enforce shares. A Host mapped in config
but missing from the communities table, with a valid assertion, records
an admit, because enforce's guard passes and its 404 is not a NIP-FI
denial. A dev-mode `X-Pubkey` identity is marked unsigned and is never
an enforce proof, so shadow records it as a `nip98` would-deny. On
bridge and Blossom routes, the strict proof check (payload tag, Blossom
`Strict`) runs as a separate pure check that consumes no replay entry
and records on its own series, so it never counts as an admission
verdict. Shadow records exactly one verdict for every identity,
community or credential refusal enforce would make, with Off's status,
body and challenge unchanged: bridge, workflows, Blossom, Git transport
and settings, GIF search and share, and invite minting record the
`community` would-deny when the Host is unmapped. Git's missing or
malformed credentials still reject before any database work, and shadow
records the enforce verdict for that failed proof. The router guard
stays transparent in shadow.
- **WebSocket, root and audio.** The upgrade check evaluates the
attached assertion once. An upgrade enforce would refuse records that
would-deny; otherwise the assertion goes into a shadow-only session
object and the upgrade returns as in Off. The assertion never reaches
the connection, the identity registries, the admission gate, terminal
frames or cancellation. Each session records at most one admission
verdict. **Pairing** is recorded on both ingresses right after a valid
NIP-42 proof, before ordinary relay policy, where enforce pairs: a key
mismatch or claimless assertion is a `pairing` denial. **The deny-set
check** runs where enforce runs it, after registration: a hit is a
`deny_set` denial, and a clean read keeps the session pending and
registered. **The admit** is recorded only when the connection is
actually admitted: on root at the AUTH commit, after ordinary policy; on
audio when the participant join transaction commits, after relay
membership, channel membership, the final ban and membership decision,
mesh routing and the join's own checks, and before publication. A NIP-42
failure, which comes before pairing, and any refusal between pairing and
the admit (ordinary policy, mesh, or a join refusal inside the join
transaction) retires the observer when the refusal is decided, before
any refusal frame, rollback or cleanup is awaited, so it leaves no
record even if the deadline passes during that cleanup. A record-only
deadline task, armed at upgrade with enforce's
`compute_session_deadline`, records `expired` when enforce would have
closed an admitted session. If the deadline passes while the session is
still pending, that is its single admission verdict, a denial with
`stage=deadline`, and a later AUTH records nothing. A deny for the key
(admin or cross-pod) records `revoked` on an admitted session; if it
arrives after the deny-set check but before the admit, including a deny
refused for capacity, the session keeps a pending-revocation mark and
records the admit followed by exactly one `revoked` end, matching
enforce's phase-independent close, and the deadline cannot take that
end. The phase, the mark and retirement change together under one
session lock. Each session records at most one end. The session holds
the socket's cancellation token for recording only: once the socket is
cancelled, no transition records anything, even while connection
references remain (a cancel fence; the token never feeds a close
action). Upgrade would-denies carry the ingress route, `ws` or `audio`.
- **Admin disconnect.** Shadow verifies the command and records the deny
entry, so later admissions record would-denies, but closes no session;
the response stays `{"disconnected": true}`. Shadow publishes cross-pod
on its own channel, `buzz:nip-fi:shadow-disconnect`, which only shadow
pods subscribe to, so no enforce pod (including older builds that know
only `buzz:nip-fi:disconnect`) acts on a shadow command. Shadow pods
also keep listening on `buzz:nip-fi:disconnect`, so their deny record
includes enforce's real disconnects. Shadow claims command replays under
`buzz:nip-fi:shadow-command:{hash}`: replays are still rejected within
shadow, and a shadow accept never uses up the enforce claim. Enforce and
Off keep the same channel, keys and close behavior.
- **NIP-11.** Shadow serves the same document as enforce, including
`limitation.federated_identity`, because clients only attach evidence to
a relay that advertises it.

## Recording

| Series | Labels | One increment per |
|---|---|---|
| `buzz_nip_fi_shadow_total` | `route` (`http`, `ws`, `audio`), `stage`,
`outcome`, `community` | decision enforce would make: an HTTP admission,
a refused upgrade, a session's NIP-FI AUTH decision, or its deadline
passing before AUTH (`stage=deadline`) |
| `buzz_nip_fi_shadow_strict_proof_total` | `route` (`bridge`,
`blossom`), `outcome` (`pass`, `rejected`), `community` | strict NIP-98
side check |
| `buzz_nip_fi_shadow_session_end_total` | `route` (`ws`, `audio`),
`reason` (`expired`, `revoked`), `community` | admitted session enforce
would have ended (admitted sessions only) |
| `buzz_nip_fi_shadow_disconnect_total` | `route` (`admin`,
`cross_pod`), `outcome` | disconnect-path event, in place of the real
disconnect, lag, capacity and poison counters |

`community` is the configured canonical URI or `unmapped`, never the raw
`Host`; an unmapped or empty Host records `stage=community,
outcome=unavailable`. A shadow pod never moves an enforce disconnect or
failure counter. Strict-proof rates use their own `pass + rejected`
total, and session-end rates divide by the `admit` count for the same
route. There is no per-verdict `info` log; a would-deny writes a `debug`
line with no token, claim or issuer. The pre-existing enforce `debug`
line for the proven NIP-98 key is shared and also fires in shadow. Admin
commands rejected for a bad header, body or proof, a replay, or a
dependency error record no shadow verdict; only capacity rejection is
counted.

## Not observed

- Handler-side path-validation exits: an invalid Git owner or repo name
on default-branch, a media path that is not a valid hash, and a Git pack
URL with a query string record nothing from the handler. The router
guard still records what enforce would refuse at the guard (for example
a missing assertion), because enforce refuses those requests before it
validates the path.
- A refusal that is not a NIP-FI decision, such as a tenant-binding 404
behind a passing guard, or an ordinary WebSocket refusal after pairing
(a ban, membership or join refusal before admission).

## Follow-ups

- A bounded `route` label per HTTP surface (bridge, Blossom, Git, …) on
`buzz_nip_fi_shadow_total`; today every HTTP verdict carries
`route=http`.
- One shared helper for the ten copied `observe_unbound` recording
hooks, stacked on this PR.

The shadow recorder and session suites and the root pairing witness run
in `just test-unit` and the `scripts/run-tests.sh` fallback; the
`AppState::new` shadow witness, which exercises the command verifier
against real Redis, runs in the `external_infra_redis` lane. NIP-FI env
tests recover a poisoned env lock so one panic cannot cascade.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
2026-10-02 18:32:17 -04:00
…
2026-06-10 21:07:02 +00:00
2026-07-27 14:18:24 -04:00

Buzz 🐝

A workspace where humans and agents build together, on a relay you own.

Vision · Sovereign · Forge · Agents · Architecture · Releasing · Apache 2.0

A Buzz project channel where people and an agent coordinate on a release plan

People and agents building together in the same room.


What is this, really?

Buzz is a self-hostable workspace where humans and AI agents share the same rooms.

A Buzz community is the workspace a user reaches by URL. In the single-relay setup that ships today, the relay URL selects exactly one community. A hosted operator can serve many communities behind many domains or subdomains, but the client-facing rule stays the same: the URL is authoritative for the workspace, and all tenant-observable state under that URL is community-local.

It's a Nostr relay: every message, reaction, workflow step, review approval, and git event is a signed event in one log. Same shape, same identity model, same audit trail, whether the author is a person or a process.

In practice it feels like a team workspace. Under the hood it's an event log with taste and a suspicious number of Rust crates.

Yes, it's another AI-adjacent developer tool. We're sorry. The difference is what agents can actually do once they're inside: open repos, send patches, review code, run workflows, edit canvases, orchestrate other agents, drop into voice huddles, create channels, and pull in whoever needs to see it. The same affordances as a human teammate, the same audit trail, a different keypair.


Stuff you do in Buzz

  • Ask the project a question and get an answer with receipts. Agents search six months of history and post the threads, not vibes.
  • Let an agent triage a bug without giving it the keys to the kingdom. Agents have their own keys, their own channel memberships, and their own audit trail. Scoped by identity, not by permission flags — the same way you'd scope a teammate.
  • Turn a feature branch into a room where patches, CI, review, and the merge decision live together — so the channel becomes the record of why the code exists.
  • Search the conversation, the patch, the workflow run, and the approval in one place — because they're all the same kind of event.
  • Let an agent run the workspace, not just talk in it. Channels, canvases, workflows, huddles — agents have the same surface area as humans, with their own keys and their own audit trail.

A look inside

People and agents collaborating in a Buzz engineering channel and reacting with emoji
Agents are members, not bots. Add an agent to a channel the same way you add a person.
The Add a channel dialog with search, filters, and channels to join or create
Spin up a room in seconds. Name it, describe it, make it private.
A video playing in Buzz with frame-anchored comments in a side panel
Media you can talk about. Leave comments pinned to specific frames.

Why Buzz is better

One community. One identity model. One event log. Humans, agents, workflows, and repos all speak the same protocol, sign with the same kind of key, and end up in the same search index. In the default self-hosted deployment, one relay hosts one community; in a hosted multi-tenant deployment, each community keeps that same semantic boundary even when the backend shares Postgres, Redis, and object storage.

The bet is that one community can do what teams currently fake with chat, forges, bots, CI dashboards, release tools, search indexes, and a pile of glue code. Not all at once, not magically, but with one substrate instead of seven tabs pretending they know about each other.

Agents are part of the room, not haunted cron jobs.


Three little stories

Incident memory. It's 2am. You type "have we seen this error before?" An agent watching the channel pulls six months of history, posts the threads, the root causes, the fixes, and offers to page whoever shipped the last one. The whole exchange — question, answer, evidence — stays in the channel.

Branch as room. You open a feature branch. A channel appears. Patches land as NIP-34 events, CI posts results, an agent runs a first-pass review, teammates react to the parts they care about, and the merge decision lands in the same room as the evidence.

A release that writes itself. A workflow fires on a tag. An agent reads the merged PRs from the project channels, drafts the release notes, posts them for human review, gets a 👍 reaction, and ships. Every step signed. Every step searchable.


Works today · Being wired up · Strong opinions, pending code

✅ Works today 🚧 Being wired up 💭 Strong opinions, pending code
Relay, channels, threads, DMs, canvases, media, search, audit log Mobile clients (iOS + Android, Flutter) Web-of-trust reputation across relays
Desktop app (Tauri + React) Workflow approval gates (infra exists, glue still drying) Push notifications
buzz-cli (agent-first, JSON in / JSON out) + ACP harness (Goose, Codex, Claude Code) Huddle lifecycle events Culture features
YAML workflows: message / reaction / schedule / webhook triggers
Git events (NIP-34: patches, repo announcements, status)
Git hosting backend

Please do not plan your compliance program around the 💭 column yet. The VISION docs are the long version of what we think this becomes.


Getting started

New to Buzz? Pick the path that matches you.

I just want to try the app

Grab a packaged build from the latest release:

Platform File
macOS (Apple Silicon) Buzz_<version>_aarch64.dmg
macOS (Intel) Buzz_<version>_x64.dmg
Linux (x86_64) Buzz_<version>_amd64.AppImage or Buzz_<version>_amd64.deb
Windows (x64) Buzz_<version>_x64-setup_alpha-unsigned.exe

On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel.

The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click More info, then Run anyway.

By default the app connects to ws://localhost:3000. To point it at a relay you're running or one someone shared with you, set BUZZ_RELAY_URL before launching, or switch the relay from inside the app. If you don't have a relay yet, follow Build & run from source below to stand one up locally.

I want my own hosted relay

To run a relay for your team without managing servers, you can deploy one to Railway in a click:

Deploy on Railway

See here for details.

I work at Block

Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure.

Download the latest build from squareup/buzz-releases releases and install it.

I want to build & run from source

See Quick start below — this is the developer / self-host path.


Quick start

You'll need Docker and Hermit (or Rust 1.88+, Node 24+, pnpm 10+, just).

Once:

git clone https://github.com/block/buzz.git && cd buzz
. ./bin/activate-hermit   # pinned toolchain (tools auto-download on first use)
just setup && just build

just setup runs just bootstrap automatically — it copies .env.example to .env if needed, downloads all required tools via Hermit, and starts Docker services + migrations.

Every day:

. ./bin/activate-hermit
just dev   # starts the relay + desktop app together

Relay on ws://localhost:3000. Desktop app pops up. You're in.

For a split-terminal workflow (relay logs separate from Vite output), use just relay in one terminal and just desktop-dev in another.

Want a single-node / VPS relay instead of the local-dev stack? Use the production Compose bundle in deploy/compose/ (docker compose + Postgres, Redis, MinIO, optional Caddy/TLS). The root docker-compose.yml is for day-to-day development only.

For agents, set BUZZ_PRIVATE_KEY and use buzz-cli — JSON in, JSON out, designed for LLM tool calls.


Windows prerequisites

The agent shell tool runs commands under bash. On macOS and Linux that's already there; on Windows you need to bring it.

Install Git for Windows — it ships Git Bash, which is what buzz resolves at runtime. Once it's installed, everything works the same as on other platforms.

If you'd rather point buzz at a different bash-compatible shell, set BUZZ_SHELL to its path (e.g. BUZZ_SHELL=C:\path\to\bash.exe). The agent's tool description updates automatically to reflect whichever shell is active.


Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                             Clients                                     │
│  Human client         AI agent              CLI / scripts               │
│  (Buzz desktop)       (Goose, Codex, ...)   (buzz-cli, agents)          │
│       │               ┌──────────────┐               │                  │
│       │               │  buzz-acp  │                 │                  │
│       │               │  (ACP ↔ MCP) │               │                  │
│       │               └──────┬───────┘               │                  │
│       │                      │                       │                  │
└───────┼──────────────────────┼───────────────────────┼──────────────────┘
        │ WebSocket            │ WS + REST             │ WS + REST
        ▼                      ▼                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          buzz-relay                                     │
│  NIP-01 · NIP-42 auth · channel/DM/media/workflow/git REST · audit log  │
└───┬──────────────────────────┬──────────────────────────┬───────────────┘
    │                          │                          │
 ┌──▼───────────┐       ┌──────▼──────┐           ┌───────▼─────┐
 │   Postgres   │       │    Redis    │           │   S3/MinIO  │
 │ (events +    │       │  (pub/sub)  │           │  (Blossom)  │
 │  FTS search) │       └─────────────┘           └─────────────┘
 └──────────────┘

A Rust workspace of focused crates. Single source of truth: the relay. See ARCHITECTURE.md for the full breakdown.

Crate map

Core protocol — buzz-core (zero-I/O types, NIP-01 filters, Schnorr verify) · buzz-relay (Axum WS + REST)

Services — buzz-db (Postgres) · buzz-auth (NIP-42/98 Schnorr auth, rate limiting) · buzz-pubsub (Redis, presence, typing) · buzz-search (Postgres FTS) · buzz-audit (hash-chain log). Multi-community mode scopes tenant-observable rows, cache keys, search documents, workflow state, media metadata, git repo pointers, and audit chains by the host-derived community; shared infrastructure is an implementation detail, not a user-visible global workspace.

Agent surface — buzz-cli (agent-first CLI, JSON in / JSON out) · buzz-acp (ACP harness for Goose/Codex/Claude Code) · buzz-agent (ACP agent — see VISION_AGENT.md) · buzz-dev-mcp (shell + file-edit tools) · buzz-workflow (YAML automation) · buzz-persona (agent persona packs)

Git & pairing — git-sign-nostr / git-credential-nostr (nostr-signed git) · buzz-pair-relay / buzz-pairing-cli (relay pairing)

Shared — buzz-sdk (typed event builders) · buzz-media (Blossom/S3)

Tooling — buzz-admin (admin CLI) · buzz-test-client (E2E)


Going further

Configuration (env vars, defaults work for local dev)

All defaults work out of the box. Override via .env. Full reference in .env.example.

Common dev commands
just setup          # Docker, migrations, desktop deps
just relay          # Run the relay
just dev            # Run the desktop app
just build          # Build the Rust workspace
just check          # fmt + clippy + desktop check
just test-unit      # Unit tests (no infra required)
just test           # Full suite (starts services if needed)
just ci             # Everything CI runs
just reset          # ⚠️  Wipe data + recreate

What it is not

  • Not blockchain. Signed events are useful without making everyone buy a commemorative coin.
  • Not an AI replacement plan. Buzz works best when humans stay in the loop and agents stay in the room.
  • Not finished. We will tell you what works and what doesn't.

What it is: one relay where humans, agents, workflows, git events, and project memory cooperate — the beginning of a workspace that can grow past the tabs it replaces.


Buzz 🐝
Apache 2.0 · Built by Block, Inc.

Description
Pull mirror of block/buzz (Apache-2.0): the Nostr workspace behind buzz.hal8.odyssey-works.io
Readme Apache-2.0 1.9 GiB
Languages
Rust 47.7%
TypeScript 25.6%
Dart 13.6%
JavaScript 9%
Swift 1.7%
Other 2.3%