## What this PR does
Implements NIP-FI stateless enforcement at the relay WebSocket upgrade
point. Every connection attempt on a NIP-FI–enabled relay must carry a
valid federated identity assertion signed for the connecting Nostr
pubkey; the relay validates it before the first frame is read, pairs it
with NIP-42 AUTH, and enforces a bounded session lifetime without
persistent state.
### Core enforcement seam
- **Assertion at upgrade** (`nip_fi_upgrade.rs`): identity header is
parsed with the same `extract_bearer_token` and denied with the same
`http_denial` as HTTP ingress (`nip_fi_http.rs`), JWT verified
(RS256/ES256/EdDSA), expiry checked, and the Nostr pubkey embedded in
the sub claim must match the NIP-42 AUTH pubkey that arrives in the
socket within the AUTH timeout. Denial returns exact byte contracts —
`401 + WWW-Authenticate: Nostr + "authentication required\n"` for
missing evidence, `403 + "evidence rejected\n"` for malformed or invalid
evidence. The active HTTP/1.1 gate runs before `bind_community`, so
denied upgrades pay zero DB cost.
- **NIP-42 pairing** (`nip_fi_gate.rs`, `audio/handler.rs`): after the
assertion is validated at upgrade, a `SessionAdmissionGate` is created
with the assertion's expiry deadline and a `CancellationToken`. The gate
is passed to every effect-acquiring path so no new effect permit is
granted after the session expires. AUTH is verified against the
assertion pubkey; a mismatched AUTH key receives `NOTICE restricted:
authorization denied` and a socket close. Every other post-upgrade AUTH
denial on an FI connection uses the same terminal shape — the canonical
spec NOTICE on the terminal queue, then close — so frame type and close
behaviour reveal nothing about private policy: ban, allowlist, or
relay-membership denial → `restricted: authorization denied`;
ban/allowlist/membership lookup failure or an internal verifier failure
→ `restricted: authorization unavailable`; an invalid NIP-42 proof →
`restricted: evidence rejected`. Connections without an assertion keep
today's `OK false` replies.
- **Session lifetime** (`nip_fi_session.rs`): three independent
deadlines bound each session — token `exp`, max assertion age from
config, and relay max connection lifetime. The first to fire cancels the
gate and sends the terminal denied frame.
- **Effect admission at expiry** (`audio/join.rs`, `handlers/req.rs`):
no new effect permit is granted after expiry. Two paths carry calculated
budgets. The audio **lease resolver** holds its permit for at most
`worst_case::PERMIT_HOLD` (14.5 s), and cancellation **during that
resolver** reaches socket close within `worst_case::EXIT` (25 s:
resolver, release of the won lease, terminal flush). Redis checkout has
one total deadline and directory scripts are single-round-trip `EVAL`,
which is what makes those budgets hold. When post-write fence
verification fails after a landed CAS, the acquire keeps the reply and
attempts an exact owner+generation cleanup on the same connection
without a serving-write lease; if that cleanup fails or its reply is
uncertain, the lease expires by its 30 s TTL. Neither number bounds
audio close in general: other admission and teardown waits (for example
the channel lookup after the resolver) and the known limitations below
have no client-side deadline. REQ liveness, search and history reads
race gate cancellation: a lost race drops the read, releases the permit,
retires the claim through `close_if_owner`, and sends no further EVENT
or EOSE. A history statement timeout ends the read with a timed-out
outcome, and its owner-checked retirement (`CLOSED`, map, index and
topic release) runs after the race with the permit dropped, so
cancellation cannot drop it halfway. Subscription registration stays one
uncancellable unit, and the permit is still taken before the claim.
- **JWKS recovery** (`main.rs`): builds on main's
`warm_nip_fi_jwks_snapshots` / `nip_fi_jwks_refresh_loop`. Issuers warm
concurrently and the first refresh is seeded from the warm result (one
interval for a warm issuer, 5 s for a cold one). A refresh that finds no
live snapshot retries at the fast cadence (5 s after a warm issuer dies,
doubling to a 300 s cap) instead of waiting a full interval, and
`run_jwks_refresh_supervisor` restarts the loop after a panic (1 s → 60
s backoff); On shutdown it is cancelled and awaited before the drain
backstop is disarmed; cancel drops an in-flight fetch and starts no
further issuer, so the join stays inside the drain bound. Logs carry
`issuer_index` only.
### Known limitations
- **Known limitations: bounded only by the dependency.** No new effect
is admitted after expiry. Two awaits that are already admitted have no
client-side bound, so if a dependency stalls at expiry, teardown waits
for it:
- **Audio join commit:** its effect permit spans `tx.commit()`, room
publication and the `event_mentions` insert. During a DB or Redis stall,
audio cleanup and socket close wait for the dependency.
- **REQ subscription setup:** the first subscriber to a topic sends
`Subscribe` to the pubsub task through a bounded channel (capacity
4096). The send blocks only if that queue is full, which happens when
the pubsub task isn't keeping up (for example during reconnect backoff
or a burst). Registration holds the connection's `conn.subscriptions`
lifecycle lock across the send, so a blocked send also blocks other
claims on that connection, including search, and any cleanup waiting on
that lock. The send itself predates this PR; holding the REQ permit
across it is new here.
### Archive/join serialization
`commit_participant_join` now takes a row-level write lock (`SELECT ...
FOR NO KEY UPDATE`) on the channel row at step 3. This serializes all
join commits against concurrent `archive_channel` calls — the archive
UPDATE (`archived_at` non-key column) blocks on the row lock, closing
the READ COMMITTED race on both the `Existing` and `AutoAddRequired`
paths.
### HTTP/2 extended-CONNECT gate (latent path)
The root WebSocket handler uses a belt-and-suspenders approach. The
HTTP/1.1 path (currently the only live WebSocket shape — workspace Axum
does not enable `http2`) is gated pre-extractor via the `Upgrade:
websocket` + `Connection: Upgrade` header predicate. A second gate point
inside the `Ok(ws)` arm is structural future-proofing: if `http2` is
ever enabled, any h2 extended-CONNECT shape the extractor accepts but
the h1 predicate misses is caught there. The active HTTP/1.1 fire-point
sits before `bind_community` (zero DB cost on denial); the latent h2
fire-point runs after it.
### Mixed-version huddle control (rolling deploy)
Owners advertise the `huddle-commit-phase` mesh capability. The ingress
selects the mode after the owner stream opens: it sends the appended
`RegisterPeerCommitPhase` variant, and later `CommitConfirmed`, only
when the owner's exact runtime gossip record carries that flag at that
point, and latches the result for the session. A successful open implies
the owner's record is present, so a record that lands during acquisition
is not missed. Otherwise it sends plain `RegisterPeer`, which every
owner decodes. A new owner publishes a plain `RegisterPeer` once at
registration (pre-commit-phase behavior) and holds a
`RegisterPeerCommitPhase` peer until its confirm; duplicate or legacy
confirms publish nothing. Existing wire variant indices are unchanged.
### Off-mode compatibility
When `BUZZ_NIP_FI_MODE=off` (or unset), the relay reads no identity
header and installs no upgrade gate, NIP-42 pairing check, or session
deadline. Existing NIP-42 AUTH flows are unchanged. Huddle
commit-before-publish is not FI-gated and applies in every mode, so
huddle join publication timing differs from main even in Off mode.
External connection cancellation (socket close, archival, revoke) now
stops an in-flight liveness, search or history read at its next await;
an uncancelled connection's frames and payload bytes are unchanged.
## Test coverage
Every test this PR adds, or whose assertions it changes, runs in exactly
one CI lane: infra-free tests in `just test-unit`, Postgres-backed tests
in `postgres_tests` modules under the PostgreSQL lane. The `buzz-relay`
bin tests run whole in `just test-unit`. `scripts/run-tests.sh` mirrors
the effective relay unit selection for machines without nextest, except
main's pre-existing `api::admin::` module. Every `buzz-relay` test
fixture reads `Config` through `Config::for_test()`, which holds
`NIP_FI_ENV_LOCK`, so a single-process `cargo test` run cannot observe a
half-written FI environment
(`fixture_config_read_waits_for_fi_env_lock`). The stub pools in the
router and AUTH fixtures use a 100ms acquire timeout, which shortens a
database fallthrough rather than removing it.
- **Upgrade/router gate** (`router::tests`): Enforce-mode `nip_fi_*`
gate tests for h1 WS upgrade, NIP-11 content negotiation, and audio
ingress. Off-mode witnesses `nip_fi_off_root_passes_without_header`,
`nip_fi_off_root_ignores_malformed_header`,
`nip_fi_off_audio_passes_without_header`,
`nip_fi_off_audio_ignores_malformed_header` prove the gate is bypassed
(status outside 401/403/503, and no-header status equals
malformed-header status); they do not prove a successful upgrade.
- **NIP-FI units**: `nip_fi_config`, `nip_fi_upgrade`, `nip_fi_gate`,
`nip_fi_session`, covering assertion contracts, NIP-42 pairing, session
lifetime, and gate behavior.
- **Effect barrier at cancellation** (`postgres_tests`, PostgreSQL
lane):
`w3_b2_req_barrier_expiry_mid_flight_blocks_subscription_registration`,
`w4_b2_count_barrier_expiry_mid_flight_blocks_count_query`.
- **AUTH denial uniformity** (`handlers::auth::tests`): each
production-handler witness asserts the exact terminal frame, no denial
on the data or ctrl channel, and cancellation, plus the unchanged
Off-mode `OK false`. Unit:
`fi_invalid_nip42_proof_emits_terminal_evidence_rejected`,
`fi_ban_check_error_emits_terminal_authorization_unavailable`,
`handle_auth_pairing_mismatch_runs_full_root_denial_path`, and
`nip42_denial_class_separates_internal_failure_from_bad_evidence`
(internal verifier failure → `authorization unavailable`; each
client-evidence NIP-42 failure → `evidence rejected`). PostgreSQL lane:
`fi_ban_denial_emits_terminal_authorization_denied` (real
`community_bans` row; Off mode keeps `OK false` on ctrl, then close),
`fi_relay_membership_denial_emits_terminal_authorization_denied`,
`fi_allowlist_check_error_emits_terminal_authorization_unavailable`,
`fi_relay_membership_check_error_emits_terminal_authorization_unavailable`,
`fix_4a_allowlist_denial_with_fi_assertion_emits_canonical_restricted_frame`.
- **Bounded audio exits** (`audio::handler::tests`, `just test-unit`):
`check_cancel_exit_with_never_ready_sink_returns_within_flush_budget`
drives the production `check_cancel!` fence with a queued FI denial into
a never-ready sink and requires it to return within
`WS_TERMINAL_FLUSH_TIMEOUT`;
`challenge_send_to_never_ready_sink_exits_within_flush_budget_on_cancel`
requires the NIP-42 challenge send to yield to cancellation and exit
within the same budget; every pre-`send_loop` audio exit write goes
through the same `connection::send_exit_frames_bounded` helper.
- **Audio NIP-42 classification** (`audio::handler::tests`, `just
test-unit`):
`audio_bad_nip42_proof_with_fi_emits_one_evidence_rejected_then_cancels`
requires exactly one canonical `restricted: evidence rejected` frame and
cancellation; `audio_bad_nip42_proof_off_mode_keeps_auth_failed_frame`
pins the Off-mode frame.
- **Effect permit before pre-auth writes** (PostgreSQL lane):
`handlers::auth::tests::postgres_tests::expiry_before_nip_oa_materialize_writes_no_rows_and_commits_no_auth`
expires the session just before NIP-OA materialize and asserts no
`users` or agent-owner row and no committed auth.
- **Community admission** (`just test-unit`):
`state::tests::on_not_run_runs_once_on_each_deny_arm_and_never_on_admit`.
- **Auth metrics contract**:
`connection::tests::auth_lifecycle_reconciles_every_terminal_and_never_leaks_gauge`
records every terminal including `PairingMismatch`;
`metrics::contract_tests::*` derive the series count from
`AuthOutcome::ALL`.
- **JWKS** (`main.rs`): `hard_dead_warm_issuer_resets_to_fast_cadence`
and `jwks_refresh_success_waits_interval_failure_uses_fast_backoff`
drive the real loop with paused Tokio time; `cold_issuer_*` cover the
backoff helper;
`f1_supervisor_loop_drives_recovery_and_restores_admission` runs
`run_jwks_refresh_supervisor` over a real `ProductionJwksSource` through
injected panic → restart → hard-dead → 5 s retry → recovery, then checks
`key_set` admission and that the first post-restart fetch follows the
cold cadence (every issuer restarts cold);
`jwks_refresh_cancel_drops_in_flight_fetch_and_skips_remaining_issuers`
cancels during a pending fetch with a second issuer due.
- **Bounded lease acquire** (`audio::handler::tests::postgres_tests`,
PostgreSQL lane):
`failed_post_cas_verification_is_fenced_and_discards_the_won_lease`
requires a won-then-unverified lease to return fenced and be deleted by
exact owner+generation;
`stalled_lease_sql_and_stalled_exit_release_close_within_exit_bound`
stalls lease SQL and the exit release and requires socket close within
`worst_case::EXIT`. `audio::join::tests` pins `PERMIT_HOLD` at 14.5 s
and `EXIT` at 25 s.
- **REQ/search cancellation** (`handlers::req::tests`, `just
test-unit`):
`expiry_during_stalled_history_read_releases_permit_and_claim` and
`expiry_during_stalled_search_read_releases_permit_and_claim` stall the
read inside production `handle_req`, expire the gate, and require
`expire()` to return within 2 s, an empty subscription map, and no EVENT
or EOSE.
- **History timeout retirement**
(`handlers::req::tests::postgres_tests`, PostgreSQL lane):
`history_timeout_retirement_survives_expiry` drives a two-channel
history REQ into a real statement timeout, pauses retirement after the
index removal and before the topic release, expires the gate, then
resumes. It requires the map and index entries gone and both channel
topic refcounts at zero.
- **Archive/join serialization** (PostgreSQL lane):
`f2a_archived_channel_rejects_existing_join`,
`f2b_join_for_update_blocks_concurrent_archive_existing_path`,
`f2c_join_for_update_blocks_concurrent_archive_auto_add_path`.
- **Huddle room and handler**: `audio::room::tests` pending-peer tests,
including `roster_revisions_are_ordered_and_snapshot_is_authoritative`;
`fix_b_w1_w3_handler_pre_commit_cancel_emits_no_delta`,
`fix_b_w2_success_emits_exactly_one_delta_with_monotone_revision`.
PostgreSQL lane: `b1_cross_pod_bootstrap_precedes_buffered_owner_delta`,
`commit_confirm_timeout_at_seam_triggers_teardown_on_stalled_owner_stream`,
`confirm_failure_combined_outcome_no_orphan_join_exactly_one_48102`,
`confirm_send_pending_expiry_terminates_client_with_denial_within_1s`,
`c1_handler_ingress_sends_only_base_frames_to_legacy_owner`,
`f7a_cross_pod_joined_payload_includes_owner_pod_peers`.
- **Mixed-version control** (`audio::join::tests`):
`shared_control_variants_are_byte_identical_to_base_wire` (against
frozen `audio::join::base_wire`),
`c1_new_ingress_speaks_base_wire_to_legacy_owner`,
`c2_base_ingress_register_publishes_once_on_new_owner`,
`c3_capable_owner_record_selects_commit_phase_hold_back`,
`capability_record_installed_during_open_selects_commit_phase`,
`duplicate_or_legacy_commit_confirmed_is_a_no_op`,
`fix_b_commit_confirmed_arm_commits_peer_and_broadcasts_joined`,
`pending_remote_stream_close_emits_no_joined_no_left`.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Implements NIP-FI HTTP ingress enforcement per the merged spec (PR
#7254). Every protected HTTP surface in enforce mode requires a valid
`Nostr-Federated-Identity: Bearer <JWS>` assertion alongside its NIP-98
authorization, bound to the same key. The NIP-98 pubkey (proven actor)
must equal the assertion's `nostr_pubkey`; absent, mismatched, or
unverifiable assertions are denied fail-closed with byte-exact
text/plain responses per the spec's rejection table.
This is S5 of the `buzz-enterprise-identity` program. WS enforcement
(S3) is separate.
## What changed
**New `nip_fi_http` module:** `admit_nip_fi_http()` covers the full
admission sequence — `Nostr-Federated-Identity` extraction, offline JWKS
verification, NIP-98 key pairing [FI-INV-05], issuer-qualified deny-map
check via `(iss, pubkey, now)` [FI-INV-14], and byte-exact denial for
all four `DenialClass` variants. `admit_nip_fi_http_on_state()` is the
single call-site wrapper for all surfaces [FI-TRACE-AUTHORITY-UNIFORM].
**Protected surfaces** (all gated via the uniform authority):
- HTTP bridge: `POST /events`, `/query`, `/count`
- GIF proxy: `POST /gifs/search`, `/gifs/share`
- Workflow reads: `GET /workflows/{id}/runs`, `GET
/workflows/{id}/runs/{id}/approvals`
- Moderation reads: `GET /moderation/reports`, `/moderation/audit`,
`/moderation/restricted`
- Media/Blossom: `PUT /upload`, `/media/upload`
- Git smart-HTTP: `info/refs`, `upload-pack`, `receive-pack`
- Git settings control plane: `GET`/`POST
/git/:owner/:repo/default-branch` (now routed through
`admit_nip_fi_http_on_state` for full assertion key-pairing)
- Invites: `POST /api/invites` (mint only; claim exempt as
pre-membership enrollment door)
**NIP-98 strictness in NIP-FI mode:**
- When NIP-FI is not Off, bridge POST handlers force `require_auth_token
= true` — the `X-Pubkey` dev-mode fallback cannot satisfy the pairing
requirement [NIP-FI.md:547-567]
- POST bodies on `/events`, `/query`, `/count` require a matching
`payload` tag in enforce mode [NIP-FI.md:579-597]
- A payload tag present with no content (one-element `["payload"]` or
empty string) is rejected as a structurally invalid claim — NIP-98
requires the content to be a valid 64-char lowercase hex SHA-256 digest.
Malformed payload diagnostics use a static error message (no
attacker-controlled content in logs).
- Off mode preserves pre-NIP-FI behavior per [FI-INV-15]
**Off-mode compatibility changes:** Admission now runs before request
validation on every protected surface, so Off mode differs from
pre-NIP-FI behavior in these ways:
- **GIF routes:** NIP-98 authentication happens before the upstream
provider lookup. An unauthenticated request on a tenant with no GIF
provider now returns 401 instead of 404. The 401 goes through the legacy
`api_error()` path, so the body is JSON `{"error":"..."}` with no
`WWW-Authenticate` header.
- **Workflow runs** (`GET /workflows/{id}/runs`): the query string is
parsed after admission. A request with a missing credential and a
malformed query (for example `?limit=abc`) now returns 401 instead of
400.
- **Authorized malformed query:** once admitted, a malformed query
returns a JSON 400 `{"error":"invalid query: ..."}` from
`parse_query_or_400` instead of axum's `text/plain` `Query` rejection.
- **Moderation reads** (`/moderation/reports`, `/moderation/audit`):
same extraction change as workflow runs. Admission comes first, then a
JSON 400 for a malformed query.
- **Malformed `payload` tag:** shared NIP-98 verification now rejects a
malformed `payload` tag in every mode, including on bodyless requests.
That means a one-element `["payload"]`, an empty digest, or anything
other than 64 lowercase hex characters. Before this change, a bodyless
request ignored the tag.
**Auth header cardinality:** Enforce mode rejects requests with more
than one `Authorization` header with 403 `evidence rejected\n`.
DenyProtected returns 503 `authorization unavailable\n` before this
check (and before NIP-98 or assertion verification) runs. The
cardinality gate is enforced at the NIP-FI admission boundary
(`admit_nip_fi_http`), not via legacy auth boolean flags, so it applies
uniformly to all Enforce-mode surfaces including Git. Off mode passes
through the first header value unchanged, preserving pre-NIP-FI behavior
[FI-INV-15].
**Denial-class remapping:** In Enforce mode, NIP-98
extraction/verification failures from the closure are mapped to the
appropriate `DenialClass`: `MissingEvidence` when no `Authorization`
header is present, `EvidenceRejected` when the header is present but
invalid. Off mode propagates legacy responses unchanged [FI-INV-15]; for
example, the Git missing-Authorization and wrong-scheme 401s keep their
legacy bytes with no `content-type` header.
**JWKS refresh scheduling:** The refresh timer's `*last` checkpoint is
set after the fetch completes, not from the pre-fetch timestamp,
preventing interval drift under network latency. The loop is extracted
to `nip_fi_jwks_refresh_loop` for unit-testability.
**Log hygiene [NIP-FI.md:777-779]:** Issuer identity is never written to
logs or error messages. All startup/refresh diagnostics use an ordinal
index (`issuer_index = N`). Config parse errors for malformed issuers
use classify()/line/column codes rather than raw config values.
**Admin SPA document routes:** `/reports`, `/reports/{id}`, and
`/feedback` are exempt from the NIP-FI assertion guard only on the admin
host (host-qualified via `is_admin_host`). These paths are not added to
the broad `NIP_FI_EXEMPT_PREFIXES` list — tenant hosts remain fully
protected.
**Deny-map seam (`HttpDenyMap` trait):** signature is `(issuer, pubkey,
now)` matching `NipFiDenyMap::is_denied` from S4 (PR #7265) so
integration is a one-liner. Stub is `AlwaysAdmitStubDenyMap` — named to
accurately describe stub-phase semantics (fail-open intentionally
pending S4).
**Route inventory:** every authenticated HTTP route is classified as
Protected or Exempt with justification. Documented in the F4 seam-test
comment in `bridge.rs`.
**Outer-guard tests:** `#[ignore = "requires Postgres"]` tests in
`bridge.rs` drive the full router (`build_router`) in Enforce mode with
valid NIP-98 but no `Nostr-Federated-Identity` header. These exercise
the **outer `nip_fi_assertion_guard` middleware** in `router.rs` — the
layer that fires before any handler runs. They prove the guard's exact
response bytes (401 `authentication required\n`, `text/plain;
charset=utf-8`, `WWW-Authenticate: Nostr`) for each protected route.
Per-handler `admit_nip_fi_http_on_state` wiring is proven separately by
the cardinality test and the media/settings/git handler-level suites.
## Seams
**S3 seam:** `nip_fi_config.rs` is duplicated (S3 not landed;
session-deadline parts stripped). Removed when S5 rebases onto S3.
**S4 seam:** `AlwaysAdmitStubDenyMap` until S4 provides the real map.
Integration is one line at the `admit_nip_fi_http_on_state` wrapper.
## Evidence
At head `ee338ea9c` (branch `hayt/nip-fi-http-enforcement`), hosted CI
run
[36167450853](https://github.com/block/buzz/actions/runs/36167450853):
- **Rust / Unit Tests** ([job
108179338458](https://github.com/block/buzz/actions/runs/36167450853/job/108179338458)):
passed. The `buzz-relay` `test-unit` step ran 171 tests with 171 passes.
Its PASS lines include all 57 non-ignored tests this PR adds to the
relay: `nip_fi_http::tests` (25), `nip_fi_config::tests` (8),
`router::tests` (8), `api::parse_query_tests` (5),
`api::transport::off_mode_precedence_tests` (5),
`tests::jwks_refresh_*` (3), and `composition_tests` (3). The same job
ran the `buzz-auth` suite, which includes
`nip_fi::jwks::tests::key_set_serves_live_snapshot_under_reader_and_refresh_overlap`.
- **Rust / Rust Lint**: passed.
- **Relay and PostgreSQL / Backend Integration (relay e2e)**: passed.
- **PostgreSQL Domain / PostgreSQL Tests** ([job
108183303436](https://github.com/block/buzz/actions/runs/36167450853/job/108183303436)):
passed. 542 tests run, 542 passed.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
Signed-off-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Authored by Brain and opened on behalf of Wes (`wesbillman`).
## Summary
Add `buzz repos default-branch get/set` and the relay operation it
needs. Git push changes branch tips but cannot select the server's
symbolic HEAD. This selects an existing branch without deleting
branches, moving refs, or changing packs.
- Reuse immutable Git manifests and captured-ETag CAS; concurrent
pushes/default changes conflict, including stale no-ops.
- Require request-specific NIP-98, body binding, fail-closed replay
protection, host tenancy, current channel membership and
repository-management authority. Push/project roles alone are
insufficient.
- Send the CLI mutation once, without redirects; ambiguous delivery
retains the observed digest and is non-retryable.
- Document the narrow HTTP exception to Nostr-first, delegation scope
and admission-time ACL revocation. Kind:30618 remains derived; the
manifest CAS is authoritative.
- Register the Postgres/Redis/MinIO route and fresh-clone tests in the
existing Backend Integration lane.
## Review fixes — `7457bda50afa6b52e36315572fa96928b2e21db6`
- Reject signed valueless `payload` tags when verifying a body; preserve
the shared verifier's intentional absent-payload compatibility. The real
settings route now proves 401, unchanged manifest pointer, and unchanged
kind:30618 event IDs for malformed and other invalid credentials.
- Require a nonempty returned branch and `head == refs/heads/{branch}`;
SET additionally requires the returned branch to equal the attempted
branch. Malformed/mismatched mutation responses remain non-retryable
`DeliveryUnknown`, retaining the attempted branch and original digest.
Invalid reads fail before any POST.
- Strengthen the later-push/fresh-clone regression to select
`release/v1` while `main` exists.
## Verification
At `7457bda50afa6b52e36315572fa96928b2e21db6`:
- Full pre-push passed (all 14 Rust test groups, serialized Tauri
tests/clippy, file-size and branch-skew gates; 522 seconds). No hooks
disabled.
- All four isolated Postgres/Redis/MinIO settings tests pass, including
real Smart HTTP `ls-remote --symref` and fresh clone on `release/v1`.
- All 15 focused NIP-98 verifier tests and three default-branch CLI
tests pass; malformed credentials and missing-branch responses were
reproduced failing before the fixes.
- Five targeted mutants are killed: restoring the valueless-payload
bypass, weakening canonical HEAD validation, accepting an empty branch,
dropping requested-branch confirmation, and omitting hydrated HEAD for
the selected non-main branch. Mutations ran in a separate worktree and
target directory; original patch bytes were verified restored.
- `cargo clippy -p buzz-auth -p buzz-cli -p buzz-relay --all-targets --
-D warnings`, formatting, diff and file-size checks pass on the fixed
tree.
Earlier validation at `bb195b140`: seven Git authorization database
regressions and 28 existing transport tests passed (three infrastructure
cases in that module not selected); four mutations killed
management/replay/CAS/redirect regressions. Full pre-push passed with
`RUST_TEST_THREADS=1` (all 14 Rust groups, Tauri clippy and 3,173 tests,
size/branch-skew gates). Serial execution avoids an unrelated
process-global Desktop discovery counter flake; no hooks disabled or
Desktop changes.
Carl and Jude's review findings are addressed in the new head; **fresh
reviewer and exact-head security confirmation remain required**. The
earlier source review is not approval of this revision. No claim of
green new-head CI.
## Rollout
Merge and **deploy the relay support first**, then build/use the updated
CLI and explicitly set/verify the desired default. No schema migration.
This PR does not deploy anything or change a live repository.
Originating conversation (channel
`8e2fda80-9142-4df8-8e14-da288bea6471`):
buzz://message?channel=8e2fda80-9142-4df8-8e14-da288bea6471&id=8a7a9cf12737abc1e16605f241300be940a1691ef0992b35197afe26c563108d
---------
Signed-off-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
## Why
PR #6660 introduced four PostgreSQL-backed persistence tests and a
focused CI selector, but an exact test-name list does not automatically
cover future database tests. Broad ignored-test execution also exposed
shared-schema races and ambiguity between desired-state and
migration-applied schema expectations.
## What
- Establish a discoverable convention: PostgreSQL unit modules use
`postgres_tests`, PostgreSQL integration binaries use a `postgres_`
prefix, and unrelated external-infrastructure tests use an
`external_infra_` prefix.
- Add a dedicated nextest PostgreSQL profile and archive-backed CI job
covering the relevant crates without enumerating test names.
- Create a run-scoped desired-state source database and a unique
PostgreSQL database per test process for parallel-safe isolation, with
cleanup on success, failure, or interruption.
- Route destructive migration and migration-parity tests to clean
`template0` databases while desired-state tests clone the desired-state
database.
- Document discovery, schema modes, required role privileges, and a
bounded portable local runner.
- Keep the existing infrastructure-free unit-test jobs unchanged.
This is intentionally limited to test and CI harness behavior. It does
not move production database code, change persistence semantics, or
implement issue #20's broader shared test-utility refactor.
## Risk
Low production risk because all changes are confined to tests, CI
configuration, documentation, and test harness scripts.
Remaining operational risks:
- Cleanup retries `dropdb --force` five times across roughly four
seconds. Exhaustion warns with the database name but deliberately does
not mask the test result; individual PostgreSQL diagnostics are
suppressed.
- The seven-package boundary is duplicated between the runner and CI
archive and must remain synchronized if PostgreSQL tests move to a new
crate.
- Portable SHA fallback branches were exercised on Linux; no macOS Blox
workstation was needed for this Linux CI artifact.
- GitHub's unchanged generic Unit Tests job currently fails while
cold-linking `buzz-voice` because `sherpa-onnx-c-api` is absent. The
branch changes no voice/build/toolchain inputs; the exact parent
previously passed that job, and Blox passes the same infrastructure-free
suite (1,450/1,450). A prior-head one-job retry and the final-head run
both reproduced the hosted-runner failure.
## Verification
Author Blox workstation `2020088`:
- `cargo fmt --all -- --check`
- `just clippy`
- `just test-unit`: 1,450 passed; PostgreSQL tests remain skipped in
fast jobs
- PR #6660 focused tests: 4/4 passed through the final runner
- Full lane: 282/282 passed in 10.087s with a non-superuser role limited
to `CREATEDB`, `CREATEROLE`, and `pg_read_all_stats`
- 272 desired-state tests
- 10 migration-applied tests
- 6 explicitly filtered external-infrastructure tests
- Cleanup fault injection: a deliberately underprivileged 282-test run
produced the expected 277 passes and 5 failures, and the post-run
catalog audit found zero lane databases after retry cleanup. The
restored successful run also left zero lane databases.
GitHub exact final head `92c231e48f299ea2af23817763f5ae80ac013d68`:
- [PostgreSQL
Tests](https://github.com/block/buzz/actions/runs/32789030203/job/97630827823):
282/282 passed across 10 binaries in 98.886s; 1,240 skipped, including 6
via the profile filter
- The shared relay/PostgreSQL archive predecessor completed successfully
- [Unit
Tests](https://github.com/block/buzz/actions/runs/32789030203/job/97626930420):
unrelated `sherpa-onnx-c-api` native-link failure described above
- Independent exact-head review on separate Blox workstation `2022762`:
no substantive findings after all initial findings were addressed. The
reviewer independently verified cleanup retry behavior, exit-status
preservation, shell portability, syntax, and a clean exact-head
worktree.
## References
- Stacked on #6660 at exact head
`561de54be4d9c2b622b7c2aa5b61bc3068f47e2a`
- https://github.com/TheSentinel454/buzz/issues/20
-
https://github.com/TheSentinel454/buzz/issues/20#issuecomment-5398526792
Generated with Codex.
## Update — August 24, 2026 review follow-up
- Centralized the repeated PostgreSQL test URL resolution in
crate-local, test-only helpers without introducing the broader shared
utility refactor from issue #20.
- Restored descriptive hybrid/Redis test function names. Structural
`external_infra_*_tests` modules now own exclusion, and the nextest
filter only recognizes module path segments.
- Added a three-second source guard that scans every Rust file and fails
CI when an ignored PostgreSQL test would be omitted or an
external-infrastructure test would be included. Fixture tests cover
accepted modules/binaries and both failure modes.
- Removed the two implementation-plan documents.
Performance profiling on Blox workstation `2027352` (same prebuilt
archive and cargo-nextest 0.9.143 for every comparison):
- Current per-test database model, 8 workers: 10.15–10.40s across three
runs. The 283 `createdb` calls and 284 `dropdb` calls consumed
28.85–29.46 aggregate seconds, about 39–40% of aggregate test-process
duration.
- Reusing one database per worker without cleanup: 6.55–6.84s, about 35%
faster, but all three runs failed because global matcher-queue tests
inherited a quiesced community from an earlier test; the failing test
varied with scheduling.
- Reusing workers with `TRUNCATE … RESTART IDENTITY CASCADE`: all 282
tests passed, but total time regressed to 10.47–10.67s. The 265 truncate
calls consumed 24.34–25.45 aggregate seconds, so truncation merely
replaced most clone/drop cost.
- Raising concurrency to 16 workers reduced one isolated-database run to
8.79s, but 24 workers exposed a cluster-global `pg_stat_activity` race.
The profile remains at the proven-safe 8 workers.
Decision: retain unique per-test databases. Harness cleanup uses `dropdb
--if-exists --force`; it drops the database rather than truncating
tables or deleting rows.
Verification at local branch head
`60086c1f6ee34a193c342255a5e8e6293b60e988`:
- Discovery inventory: exactly 282 intended tests; all 6 hybrid/Redis
tests present and structurally excluded.
- PR #6660 focused persistence tests: 4/4 passed.
- Full PostgreSQL lane: 282/282 passed three times; 1,240 skipped each
run; wall time 9.72–9.96s without profiling shims.
- Non-ignored tests with a desired-state database: 1,234/1,234 passed;
one mesh-demo timeout from the first run passed immediately in
isolation.
- `cargo fmt --all -- --check` and all-target/all-feature clippy for
`buzz-db`, `buzz-deletion`, and `buzz-relay` passed.
Final independent re-review at exact head
`42097c0136ab3fa8f0efe5720ae829c04575b9b6` on separate Blox workstation
`2028455`: no substantive residual findings. The reviewer independently
exercised ordinary strings, zero/one/three-hash raw strings, line/block
comments, and string-contained lookalikes, then reran the full 353-file
scan, shell syntax, Python AST parsing, and diff check.
Independent review found that the source guard's initial regular
expression could miss raw-string ignore reasons and treat a commented
attribute as real. Final head `42097c0136ab3fa8f0efe5720ae829c04575b9b6`
parses valid ordinary/raw Rust string literals only at attributes found
in comment-sanitized source; regression fixtures cover both cases. The
fixture suite, Python compilation, shell syntax, diff check, and full
353-file repository scan pass on Blox.
## Update — August 25, 2026 restack
PR #6660 merged, so this follow-up was rebased from its former exact
parent onto current `origin/main`
(`8d2d0ff5ad42733e9949442c4b6358d0ba87f9a8`). The final candidate head
is `05dcc2ab28948e3ab79bb44839a69f2ba44648a2`; the PR no longer carries
#6660's pre-squash history.
Fresh exact-head verification on Blox workstation `2028572`:
- Discovery guard passed across 353 Rust files; inventory remained
exactly 282 intended PostgreSQL tests.
- PR #6660 focused persistence tests: 4/4 passed.
- Full PostgreSQL lane: 282/282 passed three times; 1,243 skipped each
run; wall time 9.70–9.87s.
- `cargo fmt --all -- --check` and all-target/all-feature clippy for
`buzz-db`, `buzz-deletion`, and `buzz-relay` passed.
## Update — August 25, 2026 final hosted verification
- [GitHub PostgreSQL
Tests](https://github.com/block/buzz/actions/runs/32801679226/job/97667835080):
282/282 passed in 53.787s; 1,243 skipped.
- The final guard scanned all 353 Rust files successfully before the
hosted lane ran.
- The shared relay/archive prerequisite and infrastructure-free Unit
Tests job both passed at exact final head
`42097c0136ab3fa8f0efe5720ae829c04575b9b6`.
---------
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Signed-off-by: tornquist <tornquist@squareup.com>
NIP-OA relay admission now evaluates signed `created_at<` and
`created_at>` conditions against the already verified authentication
event timestamp. An owner-signed credential such as `created_at<1` no
longer upgrades its holder to relay membership.
The verified timestamp now flows from NIP-42, NIP-98, and Blossom
authentication through the shared membership gate used by WebSocket,
HTTP, Git, media, huddles, GIF, and workflow requests. Owner-attested
access fails closed when no signed authentication timestamp is
available. Direct relay members keep their existing admission behavior,
and `kind=` remains connection-level metadata as specified by NIP-AA.
Tests cover strict time-bound evaluation, missing timestamp rejection,
and HTTP authentication timestamp propagation.
Testing:
- `cargo test -p buzz-sdk nip_oa::tests -- --nocapture`
- `cargo test -p buzz-relay api::relay_members::tests -- --nocapture`
- `cargo test -p buzz-relay api::bridge::tests -- --nocapture`
- `just ci`
---------
Signed-off-by: Jordan Mecom <jm@squareup.com>
## Summary
Adds a durable, operator-controlled V1 for deleting an entire Buzz
community without deleting another tenant's data.
The workflow is exposed through `buzz-admin deletions`:
- `sweep` records independent fleet storage-taxonomy observations
- `submit`, `list`, `inspect`, and `approve` manage a deletion request
- `unblock` resumes a fail-closed request after an operator records
remediation identity and reason
- `run` and `drain` execute bounded work
Requests advance through a PostgreSQL-backed state machine and stop at
`retention_pending` after logical deletion has been independently
verified across PostgreSQL, object storage, and Redis.
This PR ships the engine and CLI, not a continuously running worker or
Kubernetes packaging. For V1, a cluster/VM administrator invokes
`/usr/local/bin/buzz-admin` from the existing relay image, for example
with `kubectl exec` or an equivalent container/VM exec path.
## What whole-community V1 removes
For the target community, V1 removes:
- rows from the allowlisted community-scoped PostgreSQL catalog,
including members, profiles, authored events and bodies, DMs, reactions,
mentions, memberships, tokens, workflows, moderation, audit, feedback,
and rate-limit state
- media sidecars and upload-attribution records under
`_meta/<community>/` and `_uploads/<community>/`
- Git repository pointers under `repos/<community>/`
- Redis keys under `buzz:<community>:*`
The community row survives as a permanent tombstone, and deletion
control-plane records remain as evidence of the request, approval,
execution, and result.
## Safety model
Deletion is not a broad `DELETE CASCADE` followed by optimistic cleanup.
The destructive boundaries are durable and fail closed.
### 1. Inventory and approval
- `submit` resolves the target and freezes the schema plus summary-only
storage inventory.
- Approval is bound to the exact request, community, and frozen
inventory digest.
- Unsupported manifest versions, malformed keys inside the target's
owned prefixes, live scoped-table/write-fence coverage drift,
frozen-inventory mismatch, and approval mismatch block execution rather
than guessing. Migration and catalog revision numbers are not
authorization gates; the executor validates the live safety shape
instead.
- Storage inventory is server-side prefix scoped to exactly:
- `_meta/<community>/`
- `_uploads/<community>/`
- `repos/<community>/`
- The deletion path never lists the whole shared bucket and has no
arbitrary per-community object cap. Its listing work is proportional to
the target community's bindings, not total fleet storage.
- Fleet-wide taxonomy sweeps remain independent observability. They
report unknown writer shapes but do not gate deletion submission,
fencing, or destructive progress. Maintainers must add deletion taxonomy
coverage whenever a new community-owned object-key class is introduced;
writer-coverage tests bind the current media and Git writers to that
contract.
### 2. Quiesce, fence, and destructive freeze
- Writes continue through submission, inventory, and approval. They stop
when execution moves the target into `quiescing` and then establishes
the durable fence.
- Already-admitted external effects finish under heartbeated
serving-write leases; the exact admitted lease may renew while the
community is quiescing, but new lease acquisition is rejected. The
executor drains admitted leases before destructive work.
- Invite minting after quiescing begins fails as typed `AccessDenied`
(HTTP 503 at the relay boundary) before an invite can be persisted.
- Database triggers enforce the community write fence across the
complete catalog of community-scoped tables. Startup/readiness and
destructive execution validate that catalog so a newly added but
unfenced table cannot silently escape.
- **Named isolation assumption — fresh write snapshot.** Every writer
transaction that can reach a community-fenced relation must use
PostgreSQL `READ COMMITTED`; each guarded write therefore observes a
statement snapshot no older than acquisition of the community deletion
lock. `REPEATABLE READ` and `SERIALIZABLE` can retain a pre-fence
snapshot and are unsupported for writers. The writer pool refuses
non-`READ COMMITTED` sessions at connection setup, and both SQL fence
functions reject an explicit per-transaction isolation override with
SQLSTATE `25000`. Configuration-delivered bad isolation can surface
through SQLx as a pool-acquire timeout because every `after_connect`
attempt is rejected; the precise `community writes require READ
COMMITTED isolation` reason remains observable when the SQL guard is
reached. Read-only replica transactions are outside this assumption.
- Holding the shared advisory lock until the guarded write executes is a
separate liveness condition: under `READ COMMITTED`, releasing it early
does not permit resurrection because the trigger rechecks the fence, but
it can turn a fleet sweep into a statement-wide SQLSTATE `55000` abort.
- After the fence closes writers, storage is re-enumerated into chunked
side-table rows. Per-prefix counts and digests bind those concrete keys
to the destructive manifest.
- Manifest chunk insertion, update, and deletion are protected after
freeze. This closes the race where an unbound key could otherwise appear
after the manifest was committed.
### 3. Checkpointed destruction
- Target-owned object bindings are deleted from the frozen destructive
manifest in bounded batches with durable progress.
- The concrete key list lives in chunked side-table rows rather than one
request-row JSON value. It supports large communities, resumable
execution, and terminal cleanup.
- Missing objects are accepted as idempotent crash-window outcomes;
malformed ownership, changed evidence, and unexplained target-prefix
drift fail closed.
- PostgreSQL purging remains scoped by `community_id`, including the
guarded NIP-RS hard-delete path discovered with real Desktop kind
`30078` read-state data.
- Redis cleanup explicitly scans and `UNLINK`s only
`buzz:<community_id>:*`. Natural expiry is insufficient because some
keys, including tunnel generation counters used as fencing state, are
deliberately persistent.
### 4. Independent verification
- PostgreSQL logical absence is checked after purge.
- The three target-owned storage prefixes are freshly inventoried again
and must be empty.
- Redis requires two complete empty namespace scans.
- Only after all three stores pass does the request advance through
`logically_verified` to `retention_pending`.
## What V1 deliberately does not erase
### Shared content-addressed storage
Per-community deletion removes bindings, metadata, attribution records,
and Git pointers. It does **not** physically delete fleet-shared CAS
bytes that another community may still reference:
- media blobs and thumbnails
- Git manifests, packs, and indexes (`manifests/`, `packs/`, and `idx/`)
Safe reclamation requires a separate fleet-wide reachability and
retention GC. Unknown keys elsewhere in the shared bucket do not block
one community's deletion; malformed or unrecognized keys inside that
community's three owned prefixes still fail closed.
### External retained copies
The online logical-deletion proof does not erase object
versions/replicas, database backups/WAL, CDN copies, provider retention
copies, or observability exports. Those require their own retention and
purge controls.
### Member-only erasure
This PR erases a whole community. It does not implement the different
operation "erase one npub while preserving the community."
Removing membership or accepting NIP-09 is not member erasure. A
member-only workflow would need to find and selectively remove or redact
authored event content and pubkeys, profile data, DMs, reactions,
mentions, memberships/roles, tokens, workflows/subscriptions, upload
attribution, moderation/audit history, repository attribution, and
identity embedded in tags or JSON. It would also need explicit rules for
ownership transfer, surviving replies and thread metadata, audit-chain
integrity, immutable Git history, and shared-CAS reachability. That
requires a pubkey-level fence and selective graph rewrite; it is a
separate deletion product, not a safe extension of this whole-tenant
worker.
## In scope
- migration `0029_community_deletion.sql`: requests, approvals, leases,
manifest chunks, checkpoints, tombstones, and the universal write-fence
catalog
- durable executor leases, generations, heartbeats, retry/block state,
and resumable stage transitions
- operator-driven `sweep`, `submit`, `list`, `inspect`, `approve`,
`unblock`, `run`, and `drain` commands
- serving-path fences for database writes and external effects across
event ingest, media, Git, workflow, push, invites, mesh/tunnel, and
related paths
- target-prefix-only storage inventory, summary manifests, post-fence
destructive chunks, and bounded batch deletion
- exact community Redis namespace purge and two-pass absence
verification
- cross-community isolation, crash/resume, manifest-integrity,
writer-taxonomy, and schema/migration regressions
- desired-state `schema/schema.sql` support without requiring a SQLx
migration ledger
## Deferred / not covered
- dedicated Helm/chart worker Deployment, service account, secrets,
probes, resources, and network policy
- autonomous `buzz-admin deletions worker` poll loop and worker-only
health server
- least-privilege separation among migration, relay-serving, and
destructive execution roles
- fleet-wide shared-CAS physical GC
- backup/provider/CDN/observability retention completion
- member-only erasure
- provider-native conditional-delete improvements
- a general force-continue escape hatch; permanent safety failures
remain fail closed unless an operator remediates the cause and records
an audited `unblock`
The removed continuous-worker implementation remains deferred; no remote
follow-up branch is claimed by this PR.
## Validation
### Current PR head and repository state
Current pushed head: `359d8402ee15f049768f54156f67b953c7a7e2ed`, rebased
onto `cc9a2f783375e51a6e8d1f2f9d01d5f7e22813d1` (`origin/main` at push
time). The complete PR diff is now 47 files, 9,834 additions, and 517
deletions.
The bespoke source-scanner stack was removed to keep this PR scoped to
community deletion. Tyler/team requested the underlying fenced-write
safety behavior, not `ast-grep`,
`crates/buzz-db/tests/community_fenced_writes.rs`, its 27 fixtures, or
the new `scripts/lints/community_*.yml` rules. Those scanner-specific
files, dependencies, Hermit links, and runner wiring are absent from the
current tree. The production database write fence, startup/destructive
live-catalog validation, and deletion behavior remain.
Source validation on this exact SHA passed:
- `cargo fmt --all -- --check`
- `bash -n scripts/run-tests.sh`
- `cargo nextest run -p buzz-db --all-targets`: 102 passed, 173 skipped,
0 failed
- `cargo nextest run -p buzz-deletion --all-targets`: 10 passed, 9
skipped, 0 failed
- `cargo nextest run -p buzz-admin --all-targets`: 1 passed, 0 failed
- affected-package/all-target Clippy with warnings denied
- lockfile consistency
- Helm 3.16.4 lint and all 44 chart unit tests
- Helm region controls using that fixture: default
`BUZZ_S3_REGION=us-east-1`, explicit `eu-west-2` override, and
blank-region schema rejection
The prior Kubernetes battery below was run against
`928992237358a3294621ac0280830b77155abc04`. It remains useful evidence
for the patch-equivalent production deletion implementation, but it is
**not** claimed as exact-SHA evidence for current head
`359d8402ee15f049768f54156f67b953c7a7e2ed`; the current cleanup removes
only scanner/test/tooling infrastructure. CI restarted for the new head
after the rebase and is pending. Human review remains
`CHANGES_REQUESTED`.
### Prior-head live Kubernetes deletion and safety gates
The full program used one immutable image, real PostgreSQL, Redis,
MinIO, and a three-relay Kubernetes release:
- source: `928992237358a3294621ac0280830b77155abc04` (**prior head**)
- image: `buzz-e2e:sha-928992237358`
- immutable image digest:
`sha256:a1a204f4618ac22d9e210be5e5290645a15d79831ae30b0e44379357c8e4a895`
- evidence root:
`/tmp/buzz-e2e/20260807T033025Z-928992237358-full-gates/`
- evidence-manifest digest:
`82875c5bc9bea7370b796a7aef3457b3a1c8306c84c59e0f7388bbb5ad30e865`
Passed gates at that prior head:
- **Chart/operator region:** default `us-east-1`, explicit nondefault
propagation, blank-region schema rejection, live in-pod environment, and
an in-pod taxonomy sweep over 18 objects with zero unknown.
- **Fenced writers and lifecycle:** open-write/fence ordering;
100-attempt anti-starvation; invite, push matcher, and exhausted-reaper
bystander isolation; non-`READ-COMMITTED` rejection; manifest/tombstone
contracts; eight-failure stage block and audited `unblock`.
- **Destructive lifecycle:** submit → approve → run →
`retention_pending`; PostgreSQL tombstone and Redis/S3 verification
true; zero retries/errors; terminal reruns rejected with exit 5.
- **Fresh 10,001-object crash boundary:** exactly two chunks (10,000 +
1). The executor deleted chunk 0 from MinIO while its PostgreSQL stamp
was row-lock-blocked, was killed with `SIGKILL`, left one object and
both stamps absent, then resumed the same request under generation 2 to
zero objects and terminal state.
- **Independent dead-owner recovery:** a dedicated executor claimed
generation 1, blocked before effects, and was killed through containerd
with `SIGKILL` (no TERM cleanup). The request remained owned and
unreclaimable before lease expiry; a successor claimed generation 2
after 60 seconds and completed with two attempts and zero retries.
- **Three-pod socket isolation:** ordinary NIP-42 and joined
huddle-audio target witnesses on every replica received exact `1008 /
community deleted`; healthy-tenant witnesses on those pods remained
live; deleted-host reconnect returned HTTP 404.
- **Health/provenance:** all replicas independently returned ready and
retained the exact image digest before/after destructive runs and an
audio-enabled rolling restart; PostgreSQL, Redis, and MinIO were healthy
at close.
Instrument corrections were retained as evidence rather than counted as
product failures: a foreground PostgreSQL forward caused an initial
`PoolTimedOut`; Kubernetes pod deletion exercised graceful TERM rather
than dead-owner recovery; shell-background socket witnesses died with
their parent; and the first image build hit the corporate TLS proxy.
Detached forwarding/witnesses, containerd `SIGKILL`, and the configured
internal CA/Artifactory mirror produced the discriminating runs without
weakening product security.
### Prior-head cleanup
For the prior-head Kubernetes run, the Helm release was removed,
namespace absence was verified, run-owned Screen sessions were absent,
and that source worktree remained clean. The evidence manifest was
independently recomputed and every indexed artifact passed `shasum -a
256 -c`. The current `359d8402` source worktree is also clean after the
scanner-only cleanup and push.
---------
Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Signed-off-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
Co-authored-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz>
This change rechecks the durable community ban in the shared Git HTTP
authentication path for advertise, fetch, and push requests. A banned
member is denied even if repository-channel membership still exists, and
restriction lookup errors fail closed.
The additional database lookup happens on every Git HTTP request so
access revocation does not depend on stale session state.
The check also cascades to the NIP-OA owner. Git accepts NIP-OA
attestations on the NIP-98 token, so an agent key can act for its owner
— without the cascade, a banned human would keep clone and push access
through any agent key. This mirrors the NIP-42 gate in `handlers::auth`:
either principal's ban denies the request.
The check runs inside the `GitAuth` extractor, so all three Git routes
inherit it.
## Testing
- `git diff --check origin/main...codex/security-ban-revokes-git`
- Rebased onto `origin/main` at `5c98932`
- `cargo test -p buzz-relay --lib sec005_read_gate_tests`: 8 passed, 7
ignored (Postgres)
- `cargo clippy -p buzz-relay --all-targets -- -D warnings` and `cargo
fmt --check`: clean
Pure tests cover the decision table (agent ban, inherited owner ban, no
attestation). Postgres-gated tests cover the wiring: the real ban row, a
live `compute_auth_tag` attestation, and the 503 fail-closed path.
**Not yet verified:** the three Postgres-gated tests compile and skip
but have not been run — no local Postgres, and CI does not run
`--ignored`. They need `cargo test -p buzz-relay --lib
sec005_read_gate_tests -- --ignored` against a migrated dev database.
Originating Buzz thread:
`buzz://message?channel=3928fe05-df61-4b5d-b9c7-d623b9b10ea1&id=3c6c02312f763fbe0d2bfc33a6c1a362f91d0354f3d18b039cf7a0558c1439d1`
---------
Signed-off-by: Jordan Mecom <jm@squareup.com>
Signed-off-by: Eli Foster <efoster@squareup.com>
Co-authored-by: Eli Foster <efoster@squareup.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Tal here, human. Trying to help. This bug bugged me...
## Summary
A repository's first branch becomes its symbolic `HEAD`, and Git's
bare-repository default rejects deleting that branch even when another
branch survives.
This change:
- sets `receive.denyDeleteCurrent=ignore` only for the ephemeral `git
receive-pack` process
- preserves the existing server-side `core.hooksPath` override and
authorization hook
- lets the existing CAS publication logic select a surviving branch as
the next manifest `HEAD`
- adds regression coverage using a real stateless `git receive-pack`
request and a manifest HEAD-selection test
This lets users replace an accidental default branch without deleting
the object-storage manifest pointer.
### Related issue
Fixes#3572
### Testing
- `cargo test -p buzz-relay api::` (128 passed, 5 ignored)
- `just ci`
- live E2E roundtrip against a release relay with PostgreSQL, Redis, and
MinIO:
- created a repository through signed Nostr events
- verified authorized pushes and rejected unauthorized clone/push
- pushed a surviving `master` branch
- deleted the active `main` branch over authenticated Smart HTTP
- freshly cloned the repository and verified `master` became HEAD,
`origin/main` was absent, and repository content remained intact
Signed-off-by: Tal Weiss <major.tal@gmail.com>
Buzz renders one card per `kind:30617`, so a project spanning several
repositories has no representation.
[NIP-MP](https://github.com/block/buzz/pull/3163) defines `kind:30621`
as an addressable container holding a group's name, description, channel
binding, and member coordinates. This adds the kind to `buzz-core` and
its structural validation to the relay ingest path.
## Event shape
```json
{
"kind": 30621,
"tags": [
["d", "platform"],
["name", "Platform"],
["description", "Relay, desktop, and mobile."],
["a", "30617:<owner-a-hex>:buzz"],
["a", "30617:<owner-b-hex>:buzz-infra"],
["buzz-channel", "<channel-uuid>"],
["buzz-visibility", "listed"]
]
}
```
## Validation at ingest
| Rule | Behavior |
|------|----------|
| `d` tag | exactly one, non-empty (length already bounded by the
generic `D_TAG_MAX_LEN` check) |
| member `a` tag arity | exactly 2 or 3 elements per NIP-01's `a` tag
grammar; a 4th element has no defined meaning and is rejected |
| member `a` tag coordinate | must parse as
`30617:<lowercase-64-hex-owner>:<non-empty-d>` |
| duplicate members | rejected on exact string match of the canonical
coordinate |
| member cap | 64, counted over raw `a` tags |
| metadata cardinality | at most one each of `name`, `description`,
`buzz-channel`, `buzz-visibility` |
| metadata length | `name` ≤ 256 bytes, `description` ≤ 2048 bytes,
`buzz-channel` ≤ 256 bytes, `buzz-visibility` ≤ 256 bytes |
| zero members | valid |
| unknown tags | ignored |
Rejection order is normative so a client can predict which rule fires:
`d`-cardinality → `d`-empty → member-cap → member-arity → coordinate
parse → member-duplicate → metadata cardinality → metadata length.
## Design notes
**No membership authorization.** Members are `a` tags, so one project
may name repositories owned by different pubkeys — the entire point of
the kind. That is safe because membership grants nothing: push policy
reads a repository's own `kind:30617` (`api/git/policy.rs`) and never a
project. `buzz-channel` is a metadata reference, not a routing
directive, so projects are classified global-only.
**Owner-only editing is free.** NIP-33 addressing keys replacement on
`(pubkey, kind, d)`, so one signer can never overwrite another's
project. No relay-side permission check exists or is needed, and
`test_project_same_d_under_two_authors_are_independent` pins it.
**Duplicates are rejected, not deduped.** A relay cannot rewrite tags
inside a signed event without invalidating its id and signature, so the
alternative to rejection is a stored duplicate-member head that every
consumer must apply a first-wins rule to.
**The cap is checked before the duplicate set is built.** Counting raw
`a` tags rather than distinct coordinates means an event naming one
coordinate thousands of times is refused on count, instead of being
bounded only by the relay frame limit.
**No side-effect handler.** Generic NIP-33 replacement and generic
NIP-09 coordinate soft-delete already cover replacement and deletion;
`kind:30621` needs no entry in `is_side_effect_kind`.
## Generic NIP-09 fix carried along
`soft_delete_by_coordinate` (`crates/buzz-db/src/event.rs`) previously
deleted the live coordinate head regardless of the tombstone's own
`created_at`, so a delayed or replayed `a`-tag deletion signed between
two versions destroyed the newer replacement. NIP-09 scopes an `a`-tag
deletion to versions at or before the deletion request, so the `UPDATE`
now carries `created_at <= $5` and `handle_a_tag_deletion` threads the
deletion event's `created_at` through.
The bug predates `kind:30621` and affected every
parameterized-replaceable kind on the generic path — `kind:30617`
repository announcements included — so the fix lands there rather than
as a project special case. `events.created_at` is immutable per row, so
the predicate guarantees a tombstone can never erase a version newer
than itself; the UPDATE re-evaluates its WHERE clause after any lock
wait. Under READ COMMITTED, a same-coordinate replacement racing the
deletion may cause the deletion to evaluate before the new head lands,
returning `Ok(false)` — but that outcome is state-identical to the
deletion having arrived first, a valid Nostr ordering Nostr never fixes.
The return value feeds only a debug log. No coordinate-level lock is
needed.
## Coverage
32 unit tests in `crates/buzz-relay/src/handlers/ingest.rs` pin the
envelope contract (accept: minimal, cross-owner, zero-member, same repo
`d` under two owners, colon-bearing repo `d`, cap boundary, unknown
tags, relay hint on member `a` tag, max-length metadata, stranger-owned
member, uninterpreted metadata values, non-empty content; reject: every
rule above plus valueless `d`/`a` tags). A fixture-driven test
(`project_envelope_validates_all_shared_fixtures`) runs every case in
the shared `NIP-MP.fixtures.json` oracle (11 accept + 20 reject) against
`validate_project_envelope`, so any future change that breaks a case
turns the test suite red.
6 `#[ignore]`d e2e tests in
`crates/buzz-test-client/tests/e2e_project.rs` cover behavior that only
exists past storage — coordinate round-trip, newer-wins replacement, two
authors sharing a `d`, an `a`-tag tombstone that removes the project
while leaving referenced `kind:30617`s intact, and a tombstone
timestamped between V1 and V2 that must leave V2 live. The negative e2e
case asserts on the rejection message so a refusal for an unrelated
reason cannot satisfy it; that is what proves the validator is reachable
from the live write path rather than merely correct in isolation. The
new e2e binary is wired into the Relay E2E job.
The timestamp predicate is additionally pinned at the storage layer by
`coordinate_delete_spares_head_newer_than_the_deletion` in
`crates/buzz-db/src/lib.rs`, which asserts both directions: a stale
tombstone deletes nothing and leaves the newer head readable, and a
tombstone at the head's own timestamp still deletes it. This test is
wired into the Backend Integration job.
Related: #3163 (the NIP-MP spec and shared conformance fixtures).
Independent — either can merge first.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Closes#3527.
Repos announced via vanilla NIP-34 (kind:30617 without a `buzz-channel`
tag) 404 forever: the SEC-005 read gate requires a channel-membership
ACL, and nothing tells the author why or how to fix it. Per the ruling
in the originating thread, this ships **bind/rebind tooling plus a
narrow author-only remediation carve-out** — the shelved owner-circle
approach is intentionally absent.
## Relay
- **`api/git/binding.rs` (new):** shared tri-state binding resolver —
`Bound(uuid)` / `NotBound` / `Broken`. First-tag, fail-closed: a
malformed `buzz-channel` tag is `Broken`, never conflated with "no tag".
Both gates use it.
- **Read gate (`transport.rs`):** a **never-bound** repo read by **its
own announcement author** still returns 404 (status byte-identical to
the generic denial) but the body carries remediation: `run: buzz repos
bind --id <repo> --channel <channel-uuid> — …`. This leaks nothing — the
author announced the repo, and only the author can rebind (30617 is
keyed by `(author, d)`). `Broken` bindings stay generic-denial for
everyone, including the author (revocation shape).
Bound-to-nonexistent-channel stays generic (phase 1; ingest validation
is phase 2).
- **Push gate (`policy.rs`):** unbound denial now returns
`GIT_NO_CHANNEL_BINDING_BODY`. A deploy-skew test pins that the body
carries both the new token (`no_channel_binding`) and the legacy phrase
(`"no channel binding"`) so already-shipped desktops keep matching.
**(Review r1, blocker 2)** `Broken` no longer collapses into "unbound":
it denies 403 `invalid channel binding` for *everyone — including the
announcement owner —* **before** the owner short-circuit, matching the
read gate's fail-closed posture. The remediation token stays
NotBound-only.
- **`ingest.rs`:** side-effect failure `warn!` → `error!` — prod runs
`RUST_LOG=error`, so these failures were invisible during triage.
## Contract
- **`buzz-core/git_perms.rs`:** `GIT_NO_CHANNEL_BINDING_TOKEN` /
`GIT_NO_CHANNEL_BINDING_BODY` consts as the declared cross-component
contract; relay tests and desktop matcher both build on them.
## CLI
- **`buzz repos bind --id <repo> --channel <uuid>`** — rebinds an
existing announcement, preserving other tags.
- **(Review r1, blocker 1)** **`--channel` on `buzz repos create`** —
optional; injects exactly one shape-validated `buzz-channel` tag at
creation via a pure `build_create_announcement` builder, so the primary
create command stops producing repos the relay 404s. UUID
existence/membership stays the relay's authority at git-access time
(same TOCTOU posture as `repos bind`). Overlaps with #3594 (open, head
6bbe38459) — happy to reconcile whichever lands first; this branch also
carries the bind path and tag preservation.
## Desktop
- **Rust:** new `commands/project_git_merge_error.rs` (extracted from
`project_git_workflow.rs` to respect the 1000-line ratchet); maps the
token to a structured `no_channel_binding` error carrying the bind
command.
- **TS:** new `features/projects/lib/projectBranchErrors.ts` + tests —
dual matcher (new token AND legacy spaced phrase);
`ProjectBranchDialogs.tsx` uses it.
## Tests / verification (at head f914c7066, base 581baa625)
- Workspace `cargo test` green; `clippy -D warnings` clean; desktop Rust
1859 pass; TS 3780 pass; tsc/biome/file-size checks pass. Pre-push hooks
re-ran all suites at the pushed head.
- Postgres-gated `sec005_read_gate_tests`: all 6 pass, including
`read_gate_gives_author_of_unbound_repo_remediation_body` — asserts 404
status, `text/plain` content-type, and exact body bytes, distinguishing
remediation from generic denial (a blind `is_err()` can't).
- **New (review r1):** `buzz-cli` emitted-event tests —
`create_with_channel_emits_exactly_one_binding_tag`,
`create_without_channel_emits_no_binding_tag`,
`create_rejects_malformed_channel_uuid` (266/266 pass). Postgres-gated
`push_gate_denies_owner_through_broken_binding` — owner +
malformed-first/valid-second binding → 403 generic body without the
remediation token; never-bound control stays 200, pinning the denial to
`Broken` specifically.
- e2e git tests now bind announcements to a real channel via a
`create_test_channel` helper.
---------
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
## Summary
- add one strict `BUZZ_S3_ADDRESSING_STYLE=path|virtual` setting shared
by media and Git/CAS storage
- preserve path-style defaults for bundled Compose/Helm MinIO while
supporting Railway's virtual-hosted bucket contract
- fail startup on invalid or non-Unicode values before dependency
connection, and validate the Helm value with the same two choices
- document operator mappings and why endpoint and bucket remain separate
for routing and SigV4 signing
## Best-practice rationale
AWS documents both URL forms and favors virtual-hosted addressing for
S3, while compatibility endpoints such as the bundled MinIO deployment
can require path style. `rust-s3` defaults to virtual/subdomain
addressing and provides `with_path_style()` for the explicit
compatibility case.
Some providers buckets only support as virtual-hosted bucket styles.
This PR therefore uses one explicit, provider-neutral switch rather than
endpoint heuristics or fallback behavior, while retaining `path` as
Buzz's backward-compatible default.
Sources:
-
https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html
- https://docs.rs/rust-s3/0.37.0/s3/bucket/struct.Bucket.html
- https://docs.railway.com/storage-buckets#url-style
-
https://github.com/minio/minio/blob/master/docs/config/README.md#domain
## Validation
- `cargo fmt --all`
- `cargo check --workspace --all-targets`
- targeted `buzz-media` and `buzz-relay` parsing/client-construction
tests for defaults, strict errors, and both URL styles
- Helm unittest: 45/45 passed
- Compose config/render validation passed
- local MinIO path-mode relay startup passed the Git A3 conformance
probe and became ready
- unreachable object storage failed startup and readiness never opened
- push hooks completed the broader Rust and desktop suites successfully
---------
Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz>
Fixes all six HIGH findings from the buzz security report, one commit
per finding. Independently reviewed to approval by Max at `0158ae542`,
plus a deep isolated live pass (clean-room compose stack, weird ports,
full product matrix) at the same head — see the buzz-security thread for
evidence. `fe65c07c3` merges current `origin/main` on top (new commit,
no rebase), inheriting the nostr 0.44.6 bump (#3135) and relay-admin ban
gate (#3128).
## Findings and fixes
| Finding | Commit | Fix |
|---|---|---|
| 003 — quinn-proto RUSTSEC-2026-0185 | `e5dcdec72` | Bump quinn-proto
0.11.14 → 0.11.16 (lockfile-only) |
| 002/004 — linkify-it quadratic-parse DoS (GHSA-22p9-wv53-3rq4,
GHSA-v245-v573-v5vm) | `923b3c20f` | pnpm override `linkify-it: ^5.0.2`;
`pnpm why` confirms a single 5.0.2 copy |
| 001 — media reads served unauthenticated by default | `0f277e3e2` |
Helm `requireMediaGetAuth` defaults to `true` + rendered-chart test
pinning the default |
| 006 — removed workflow owners retain webhook-exfiltration authority |
`4749bd56c` | Fail-closed per-fire authority gate (current owner/admin
membership) on **all four** trigger doors (on_event, scheduler
pre-claim, manual trigger, webhook — masked as generic 404), save-time
gate for `call_webhook` defs, durable disable-on-removal wired to kinds
9001 + 9022 |
| 005 — git Smart-HTTP reads ignore channel membership | `e648f2dba` +
`0158ae542` | `authorize_git_read`: caller's **current active
membership** in the repo's bound channel, checked before any
hydration/subprocess on all three read doors (`info_refs` for both
services + `upload_pack` POST). Uniform generic 404 denials (no
membership probing), no repo-owner bypass, first-`buzz-channel`-tag
binding semantics fail closed on ambiguous duplicates (mutation-verified
test). Resolution follows the live kind:30617 announcement, so
deleted/replaced announcements deny immediately. The committed
`e2e-git-perms.sh` guest scenario previously asserted the vulnerability
— now asserts denial. |
## Behavior changes to be aware of
1. **Unbound repos fail closed for git reads.** `buzz repos create`
emits no `buzz-channel` tag, so CLI-created repos without a binding are
unreadable via git HTTP. Correct per finding 005's fail-closed posture;
a follow-up could bind CLI-created repos at creation time.
2. **006 is conservative:** a workflow disabled on owner removal does
not auto-re-enable if the owner is re-added — explicit re-enable
required.
3. Merge conflict resolution in `fe65c07c3`: kept main's
`@radix-ui/react-dismissable-layer` 1.1.19 bump alongside the linkify-it
security override (`pnpm-workspace.yaml` + lockfile).
## Verification at the merge head `fe65c07c3` (same shell)
- buzz-relay `--lib`: 761 passed / 1 failed — the lone red is the known
pre-existing `mesh_demo::demo_join_forwarded_arm_round_trips_echo` 504
flake, present on main
- SEC-005 module incl. PG behavioral matrix: 8/8 (removed-member,
never-member, owner-no-bypass, deleted-30617, malformed/ambiguous
binding, owner-mismatch all denied)
- buzz-workflow 153/0, buzz-db 84/0; `clippy --all-targets -D warnings`
+ `fmt --check` clean
- Desktop JS 3637/3637, tsc clean, biome clean,
file-size/px-text/pubkey-truncation gates clean
- `helm lint` + `helm unittest` (40/40) on `deploy/charts/buzz`
- All five pre-push hooks green (desktop-check, desktop-test,
rust-tests, desktop-tauri-test, branch-skew)
Prior review evidence at `0158ae542` (pre-merge): Max's independent
exact-head approval + clean-room live regression pass
(`WORK_LOGS/2026-07-27_SECURITY_HIGH_LIVE_TEST.md` in his workspace).
Max will re-run the deep local pass at this post-merge head before
merge.
---------
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
