mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
bf8b356f3bf551a01cc24041605ca77d009681dd
44 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Bradley Axen
|
8b60dcf8e3 |
feat(relay): deliver full signed events to operator listeners (#8309)
## Summary
Operator-listener mention notifications now carry the full signed event,
so a listener such as kgoose can wake an agent without first fetching
the event (one signed round trip less on time-to-first-reply).
Because the notification now carries message content, two mitigations
come with it, as agreed in #buzz-security:
1. **Each target signs its own registration.** `POST
/operator/listener/pubkeys` takes `{"registrations": [proof, …]}`. Each
proof is a kind `24245` event signed by the target pubkey, with exactly
one `p` tag (the listener pubkey) and one `u` tag
(`{RELAY_OPERATOR_API_ORIGIN}/operator/listener/pubkeys`), and a
`created_at` within ±60s. The old unsigned `{"pubkeys": […]}` POST body
is rejected with 400. `DELETE` is unchanged.
2. **Only notify a target that can read the event.** Before each
delivery the relay re-checks, on the writer, that:
- the registration still exists;
- the event isn't deleted;
- neither the target nor its recorded NIP-OA owner is banned;
- the target passes the relay-membership gate (with the owner path when
`allow_nip_oa_auth` is on);
- the channel is open, or the target is an active member.
A delivery that fails the check retries with backoff instead of being
dropped, because a mention can commit just before the target is added to
the channel. It fails permanently, with no HTTP request, once attempts
are exhausted.
The payload is now `v: 2`: the v1 fields plus `event`, the full signed
NIP-01 event.
There's no migration. Existing registrations stay in place and age out
under the 30-day retention; only a configured listener could have
created them. Listeners must sign proofs to register new targets, so
receivers have to accept `v: 2`.
Companion kgoose change: squareup/cash-server#136794 signs registrations
with the agent key. The kgoose receiver in squareup/cash-server#136004
currently accepts only `v: 1` and bodies up to 4 KB, so it needs to
accept v2 and a larger body.
## Testing
- Unit tests:
- proof verification (forged signer, wrong kind, stale or future, other
or ambiguous listener, other or missing endpoint);
- signed-only request bodies;
- delivery payload carries a verifiable event;
- unreadable or failed loads retry, then fail, without an HTTP request.
- Postgres tests:
- every access predicate (open, member and non-member private,
channel-less, closed community with target or owner membership, owner
ban, removed member, deleted channel or event, removed registration);
- the route test (signed registration accepted; a proof for another
listener and the unsigned body both rejected).
- Mutation runs confirmed that the membership, owner, ban and
reader-visibility guards each fail a test when removed.
- Local relay exercise with a fake listener:
- signed registration returned 200; unsigned, stale and wrong-listener
proofs returned 400;
- open-channel mentions delivered v2 with an event whose id and
signature verify independently;
- a private-channel mention of a non-member retried without delivering,
then delivered once the target was added.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Bradley Axen <baxen@squareup.com>
Co-authored-by: murderbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Jarrod Sibbison
|
8599efb6a3 |
feat(relay): allow HTTP operator listener delivery URLs (#8287)
## Summary Operator listener configuration currently rejects HTTP URLs, preventing delivery to sibling services over internal HTTP connections. Accept HTTP and HTTPS delivery URLs while retaining the existing pubkey, duplicate-route, and URL restrictions. Add configuration coverage for internal hostnames and custom ports, verify signed delivery through the real HTTP client, and include the listener suite in both unit-test runners. Shared NIP-98 signing and push gateway code are unchanged. ### Related issue Related: #7934 refactors operator listener delivery tests. No matching open issue found. ### Testing - `cargo test -p buzz-relay --lib operator_listener`: 13 passed, including signed HTTP delivery and redirect rejection. - `cargo fmt --all -- --check`, shell syntax, Justfile parsing, and diff whitespace checks passed. - `just test`: passed. - `just ci`: stopped at mobile analysis with an existing undefined `buzzDevPushAppProfile` in `mobile/test/features/settings/notification_toggle_test.dart:81`. Workspace and native desktop formatting/Clippy, desktop frontend checks, and web checks passed. The mobile file is unchanged by this PR. Locally verified with a spiked listener client Signed-off-by: jsibbison-square <jsibbison@squareup.com> |
||
|
Tom Brow
|
5b907b838a |
fix(push): isolate push enqueue and stop workers on rollback (#8216)
Push enqueueing previously ran inside the message transaction, so a push lock or database error could block or reject a message. Configured relay writers now submit bounded best-effort work only after commit. A dedicated connection and bounded queue isolate enqueue waits and failures from message-write connections. Overload, errors, process loss, or shutdown may drop a notification; the committed message remains authoritative. `BUZZ_PUSH_ENABLED=false` disables the application producer and workers. Updated writer connections always bypass any surviving enqueue trigger. The overlap migration updates an existing function without resurrecting retired objects. Apply it before relying on isolation with a surviving trigger, and replace every old enabled process and writer connection: missing settings preserve legacy behavior during overlap. SIGTERM/Ctrl-C cancels enqueue, matcher, and delivery work before listener drain. Durable queues, claims, leases, preferences, and delivery configuration remain intact. Claims expire normally; gateway requests already accepted may still deliver. Reactivation requires an explicit operator decision about retained work. There is no automatic purge, startup interlock, or backfill of messages written while disabled. Built on merged [#8135](https://github.com/block/buzz/pull/8135), including [#8178](https://github.com/block/buzz/pull/8178). Trigger retirement [#8210](https://github.com/block/buzz/pull/8210) remains open; this change is tested with the trigger present and after removal. Operational details are in [the rollback and reactivation guide](https://github.com/block/buzz/blob/relay-push-rollback/docs/push-gateway-deployment.md#relay-rollback-and-reactivation). ### Validation [ |
||
|
Ian Oberst
|
a8fe2b612c |
feat(db): advance an empty partition catch-all under bounded locks (#7993)
## Summary Adds bounded, serialized advancement of an empty right-edge partition catch-all for `events` and `delivery_log`. The repaired catalogs currently end in an empty catch-all from 2027-01-01, and the merged manager (#6515) only audits them. Without this change, January writes land in the catch-all, after which only an operator can move them. When `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED=true`, each relay keeps six dedicated monthlies after the current month. It does this at startup and on every audit interval by replacing the empty catch-all with canonical monthlies and a new canonical `<table>_p_future` after the horizon. From today's catalogs the first run creates January–March 2027 and moves the catch-all to April 2027. ## Safety design - **Refuse before locking.** The following return `operator_required` without taking any table lock: - a populated catch-all, or one whose occupancy is unknown; - a catch-all with child-only triggers, which the replacements would not inherit; - a misaligned bound; - a `DEFAULT`, anomalous or pending-detach child; - a name collision. - **One winner.** A schema-scoped `pg_try_advisory_xact_lock` is taken before any DDL. Losers report `skipped_locked` and never issue DDL. The winner re-plans from a fresh audit before taking any table lock and checks for collisions. A runner that lost a race therefore sees the committed layout as a no-op, not as colliding names. - **Writer lock order.** First `ONLY` the parent is locked `ACCESS EXCLUSIVE` under a 2 s `lock_timeout`. Then each foreign-key counterpart (today only `communities`) is locked in OID order `SHARE ROW EXCLUSIVE NOWAIT`, then the catch-all `NOWAIT`. - The counterparts are read again once the parent is locked, and that set is the one locked. Any FK DDL on or to the parent conflicts with the parent lock, so the set can't change after that. An FK committed while maintenance waited is therefore locked `NOWAIT` too, so attaching never waits on it under the parent lock. The read before the lock remains as a cheap early refusal. - This is the order an insert takes locks (parent, then the FK check on `communities`), so maintenance cannot deadlock with ingest. The internal manual runbook used to lock `communities` first; it now uses this order too. - `SHARE ROW EXCLUSIVE` is the strongest lock the DDL takes on `communities`; measured on PG 16 and 17, the catch-all `DROP` takes none, as long as the catch-all's only foreign keys are those inherited from the parent. A catch-all with its own foreign key, or one that another table's foreign key references, is refused: once before any lock, and again after the catch-all lock, where the check is authoritative. Readers and other writers' `FOR KEY SHARE` checks proceed. A concurrent `communities` writer makes the run fail fast with `lock_timeout`. - A `40P01` deadlock victim is also reported as `lock_timeout`. - Parents that are referenced by a foreign key, or that reference an inherited table, are refused, including when such a key commits while maintenance waits on the parent. - **Re-plan under lock.** The catalog is re-audited under those locks. The transaction proceeds only if the plan is unchanged. - **Bounded.** `statement_timeout` 5 s, `idle_in_transaction_session_timeout` 5 s, and a 10 s client deadline per table. A deadline-abandoned connection is closed, never returned to the pool, and the server rolls back anything uncommitted. If the deadline fires after `COMMIT` was sent, the change may have landed; the next audit reports the truth. The orphaned backend keeps its locks until its in-flight statement ends, at most `statement_timeout`. - **Verify before commit.** The changed catalog is re-audited inside the transaction for bounds, canonical kinds, trigger parity and catch-all emptiness, and serving safety must not regress. It also checks that every new child has valid, ready clones of each parent index and validated clones of each PK/unique/FK constraint. - **Audits tolerate a concurrent advance.** Catalog rows come from the statement snapshot, but `pg_get_expr` renders from current catalog state. So an audit that overlaps another relay's `DROP` of a catch-all can see a partition that no longer exists. The read-only audit detects this (a NULL bound, or `42P01` from the emptiness probe) and retries up to twice with a fresh snapshot. - **Fail fast on audit probes.** If another relay's audit is probing the catch-all's emptiness, the catch-all `NOWAIT` lock fails and the run reports `lock_timeout`. The parent lock is then released immediately, and the next runner or interval completes the advance. Uncovered-month creation (`BUZZ_PARTITION_MANAGER_CREATE_ENABLED`, default `true`) now runs through the same locked transaction, replacing the unlocked per-month `CREATE`. That flag remains the kill switch for all automatic partition DDL: with it off, advancement is disabled too. With advancement disabled (the default) the periodic loop stays read-only and startup behaves as before, apart from the locking. A runner that planned DDL always re-audits before returning, even when a peer did the work, so it never reports the pre-maintenance layout. A serialized re-plan that finds only disabled work reports `skipped_disabled`. ## Behavior changes - The relay audit horizon and the `buzz-admin partition-audit` default are now 6 months (`PARTITION_MANAGER_MONTHS_AHEAD`), up from 3. Until advancement is enabled, months already covered by the catch-all make `degraded` true. That correctly reports a runway under six months. - New metric `buzz_partition_maintenance_runs_total{table,outcome}` with outcomes `noop | skipped_disabled | created | advanced | skipped_locked | operator_required | lock_timeout | deadline | error`. `buzz_partition_create_attempts_total` keeps its per-month `created | skipped_covered | error` labels. `error` there still means a DDL or collision failure; `lock_timeout` and `deadline` runs add nothing to it, and only `buzz_partition_maintenance_runs_total` records them. - `ensure_future_partitions` (free function and `Db` method) is removed; `maintain_partitions` is the one entry point. - New advisory-lock label `lock_type="partition_maintenance"`. ## Rollout Enable `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED` per environment in BPCI: `bb-block-staging` first, then production. The staging canary proves a real advance because the six-month horizon already extends past January. Fallback: rerun the internal manual runbook, which uses the same parent-first lock order as of 2026-10-05. It keeps `ACCESS EXCLUSIVE` on the counterparts, because its detach/attach can drop FK triggers on them. ## Testing - `cargo test -p buzz-db --lib partition`: 17 unit tests. The 10 new ones cover: - plans: repaired layout, policy gating, no-op, refusal states including child-only triggers, December→January; - the create kill switch also stopping advancement; - the month cap at exactly 120; - per-month metric outcomes for transient failures versus DDL failures; - the mid-audit-drop retry classifier. - `cargo test -p buzz-db --lib partition -- --ignored` (PostgreSQL 17): 48/48 on 9 consecutive runs, including 17 new tests: - advance to the canonical layout, with routing and idempotent rerun; - replacing a canonically named catch-all in one transaction; - populated catch-all refused in under 1 s while the parent is locked; - a `communities` writer making the run fail fast and roll back; - a `communities` reader not blocking the advance; - an in-flight insert's FK check, and its write to `communities`, behind queued maintenance completing without a deadlock; - a foreign key committed while maintenance waits on the parent, with a writer on its new counterpart, failing fast as `lock_timeout` on that table instead of waiting under the parent lock; - a foreign key referencing `events`, committed while maintenance waits on the parent, being refused under the parent lock; - a catch-all's own foreign key, or a foreign key referencing it, being refused before the parent lock with the key intact, and a catch-all key committed while maintenance waits on the parent being refused after the catch-all lock; - a `40P01` deadlock victim being classified as `lock_timeout`; - parent contention timing out without blocking the other table; - held advisory lock skipping with no DDL; - concurrent runners converging, with at most a transient `lock_timeout`; - a stale audit after a concurrent advance being a no-op, and the losing runner returning the committed layout; - a re-plan that finds only disabled work reporting `skipped_disabled`; - an audit retrying when a listed partition is dropped mid-audit; - a catch-all with a child-only trigger being refused; - client deadline abandoning the attempt and closing the session (scoped to the test's own blocked backend); - catch-all name collision; - incoming foreign key refused. - Mutation checks: - restoring the old counterpart-first `ACCESS EXCLUSIVE` order fails the reader test (`lock_timeout`) and the writer test (a ~1 s deadlock wait); - dropping `NOWAIT` from the counterpart lock, or allowing a populated catch-all, fails the matching tests; - removing the parent-locked counterpart re-read fails the committed-FK tests (the parent is held 2.06 s waiting on the new counterpart), and ignoring its refusal fails the referencing-FK test; - locking the counterparts `SHARE ROW EXCLUSIVE` before the parent fails the writer test; - removing either catch-all foreign-key check fails its test. - Full PostgreSQL lane: 716/718. Both failures are timing-sensitive tests outside partition code, and each passes 3/3 when run alone: - `hanging_redis_peer_times_out_before_the_health_listener_binds`; - `readiness_check_cancellation_balances_waiter_and_inflight_connection`. - `cargo clippy -p buzz-db -p buzz-relay -p buzz-admin --all-targets -D warnings` and `cargo fmt --check` are clean. ## Related Refs #2396. This PR conflicts textually with #7994 (partition audit observability follow-ups) in `crates/buzz-db/src/store/partition.rs` and `crates/buzz-relay/src/main.rs`. Whichever PR merges second will be rebased onto the other. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Will Pfleger
|
032198ce43 |
feat(relay): add community reads to the admin API (#8011)
The in-app Admin Console needs to work on any community the relay
serves, not only the one the desktop is connected to. This adds four
read-only routes under `/api/admin/v1` that the stacked desktop PR
builds on, puts every admin route under one explicit access rule, and
binds every admin request's signature to its real method, which fixes
signed `HEAD` on reads.
## Access rule
| Mode | Moderation reads | Writes | Staffing (`/operators`) |
|---|---|---|---|
| `nip98` (default) | signed, any staff role | signed, role-checked |
signed Operator |
| `disabled` | served to anyone who can reach the relay | `403` | `403`
|
"Moderation reads" means every read: reports, feedback and attachments,
restrictions, and the four community routes below. `nip98` behavior is
unchanged, and the Operator API (`/operator/*`) is untouched.
**Existing `BUZZ_ADMIN_AUTH=disabled` deployments start serving the four
new reads** (community directory, member search, member lookup without
staff status, and stored events by ID) with no config change. The
startup `WARN`, `docs/admin/README.md`, `.env.example`,
`deploy/compose/.env.example` and the `just admin` comment list exactly
what disabled mode exposes, say it should be reachable only by people
allowed to see all moderation data in every community, and say the whole
relay port must be private: the admin API is on the relay's main
listener and its hostname check uses the caller's `Host` header. `just
admin` binds the relay to `127.0.0.1` unless `BUZZ_ADMIN_AUTH` is
exactly `nip98`, so an unset, padded or misspelled value stays local.
In code, `authorize()` returns `AdminAccess`, either `NetworkTrusted`
(disabled mode, no identity) or `Staff(AdminPrincipal)` (`nip98`, any
rostered Operator or Moderator). Each route makes one check: **view**
(`authorize()` succeeding), **act** (`AdminAccess::act()`, must be
`Staff`, then the existing role checks) or **operator**
(`AdminAccess::operator()`, must be an Operator). Act and operator
refuse `NetworkTrusted` with `403 this endpoint requires
BUZZ_ADMIN_AUTH=nip98`; no principal is ever created for an anonymous
caller. `every_route_makes_its_check_in_both_auth_modes` lists every
mounted admin route with its check and asserts the result in both modes,
and `every_route_is_listed` fails when `router()` mounts a route that
list does not name.
## Routes
All four live in `api/admin/reads.rs`; `mod.rs` only registers them.
Their queries reject unknown fields, like `ReportQuery`. The three
`?communityHost=` routes share `CommunityQuery`, which now rejects
unknown fields too, so the existing unban, untimeout and direct-action
routes that use it do the same; their desktop callers send only
`communityHost`. Every admin route that takes `limit` validates it with
the module's shared `limit()` helper, so an out-of-range value is `400
invalid_limit`.
| Route | Returns |
|---|---|
| `GET /communities?q=&cursor=&limit=` | `{items: [{id, host,
icon\|null}], nextCursor\|null}` |
| `GET /members/search?communityHost=&q=&limit=` | `{items: [{pubkey,
displayName\|null, nip05\|null, avatarUrl\|null}]}` |
| `GET /members/{pubkey}?communityHost=` | `{pubkey, profile:
{displayName, nip05, avatarUrl, about}\|null, role:
"owner"\|"admin"\|"member"\|null, banned, mutedUntil\|null, isStaff:
bool\|null}` |
| `GET /events/{id}?communityHost=` | `{id, authorPubkey, kind, content,
createdAt, deletedAt\|null, channelId\|null}` |
- **One community per read.** Community-scoped routes bind
`communityHost` through the tenant binder (`400 unknown_community_host`)
and read only that community's `users`, `relay_members`,
`community_bans` and `events` rows.
- **Directory.** It uses the binder's liveness rule (not archived, not
deleted, `deletion_state = 'active'`), so every listed host can be
opened. `q` is an escaped, case-insensitive host prefix of at most 255
characters (`400 invalid_query`). Results are ordered and paged by
`lower(host)`, which is unique, so the opaque base64url cursor carries
only the last host; a malformed cursor is `400 invalid_cursor`. `limit`
is 1 to 100 with a default of 50.
- **Member search.** It reuses `search_users`, which already escapes,
binds and scopes, with no second escaping layer. `q` is trimmed and must
be 1 to 100 characters (`400 invalid_query`), and `limit` is 1 to 50
(default 20). It searches community profiles, so former members appear.
- **Member lookup.** `isStaff` uses the same roster lookup as the
direct-action staff guard and is returned to signed staff only: it is
`null` in disabled mode, where the roster is never read. A roster lookup
failure is a 500, never "not staff".
- **Event preview.** It reads the event by `(community_id, id)` and
never returns a private kind (see below). A malformed id, an id stored
only in another community and a private kind are all `404
event_not_found`, the same answer as an absent event.
## Private events
Admin reads never reveal events the relay keeps private.
`admin_hidden_kinds()` combines the four private kind sets in
`buzz_core::kind`: `AUTHOR_ONLY_KINDS`, `RESULT_GATED_KINDS`,
`P_GATED_KINDS` and `SHARED_GATED_KINDS`. The whole shared-gated set is
hidden, including events tagged `["shared","true"]`; the shared tag is
never evaluated. Every HTTP response applies it the way it treats a
missing event: event preview is `404 event_not_found`, report detail
returns the report with `message: null` and `targetAuthorPubkey: null`,
and the report list returns a null target author.
Moderators can still act on these events, they just cannot read them.
Report detail is one query with the hidden-kind list as a parameter,
exposed two ways: `admin_get_report` hides private kinds and backs every
HTTP response (GET detail, the resolve response, reopen, cancel and the
reporter-notice worker), while `admin_get_report_for_enforcement` reads
the target unfiltered and is used only by the resolve handler's
enforcement load (the staff-target check and ban, timeout and kick
target derivation) and by the recovery worker for interrupted actions.
Its result is never serialized, so a ban from a report on a private
persona bans the real author while GET still shows nothing.
## Method binding
Axum serves `HEAD` through `GET` handlers, but the read handlers
authorized with a literal `"GET"`, so a correctly signed `HEAD` was
rejected. Admin reads now authorize through `authorize_read` and writes
through `authorize_write`, both of which use the request's real method
and full target, so no handler hardcodes a method string. A database
error behind any admin route is now logged before it becomes a 500.
## Directory query plan
`idx_communities_host` is on `lower(host)` in the default collation, so
the prefix `LIKE` is a filter, not an index condition. The plan walks
the index in order and stops once it has `limit` matches. A common
prefix is cheap (on local Postgres 16 with 21,359 communities, a first
page skipped 2,728 index rows in 3.9 ms), but a prefix with few or no
matches walks every index entry, so its cost grows with the number of
communities. A `COLLATE "C"` index would turn the prefix into an index
range; it is left out because the admin directory is low-traffic and the
cost is small at current scale.
The cursor condition is `lower(host) > COALESCE($2, '')`, with no `OR`,
so it remains an index condition under a generic plan:
```text
-> Bitmap Index Scan on idx_communities_host
Index Cond: (lower((host)::text) > COALESCE($2, ''::text))
```
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
|
||
|
Luke Tornquist
|
6d07a29137 |
perf(relay): bound fleet usage metrics collection (#7176)
## Why
The usage poller pays memory and query costs proportional to community
count even when per-community emission is disabled. Every pod loads the
full host map, and the leader materializes all per-community aggregates
before suppressing their labels. That makes telemetry collection itself
a scaling constraint.
## What changed
- Fleet-only usage metrics are now the default. `all` remains available
as an explicit rollback mode.
- In fleet-only mode, the poller skips the community host map and all
community-sized query results.
- Exact community counting is replaced with the PostgreSQL planner's
table-size estimate.
- Stock totals are collected as one fixed row, scanning each stock table
once per hour.
- 1d/7d/30d active-user totals come from one bounded 30-day scan per
day.
- Fleet database snapshots run only on a proved read replica, with a
telemetry-specific freshness budget and server-side statement timeouts.
If no such replica is available, the collection is skipped; it never
falls back to the writer.
- Successful fixed-row snapshots are cached and re-emitted on each poll
tick, so gauges stay scrape-visible without raising query frequency.
- `buzz_usage_snapshot_available{family}` reports whether the latest due
refresh of that family succeeded. After a skipped or failed refresh, the
last values stay visible (with `buzz_usage_snapshot_age_seconds`) but
the family reads unavailable until a 60-second retry succeeds.
- The `all` rollback collector publishes the same availability gauges,
so dashboards gated on availability keep working during a rollback.
- Each due refresh outcome is applied by `apply_stock_refresh` /
`apply_activity_refresh`, the single production mapping from `Ok(Some)`
/ `Ok(None)` / `Err` to cache, availability, skip counter, and retry;
the regression test drives that mapping directly.
- `buzz_usage_snapshot_age_seconds` is emitted in the `all` rollback
too, so age-based alerts keep firing through a rollback.
- Every fleet telemetry attempt that completes lands in
`buzz_db_route_decision`; a query error on a proved reader records
`skipped/replica_error`. An attempt abandoned at the relay deadline
records no route decision and appears only as
`buzz_usage_query_skipped_total{reason="timeout"}`. The shared
`proved_reader` warnings no longer claim "routing to writer", because
the telemetry caller never does (serving callers still record their
writer decision in the route counter).
- The PostgreSQL production-method matrix now exercises
`usage_fleet_stock_snapshot`, so the new `reader/maintenance` pool pair
is proven to be emitted by a real Db method.
- Each fleet collection, reader proof included, runs under a relay-side
`tokio::time::timeout` (10s stock, 20s activity, both above their
server-side `statement_timeout`). A reader that stops answering expires
into the same failure mapping with
`buzz_usage_query_skipped_total{reason="timeout"}`, so it cannot stall
the leader's poller, hold the advisory lock past a liveness check, or
leave a frozen snapshot reporting available.
- Fleet telemetry reader checkouts (`ReaderOperation::Maintenance`) opt
into SQLx `close_on_drop` before their first post-acquire await, so an
abandoned collection's connection is closed within SQLx's 5s
`CLOSE_ON_DROP_TIMEOUT` instead of going through the unbounded
return-to-pool ping while it holds a slot in the reader pool that
serving reads share. Serving reads keep pooled reuse; the cost is one
reader reconnect per fleet refresh.
- On leader demotion, cached snapshots and availability are cleared, and
a fresh collection is forced after the lock is reacquired.
- Fleet-only mode passes no host map to the worker-snapshot storage
emitter. It therefore omits community attribution instead of reporting
every byte as `buzz_storage_unmapped_community_bytes`.
- Lifetime stored-message and active-channel totals are no longer
queried in fleet-only mode. Event throughput remains available through
the existing stored-event counter.
- No test-selector changes: main's `config::tests::` selector in `just
test-unit` and the `run-tests.sh unit` fallback already runs the new
replica-budget config test, and the new fleet schedule, availability,
and storage tests run through the existing bin `tests::` and
`storage_sweep::tests::` selectors.
- Added seeded PostgreSQL contract tests and operator docs covering
rollout, rollback, availability, the no-replica behavior, and leader
election.
No migrations, tables, indexes, or new persistent storage are added.
### What got simpler
The relay-side storage half of the original branch (totals-only S3
listing and `buzz_storage_community_breakdown_available`) was dropped.
Main moved S3 listing out of the relay in #7543 and #7845, so that code
was dead on arrival. The rework adds no storage code path; it only gates
attribution on the presence of a host map.
## Risk
Medium. The default changes which telemetry is emitted:
- In fleet-only mode, `buzz_communities_estimated` replaces the exact
`buzz_communities_total`.
- `buzz_total_messages` and `buzz_total_active_channels` are not
refreshed in fleet-only mode (also called out in the rollout steps of
`docs/usage-metrics.md`).
- Deployments without a verified reader (including local compose and
single-node self-hosts) skip database-derived fleet snapshots instead of
querying the writer, and report both families unavailable.
- Multi-pod dashboards must filter database and storage series to
`buzz_usage_poller_is_leader == 1`.
Setting `BUZZ_USAGE_METRICS_PER_COMMUNITY=all` restores the legacy
collector for temporary rollback.
## Verification
At head `1879ee3aedf4227942120a3b521c8d35fc7839fe` (tree `78477e74`;
merges main `da651ad6`; the PR's diff against main is byte-identical to
`1613823f`'s, minus the Justfile and `run-tests.sh` selector lines that
main's `config::tests::` selector now subsumes):
- On Blox: `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay --all-targets -- -D warnings` pass. Relay bin `tests::`
34/34, `config::tests::` 55/55, `storage_sweep::tests::` 10/10 and
`store::usage::tests` pass, as do the four `fleet_` and lock PostgreSQL
tests that use scratch databases, including
`abandoned_fleet_collection_releases_its_reader_slot`.
- Against a freshly migrated PostgreSQL database:
`store::usage::postgres_tests::` 14/14,
`storage_sweep::postgres_tests::` and `buzz-db --test
observability_source` 14/14 pass. The new reader code passes main's
crate-wide typed-acquisition and serving-table policy checks.
- E2E at `1613823f` is reused for this head. It ran in an isolated relay
against a real PostgreSQL primary and streaming standby. In 2/2 runs a
stock read went dark mid-result; the deadline abandoned it at about 10s,
the reader pool's active and size counts dropped to zero about 1s later,
and a fresh activity read and the scheduled stock retry succeeded on new
backends while the dark session stayed open. The `all`-mode and
healthy-replica controls also passed. Main's merged changes are on the
writer path and in startup step timing, and touch none of the reader
checkout, the `close_on_drop` seam, the fleet poller or the deadlines.
At head `1613823fa8ef1a423bc1a499c57bc40d692fcd4f` (tree `12bc6d1b`;
test/docs-only on top of `57749418`), on a Blox workstation:
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay --all-targets -- -D warnings` pass. Relay bin `tests::` 34/34
pass.
- New PostgreSQL test
`abandoned_fleet_collection_releases_its_reader_slot`: a one-slot reader
pool behind a loopback proxy that goes dark mid-session; the collection
is cut at a deadline, then a later acquire must succeed in under 8s on a
fresh connection. The dark call runs on a fresh `Db` with a cold Aurora
capability cache, so the first post-acquire await is the capability
probe. It passes in 3.4s. With `close_on_drop` removed, or moved below
the probe await, it fails with `PoolTimedOut` after the 15s acquire
timeout. The other three `fleet_` PostgreSQL tests that use scratch
databases pass.
- `each_fleet_family_is_abandoned_at_its_own_deadline` now has an outer
timeout: replacing the stock deadline with a bare `.await` fails it
(`Elapsed`) instead of hanging.
At head `fd194ec79d6c6796d6764564de225ad56485cfc7` (tree `79e94727`,
main merged at `932228936`), on a Blox workstation:
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-relay -p
buzz-db --all-targets -- -D warnings` pass.
- Relay bin `tests::` nextest selection: 27/27 pass (plus
`store::usage::tests`), including
`fleet_refresh_outcomes_drive_availability_through_production_mapping`
and the new `stalled_fleet_collection_times_out_into_failed_refresh`
(paused time: two never-completing collections are abandoned at exactly
10s + 20s; both families read unavailable with cached values kept,
`reason="timeout"` skips, retry scheduled; the next tick runs and
recovers; no route decision is recorded for an abandoned attempt) and
`each_fleet_family_is_abandoned_at_its_own_deadline`. Compile-time
asserts keep each deadline above its `statement_timeout`.
- Falsifiability: replacing the stock deadline with a bare `.await`
makes the new test fail (the tick stalls); swapping the two deadlines
fails the per-family test; deleting `schedule.mark_stock_failure(now)`
from the `Ok(None)` arm fails the mapping test. Trees restored and
re-verified.
- At `970414db`: CI PostgreSQL Tests 809/809 including the
production-method matrix with `reader/maintenance`; Unit Tests ran
`finish_usage_read_records_every_attempt_in_route_decisions`.
- `just ci` / `just test` at `5b18914a` on Blox: not green for
environmental reasons (`mobile-check` could not reach pub.dev;
`_ensure-services` failed so service-backed suites did not run;
`buzz-acp git_bootstrap` fails identically on `origin/main`). PostgreSQL
and integration coverage comes from GitHub CI at this head.
E2E at the previous head `0a01e5f22` (isolated relay, real Postgres
primary plus streaming replica, worker storage snapshot, desktop
frontend in relay mode): with no reader, both families report
skipped/unavailable and route `skipped/disabled` (no writer queries);
fleet-only storage emits totals without `buzz_community_storage_*` or
unmapped bytes; `all` mode reports both availability gauges `1` with
per-community storage series; with a fresh replica, both fleet paths
route `replica/fresh` on `reader/maintenance` acquisitions; desktop
send/receive works. The cached success → due failure → recovery
transition was not reached live (hourly/daily cadence); it is covered by
the seam test above.
The original branch also recorded live verification at `aa3e7023`,
before this rework: reduced-idle cache re-emission, two-relay leader
takeover with demoted availability clearing, the no-reader skip with
60-second retry, `all` rollback restoring per-community series, and a
desktop smoke test. Those runs predate the availability and storage
changes and are not claimed for this head.
## Linear tracking
Related work: BUZZ-167.
---------
Signed-off-by: tornquist <tornquist@squareup.com>
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
||
|
Ian Oberst
|
14a752fb24 |
fix(db): audit partition catalog before creation (#6515)
## Why
The partition manager checks partition existence by name only and treats
Postgres 42P17 "would overlap partition" errors as success. On databases
where a catch-all partition (`events_p_future` /
`delivery_log_p_future`) covers future months, the monthly creation path
silently does nothing while real coverage erodes toward a write outage.
There is currently no visibility into partition catalog health and no
signal before writes start failing.
## What
- **Read-only catalog audit** in `buzz-db` (`partition.rs`): classifies
the live partition catalog for `events` and `delivery_log` via
`pg_inherits` and bound parsing — coverage bounds, catch-all detection,
anomalous children, and trigger parity checked recursively down to
routable leaf partitions.
- **Creation gated on the audit verdict**: the 42P17-to-success swallow
is removed; monthly CREATE runs only for months the audit proves
uncovered, a canonical-name mismatch is an error, and per-month outcome
capture replaces first-error abort. New
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED` kill switch (default `true`).
- **Prometheus metrics** per table: `buzz_partition_serving_safe`,
`buzz_partition_uncovered_months` (audit window is now through +3
months), `buzz_partition_catch_all_covered_months`,
`buzz_partition_anomalous_children`,
`buzz_partition_trigger_parity_missing`/`_extra`,
`buzz_partition_audit_duration_seconds`,
`buzz_partition_audit_runs_total{outcome}`,
`buzz_partition_audit_failures_total`,
`buzz_partition_create_attempts_total`, and
`buzz_partition_audit_last_success_timestamp_seconds` for staleness
alerting.
- **Cached diagnostics**: `/_status.partition_catalog` reports partition
serving-safety, audit age, and freshness. `/_readiness` stays
process-local, consistent with #7341; missing, unsafe, or stale audits
do not remove pods from service.
- **Periodic audit task** in the relay, plus config surface for the
audit interval and the monthly-create kill switch. Catch-alls are
detected from catalog bounds rather than configured names.
- Doc correction: the mesh heartbeat comment no longer claims readiness
gating (it never was gated; behavior unchanged).
## Behavior changes & risk
- No DETACH/DROP/ATTACH and no schema migrations. The only DDL is the
pre-existing monthly CREATE path, now strictly narrower
(audit-proven-uncovered months only) and behind a kill switch.
- Partition audit health does not gate readiness. Until the first
completed audit, read-only retries back off from 5 seconds to a
60-second cap. Any completed audit, including unsafe coverage, restores
the normal interval.
- Partition-overlap errors now surface as errors instead of being
swallowed as success.
- Rollback: revert the commit, or set
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED=false` to disable the creation
path while keeping the audit and metrics.
## Validation
- New Postgres-backed regression tests for catalog classification and
trigger parity (disabled child trigger, `ENABLE ALWAYS` parent, disabled
nested leaf clone in a multi-level layout, healthy-catalog zero-parity
checks). Each new regression was verified to fail against the previous
logic before the fix.
- Local affected-target validation passed against a disposable
current-schema database, with the known mesh forwarded-echo and tracing
callsite failures excluded. This is not a claim that the full suite
passes.
Follow-up work (separate PRs): bounded catch-all advance (empty-only
golden path; non-empty reattach only behind an explicit operator flag,
default off), fresh-install convergence, and proactive coverage-runway
alerting.
---
## Update — 2026-08-21
- Fixed nested-leaf trigger parity auditing so child-only row triggers
are checked on every routable descendant leaf and reported as
`{leaf}:{trigger}` under the owning root child.
- Added a Postgres regression test for a child-only trigger on a nested
leaf; mutation-reverting to immediate-child-only extra-trigger scanning
makes it fail.
Generated with Codex
Signed-off-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Signed-off-by: Ian Oberst <ioberst@squareup.com>
Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Jarrod Sibbison
|
c4c86006f2 |
feat(relay): deliver pubkey mentions to relay companions (#7793)
We are introducing a companion service to the relay that needs pubkey event mentions for pubkeys it manages. This PR introduces an outbox for pubkey mentions that will be sent to these "operator listener" services. ## Summary Deliver mentions of registered pubkeys to deployment-global operator listeners. - Configure listeners with `BUZZ_OPERATOR_LISTENERS`; listeners manage target pubkeys through NIP-98-authenticated `POST`/`DELETE /operator/listener/pubkeys` requests. - During supported event ingestion, match `p` tags to registrations and insert one outbox row per listener in the event transaction. - A background worker claims up to 10 rows, sends NIP-98-signed notifications, and retries transient failures with exponential backoff for up to 9 attempts (idle polling backs off from 250 ms to 2 s). - Outbox rows are ineligible for delivery after 15 minutes and are reaped every 5 minutes. Registered pubkeys live for 30 days; re-registering renews them, and a daily reaper removes expired registrations. - Keep persistence app-managed: no database triggers, functions, foreign keys, or separate match queue. ## Changed behavior Listener outbox rows are enqueued inside the shared event-insert and thread-metadata transactions; enqueue failure rolls back the event. `event_mentions` behavior is unchanged: ordinary event/reaction mention indexing remains post-commit best effort with warning-only failures. ## Testing - `cargo fmt --all` - `cargo test -p buzz-db --lib` - `cargo test -p buzz-relay --lib config::tests::operator_listener_routes` - `cargo check -p buzz-db -p buzz-relay` |
||
|
Tom Brow
|
781d39510c |
feat(push): support configurable HTTP(S) delivery URLs (#7877)
The relay can use a configurable HTTP(S) gateway delivery URL and signs the same URL it sends. The gateway verifies NIP-98 against the received request URL, including authority, port, path and query. App Attest, scoped grants, replay checks and delivery quotas are unchanged. Forwarding proxies must preserve Host and supply the original request scheme through X-Forwarded-Proto. The NIP relay-delivery wording now defines the configured HTTP(S) URL as the signed URL and labels the public URL as an example. <!-- buzz-review-completed --> --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Tom Brow
|
76f7e5ef0c |
feat(push): make gateway origin configurable (#7261)
Make the push gateway origin configurable across the relay, gateway, mobile client, and deployment chart. This is a fresh deployment. The gateway has only run on personal development infrastructure, and push has no existing enabled users. Legacy migration, configured-to-unconfigured mobile transitions, and rolling upgrades from pre-launch gateway binaries are not supported. Populated pre-launch databases are rejected with an informative error before migrations run; their data is untouched. Mobile builds may omit the gateway URL, leaving push unavailable. Once push is enabled, subsequent builds must retain that configuration. ### Validation - All applicable CI checks pass at `5f546cfaf403f0386f146d213adfb4392f4ec29c`, including Rust, mobile/Swift, desktop/integration, security, image builds, and Helm checks. An unrelated agent cancellation test passed on retry. - PostgreSQL CI passed 384 tests, including the production-entry-point regression for refusing populated pre-launch databases before migrations, fresh initialization, and subsequent initialized deployments. - Focused local validation passed all 10 gateway PostgreSQL tests under the CI discovery filter and all 3 unconfigured-mobile tests. Repository-wide PostgreSQL discovery validation and the complete pre-push hooks passed. - Physical-device APNs/App Attest behavior and deployment to a live cluster were not verified in this work. --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Codex <noreply@openai.com> |
||
|
Salman Mohammed
|
674c173eb7 |
feat(buzz-acp): give each channel thread its own agent session (#6732)
## What this does In a channel, people often run several unrelated conversations at once (separate threads). Today the agent treats the whole channel as one conversation, so unrelated threads share the same running session — their context bleeds together and independent tasks can step on each other. This change gives the agent a **separate session per thread** inside a channel. Direct messages stay as one conversation (unchanged). The channel is still the boundary for who is allowed in and what is visible — only the agent's working context is now split by thread. ## How it is turned on Off by default. Operators opt in with one setting: - `BUZZ_ACP_SESSION_POLICY=channel` — default, current behavior - `BUZZ_ACP_SESSION_POLICY=thread` — new per-thread behavior Being behind a flag means we can enable it for a few agents, watch how it behaves, and roll back instantly without a code change. ## Key design decisions - **Decide the thread once, up front.** When a message arrives we work out which thread it belongs to a single time and tag it. Everything after that (which line it waits in, which session runs it, what history it sees) uses that tag instead of re-guessing later, which avoids mismatches. - **Default stays identical to today.** Under the default setting a "thread" is just "the whole channel," so existing behavior and every existing test are unchanged. The new, riskier behavior is strictly opt-in. - **Give the agent only its thread's history.** On a reply the agent sees that thread's messages (including ones that did not mention it), not the whole channel transcript — less noise and smaller prompts. - **Don't let one channel use more memory than before.** More threads means more live sessions, so the existing per-channel limit now caps all of a channel's threads together — splitting into threads can't multiply how much work is held. ## Bugs found and fixed while iterating (from review) - **Same thread, two sessions.** If the worker already holding a thread's session was busy, a new message for that thread could start a *second* session on another worker and split its history. Now it waits for the right worker instead of forking. - **Interrupting the wrong thread.** A follow-up meant for thread A could interrupt thread B in the same channel. Interrupts now target the exact thread. - **Stuck thread after a crash.** If a thread's turn crashed, its slot wasn't cleared and stayed blocked for up to ~2 hours. It now clears right away and retries. - **Lost the original request.** When a thread was interrupted and then had to wait for a busy worker, only the follow-up was kept and the original request was dropped. The full request is now preserved on retry. - **Same thread seen as two.** Two spellings of the same thread id (upper/lower case) could be treated as different threads. Normalized so they count as one. ## Not in this PR - The desktop Settings toggle and rollout wiring for managed agents — https://github.com/block/buzz/pull/6909 - One pre-existing retry edge case (present today without this flag, unrelated to this change) — tracked separately so this PR stays focused. ## Testing The full `buzz-acp` test suite passes (830+ unit and integration tests), plus new focused tests for thread routing, session reuse, interrupt targeting, crash recovery, and request preservation. Behavior with the flag off is unchanged. --------- Signed-off-by: Salman Mohammed <smohammed@squareup.com> Signed-off-by: Leo <5faf251baee50ee6bcde338aef6acdd70bb3e60115664c2cd490d94a55dfc488@buzz.block.builderlab.xyz> Co-authored-by: Leo <5faf251baee50ee6bcde338aef6acdd70bb3e60115664c2cd490d94a55dfc488@buzz.block.builderlab.xyz> |
||
|
Arjun Mahanti
|
5673c3307e |
feat(desktop): add protected-build Bestie experiment (#6902)
## Summary Introduces a protected-build boundary for the default-off Bestie experiment without adding any Bestie product surface. - Official OSS builds select an empty protected-feature module and emit no Bestie/Chief metadata or implementation content. - Protected internal builds select a separate module graph containing the Bestie experiment definition. - Within an internal build, Bestie remains disabled until the user opts in under Settings → Experiments. - The production build runs an artifact matrix and fails if OSS output contains protected content or internal output lacks the Bestie manifest. ## Build contract | Build variant | User opt-in | Result | | --- | --- | --- | | Official OSS | Any/forged | Bestie absent from the compiled artifact | | Protected internal | Off | Bestie available but disabled | | Protected internal | On | Bestie enabled | The companion protected-release change is squareup/buzz-releases#91. It sets `VITE_BUZZ_BESTIE=1`, requires that exact value, forwards it into the signed macOS build, and asserts the contract in release validation. ## Why this is separate This gives later Bestie PRs one build-selected import seam. Protected implementations must be reachable only from the internal module so they never enter the official OSS module graph. ## Non-goals - No Bestie persona or provisioning - No sidebar, app-chrome, or message-toolbar UI - No entitlement or secrecy claim: the source is public; this boundary controls official Block artifacts ## Verification - Exact commit `523cf49ced03cba9be43836a54d6aa5d6923cc82` - Full `just ci`: 5,673 Desktop tests, 2,773 Tauri tests, 1,860 mobile tests, Rust/Tauri/web/mobile static checks and builds - OSS production artifact: scanner confirms no `Bestie`, `Chief of Staff`, or `builtin:bestie` content - Internal production artifact: scanner confirms the protected Bestie manifest is emitted - Both build orders verified; `dist` retains the requested variant for Vite/Tauri packaging --------- Signed-off-by: Arjun Mahanti <arjun@squareup.com> Signed-off-by: Fizz <fizz@buzz.local> Signed-off-by: Fizz <dae5f6af70b8695a8b83c8deae555f63be41630ec2b8cd493e41a439c9527dd8@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Fizz <fizz@buzz.local> Co-authored-by: Fizz <dae5f6af70b8695a8b83c8deae555f63be41630ec2b8cd493e41a439c9527dd8@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
3ed623bb21 |
feat(db): configurable writer session timeouts (lock, idle-txn, statement) (#6229)
## Why A wedged relay boot pod holding a relation lock can park every other writer in the fleet behind it: DB load pins at pool capacity in `Lock:relation` waits while CPU stays flat, and nothing server-side releases the lock until the holder dies. We hit exactly this in production — ~1,400 sessions queued behind one crash-looping pod's boot transaction for ~20 minutes until kubelet killed the container. ## What Applies session-level Postgres timeouts to every **writer** connection inside the existing single `after_connect` hook in `buzz-db`, all env-tunable through the same `Config::from_env → DbConfig` path as the existing pool-size knobs: | Env var | GUC | Default | Effect | |---|---|---|---| | `BUZZ_DB_LOCK_TIMEOUT_MS` | `lock_timeout` | 5000 | statements waiting on any lock fail fast instead of parking behind a wedged holder | | `BUZZ_DB_IDLE_TXN_TIMEOUT_MS` | `idle_in_transaction_session_timeout` | 60000 | reaps wedged clients idling inside an open transaction while holding locks | | `BUZZ_DB_STATEMENT_TIMEOUT_MS` | `statement_timeout` | 0 (off) | opt-in runaway-statement cap; off by default because startup migrations/backfills legitimately run long statements | `0` disables a timeout (Postgres semantics) and deliberately passes through the env parsing — unlike the pool-size knobs where `0` falls back to the default. The reader pool is untouched: replica sessions never take contended locks and already fail acquire in 150 ms. Deployers tune these via plain env vars (`.env`, or `relay.extraEnv` in the Helm chart) — no code changes needed. ## Behavior change to note With the 5 s default `lock_timeout`, a boot-time migration or backfill that waits >5 s on a lock now errors (surfacing in logs / crash-looping the pod) instead of stalling silently. That is the intended visible-failure-over-fleet-stall tradeoff; deployers with slow contended migrations can set `BUZZ_DB_LOCK_TIMEOUT_MS=0`. ## Testing - `cargo test -p buzz-db -p buzz-relay` — buzz-db green; buzz-relay has 9 failures that also fail on clean `main` in this environment (api::admin/api::media/mesh_demo — unrelated, pre-existing). - New config test covers override / `0`-passthrough / invalid-fallback for all three env vars. - Extended the existing `writer_pool_safety_hook_is_single_and_composed` source-shape test so the timeouts can't drift out of the single `after_connect` hook (SQLx replaces hooks — a second hook would silently disarm the floor guard). - `cargo fmt --check` and `cargo clippy --all-targets` clean for the touched crates. Closest existing PR/issue: none found. --- **Update Aug 28, 17:06 EDT:** Rebased onto `main` at `a3730784fc` and addressed the latest correctness review. - Ported the timeout policy onto the refactored `buzz-db::runtime` pool constructor and kept the shared env overlay for relay, admin, deletion, and audit writers. - Migration/schema-destruction connections now disable `lock_timeout` and `statement_timeout` for their intentional long wait/DDL path. This supersedes the earlier “Behavior change to note”: contended boot migrations wait for the current migration owner rather than crash-looping after five seconds. - The audit worker now preserves and retries the same entry on PostgreSQL `55P03` lock timeouts, using exponential backoff capped at one second. Other database errors retain the existing terminal error behavior, and retries emit `buzz_audit_log_lock_retries_total`. - Added CI-backed PostgreSQL regressions for writer GUC installation/migration exemption, audit-pool lock timeouts, and worker recovery. The worker regression holds the real audit advisory lock past `lock_timeout`, observes a retry, releases the lock, and proves the original entry is appended exactly once. Current verification supersedes the earlier testing notes: workspace Rust clippy passed with warnings denied; all nine infrastructure-free backend unit-test lanes passed; all three focused PostgreSQL regressions passed against PostgreSQL 17; formatting, diff checks, file-size guards, and desktop frontend checks passed. The Linux Blox workstation could not run the unrelated Tauri native lane because `glib-2.0` is absent, so that platform check is left to PR CI. --- **Update Aug 31, 11:09 EDT:** Rebased onto current `main` at `c3132c3ee9` and reran the requested audit-lock contention scenario on Blox at head `896c3fe9ed`. - `git range-diff` reports both PR commits unchanged by the rebase; the branch remains two commits and the worktree is clean. - `cargo fmt --all -- --check` and clippy with warnings denied passed for `buzz-db`, `buzz-relay`, `buzz-admin`, and `buzz-deletion`. - All three focused PostgreSQL 17 regressions passed: writer session timeout/migration exemption, audit writer timeout bounds, and audit worker recovery of the original entry exactly once. - Live protocol verification used a head-built relay and CLI, native PostgreSQL 17/Redis, `BUZZ_DB_LOCK_TIMEOUT_MS=300`, and an eight-second hold on the community audit advisory lock. The real message was accepted and persisted once while the lock was held; its audit-row count remained zero during contention while retries accumulated. After release, exactly one `event_created` audit row appeared and remained exactly one after an additional two-second duplicate check. The run recorded nine lock-timeout retries, zero audit failures, and event ID `7f8c4ffae28e78555fcf2d56396d6e6c01b3712e5411288dc79e9a54af9d9444`. Generated with Codex --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> |
||
|
Tom Brow
|
c432a111ca |
feat(mobile): push notifications MVP (#6269)
This PR implements MVP, iOS-only,
[NIP-PL](
|
||
|
Will Pfleger
|
86b9142a09 |
refactor(relay): NIP-98 admin auth with Operator/Moderator roles and NIP-11 discovery (#3777)
Adds authenticated, role-based moderation to the relay admin API. On
`main` the admin API is read-only and gated only by `Host`/`Origin`
matching; this branch adds NIP-98 authentication, a two-tier
Operator/Moderator principal model, mutation and staffing endpoints, and
NIP-11 auto-discovery so clients never type the admin URL by hand.
## Authentication (`BUZZ_ADMIN_AUTH`)
`BUZZ_ADMIN_AUTH` accepts `nip98` or `disabled`. Leaving it unset
defaults to `nip98` (fail-secure). Configuration fails closed: any other
value aborts startup, while a lingering `BUZZ_ADMIN_TOKEN` is ignored
with a startup warning — token (bearer) authentication is not supported.
`Host`/`Origin` matching is retained in every mode as defense-in-depth.
- **`nip98`** (default) — per-request signed NIP-98 (kind 27235) events,
resolved to an Operator or Moderator principal with per-person
attribution and individual revocability. Read-write per resolved
principal.
- **`disabled`** — no credential; relies entirely on network-layer
controls (reverse proxy, VPN, firewall) and logs a `WARN` on every boot.
Always read-only: `authorize()` resolves no principal, so mutation and
staffing routes always `403`.
## Roles
Buzz has two independent authority axes after this change.
**Relay-level** roles (new here) are deployment-global: they act across
every community on the relay, through the admin API. **Community-level**
roles (pre-existing, unchanged by this PR) are tenant-scoped: they act
inside one community, through signed Nostr moderation commands.
### Relay level (new)
| Role | Description |
|---|---|
| **Operator** | Full control of the deployment's moderation surface:
read all reports, feedback, and attachments across every community;
resolve reports with enforcement (`delete`/`kick`/`ban`/`timeout`) or
decisions (`dismiss`/`escalate`); reopen and cancel; update feedback
status; and manage the Operator/Moderator roster via the staffing
endpoints. |
| **Moderator** | Day-to-day triage: everything an Operator can do
except staffing — cannot view or change the roster. |
How a pubkey acquires a relay role (resolution order; config always
outranks DB):
1. Listed in `RELAY_OPERATOR_PUBKEYS` → **Operator** (source `config`)
2. Equals `RELAY_OWNER_PUBKEY` while `RELAY_OPERATOR_PUBKEYS` is empty →
**Operator** (source `owner_fallback`, a break-glass grant for
self-hosters that deactivates once any operator is configured)
3. Row in the `relay_operators` table → **Operator** or **Moderator**
(source `db`, managed via the staffing endpoints)
4. No match → `403`
### Community level (pre-existing, unchanged)
| Role | Description |
|---|---|
| **Owner** (community) | Full authority within their community: every
moderation action (delete, kick, ban/unban, timeout/untimeout, resolve
reports, view queue) plus member, role, and invite management. No guard
rails. |
| **Admin** (community) | Same community-wide moderation capabilities as
owner, except an admin cannot ban or time out the owner or a fellow
admin — only the owner may action an admin. Manages members and invites;
only the owner grants the admin role. |
| **Member** (community) | Standard participant; no moderation
capability. |
| **Owner / Admin** (channel) | Channel-local authority only: delete
messages and kick users within their own channel. |
| **Member / Guest / Bot** (channel) | No moderation authority. |
There is no community-level Moderator tier in v1; relay-level Moderator
is the only role by that name.
## Escalation scoping
The operator report queue is an escalation backstop, not the community's
day-to-day triage surface (per `VISION_MODERATION`, the severe class is
the platform's to review rather than the community's). Two rules enforce
that:
- **Escalated-by-default listing.** `GET /reports` with no `status`
parameter returns only `escalated` reports. An explicit
`status=<open|resolved|dismissed|escalated>` filter is always honored as
given, and full visibility across every status stays available for
platform-safety and legal review via `scope=all` (which lists reports
regardless of status). `scope` accepts only `all` and is ignored when an
explicit `status` is present.
- **Auto-escalated `illegal` reports.** Member reports whose category is
`illegal` are ingested with `status=escalated` rather than `open`, so
the severe class reaches the operator backstop without waiting for a
community admin to forward it. Every other category still lands `open`.
Auto-escalation only sets the queue status — it records no moderator
decision and stamps no resolver, so an auto-escalated report is
indistinguishable downstream from an admin-escalated one: the reopen
route returns it to `open` on the same terms, keyed only on status,
never on how the report became escalated.
## Principal resolution and NIP-98 admission
`resolve_admin_principal()` returns `AdminPrincipal { pubkey, role,
source }` per the resolution order above; `None` never falls through as
a role. Admission is ordered so the replay guard is a privilege, not a
public surface: signature/URL/method/payload-hash verification first,
roster check second, and only then is the deployment-scoped replay id
atomically consumed — a validly-signing but unrostered key never
allocates a replay slot. Redis failure fails closed.
## Report resolution, recovery, and enforcement provenance
`POST /reports/{id}/resolve` is a crash-safe enforcement state machine:
decision-only outcomes (`dismiss`/`escalate`) are a single
CAS-plus-audit transaction; enforcement
(`delete`/`kick`/`ban`/`timeout`) claims the report
(`open`→`processing`), runs the durable mutation, then finalizes — a
re-drive resumes at the step marker and converges to exactly-one
enforcement, fenced by a lease and an outbox claim token.
Person-directed enforcement on an `event`-kind report derives its target
from the stored event's author (server-owned truth, never the reporter's
`p` tag) via a single `derive_enforcement_target` shared by the HTTP
driver and the recovery worker. If the reported event was purged before
its author could be read, person-directed actions are rejected pre-claim
and the report stays `open`; `delete` needs only the event id and is
exempt.
`GET /reports/{id}` and the resolve response carry an `activeAction`
field surfacing the enforcement that actually executed — a report
dismissed after a reopen still reports the ban that ran. `POST
/reports/{id}/reopen` returns a terminal report to `open` (idempotent on
`requestId`). `POST /reports/{id}/cancel` is the sole recovery path for
a pre-mutation `failed` action, attributed via
`relay_admin_actions.cancelled_by`.
## Feedback
`GET /feedback` and `/feedback/{id}` survive a tenant purge: provenance
columns are severed to `NULL` rather than cascade-deleted, and the
attachment path fails closed to `404` on a severed row. `PATCH
/feedback/{id}` updates lifecycle `status`
(`new`/`reviewed`/`archived`).
## Staffing and probe
`GET/PUT/DELETE /operators/{pubkey}` are Operator-only; mutating a
config-backed pubkey returns `409 Conflict`. `GET /operators` returns
the union of config and DB principals with per-entry `source`. `GET
/probe` reports auth mode, role, source, `canAct`, and `canStaff` for
the desktop console.
## NIP-11 auto-discovery
The NIP-11 relay-information document gains an optional `admin_api`
field carrying the canonical admin origin (`scheme://host[:port]`, no
path), present iff `BUZZ_ADMIN_HOST` is set and omitted otherwise. The
scheme follows the same loopback rule as NIP-98 `u`-tag verification via
a shared `scheme_for_host` helper, so the advertised origin and the
origin the relay verifies against can never diverge.
## Operator API origin decoupling
`RELAY_OPERATOR_API_ORIGIN` is no longer required at boot when
`RELAY_OPERATOR_PUBKEYS` is set — it is used only by the
community-provisioning endpoints, which fail closed at request time
(with a boot-time `WARN`) until it is set. The admin console needs no
origin.
## Admin-web adaptation
The standalone `admin-web` dashboard signs each request as a NIP-98
event via a NIP-07 browser extension, discovers the auth mode with a
single unauthenticated probe (`200` → `disabled`, anything else →
`nip98`, fail-secure), and carries no token entry surface. Playwright
coverage exercises the NIP-98 and CSP paths.
## Security hardening
Three findings from security review are folded in:
- **Append-only roster audit.** `PUT`/`DELETE /operators/{pubkey}`
mutate the deployment-wide root of trust, but the upsert overwrites
`role`/`added_by` in place and the delete removes the only row — so a
grant→revoke sequence left no trace of who was ever granted or by whom.
Each mutation now writes an `relay_operator_audit` row (actor, target,
`grant`/`revoke`, pre-image `prev_role`, `new_role`, timestamp) inside
the same transaction as the mutation. A per-target transaction-scoped
advisory lock serializes concurrent mutations of the same pubkey before
the pre-image read, so the recorded `prev_role` is always the true
predecessor even under a concurrent-grant race. Chronology is keyed on a
`BIGINT GENERATED ALWAYS AS IDENTITY` `seq` column, not the wall clock:
the serializing lock guarantees insertion order and `seq` captures it,
so ordered reads (`ORDER BY seq`) follow the true privilege chain even
across a backward NTP step that a `clock_timestamp()` ordering would
invert. `created_at` (`clock_timestamp()`) is retained as informational
occurrence time only. Append-only by construction — no `UPDATE`/`DELETE`
path and no API surface. A no-op delete writes nothing.
- **`expirationSecs` overflow.** The timeout path built `Utc::now() +
Duration::seconds(secs as i64)` from an attacker-controlled `u64`:
`i64::MAX` panicked the handler, and a wrapped-negative magnitude minted
a *past* expiry that still passed validation. `compute_timeout_until`
now rejects zero, rejects magnitudes above a documented
`MAX_TIMEOUT_SECS` (365 days), and uses checked
`try_seconds`/`checked_add_signed` so no input can panic or produce a
past expiry — over-cap, zero, `i64::MAX`, and wrapping-negative inputs
all return a clean `4xx`.
- **Uppercase-hex config-backed bypass.** Config pubkeys are lowercased
at parse, but the `409` immutability check raw-string-compared the path
param while `decode_hex_pubkey` accepted uppercase — so `PUT
/operators/{UPPERCASE}` skipped the guard and wrote a shadow row for the
same 32 bytes. The validated param is now canonicalized (lowercased)
before the `409` check, DB write, `DELETE`, and response body.
## Migrations
- `0035_relay_operators.sql` — `relay_operators` roster table
(deployment-global), `actor_authority` on `moderation_actions`,
`processing` status plus `active_action_id` on `moderation_reports`,
`status` on `product_feedback`.
- `0036_relay_admin_actions.sql` — enforcement-action table with a
`request_id` idempotency key, a `step_marker` for crash recovery, and a
`cancelled_by` attribution column.
- `0037_relay_admin_action_lease.sql` — lease fencing for the action
worker.
- `0038_relay_admin_outbox_claim_token.sql` — fenced claim token on the
outbox worker.
- `0039_relay_operator_audit.sql` — append-only `relay_operator_audit`
trail for roster mutations (see Security hardening).
`docs/admin/README.md` documents the full principal model, NIP-98 event
requirements, capabilities by role, the startup error matrix, and the
discovery field.
## Production blast radius
A relay without `BUZZ_ADMIN_HOST` is completely unaffected — the admin
surface stays disabled and `BUZZ_ADMIN_AUTH` is ignored; a lingering
`BUZZ_ADMIN_TOKEN` logs a startup warning and must be removed. Where
`BUZZ_ADMIN_HOST` **is** set, unset `BUZZ_ADMIN_AUTH` defaults to
`nip98` (per-person signed auth); `BUZZ_ADMIN_AUTH=disabled` reproduces
`main`'s prior `Host`/`Origin`-only gating but is read-only (mutation
routes `403`). The five migrations add tables and columns without
touching existing data.
---
Related: [block/buzz#4768](https://github.com/block/buzz/pull/4768)
(desktop admin console consuming the `admin_api` field),
[squareup/bb-public#339](https://github.com/squareup/bb-public/pull/339)
(Phase 4 rollout config)
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Kalvin C
|
be8d7756d5 |
feat(agent): discover Databricks Unity Catalog models (#6918)
## Summary Add Unity Catalog model-service discovery to `buzz-agent` Databricks v2, preserve full Unity Catalog FQNs through ACP model discovery, route model services through MLflow chat, and add the optional `DATABRICKS_MODEL_FILTER` visibility filter. The resilience follow-up bounds each catalog attempt across headers, bounded body reads, and JSON parsing; retries transient 499/5xx, transport/body failures, malformed JSON, and attempt timeouts; and emits only safe catalog/status diagnostics. ## Validation The live validation and test gates below were run on the pre-rewrite tip `1cf6c66e7c5bf9a00c357bda24df48e3beeb310b`. The rewritten tip `0364cf44f3eb5cfbd5771b0a0dd7868ce0ad2c19` has identical trees for both commits, so the attribution rewrite changes no tested code. - 31 catalog tests passed - 19 Databricks OAuth/routing integration tests passed - `cargo check -p buzz-agent` passed - `cargo clippy -p buzz-agent --all-targets -- -D warnings` passed - `cargo fmt --check` passed - `git diff --check` passed - Desktop Tauri validation passed through the repository-supported setup path, including 2,777 library tests and auxiliary terminal suites The full `buzz-agent` package has one separately reproduced pre-existing failure: `regressions::cancel_kills_inflight_tool_via_mcp_notification`, failing on both baseline `eb8e97cfe` and the follow-up worktree. No Databricks path reaches that test. The live validation proves discovery and ACP exposure, not inference against every listed model. --------- Signed-off-by: Kalvin Chau <kalvin@block.xyz> Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
8471049c43 |
feat(desktop): add KLIPY GIF search to composers (#5554)
## Summary - Supersedes #1913 with a KLIPY-hosted URL implementation. - Adds KLIPY GIF search and trending results to desktop message and forum composers. - Keeps selected GIFs hosted by KLIPY; Buzz stores only the external URL and media metadata (no imeta tag, since relays only accept hash-backed local `/media/` entries). - Aligns the Emoji/GIF picker with Buzz's standard segmented control, theme surfaces, and motion behavior. ## Relay-to-provider boundary - The relay proxies KLIPY search/share so the `BUZZ_KLIPY_API_KEY` never reaches the desktop; the key stays server-side behind a redacted `Debug` impl. - The dedicated GIF `reqwest` client sets `redirect::Policy::none()`. Because the API key rides in the request path, following a provider `3xx` could replay a key-bearing URL to an attacker-chosen host (an SSRF/key-disclosure primitive). With redirects disabled, a `3xx` returns as a non-success status that the handlers map to a generic `502`; the `Location` target is never read or forwarded. - Admission reuses the established NIP-98, membership, replay, and per-pubkey rate-limit gates, with an upstream response-size cap and allowlisting so KLIPY error bodies never cross the relay boundary. ## Accessibility - Under `prefers-reduced-motion: reduce`, the picker grid renders a static provider poster (a normalized `jpg` asset) instead of the animated preview, or a named static placeholder when no poster is available. It reacts to preference changes while mounted. `no-preference` keeps the animated preview. - Selected GIFs carry their title through `ImetaMedia`'s `displayLabel`, so the composer thumbnail, preview dialog, editor, lightbox, and remove control all derive one non-empty accessible name instead of an empty `Attachment ` label. Ordinary hashed uploads keep their existing hash-derived names. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Signed-off-by: Kenny Lopez <klopez4212@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Signed-off-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz> Co-authored-by: Princess Donut <b238ea756dee4d98afa5883fc7f1de61eeabe65bf700e3a5a5a80db5e42e2c2b@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <9e1c23a3fd83f61da34420e4e88ff1b16e45cafcc0cd9019eb07d4ecfa8ca9b0@buzz.block.builderlab.xyz> |
||
|
Jordan Mecom
|
ee6ca5fa28 |
Remove public relay signing key fallback (#6729)
Require `BUZZ_RELAY_PRIVATE_KEY` on every relay startup and remove the shared public fallback key completely. For local development, `just bootstrap` now generates a random key once in the gitignored `.env` file. Re-running bootstrap preserves that key, so relay-authored events retain the same identity across restarts. The local relay recipes reload `.env`, while isolated CI and E2E launchers pass explicit per-run keys. Startup fails immediately when the key is missing or invalid, before connecting to Postgres or Redis. Deployed relays continue to receive their stable key from the existing chart-managed secret. Validated with: - `./scripts/test-ensure-local-relay-key.sh` - `cargo test -p buzz-relay --bin buzz-relay` - `cargo clippy -p buzz-relay --bin buzz-relay -- -D warnings` - `cargo fmt --all -- --check` - `shellcheck scripts/ensure-local-relay-key.sh scripts/test-ensure-local-relay-key.sh` Signed-off-by: Jordan Mecom <jm@squareup.com> |
||
|
Jordan Mecom
|
769ac70b74 |
fix(media): require authenticated reads (#4610)
This change requires a valid signed Blossom authorization request and current relay membership for every media GET and HEAD request. It removes the unauthenticated compatibility path and updates desktop reads to send the required authorization. This blocks anonymous retrieval and access after relay-membership revocation. It does not yet bind a blob to its originating channel, so someone removed from a private channel can still read a known blob while remaining a relay member. That channel-ACL follow-up remains required before closing the full finding. ## Testing - `git diff --check origin/main...codex/security-media-read-auth` - Rebased onto `origin/main` at `5c98932` - Full CI pending Originating Buzz thread: `buzz://message?channel=3928fe05-df61-4b5d-b9c7-d623b9b10ea1&id=3c6c02312f763fbe0d2bfc33a6c1a362f91d0354f3d18b039cf7a0558c1439d1` --------- Signed-off-by: Jordan Mecom <jm@squareup.com> Signed-off-by: Alex Rosenzweig <arosenzweig@squareup.com> Signed-off-by: Eli Foster <efoster@squareup.com> Co-authored-by: Eli Foster <efoster@squareup.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Kalvin C
|
7012d86d52 |
feat: configure S3 URL addressing style (#3400)
## Summary - add one strict `BUZZ_S3_ADDRESSING_STYLE=path|virtual` setting shared by media and Git/CAS storage - preserve path-style defaults for bundled Compose/Helm MinIO while supporting Railway's virtual-hosted bucket contract - fail startup on invalid or non-Unicode values before dependency connection, and validate the Helm value with the same two choices - document operator mappings and why endpoint and bucket remain separate for routing and SigV4 signing ## Best-practice rationale AWS documents both URL forms and favors virtual-hosted addressing for S3, while compatibility endpoints such as the bundled MinIO deployment can require path style. `rust-s3` defaults to virtual/subdomain addressing and provides `with_path_style()` for the explicit compatibility case. Some providers buckets only support as virtual-hosted bucket styles. This PR therefore uses one explicit, provider-neutral switch rather than endpoint heuristics or fallback behavior, while retaining `path` as Buzz's backward-compatible default. Sources: - https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html - https://docs.rs/rust-s3/0.37.0/s3/bucket/struct.Bucket.html - https://docs.railway.com/storage-buckets#url-style - https://github.com/minio/minio/blob/master/docs/config/README.md#domain ## Validation - `cargo fmt --all` - `cargo check --workspace --all-targets` - targeted `buzz-media` and `buzz-relay` parsing/client-construction tests for defaults, strict errors, and both URL styles - Helm unittest: 45/45 passed - Compose config/render validation passed - local MinIO path-mode relay startup passed the Git A3 conformance probe and became ready - unreachable object storage failed startup and readiness never opened - push hooks completed the broader Rust and desktop suites successfully --------- Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz> Signed-off-by: Kalvin Chau <kalvin@block.xyz> Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz> |
||
|
Tyler
|
2ce2d71cc3 |
feat(relay): make Postgres pool size configurable, default 50 (#3191)
## Summary - Raise the relay's Postgres pool cap from the `buzz-db` default of 20 to 50 per pool, and expose `BUZZ_DB_POOL_SIZE` for per-deploy tuning - Applies to the writer pool and, when `READ_DATABASE_URL` is set, the reader pool; zero/unparsable values fall back to the default - The `buzz-db` library default is unchanged — only the relay opts into the larger cap ## Why During the 2026-07-27 18:40–19:05Z traffic burst on bb-public, per-pod PG pools pinned at 20 fleet-wide and ~380 requests failed on the 3s acquire timeout — membership checks, channel access lookups, and historical queries returning errors to users. The database was nowhere near a limit: Aurora (db.r8g.8xlarge, ~5,000 max connections) sat at 19% CPU, 201 connections (~4% of capacity), commit latency flat at 0.01ms. The 20-connection default was sized for "four relay pods against PG max_connections=100" (the comment in `buzz-db` says exactly that). Production now runs 12–15 pods against Aurora — the per-pod cap is the binding constraint, not the DB. Budget at the new default: 15 pods × (50 writer + 50 reader + 5 audit) ≈ 1,575 potential connections, ~30% of Aurora's ceiling — and actual usage stays demand-driven (`min_connections` stays 2, connections only open under load). Same shape as #2521 (`BUZZ_REDIS_POOL_SIZE`), which fixed the identical class of ceiling on the Redis side. ## Testing - `cargo test -p buzz-relay`: 762 passed, 1 failed — the lone red is `api::mesh_demo::tests::demo_join_forwarded_arm_round_trips_echo`, the known pre-existing flake; it fails identically on clean `main` at the same SHA (verified via `git stash` / rerun) - New test `db_pool_size_env_override_and_invalid_fallback` covers override, zero, and unparsable fallback - `defaults_are_valid` extended to pin the new default - `cargo clippy -p buzz-relay --all-targets -- -D warnings` and `cargo fmt --check` clean Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
Dave Grochowski
|
e94b9aeda0 |
feat(tracing): add datastore tracing plumbing (#2760)
Configure a dedicated datastore tracing target on the OTLP layer while preserving explicit logging filters and avoiding span overhead when OTLP is disabled. This is in preparation for adding trace spans for datastores used in Buzz ## Update — 2026-07-27 - Export HTTP requests as `INFO` server spans under `buzz_relay`, preserving request parentage for datastore spans. - Configure OTEL span filtering independently with `BUZZ_OTEL_FILTER`, so `RUST_LOG` changes cannot break trace topology. - Verify exported HTTP and datastore spans share a trace ID and have the expected parent/child relationship. Co-authored-by: Amp <amp@ampcode.com> |
||
|
Tyler
|
bcc3e13069 |
feat(relay): make Redis pool size configurable, default 16 (#2521)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
Tyler
|
0fb820f9bf | Revert "feat(relay): inventory unreachable Git objects" (#2275) | ||
|
thomaspblock
|
3afc9dae15 | feat(relay): inventory unreachable Git objects (#2264) | ||
|
thomaspblock
|
a4d82ec722 | perf(relay): cache Git pack hydration (#2169) | ||
|
Tyler
|
29c48883d3 |
Route lag-tolerant reads to an optional Postgres read replica (#2084)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
Jordan Mecom
|
73fc0ec6cf | [codex] Enforce shared relay admission limits (BUZZ-SEC-019) (#1917) | ||
|
morgmart
|
85fc64835e |
Welcome new users with a live agent team kickoff (#1998)
Signed-off-by: morgmart <98432065+morgmart@users.noreply.github.com> |
||
|
Tyler
|
6c2d667575 |
Apply optional relay join policy across join flows (#1894)
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Tyler Longwell <tlongwell@squareup.com> Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: klopez4212 <klopez4212@gmail.com> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> |
||
|
Tyler
|
f308762852 |
feat(media): require auth for relay media reads (#1926)
Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> |
||
|
Jordan Mecom
|
29368cf17b | Harden relay attack surfaces (#1369) | ||
|
Wes
|
8f580f308c |
Rename desktop app to Buzz (#960)
Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Pinky <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co> |
||
|
Will Pfleger
|
f1c672fea5 | chore: deprecate sprout-mcp — fill CLI gaps, remove crate and all references (#850) | ||
|
tlongwell-block
|
86f4c5d9a9 |
docs: fix stale claims, remove counts, purge LiveKit references (#742)
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com> |
||
|
Wes
|
809275795a |
feat(web): browse git repos in-browser via isomorphic-git (#554)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Wes
|
0dcfa86f84 |
chore: remove dead API token references from GUI, docs, and config (#498)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Wes
|
3a12369491 |
feat: serve web UI from relay + repos page redesign (#479)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Bradley Axen
|
c07b14bd9f |
feat: ephemeral channels with TTL-based auto-archiving (#232)
Co-authored-by: Wes <wesbillman@users.noreply.github.com> |
||
|
tlongwell-block
|
e33c6bc0f2 | feat(acp,desktop): bump idle timeout default from 300s to 320s (#195) | ||
|
Wes
|
70206a4a9b | docs: add ACP env var section to .env.example (#161) | ||
|
Alec Thomas
|
6093fc6066 |
feat: migrate from MySQL to Postgres with pgschema (#114)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Tyler Longwell <tlongwell@squareup.com> |
||
|
Wes
|
3eee3f738a | fix(forum): preserve resolved authors across restarts (#113) | ||
|
tlongwell-block
|
058c4b92a9 | Initial release — Sprout Nostr relay with enterprise extensions (#2) |