jerry/buzz - buzz - Gitea: Git with a cup of tea

jerry/buzz

mirror of https://github.com/block/buzz.git synced 2026-10-27 12:40:30 +00:00

Author	SHA1	Message	Date
ravarora2	2d289f720d	BUZZ-301: Add user deletion request intake (#8496 ) ## Summary Add the first testable piece of user deletion: submit a synthetic request, list requests, and inspect one by UUID. Operators can verify that intake reaches PostgreSQL before we add inventory, approval, and execution. Add one operator-global `entity_deletion_requests` table with an additive migration and matching desired-state schema. Identical submissions return the same row. Conflicting submissions fail without changing it. Only `demo:` targets are accepted, and requests stay `submitted`. Keep the commands in `buzz-deletion`, reached through a thin `buzz-admin user-deletions` adapter. Reuse the existing database connection setup. This change adds no worker, stage history, community gates, or deletion operations. ### Rollout plan This is an expand-only schema change plus an operator CLI. Existing relay code does not use the new request table. No backfill or real deletion is part of this release. 1. Order the PRs. Land #8496 first, using 0068; main now owns 0067 for channel join roles. Rework #8489 to build on this table with the next available migration number. Renumbering its existing CREATE TABLE migration alone is insufficient; both PRs currently create the same request table. 2. Apply on staging. For deployments that manage schema separately, apply the reviewed additive migration before using the new CLI. Where auto-migration is enabled, the new relay binary applies it at startup. Keep existing tables and records unchanged. 3. Verify on staging. Use the new `buzz-admin` binary with `DATABASE_URL` pointing to staging. Submit a unique synthetic label, list requests, and inspect the returned UUID: ```bash buzz-admin user-deletions submit --target demo:staging-intake-1 --requested-by <operator> buzz-admin user-deletions list --limit 5 buzz-admin user-deletions inspect <returned-uuid> ``` Verify one `entity_deletion_requests` row with status `submitted`. Repeat the identical submission and verify the same UUID and timestamps. Use a new label if this label was used by an earlier staging test. 4. Apply beyond staging. After verification, use the deployment's reviewed schema/release workflow to add the table in the remaining environments. Requests run only through the operator CLI; this release starts no deletion worker. No feature flag is needed for this synthetic intake: the database accepts only `demo:` targets, and real deletion is absent. 5. Rollback. Keep the additive table and saved requests; stop using the new commands and roll back the binary if needed. After 0068 is applied, an older binary with auto-migration enabled can fail at startup with sqlx `VersionMissing`. A binary rollback therefore requires migration execution to be disabled on the older relay (`BUZZ_AUTO_MIGRATE=false`, including the corresponding deployment configuration). ### Related work This is a smaller first slice of the full-workflow draft [#8489](https://github.com/block/buzz/pull/8489), which is preserved for later extraction. The two PRs overlap and should not both be merged as written. ### Testing The author submitted `demo:rarora-manual-1` to a separate local PostgreSQL 17 database. Repeating the same submission returned its original UUID and timestamps. List, inspect, and a direct SQL query all returned the same saved request. <details> <summary>Actual command inputs and outputs</summary> Commands ran from the new worktree with `DATABASE_URL` set to the local `buzz_user_deletion_submit` database. Outputs below are stdout; connection diagnostics on stderr are omitted. Submit again — returns the existing request without creating a duplicate ```bash ./target/debug/buzz-admin user-deletions submit \ --target demo:rarora-manual-1 --requested-by rarora ``` ```json { "id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89", "entity_type": "user", "target": "demo:rarora-manual-1", "status": "submitted", "requested_by": "rarora", "reason": null, "created_at": "2026-10-20T19:57:44.516698Z", "updated_at": "2026-10-20T19:57:44.516698Z" } ``` List saved requests ```bash ./target/debug/buzz-admin user-deletions list --limit 5 ``` ```json [ { "id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89", "entity_type": "user", "target": "demo:rarora-manual-1", "status": "submitted", "requested_by": "rarora", "reason": null, "created_at": "2026-10-20T19:57:44.516698Z", "updated_at": "2026-10-20T19:57:44.516698Z" } ] ``` Inspect one request ```bash ./target/debug/buzz-admin user-deletions inspect 2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89 ``` ```json { "id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89", "entity_type": "user", "target": "demo:rarora-manual-1", "status": "submitted", "requested_by": "rarora", "reason": null, "created_at": "2026-10-20T19:57:44.516698Z", "updated_at": "2026-10-20T19:57:44.516698Z" } ``` Read the table directly ```bash docker exec buzz-user-deletion-dev-20261020 \ psql -X -U buzz -d buzz_user_deletion_submit -c \ "SELECT id, entity_type, target, status, requested_by FROM entity_deletion_requests ORDER BY created_at DESC;" ``` ```text id \| entity_type \| target \| status \| requested_by --------------------------------------+-------------+----------------------+-----------+-------------- 2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89 \| user \| demo:rarora-manual-1 \| submitted \| rarora (1 row) ``` </details> A separate CLI smoke test also verified that a conflicting operator and a non-demo target fail without changing the saved request. The existing community-deletion `list --limit 1` command passed against the smoke database. See [the intake walkthrough](`729233c726/docs/operator-user-deletion-intake.md`) for setup, commands, expected fields, and the SQL query. Generated with Codex --------- Signed-off-by: Ravneet Arora <rarora@squareup.com>	2026-10-22 10:49:27 -04:00
Logan Johnson	d97f6b4c6e	feat(relay): add announce channels with a join_role setting (#8143 ) 🤖 ## Summary Channels are Buzz's main way to control access, but today they cannot separate reading from writing. In an open channel, anyone who can read can also write. So there is no way to say "many people read, a few people write". Announcements, release feeds, plugin release records and shared configuration all need that. This PR adds announce channels: anyone can follow and read them, but only members can post. It does this with one new channel setting and without changing what any role means. - Roles mean the same thing in every channel. A guest can read and leave, but cannot write. A member (or bot) can write. Admins and owners also manage the channel. The relay already had the guest role, but no client used it until now. - A new setting, `join_role` (`member` \| `guest`), picks the role a new person gets when nobody chooses one: when they join by themselves, or when someone adds them without a role. The default, `member`, keeps today's behavior for every existing channel. - An open channel with `join_role = guest` is an announce channel. Anyone can find, read and join it. People who join are guests, so they cannot write until someone makes them a member. The relay enforces these rules, so a client cannot get around them. What each role can do (the same in every channel): \| Action \| Guest \| Member or bot \| Admin / owner \| \| --- \| --- \| --- \| --- \| \| Read, leave \| ✅ \| ✅ \| ✅ \| \| Post, reply, edit, delete own, react, artifacts, canvas, forum, pins, topic, typing, workflow messages, save or run workflows, huddles \| ❌ \| ✅ \| ✅ \| \| Add people \| ❌ \| ✅ as member, guest or bot \| ✅ any role \| \| Make a guest a member (person or agent) \| ❌ \| ✅ \| ✅ \| \| Any other role change \| ❌ \| ❌ \| ✅ \| \| Change name, about, visibility, TTL, `join_role` \| ❌ \| ❌ \| ✅ \| Other rules: - Joining by yourself. A person who joins (kind:9021), or adds themselves (kind:9000 with their own key), gets the channel's `join_role`. They may ask for `join_role` or a lower role, never a higher one. So in an announce channel nobody can join as a member by themselves. - Non-members. A non-member may write only where joining would let them write: an open channel with `join_role = member`. That is today's behavior for open channels. - Agents. Agents follow the same rules as people. Any member who can write may make a guest agent a member. Limits that only an agent's owner may lift belong to the later IFC work (its "restricted" agent level), not to this PR. That work only refuses more actions, so it can be added later without breaking anything here. - Leaving always works. A member may leave an archived channel too (kind:9022, or kind:9001 naming themselves). The last owner still cannot leave. - Agent owners keep edit and delete. Today the human owner of an agent can edit (kind:40003) and delete (kind:9005) that agent's messages without being a member. This PR keeps that existing behavior in announce channels too, and documents and tests it. It does not let the owner post. - Changing the setting changes no roles. Existing members can still write after a change to `guest`. Existing guests stay read-only after a change back to `member`. A refused write gets `restricted: only members can post in this channel`. ## Details - One write rule. The relay's channel write check decides from the actor's role and the channel's `join_role`. Some event types skip that check and run their own: message edit, deleting your own message (kind:9005), topic and purpose (kind:9002), add-member (kind:9000), workflow messages and joining a huddle. Each of these now uses the same rule. Membership changes (add, remove, leave) do not need write access, so a guest can always leave. If the relay finds a stored `join_role` value it does not know, it treats it as `guest`, so an unknown value can never give more access. - Admission is decided in the database transaction. The membership write (`put_member`) makes the whole admission decision inside its transaction, after it takes the channel's membership lock: `join_role` for a missing role tag, the self-join ceiling, guests cannot add people, who may grant admin or owner, and role changes (`MemberRole::may_change_role`). The relay's request check uses the same rules, but the database is the authority, so a write that skips the request check cannot get around them. A change to `visibility` or `join_role` takes the same lock, so an add and a setting change cannot interleave. - No creator exception. `put_member` used to give owner to a channel's creator who added themselves to a private channel without an active membership. A removed creator could use that: their join (kind:9021) passes the open-channel check, the channel becomes private before the join takes the membership lock, and the join restored them as owner. Channel creation already makes the creator owner in the same transaction, so the exception is removed. A removed creator is now admitted like anyone else. - Huddles. Joining a huddle needs write access in the huddle channel and in its parent channel. The relay checks both before the join. When it commits the join, it applies the same rule again while it holds both channels' membership locks: a member needs a role that can post, and a non-member needs an open channel whose `join_role` is `member`. A guest in either channel cannot join. A join is also refused when the joiner loses membership or the channel's `join_role` changes before the commit. - One lock order. Each writer that takes more than one of a channel's locks takes them in this order: membership lock, TTL lock, channel row. Settings changes, roster publication, TTL refresh and huddle joins all use this order. Before this change, a `join_role` change on an ephemeral channel at the same time as a roster publication could deadlock. - Workflows. Saving a workflow (kind:30620) skips the generic write check, so it now checks the sender's role itself and needs a writing role before anything is stored. Every path that starts a workflow run, including the manual trigger (kind:46020), needs the workflow's owner to hold a writing role now (and an elevated role for an elevated workflow). So a workflow saved while its owner was a member stops running after the owner is made a guest. - Channel names are checked before storage. kind:9002 and kind:9007 refuse a name longer than the `channels.name` column (255 characters) before the event is stored. Before this, the relay accepted the event, then the name update failed, and the rest of the update (for example `join_role`) was silently skipped. This PR only adds this check. Storing the event and applying the update in one transaction is a larger change, and it is not in this PR. - Changing the setting. Owners and admins send a channel-metadata update (kind:9002) with `["join_role", "guest"]` or `["join_role", "member"]`. Buzz applies these updates as partial updates; a missing tag means "no change". The relay posts a `join_role_changed` system message in the channel. - What clients see. The relay signs a channel description event (kind:39000) that clients read. One shared builder now writes it for both the relay and the `buzz-admin` backfill, which had a copy that had drifted from the relay's. It always publishes an explicit `["join_role", …]` tag, plus the standard NIP-29 group access flags: \| `visibility` \| `join_role` \| `public` / `private` \| `restricted` \| `closed` \| \| --- \| --- \| --- \| --- \| --- \| \| open \| `member` \| public \| \| \| \| open \| `guest` \| public \| ✓ \| \| \| private \| either \| private \| ✓ \| ✓ \| This also fixes an old bug: `closed` ("join requests are ignored") was published for every channel, even open channels that accept joins. - CLI and SDK. `buzz channels update --join-role guest\|member`. `buzz_sdk::build_update_channel` takes a new `join_role` argument and always writes it as an explicit tag. - Docs. `NOSTR.md` has an "Announce channels" section with the role table and the flag table. - Tests. Relay end-to-end tests, run in the CI relay E2E step: - `test_nip29_announce_channel_wire`: who can set `join_role`, the kind:39000 flags, joining as a guest, refused guest and non-member posts, reactions and adds, member posts, a member making a guest a member who then posts, changing the setting back without changing roles, and leaving. - `test_nip29_member_raises_guest_agent`: a member who is not the agent's owner can make a guest agent a member. - `test_nip29_announce_channel_wire_workflow_send_needs_write_role`: a guest's workflow message is refused; a member's is accepted. - `test_nip29_departure_wire_archived_channel_allows_self_leave`: guests and members can leave an archived channel. - `test_private_channel_any_member_can_invite` now expects a private-channel guest to be refused when it adds someone (it used to expect success). - `test_nip29_announce_channel_wire_workflow_commands_need_write_role`: saving and triggering a workflow, by a member and by a guest, in announce and normal channels; a guest cannot run a workflow saved while it was a member. - `test_nip29_announce_channel_wire_overlong_name_is_refused_before_storage`: a kind:9002 with an overlong name and `join_role` is refused, and kind:39000 still shows the old settings. - `test_nip29_announce_channel_wire_agent_owner_exception`: an agent's owner, as a guest and as a non-member, can edit and delete the agent's messages, while ordinary guests are still refused. - `test_nip29_guest_is_read_only_in_normal_channel`: a guest in an ordinary channel cannot write, and joining an ordinary channel still gives `member`. Unit tables cover the self-join rule for each `join_role`, and every actor, current role and requested role for the role-change rule. Database tests check the admission rules in `put_member` (`put_member_enforces_announce_admission`), that an add which waits on an uncommitted change to `guest` is decided on the new setting (`put_member_waits_for_a_concurrent_join_role_change`), and that a member can make a guest a member but not an admin. Unit tests cover the name limit and the workflow owner rule (`owner_authority_denies_roles_that_cannot_write`). A relay test covers huddle joins with different roles in the huddle and its parent, and a commit after a demotion, a `join_role` change or a lost membership (`huddle_join_needs_a_write_role_in_the_huddle_and_its_parent`). Lock-order tests: `join_role_change_waits_for_a_roster_publication_on_an_ephemeral_channel` reproduced a real deadlock (`40P01`) against the old order. `f2d_join_lock_order_serializes_membership_and_settings_writers` checks that membership and settings writers wait for a join and then succeed. `removed_creator_cannot_rejoin_a_private_channel` checks that `put_member` refuses a removed creator on a private channel, and `removed_creator_join_racing_a_private_switch_is_refused` holds the membership lock so a removed creator's kind:9021 passes the open-channel check, then makes the channel private before the join continues. Each new test was run against the old code and failed. ## Rollout 1. Before the deploy, count guest rows (`SELECT count() FROM channel_members WHERE role = 'guest' AND removed_at IS NULL`). We expect none. Any guest that exists loses the ability to post, change the topic and add people when this ships. If some exist, decide first whether to make them members. 2. Deploy. Every channel stays `join_role = member`, so nothing else changes. 3. Do not set any channel to `join_role = guest` until no old relay is running.* Old relays ignore the column, so they would let guests and non-members write in that channel. 4. Rerun the `buzz-admin` channel reconcile so open channels drop the wrong `closed` flag. ## Migration and rollback - Migration `0067` adds `channels.join_role TEXT NOT NULL DEFAULT 'member'` with a CHECK constraint. Postgres stores a constant default in table metadata, so existing rows are not rewritten and there is no backfill. The relay runs the migration at startup. - Existing channels keep the old `closed` flag until their next change, or until the reconcile in Rollout step 4. - Before rolling back the relay, set every announce channel back to `join_role = member`. An older relay ignores the column and does not stop guests from writing, so those channels would accept posts from their guests and give `member` to new joiners again. ## Not in this PR Client UI: a Follow button, hiding the composer for guests, and an admin toggle. Clients that ignore the new tag keep working. A guest who tries to post sees the `restricted` error. ## Stack 1. #8143 (this PR): announce channels 2. #8494: store the bot role only for keys with a recorded agent owner --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>	2026-10-21 16:01:11 -04:00
Luke Tornquist	039e1c8d59	feat(relay): write the NIP-43 member list in the membership transaction (#8482 ) ## Summary The relay-signed NIP-43 member list (kind:13534) was published after each membership change had already committed. If that publish failed, or if a change never republished at all (startup `bootstrap_owner`), the list stayed wrong until the background reconcile sweep found and fixed it. #8467 moved that sweep off the startup path. This PR removes the reason the sweep exists. Now every `buzz-db` function that changes `relay_members` writes the new list in the same transaction, as its last statement before commit: - It takes the per-community list lock last, reads the members, signs the list, retires the old one and inserts the new one. - If the list write fails, the membership change rolls back and the caller gets an error. - A call that changes nothing (already a member, role mismatch, host exists, limit reached) writes no list. The functions return `MembershipWrite<T> { outcome, snapshot }`. Callers push `snapshot` to live subscribers after commit. That push is the only thing left outside the transaction; if it fails, live clients see the change on their next fetch, and the stored list is already correct. Write paths covered (RPR #2, every write path): `add_relay_member`, `claim_relay_membership`, `claim_relay_invite`, `remove_relay_member`, `remove_relay_member_if_role`, `update_relay_member_role`, `bootstrap_owner` / `provision_owner`, `transfer_ownership`, `backfill_from_allowlist`, and the owner insert in `create_community_with_owner[_and_nip_fi_issuers]`. Community deletion removes the members together with the community, so it writes no list. The relay keypair is passed in explicitly instead of being stored on `Db`, so the compiler finds every caller. ### Behavior changes - Open relays get an accurate list too. Provisioning and ownership transfer used to publish only when `require_relay_membership` was on. The write is now unconditional. - Same-second changes can't reuse an id. The list's `created_at` is one second past the newest earlier list for that community, including retired ones, or the current time if that is later. Before, adding and then removing a member in the same second could produce an event with the same id as an earlier list. The insert then failed, and the stale list stayed live. A burst of changes can push `created_at` ahead of the clock by up to one second per change; it returns to the clock once the burst ends. - `buzz-admin add-member` / `remove-member` no longer build their own list (`publish_membership_list_with_bump` is deleted). They push the list the database wrote. Concurrent CLI runs now serialize on the database lock rather than depending on `run.sh` running one at a time. - Owner mutations lock the community row `FOR NO KEY UPDATE`. `transfer_ownership` and `bootstrap_owner` used `FOR UPDATE`. The list insert takes `FOR KEY SHARE` on that row through `events.community_id`'s foreign key, so a repair holding the list lock, or a removal or role update holding a member row, deadlocked with a concurrent owner change. `FOR NO KEY UPDATE` still conflicts with archive, unarchive and deletion admission, so the lifecycle ordering is unchanged. - Startup delivers its list to other replicas. When backfill or `bootstrap_owner` changes members at startup, the relay fans the newest list out through pub/sub once its state is ready, so clients connected to other replicas during a rolling deploy see an owner rotation without a refetch. - New `buzz-admin republish-members`. Rewrites the community's list from the stored members in one community-write transaction and pushes it live. This replaces the sweep as the repair for a list that is missing or stale: a `relay_members` change made directly in SQL, a restore, or a deployment that turns on `BUZZ_REQUIRE_RELAY_MEMBERSHIP` after running without it. ## Rollout Nothing rewrites a list automatically once the sweep is gone. A community whose list drifted before this deploys stays wrong until its next membership change or until an operator runs `buzz-admin republish-members`. Either deploy #8467 and let its sweep complete one pass first, or run `republish-members` for each community after this deploys. The command repairs only the community that `RELAY_URL` maps to, so on a relay hosting many communities, run it once per community host. ## What got simpler Deleted: - `reconcile_nip43_membership_snapshots`, `spawn_nip43_membership_reconciler`, its leader lock key and #8467's standby retry - the three `nip43_membership_snapshot_needs_reconciliation` drift reads (one already deprecated) - `publish_nip43_membership_list` / `publish_nip43_membership_locked`, the separate after-commit republish at every call site, and `buzz-admin`'s duplicate publisher - the reconcile metrics `buzz_nip43_membership_reconciliations_total` and `buzz_nip43_membership_reconciliation_failures_total` - the after-commit publish metrics `buzz_nip43_membership_publications_total` and `buzz_nip43_membership_publication_seconds`, and the `publish_nip43_membership_locked` transaction metric label. A failed list write is now a failed membership write and shows up in the caller's existing error handling and the transaction metrics of its write path. - the `BUZZ_NIP43_RECONCILE_INTERVAL_SECS` setting There is now one way the list gets written, and it can't disagree with `relay_members` after commit. Unused deploy config (outside this repo):* hosted deployments that set `BUZZ_NIP43_RECONCILE_INTERVAL_SECS` can drop it. It is now ignored. Dashboards or alerts on the four deleted metrics will stop receiving data. ## Testing New Postgres tests (`buzz-db`): - `relay_members::every_membership_write_commits_its_list`: for each write path, the returned list is the single live kind:13534 and names exactly the committed members. No-op calls return no list and leave the live list alone. - `relay_members::a_failed_list_write_rolls_back_the_membership_change`: holds the list lock, waits for the join to block on it, cancels that statement, and checks that the member was not added and the previous list is still live. - `relay_members::concurrent_joins_all_land_in_the_list`: 8 concurrent joins; the final list names all 8. - `relay_members::same_second_return_to_an_earlier_membership_writes_a_new_list`: add, remove, add, remove within a second; every list is newer than the last and no id repeats. - `relay_members::owner_transfer_and_list_repair_both_commit` and `relay_members::owner_transfer_and_member_removal_both_commit`: hold a repair (list lock) or removal (member row) open, wait until a transfer blocks behind it, then finish it. Both transactions commit; with `FOR UPDATE` back in place, one of them fails. - `community::create_community_with_owner_is_atomic_and_create_only` and `relay_invite::bounded_claim_exhausts_and_existing_member_retry_does_not_consume` now also check the list. Each new test was checked to fail against a mutation of the new code that brings back the old behavior (details in the thread). Stacked on #8467. Merge order: #8467, then #8468, then this. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>	2026-10-20 11:39:13 -04:00
Will Pfleger	6286a54321	feat(relay): store per-community NIP-FI issuer bindings (#8372 ) Stack: this PR → #8377 → #8399 → #8401 bb-public creates communities all the time, so a hand-edited `BUZZ_NIP_FI_COMMUNITIES` map can't list them. This PR stores an explicit NIP-FI issuer allowlist for each community in the database and writes it when the community is created. Login checks don't read the table yet; that comes in #8377 together with the database-backed resolver. Operator steps are in [`docs/operator-nip-fi-bindings.md`](docs/operator-nip-fi-bindings.md). Storage. Migration `0066_community_nip_fi_issuers` adds `community_nip_fi_issuers (community_id, issuer, created_at)`, with primary key `(community_id, issuer)` and the community write fence attached. The assertion audience is `https://<communities.host>`, so the table has no `aud` column. Zero rows means no binding. The table is included in whole-community deletion and purge. One bindable rule. `BUZZ_NIP_FI_PROVISION_BINDING` (`{"host_suffix": ..., "authorized_issuers": [...]}`) is parsed in `buzz-auth` with `BUZZ_NIP_FI_ISSUERS`, whatever `BUZZ_NIP_FI_MODE` is set to, and both the relay and `buzz-admin` use the same parser. Each authorized issuer must be a configured issuer (1 to 2048 bytes), and the suffix must not cover any host declared in `BUZZ_NIP_FI_COMMUNITIES`; an unreadable declaration refuses startup even in mode `off`. `ProvisionBindingPolicy::is_bindable(host)` is true for a host exactly one label under the suffix that is not reserved. It decides creation-time binding, explicit `bind`, and what the lookup route reports. Binding on creation. Rows are written in the same transaction as the community insert on all three creation paths (`POST /operator/communities` `create_only`, its legacy convergence branch, and startup bootstrap), and only when the call actually created the community, so existing rows are never rewritten. Reserved host. A host whose `https://<host>` equals a configured `command_audiences` value never holds a binding. Operator provisioning of it is rejected with a 400 and the `reserved_host` error code, bootstrap creates it unbound, and the availability route reports it unavailable with an error code. `main.rs` calls `verify_reserved_hosts_unbound` at startup; if a reserved host has rows the relay refuses to start and names the fix (`buzz-admin nip-fi unbind --host <h> --issuer <i>`, or revert the command audience). A database failure during that check is a separate `Check` error. Operator commands. - `buzz-admin nip-fi backfill` pages unbound communities by host (1000 per page) and binds each page with one `INSERT ... SELECT` limited to active communities. It skips communities being deleted, always prints `{rows_bound, unbound, unbound_hosts, truncated}`, and exits non-zero while any bindable community is still unbound. Archived communities are bound on purpose. Both bind paths take the shared deletion lock and then lock the community row (`FOR NO KEY UPDATE`, in id order for a batch), the same order deletion uses, and the backfill insert also requires the community to still have no binding, so an operator `bind` that lands after the backfill listed a community wins and the default issuer is not added beside it. - `buzz-admin nip-fi bind --host --issuer` refuses a reserved host, a host outside the suffix, or an unconfigured issuer. - `buzz-admin nip-fi unbind --host --issuer` refuses to remove a community's last issuer (archive or delete the community instead), with one exception: when a valid provisioning policy shows that backfill can't bind the host (it's reserved, or outside the suffix), the last issuer can be removed. That is the reserved-host recovery path in the runbook. A missing or malformed policy doesn't block `unbind`; it just means the exception never applies. Lookup route. `GET /operator/communities/nip-fi-binding?host=` (NIP-98, `RELAY_OPERATOR_PUBKEYS`) reads the database and returns `{host, active, issuers[]}`. `host` is the stored host. It returns 404 for an unknown host, 200 with `active:false` for an archived or quiescing community, and `issuers:[]` when no provisioning policy is set. It lists only issuers that are configured and on a bindable host. kgoose depends on this contract; it is also documented in the runbook. NIP-FA. The `relay_url` bullet now also says: "An adapter MAY obtain its exact relay list from a configured, authenticated source; it MUST still compare `relay_url` exactly." Deployment dependency. The kgoose config side is already handled by [cash-server#137671](https://github.com/squareup/cash-server/pull/137671). In public-community mode it requires the command audience not to be a community audience. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>	2026-10-17 13:13:49 -04:00
Logan Johnson	71d7aa0436	feat(relay): add the system channel type (#8180 ) 🤖 ## Summary Buzz uses channel membership to decide who can read what, and every channel a person belongs to shows up in their client as a chat. So there is no way to make a group that exists only for access control, for example "only this agent and its owner can read the agent's config". Such a channel would appear in both people's sidebars, search results and unread counts. This stack adds a system channel: an ordinary channel that queries leave out unless they ask for it. This first PR adds the type itself. - `kind:9007` (create channel) with `["channel_type", "system"]` creates a system channel. - It is private unless the creator sends `["visibility", "open"]`. - While it is private, its channel metadata event (kind:39000) carries the NIP-29 `hidden` marker, which means "hide this group's metadata from non-members". Until now the relay set `hidden` only for DMs. On its own this PR does not yet hide system channels from queries. Part 2 does that. Part 1 of 3, stacked on #8176 (the channel labels and creator lookup stack, which this feature uses). ### Related issue None found. ### Details - Migration 0062 adds `system` to the `channel_type` Postgres enum. No new columns. - `buzz_core::channel::ChannelType::System`, and `system` is now a reserved label name (labels cannot equal a channel type name). - `buzz-admin` reconcile writes `hidden` with the same rule as the relay. Usage metrics count system channels. - Rollback. A relay older than this change cannot read `channel_type = 'system'`. Do not roll back past it while system channels exist. - Spec. NIP-CL gets a "System channels" section. `ARCHITECTURE.md` lists the `System` type. ### Testing - Relay test: a system channel is private and `hidden` by default; opening it removes `hidden`; its first `t` tag is `system`. - Migration embed and usage-metrics tests. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. Stack 1: channel labels and creator lookup (#8174–#8176) 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel Stack 2: system channels (#8180–#8182), stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type ← this PR 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>	2026-10-14 19:03:34 -04:00
Logan Johnson	2f79374f48	feat(relay): channel labels (#8175 ) 🤖 ## Summary A plugin or agent often needs to tag its own channels and then find them again, for example "my attention-config channel" or "all channels of this workspace". Today a channel has no labels: the relay ignores any `t` tags in create or edit requests. This PR adds channel labels. - A create request can carry up to 8 labels as `["t", "<label>"]` tags. - An edit request with `t` tags replaces the label set. `["t", ""]` alone clears it, the same convention as `["ttl", ""]`. Only owners and admins can change labels, as for the name. - The relay publishes each label as a `t` tag on the channel's state events, after the channel type tag. Part 2 of 3, stacked on part 1. ### Related issue None found. ### Details - Rules. A label uses 1–64 characters from `a-z`, `0-9`, `.`, `:` and `-`. The relay refuses a label equal to a channel type name (`stream`, `forum`, `dm`, `workflow`, `system`), so a label cannot make a channel look like another type. - No trust from labels. Any admin can set a label. Only the relay-written creator tag (part 1) says who made a channel. - Storage. New migration `0061_channel_labels.sql` adds `channels.labels TEXT[] NOT NULL DEFAULT '{}'`. Channels without labels look and behave as before. - Readers. The CLI and `buzz-acp` read the channel type from the first `t` tag. Before this PR they used the last one, which would now be a label. - Spec. NIP-CL gets a "Channel labels" section with these rules. ### Testing - Relay tests: labels at creation appear after the type tag, edit replaces and clears them, invalid labels are refused, and members who are not admins cannot change them. - Unit tests for the label rules: allowed characters, length, reserved type names, the 8-label cap and the clear marker. - `buzz-acp` test: the channel type comes from the first `t` tag, not a label. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. Stack 1: channel labels and creator lookup (#8174–#8176) 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels ← this PR 3. #8176 database-side label and creator match, and the rule for choosing one channel Stack 2: system channels (#8180–#8182), stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>	2026-10-14 17:25:46 -04:00
Logan Johnson	8bb49a1b6a	feat(relay): publish channel type and creator on every channel-state event (#8174 ) 🤖 ## Summary Clients cannot tell who created a channel, and only the channel-metadata event says what type of channel it is. This PR makes every relay-signed channel-state event carry both facts, so a client can find "the channel that this key created" with one ordinary Nostr filter. - Channel metadata, admin list and member list events (kinds 39000, 39001 and 39002) now all carry the channel type as their first `t` tag, then the creator as a `P` tag. - The creator comes from the relay's own record of who signed the channel's create request. No client can set it, and it does not change when ownership moves. - Channel metadata (kind:39000) also carries `["created_at", <unix seconds>]`, the time the channel was created. The event's own `created_at` changes on every edit, so before this a client could not tell which of two channels is older. This is part 1 of a 3-part stack (channel labels and creator lookup). The system channels stack (#8180–#8182) builds on it. ### Related issue None found. ### Details - Why `P`. Several Nostr specs (NIP-22, NIP-34, NIP-72) use uppercase `P` for the author of the root object, and lowercase `p` for people involved. Lowercase `p` is already used on these events for members and DM participants, and a filter matches tag values, not their meaning, so `#p` would mix them. - Trust. A client that trusts a channel because of its creator must also check that the relay key signed the event. - Order. The type `t` tag comes first, so clients that read the first `t` tag still get the type after part 2 adds labels as more `t` tags. - Creation time. The relay reads it from `channels.created_at`, so an edit does not change it. #8176 uses it to pick one channel when two creates race. - `buzz-admin reconcile` writes the same tags when it rebuilds these events. `NOSTR.md` documents the tags. - Spec. New `docs/nips/NIP-CL.md` defines these tags, the trust rule and lookup by creator. Later PRs in both stacks add their parts to it. `NOSTR.md` links to it. ### Testing - Relay tests in `group_state_postgres_tests.rs`: each of the three kinds carries the type first, then the creator; the creator stays the same after an ownership transfer; kind:39000 carries the stored creation time, and still does after an edit. - Unit test for the shared tag builder that both the relay and `buzz-admin reconcile` use. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. Stack 1: channel labels and creator lookup (#8174–#8176) 1. #8174 channel type and creator tags, and the channel creation time ← this PR 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel Stack 2: system channels (#8180–#8182), stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>	2026-10-14 09:22:42 -04:00
Ian Oberst	85b7331784	fix(relay): partition audit observability follow-ups (#7994 ) > [!IMPORTANT] > The alert rules changed in c3a314e71. If you reviewed the alert guidance earlier, please look again. > - The two range-based rules now apply only to live pods. A range selector such as `[30m]` keeps reading a terminated pod's samples until they leave the window. So `BuzzPartitionAuditNotRunning` could fire for about 10 minutes after normal pod turnover, and `BuzzPartitionAuditFailing` could keep firing for up to an hour after the pod was gone. Each rule now adds `and <counter>`, which requires the counter to be exported right now: > ```yaml > expr: \| > sum without (outcome) (increase(buzz_partition_audit_runs_total[30m])) == 0 > and sum without (outcome) (buzz_partition_audit_runs_total) > ``` > ```yaml > expr: \| > increase(buzz_partition_audit_failures_total[1h]) >= 2 > and buzz_partition_audit_failures_total > ``` > - The audit counters start at 0. When the relay starts, it exports every `buzz_partition_audit_runs_total{table, outcome}` series and `buzz_partition_audit_failures_total` at 0, before the first audit. `increase()` ignores a new series' first sample. Before this change, it could miss the first periodic failure, or read a table's first `degraded` run as a stopped loop. This change is also what makes the `and <counter>` gate work: the counters now exist for a relay's whole life. > > The doc's old claim that pod turnover can't leave an alert firing was true only of the instant-selector stale rule. The doc now explains the difference. ## Why These are the observability follow-ups deferred from #6515 ("fix(db): audit partition catalog before creation"): - Timestamp-age alerts go silent ([review comment](https://github.com/block/buzz/pull/6515#discussion_r4127212141)). Partition gauges are set only when a table audits successfully, and the exporter evicts idle gauges after 2,700 s by default. After sustained failures, `buzz_partition_audit_last_success_timestamp_seconds` disappears. A table that has never audited successfully never has it. An alert on `time() - last_success` then has no input. - Startup fallback failures are not counted ([review comment](https://github.com/block/buzz/pull/6515#discussion_r4127212158)). When the startup fallback audit fails, it logs an error but does not increment `buzz_partition_audit_failures_total`. Periodic failures do. - Repaired environments are always `degraded`. `PartitionTableAudit::degraded()` counts every legacy leaf, even one wholly in the past. The `bb-block-staging` audit of the repaired catalog reports `degraded` only because of `events_p2026_07_08_legacy` `[2026-07-01, 2026-09-01)`. With this rule, every repaired environment stays `degraded`, so nobody can alert on the outcome. - Partition config tests can see a poisoned env ([review comment](https://github.com/block/buzz/pull/6515#discussion_r4137416297)). Two partition config tests take only `ENV_MUTEX`. A NIP-FI test running in the same process can make them see its deliberately invalid environment. Partition health is still diagnostic only, through `/_status.partition_catalog` and metrics. Nothing here affects `/_readiness`. Handling concurrent creation (#6515 item 4) is out of scope and stays with the catch-all advancement work. ## What - Success timestamp survives failures. Each failed table audit refreshes `buzz_partition_audit_last_success_timestamp_seconds` with `increment(0.0)`, which leaves the value unchanged. It still resets the exporter's idle tracking, the same mechanism as `refresh_legacy_active_gauge_recency`. Sustained failures therefore keep the last success exported and its age rising. A table that has never succeeded exports `0`. A failed audit still never makes the last success look fresh. - Alert guidance. New `docs/partition-catalog-monitoring.md` covers the metrics, how long each series lives, and the audit outcomes. It notes that `runs_total` has no `unsafe` outcome. It gives three alert rules: - timestamp age, which also covers never-successful tables and sustained failures; - `sum without (outcome) (increase(buzz_partition_audit_runs_total[30m])) == 0` for a stopped audit loop. Counters are never idle-evicted, and this rule fires before the stopped loop's timestamp is evicted, so there is no false resolve; - an aggregate failure-counter rule. Both range-based rules also require their counter to be exported right now (`and <counter>`), so a terminated pod's leftover samples can't raise or hold an alert. - Zero counter baselines. `try_install` exports each `buzz_partition_audit_runs_total{table, outcome}` series (`ok`, `degraded`, and `error` for both tables) and `buzz_partition_audit_failures_total` at 0 before the startup audit, the same way the auth series already start at 0. A failure in the startup audit itself can still happen before the first scrape. The doc says so; the stale rule covers that case. - Failure counter. `buzz_partition_audit_failures_total` now counts failed explicit audit attempts: the startup fallback audit and each periodic audit. Both paths go through one helper, `counted_partition_audit`. A failed `ensure_future_partitions` call, such as a DDL error that the fallback then recovers, is not counted by itself. A single attempt never counts twice. - Historical legacy leaves. A legacy leaf is historical when its finite upper bound is at or before the current month start. It is still reported under `children` as `legacy_leaf`, but it no longer degrades the table. A legacy leaf that still covers the current month or any lookahead month stays degraded. All other degradation causes are unchanged. The verdict now lives in `PartitionAuditReport::outcome()`, which `buzz-admin partition-audit` uses. The JSON outcome names and exit codes are the same as before. - Test hygiene. Both partition config tests now use `env_guards()`. ## How it was tested - `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p buzz-relay -p buzz-admin --all-targets -- -D warnings`. - New unit tests: - The repaired incident layout with an injected clock: `degraded` in August 2026 and `ok` on 2026-09-01T00:00Z, when the upper bound equals the current month start, and on 2026-09-15. The legacy leaf is still in the serialized report. The `outcome` label on the run counter follows the verdict. - A table test of the legacy-leaf predicate: the upper bound before, at, and one second after the month start; lookahead and beyond-horizon leaves; historical leaves with trigger drift; anomalous and `DEFAULT` children; no audited month. - Outcome precedence (error, then unsafe, then degraded) and the `buzz-admin` exit-code mapping. - Startup: a failed fallback counts once, a successful fallback after a DDL failure counts nothing, and a successful ensure never runs the fallback. Each periodic failure counts once. Against an unreachable database, startup counts one aggregate failure while each table still records two `error` runs. - On the production exporter, never-successful audits leave run counters and a `0` success timestamp for both tables. - On the production exporter, all six run-counter series and the failure counter read 0 before any audit. After one failed audit, `error` reads 1 and `ok`/`degraded` stay at 0. Removing the initialization fails this test. - PostgreSQL lane (`scripts/postgres-test-run.sh`) with new or extended tests: - The repaired layout is not degraded in September. The same catalog is degraded in August. - With a 1 s idle timeout, a failed audit leaves the success timestamp unchanged, and sustained failures keep it exported past the idle timeout. When audits stop, the timestamp is evicted but the run counters remain. The next success exports a fresh timestamp. - Mutation check: removing the failure-path refresh fails both alert fixtures. - These fixtures check the emitted series. They do not evaluate the rules themselves, including the `and <counter>` gate. `promtool` isn't available locally or in CI, and the doc points to `promtool test rules` for that. - `scripts/test-postgres-test-discovery.sh`. - `config::` and `nip_fi_config::` tests run together in one process five times. ## Related This PR conflicts textually with #7993 (catch-all advancement) in `crates/buzz-db/src/store/partition.rs` and `crates/buzz-relay/src/main.rs`. Whichever PR merges second will be rebased onto the other. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 13:57:54 -07:00
Ian Oberst	a8fe2b612c	feat(db): advance an empty partition catch-all under bounded locks (#7993 ) ## Summary Adds bounded, serialized advancement of an empty right-edge partition catch-all for `events` and `delivery_log`. The repaired catalogs currently end in an empty catch-all from 2027-01-01, and the merged manager (#6515) only audits them. Without this change, January writes land in the catch-all, after which only an operator can move them. When `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED=true`, each relay keeps six dedicated monthlies after the current month. It does this at startup and on every audit interval by replacing the empty catch-all with canonical monthlies and a new canonical `<table>_p_future` after the horizon. From today's catalogs the first run creates January–March 2027 and moves the catch-all to April 2027. ## Safety design - Refuse before locking. The following return `operator_required` without taking any table lock: - a populated catch-all, or one whose occupancy is unknown; - a catch-all with child-only triggers, which the replacements would not inherit; - a misaligned bound; - a `DEFAULT`, anomalous or pending-detach child; - a name collision. - One winner. A schema-scoped `pg_try_advisory_xact_lock` is taken before any DDL. Losers report `skipped_locked` and never issue DDL. The winner re-plans from a fresh audit before taking any table lock and checks for collisions. A runner that lost a race therefore sees the committed layout as a no-op, not as colliding names. - Writer lock order. First `ONLY` the parent is locked `ACCESS EXCLUSIVE` under a 2 s `lock_timeout`. Then each foreign-key counterpart (today only `communities`) is locked in OID order `SHARE ROW EXCLUSIVE NOWAIT`, then the catch-all `NOWAIT`. - The counterparts are read again once the parent is locked, and that set is the one locked. Any FK DDL on or to the parent conflicts with the parent lock, so the set can't change after that. An FK committed while maintenance waited is therefore locked `NOWAIT` too, so attaching never waits on it under the parent lock. The read before the lock remains as a cheap early refusal. - This is the order an insert takes locks (parent, then the FK check on `communities`), so maintenance cannot deadlock with ingest. The internal manual runbook used to lock `communities` first; it now uses this order too. - `SHARE ROW EXCLUSIVE` is the strongest lock the DDL takes on `communities`; measured on PG 16 and 17, the catch-all `DROP` takes none, as long as the catch-all's only foreign keys are those inherited from the parent. A catch-all with its own foreign key, or one that another table's foreign key references, is refused: once before any lock, and again after the catch-all lock, where the check is authoritative. Readers and other writers' `FOR KEY SHARE` checks proceed. A concurrent `communities` writer makes the run fail fast with `lock_timeout`. - A `40P01` deadlock victim is also reported as `lock_timeout`. - Parents that are referenced by a foreign key, or that reference an inherited table, are refused, including when such a key commits while maintenance waits on the parent. - Re-plan under lock. The catalog is re-audited under those locks. The transaction proceeds only if the plan is unchanged. - Bounded. `statement_timeout` 5 s, `idle_in_transaction_session_timeout` 5 s, and a 10 s client deadline per table. A deadline-abandoned connection is closed, never returned to the pool, and the server rolls back anything uncommitted. If the deadline fires after `COMMIT` was sent, the change may have landed; the next audit reports the truth. The orphaned backend keeps its locks until its in-flight statement ends, at most `statement_timeout`. - Verify before commit. The changed catalog is re-audited inside the transaction for bounds, canonical kinds, trigger parity and catch-all emptiness, and serving safety must not regress. It also checks that every new child has valid, ready clones of each parent index and validated clones of each PK/unique/FK constraint. - Audits tolerate a concurrent advance. Catalog rows come from the statement snapshot, but `pg_get_expr` renders from current catalog state. So an audit that overlaps another relay's `DROP` of a catch-all can see a partition that no longer exists. The read-only audit detects this (a NULL bound, or `42P01` from the emptiness probe) and retries up to twice with a fresh snapshot. - Fail fast on audit probes. If another relay's audit is probing the catch-all's emptiness, the catch-all `NOWAIT` lock fails and the run reports `lock_timeout`. The parent lock is then released immediately, and the next runner or interval completes the advance. Uncovered-month creation (`BUZZ_PARTITION_MANAGER_CREATE_ENABLED`, default `true`) now runs through the same locked transaction, replacing the unlocked per-month `CREATE`. That flag remains the kill switch for all automatic partition DDL: with it off, advancement is disabled too. With advancement disabled (the default) the periodic loop stays read-only and startup behaves as before, apart from the locking. A runner that planned DDL always re-audits before returning, even when a peer did the work, so it never reports the pre-maintenance layout. A serialized re-plan that finds only disabled work reports `skipped_disabled`. ## Behavior changes - The relay audit horizon and the `buzz-admin partition-audit` default are now 6 months (`PARTITION_MANAGER_MONTHS_AHEAD`), up from 3. Until advancement is enabled, months already covered by the catch-all make `degraded` true. That correctly reports a runway under six months. - New metric `buzz_partition_maintenance_runs_total{table,outcome}` with outcomes `noop \| skipped_disabled \| created \| advanced \| skipped_locked \| operator_required \| lock_timeout \| deadline \| error`. `buzz_partition_create_attempts_total` keeps its per-month `created \| skipped_covered \| error` labels. `error` there still means a DDL or collision failure; `lock_timeout` and `deadline` runs add nothing to it, and only `buzz_partition_maintenance_runs_total` records them. - `ensure_future_partitions` (free function and `Db` method) is removed; `maintain_partitions` is the one entry point. - New advisory-lock label `lock_type="partition_maintenance"`. ## Rollout Enable `BUZZ_PARTITION_MANAGER_ADVANCE_ENABLED` per environment in BPCI: `bb-block-staging` first, then production. The staging canary proves a real advance because the six-month horizon already extends past January. Fallback: rerun the internal manual runbook, which uses the same parent-first lock order as of 2026-10-05. It keeps `ACCESS EXCLUSIVE` on the counterparts, because its detach/attach can drop FK triggers on them. ## Testing - `cargo test -p buzz-db --lib partition`: 17 unit tests. The 10 new ones cover: - plans: repaired layout, policy gating, no-op, refusal states including child-only triggers, December→January; - the create kill switch also stopping advancement; - the month cap at exactly 120; - per-month metric outcomes for transient failures versus DDL failures; - the mid-audit-drop retry classifier. - `cargo test -p buzz-db --lib partition -- --ignored` (PostgreSQL 17): 48/48 on 9 consecutive runs, including 17 new tests: - advance to the canonical layout, with routing and idempotent rerun; - replacing a canonically named catch-all in one transaction; - populated catch-all refused in under 1 s while the parent is locked; - a `communities` writer making the run fail fast and roll back; - a `communities` reader not blocking the advance; - an in-flight insert's FK check, and its write to `communities`, behind queued maintenance completing without a deadlock; - a foreign key committed while maintenance waits on the parent, with a writer on its new counterpart, failing fast as `lock_timeout` on that table instead of waiting under the parent lock; - a foreign key referencing `events`, committed while maintenance waits on the parent, being refused under the parent lock; - a catch-all's own foreign key, or a foreign key referencing it, being refused before the parent lock with the key intact, and a catch-all key committed while maintenance waits on the parent being refused after the catch-all lock; - a `40P01` deadlock victim being classified as `lock_timeout`; - parent contention timing out without blocking the other table; - held advisory lock skipping with no DDL; - concurrent runners converging, with at most a transient `lock_timeout`; - a stale audit after a concurrent advance being a no-op, and the losing runner returning the committed layout; - a re-plan that finds only disabled work reporting `skipped_disabled`; - an audit retrying when a listed partition is dropped mid-audit; - a catch-all with a child-only trigger being refused; - client deadline abandoning the attempt and closing the session (scoped to the test's own blocked backend); - catch-all name collision; - incoming foreign key refused. - Mutation checks: - restoring the old counterpart-first `ACCESS EXCLUSIVE` order fails the reader test (`lock_timeout`) and the writer test (a ~1 s deadlock wait); - dropping `NOWAIT` from the counterpart lock, or allowing a populated catch-all, fails the matching tests; - removing the parent-locked counterpart re-read fails the committed-FK tests (the parent is held 2.06 s waiting on the new counterpart), and ignoring its refusal fails the referencing-FK test; - locking the counterparts `SHARE ROW EXCLUSIVE` before the parent fails the writer test; - removing either catch-all foreign-key check fails its test. - Full PostgreSQL lane: 716/718. Both failures are timing-sensitive tests outside partition code, and each passes 3/3 when run alone: - `hanging_redis_peer_times_out_before_the_health_listener_binds`; - `readiness_check_cancellation_balances_waiter_and_inflight_connection`. - `cargo clippy -p buzz-db -p buzz-relay -p buzz-admin --all-targets -D warnings` and `cargo fmt --check` are clean. ## Related Refs #2396. This PR conflicts textually with #7994 (partition audit observability follow-ups) in `crates/buzz-db/src/store/partition.rs` and `crates/buzz-relay/src/main.rs`. Whichever PR merges second will be rebased onto the other. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 12:59:10 -07:00
Luke Tornquist	c6340a45a0	fix(deletion): give the scheduled drain the cold-start DB connector (#8146 ) ## Summary The scheduled `buzz-admin deletions drain` Job never gets past its first database connection in a hosted staging deployment. Every run exits 5 after exactly 3 seconds with `pool timed out while waiting for an open connection`. The drain starts in a fresh pod with no mesh sidecar, and that pod's first connection can take longer than the relay's 3-second acquisition budget. `connect_store()` in `buzz-deletion` inherits that budget from `DbConfig::default()`, with no retry. In the same pod, the same command run 15 seconds later succeeds. TCP to Postgres, Redis, and S3 from a sidecar-less pod is fine. #7770 fixed the same failure for `buzz-admin storage-snapshot`, but only inside `buzz-admin`. This PR moves that connector into `buzz-db` as `Db::connect_cold_start` and uses it from both run-once workers: - Pool settings: a 30s acquisition budget, and no idle spare connections (`min_connections: 0`). - Retries: startup timeouts and transient transport errors retry at most twice, after 2s and 5s, so startup waits at most 97s. - No retry: configuration, authentication, TLS, and protocol errors fail immediately, and the error message stays free of credentials. - Per-worker settings: each worker keeps its own pool size. Storage-snapshot stays at 1 connection, and deletion stays at `BUZZ_DB_POOL_SIZE` (default 20). Each worker also keeps its session timeouts. - Startup events: the stderr events carry a worker prefix. Storage-snapshot still emits `storage_snapshot_db_connect_`, and the deletion worker now emits `deletion_db_connect_`. - Error text: the exhaustion message is now `{worker} database startup failed after …`. For storage-snapshot that reads `storage_snapshot database startup failed …` instead of `storage snapshot database startup failed …`. Nothing in this repo matches on that string. - Session-setup failures: SQLx retries a failed `after_connect` hook (writer floor guard, session timeouts, isolation assertion) until the acquire deadline. Those failures therefore surface as `timeout`, are retried, and fail closed after at most 97s instead of immediately. Relay pool defaults are unchanged. The deletion CLI's operator commands (`list`, `approve`, …) use the same connector. They normally run inside the warm relay container, where the first attempt succeeds. What got simpler: there is now one cold-start connector instead of a `buzz-admin`-private copy that the deletion worker couldn't reach. `storage_snapshot_startup.rs` is gone, and its unit tests moved with the code. The two Postgres startup tests now share one harness with the drain's tests (`postgres_storage_snapshot.rs` → `postgres_cold_start.rs`). ### Related issue Follow-up to #7770 (same failure, other worker) and #7827 (the drain CronJob). No duplicate found. ### Testing All at head `1545de7b5`: - GitHub CI: green on attempt 1, including `PostgreSQL Domain / PostgreSQL Tests`. - Blox: - fmt, `clippy --workspace --all-targets -D warnings`, and the `buzz-db` / `buzz-admin` / `buzz-deletion` tests pass. - All four `postgres_cold_start` tests pass, each against a freshly migrated database. - Falsification: with main's `connect_store` restored, the drain retry test fails. - Agent review: two rounds. All findings are fixed or disclosed; the error-text change is listed above. - Agent exercise: real `buzz-admin deletions list` / `deletions drain` against local Postgres 17 behind a fault proxy. - 5s delay: succeeds on attempt 1 (3/3). Main fails at about 3.5s. - Dropped handshake, then 5s delay: attempt 1 fails with `UnexpectedEof`, then a 2s backoff, then attempt 2 succeeds (3/3). - Wrong password, missing database or malformed URL: exit 5 after one attempt. - Persistent rejection: exit 5 after three attempts, with 2s and 5s backoffs. - The deletion queue stayed empty throughout. - Not covered: an approved deletion, real S3 / Pod Identity, TLS and session-setup failures, and a live 97s stall. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>	2026-10-08 10:30:38 -04:00
Luke Tornquist	1972b7d256	feat(admin): add community archive commands (#7107 )	2026-10-07 18:31:00 -04:00
Ian Oberst	14a752fb24	fix(db): audit partition catalog before creation (#6515 ) ## Why The partition manager checks partition existence by name only and treats Postgres 42P17 "would overlap partition" errors as success. On databases where a catch-all partition (`events_p_future` / `delivery_log_p_future`) covers future months, the monthly creation path silently does nothing while real coverage erodes toward a write outage. There is currently no visibility into partition catalog health and no signal before writes start failing. ## What - Read-only catalog audit in `buzz-db` (`partition.rs`): classifies the live partition catalog for `events` and `delivery_log` via `pg_inherits` and bound parsing — coverage bounds, catch-all detection, anomalous children, and trigger parity checked recursively down to routable leaf partitions. - Creation gated on the audit verdict: the 42P17-to-success swallow is removed; monthly CREATE runs only for months the audit proves uncovered, a canonical-name mismatch is an error, and per-month outcome capture replaces first-error abort. New `BUZZ_PARTITION_MANAGER_CREATE_ENABLED` kill switch (default `true`). - Prometheus metrics per table: `buzz_partition_serving_safe`, `buzz_partition_uncovered_months` (audit window is now through +3 months), `buzz_partition_catch_all_covered_months`, `buzz_partition_anomalous_children`, `buzz_partition_trigger_parity_missing`/`_extra`, `buzz_partition_audit_duration_seconds`, `buzz_partition_audit_runs_total{outcome}`, `buzz_partition_audit_failures_total`, `buzz_partition_create_attempts_total`, and `buzz_partition_audit_last_success_timestamp_seconds` for staleness alerting. - Cached diagnostics: `/_status.partition_catalog` reports partition serving-safety, audit age, and freshness. `/_readiness` stays process-local, consistent with #7341; missing, unsafe, or stale audits do not remove pods from service. - Periodic audit task in the relay, plus config surface for the audit interval and the monthly-create kill switch. Catch-alls are detected from catalog bounds rather than configured names. - Doc correction: the mesh heartbeat comment no longer claims readiness gating (it never was gated; behavior unchanged). ## Behavior changes & risk - No DETACH/DROP/ATTACH and no schema migrations. The only DDL is the pre-existing monthly CREATE path, now strictly narrower (audit-proven-uncovered months only) and behind a kill switch. - Partition audit health does not gate readiness. Until the first completed audit, read-only retries back off from 5 seconds to a 60-second cap. Any completed audit, including unsafe coverage, restores the normal interval. - Partition-overlap errors now surface as errors instead of being swallowed as success. - Rollback: revert the commit, or set `BUZZ_PARTITION_MANAGER_CREATE_ENABLED=false` to disable the creation path while keeping the audit and metrics. ## Validation - New Postgres-backed regression tests for catalog classification and trigger parity (disabled child trigger, `ENABLE ALWAYS` parent, disabled nested leaf clone in a multi-level layout, healthy-catalog zero-parity checks). Each new regression was verified to fail against the previous logic before the fix. - Local affected-target validation passed against a disposable current-schema database, with the known mesh forwarded-echo and tracing callsite failures excluded. This is not a claim that the full suite passes. Follow-up work (separate PRs): bounded catch-all advance (empty-only golden path; non-empty reattach only behind an explicit operator flag, default off), fresh-install convergence, and proactive coverage-runway alerting. --- ## Update — 2026-08-21 - Fixed nested-leaf trigger parity auditing so child-only row triggers are checked on every routable descendant leaf and reported as `{leaf}:{trigger}` under the owning root child. - Added a Postgres regression test for a child-only trigger on a nested leaf; mutation-reverting to immediate-child-only extra-trigger scanning makes it fail. Generated with Codex Signed-off-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz> Signed-off-by: Ian Oberst <ioberst@squareup.com> Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-29 13:43:40 -07:00
ravarora2	a929532a04	fix(admin): allow cold storage worker DB startup (#7770 ) ## Summary The standalone S3 accounting worker can exit before its scan begins because database startup inherits the relay’s three-second connection budget. A cold DNS probe took about five seconds. Give `buzz-admin storage-snapshot` its own pool: one connection, no minimum idle connections, and a 30-second acquisition budget. Retry startup timeouts and transport failures at most twice, after two and five seconds. Total startup wait is bounded to 97 seconds. Configuration, authentication, TLS, and protocol errors fail immediately. Log each startup attempt’s elapsed time and error category without connection credentials. Relay defaults stay unchanged. Retries stop before lock acquisition; the same lock-owning database session still publishes the completed snapshot. ### Related issue Follow-up to #7543. No duplicate startup fix found. ### Testing Archived-command check: built a nextest archive, temporarily moved `target/debug/buzz-admin` out of the checkout, and ran `scripts/postgres-test-run.sh --archive-file <archive>`. Both tests failed before the path fix and passed afterward. The tests use nextest’s relocated executable path, with the original Cargo path retained for local runs. Manually restored the old startup call and ran `scripts/postgres-test-run.sh -p buzz-admin --test postgres_storage_snapshot -E 'test(storage_snapshot_retries_then)'`. It failed before reaching S3; restoring the fix passed. This confirms the test exercises the actual worker command. Local `just ci` passed all stages before mobile tests, which stopped at SDK discovery in the `objective_c` native hook. The default Xcode tools report an unaccepted license. Two existing PostgreSQL tests hit connection timeouts in the parallel run and passed individually. After merge, build a worker image and repeat the cold-pod staging smoke test through snapshot publication before changing relay metrics mode. Generated with Codex --------- Signed-off-by: Ravneet Arora <rarora@squareup.com>	2026-09-22 13:32:20 -04:00
ravarora2	6b1e45d2f6	Isolate S3 storage metrics from the relay (#7543 ) ## Why This PR moves the S3 storage scan out of the serving relay and into a separate batch job. The job saves a complete result in PostgreSQL. The relay reads that result, exports the existing storage totals, and reports the snapshot's freshness and database-read health. ```text Today: relay -> scan S3 -> calculate totals -> Datadog metrics After rollout: CronJob -> scan S3 -> PostgreSQL snapshot -> relay -> Datadog metrics ``` The code in this PR is the new pipeline. It adds the worker, database snapshot, relay read mode, and Helm CronJob template. It does not enable the pipeline in production. The chart keeps the CronJob disabled, and the relay keeps its current inline mode by default. [squareup/builderbot-platform-core-infrastructure#244](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/244) will set the production limits, enable the job, and switch the relay to the saved snapshot after this PR merges. The relay currently lists the media bucket once an hour and calculates storage totals inside the serving process. The production bucket now exceeds the relay's one million object cap, so each sweep fails and the last good per-community values remain stale. The Datadog dashboard can no longer show current storage use by community. Raising that cap inside the serving relay would move a larger, memory-heavy scan into the process that owns live WebSocket connections. The scan retains state for unique blob hashes and community bindings, so its memory use grows with the bucket. A large scan could raise relay memory use, trigger restarts, and drop user connections. The goal is to restore current metrics without putting a production-scale bucket scan in the relay's failure domain. No media objects move or change. Only the process that calculates the totals and the handoff to the relay change. ## What - Add `buzz-admin storage-snapshot`, a run-once worker that scans S3 and saves one complete accounting snapshot in PostgreSQL. - Add `external` relay mode to read the saved snapshot. Keep the current `inline` mode as the default, and add `off` as a fail-closed option. - Add an opt-in Helm CronJob for the worker. The 10 million object cap exists only on this Job. - Keep the current fleet and per-community gauges, and add snapshot age, generation duration, configured cap, cap utilization, and database-load health gauges. - Add an additive migration for the latest complete snapshot and its run metadata. ## Overall plan 1. Merge this PR to add the worker, database handoff, relay modes, and disabled-by-default chart support. This step does not change production behavior. 2. Publish the relay image and Helm chart `0.1.9` from `main`. 3. Update [squareup/builderbot-platform-core-infrastructure#244](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/244) to pin those final artifacts and rerender its manifests. 4. Use the infrastructure PR to deploy the CronJob with a 10 million object cap, a one-hour deadline, and a 20 GiB memory limit. 5. Start one manual Job during a staffed window and verify its snapshot, duration, memory use, relay stability, and Datadog output. 6. Keep the daily schedule and `external` relay mode only after that gate passes, then add alerts for stale snapshots, snapshot-load failure, failed jobs, cap pressure, duration, and worker memory. The code and infrastructure remain separate so this PR can ship the mechanism without silently enabling a production-scale scan. The infrastructure PR owns the production settings and rollout gate. ## How ### Complete snapshot handoff The worker takes a deployment-wide PostgreSQL advisory lock before it lists S3. This prevents scheduled and manual workers from overlapping, even if Kubernetes starts both. The final database replacement runs through that same lock-owning session. If the session dies during the scan, publication fails instead of letting the stale worker overwrite its successor. A worker that no longer owns the lock cannot publish through another pooled connection. The worker reads S3 in pages of 1,000 objects and folds each page into physical totals, logical per-community usage, and anomaly counts. It stops before folding a page that would exceed `--max-objects` and emits structured progress every 100,000 objects. The worker replaces the singleton database row only after the full scan succeeds. A listing error, malformed page, cap breach, serialization error, lost lock session, or database error cannot publish a partial snapshot. The last completed snapshot stays available. Each stored result includes its completion time, duration, cap, and image or source revision. PostgreSQL is the handoff between the batch worker and the relay. The worker writes one complete deployment-wide snapshot, and the relay reads that snapshot on its normal leader metrics tick. No new network service or worker endpoint is required. ### Relay behavior `BUZZ_STORAGE_METRICS=inline` keeps the existing in-process sweep and its one million object default. `external` loads the newest completed worker snapshot on each leader metrics tick. `off` emits no storage-family metrics. An unknown value fails closed to `off`. External mode republishes the existing fleet, per-community, and anomaly metric names, so those current queries remain valid. Snapshot handoff health uses a separate family: `buzz_storage_snapshot_age_seconds`, `buzz_storage_snapshot_duration_seconds`, `buzz_storage_snapshot_max_objects`, `buzz_storage_snapshot_cap_utilization`, and `buzz_storage_snapshot_load_ok`. The `buzz_storage_sweep_` gauges remain inline-only because they describe a scan attempted by the relay process. External mode does not synthesize `sweep_ok=1` from the last successful database row: an old row proves only that an earlier worker completed. Kubernetes CronJob/Job status is authoritative for the latest worker result, including non-zero exit, deadline expiry, and OOM termination. The relay keeps the last complete snapshot available when a worker run fails. Snapshot age shows that no replacement arrived on schedule, while `buzz_storage_snapshot_load_ok` distinguishes staleness from a missing row, database read failure, or invalid payload. If a community disappears, changes host, or leaves the configured emission scope, the relay zeroes the old labeled series instead of leaving stale values behind. Per-community bytes are logical referenced media usage. If two communities reference the same blob, each community receives the blob's bytes. Fleet physical totals still count each S3 object once. This PR preserves those existing metric semantics. ### Deployment boundary The CronJob is disabled by default. When enabled, it uses `concurrencyPolicy: Forbid`, no retry, a configurable active deadline, bounded history, explicit CPU and memory resources, a read-only root filesystem, and no Istio sidecar by default. The worker receives only `DATABASE_URL`, S3 settings, optional S3 credential references, and the code revision. It does not receive the relay private key, Git hook HMAC secret, Redis URL, or the full relay secret as environment variables. Operators may reuse the Buzz service account or select a dedicated one. ### Non-goals - This PR does not enable the worker or `external` mode in production. - This PR does not change media uploads, object ownership, retention, or deletion. - This PR does not change the names or meaning of the existing fleet, per-community, or anomaly storage gauges. - This PR does not add a lease service or allow multiple workers to publish concurrently. ## Risk The application and chart defaults do not enable the new worker or raise the serving relay's cap. The database change is an additive deployment-global singleton table. The relay's default remains `inline`, so merging this PR alone does not change the active metrics path. The main rollout risk is worker memory use because the fold retains state for distinct blob hashes and community bindings. The first production-sized run must confirm peak memory and duration before the schedule is trusted. The 10 million cap and one-hour deadline bound the first deployment, but they do not replace measurement. External mode also needs stale-snapshot, snapshot-load, and job-failure monitors because the relay intentionally keeps publishing the last complete result after a failed job or read error. This behavior avoids gaps and partial data, while snapshot age and load health tell operators when the result or PostgreSQL handoff needs attention. Job status remains necessary for immediate failures and termination reasons that a killed process cannot record itself. ## Testing No production S3 scan was run from this PR. Automated regressions verify that external mode keeps stale last-good totals visible without emitting inline attempt-health gauges, reports database-load failure while retaining the cache, and leaves inline telemetry unchanged. Mutation checks also prove those tests fail if external mode is routed through inline emission or a failed load is reported as healthy. After deployment, create one manual Job from the CronJob during a staffed window. Require a completed Job, exactly one new snapshot, fewer objects than the configured cap, acceptable duration and peak memory, no relay restart regression, restored Datadog points, and no OpenMetrics sample truncation. ## Next steps 1. Complete review and merge this PR. 2. Publish chart `0.1.9` and the final main-branch relay image. 3. Replace the provisional pins in [squareup/builderbot-platform-core-infrastructure#244](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/244), rerender, and require clean CI. 4. Run the staffed manual Job gate and record object count, peak memory, duration, snapshot revision, and relay stability. 5. Confirm the relay reads the new snapshot in `external` mode and the existing `block.buzz_relay.` storage series update again. 6. Enable the daily 03:17 UTC schedule and add dashboard panels and monitors for snapshot age, snapshot-load health, Job failures, cap pressure, duration, and worker memory. If production must roll back to an older relay image, set `BUZZ_STORAGE_METRICS=off` first. Older images do not understand `external` and would resume the failing inline scan. ## Bigger picture This is the application half of the storage-metrics repair. The paired infrastructure PR owns the production resource limits, schedule, artifact pins, secret boundary, and monitored rollout. Keeping those changes separate lets this PR remain safe by default while still providing a complete path to current per-community S3 totals. Related issue: N/A. No matching issue or pull request exists in `block/buzz`. Generated with Codex --------- Signed-off-by: Ravneet Arora <rarora@squareup.com> Co-authored-by: Codex <noreply@openai.com>	2026-09-17 10:34:31 -04:00
Luke Tornquist	3ed623bb21	feat(db): configurable writer session timeouts (lock, idle-txn, statement) (#6229 ) ## Why A wedged relay boot pod holding a relation lock can park every other writer in the fleet behind it: DB load pins at pool capacity in `Lock:relation` waits while CPU stays flat, and nothing server-side releases the lock until the holder dies. We hit exactly this in production — ~1,400 sessions queued behind one crash-looping pod's boot transaction for ~20 minutes until kubelet killed the container. ## What Applies session-level Postgres timeouts to every writer connection inside the existing single `after_connect` hook in `buzz-db`, all env-tunable through the same `Config::from_env → DbConfig` path as the existing pool-size knobs: \| Env var \| GUC \| Default \| Effect \| \|---\|---\|---\|---\| \| `BUZZ_DB_LOCK_TIMEOUT_MS` \| `lock_timeout` \| 5000 \| statements waiting on any lock fail fast instead of parking behind a wedged holder \| \| `BUZZ_DB_IDLE_TXN_TIMEOUT_MS` \| `idle_in_transaction_session_timeout` \| 60000 \| reaps wedged clients idling inside an open transaction while holding locks \| \| `BUZZ_DB_STATEMENT_TIMEOUT_MS` \| `statement_timeout` \| 0 (off) \| opt-in runaway-statement cap; off by default because startup migrations/backfills legitimately run long statements \| `0` disables a timeout (Postgres semantics) and deliberately passes through the env parsing — unlike the pool-size knobs where `0` falls back to the default. The reader pool is untouched: replica sessions never take contended locks and already fail acquire in 150 ms. Deployers tune these via plain env vars (`.env`, or `relay.extraEnv` in the Helm chart) — no code changes needed. ## Behavior change to note With the 5 s default `lock_timeout`, a boot-time migration or backfill that waits >5 s on a lock now errors (surfacing in logs / crash-looping the pod) instead of stalling silently. That is the intended visible-failure-over-fleet-stall tradeoff; deployers with slow contended migrations can set `BUZZ_DB_LOCK_TIMEOUT_MS=0`. ## Testing - `cargo test -p buzz-db -p buzz-relay` — buzz-db green; buzz-relay has 9 failures that also fail on clean `main` in this environment (api::admin/api::media/mesh_demo — unrelated, pre-existing). - New config test covers override / `0`-passthrough / invalid-fallback for all three env vars. - Extended the existing `writer_pool_safety_hook_is_single_and_composed` source-shape test so the timeouts can't drift out of the single `after_connect` hook (SQLx replaces hooks — a second hook would silently disarm the floor guard). - `cargo fmt --check` and `cargo clippy --all-targets` clean for the touched crates. Closest existing PR/issue: none found. --- Update Aug 28, 17:06 EDT: Rebased onto `main` at `a3730784fc` and addressed the latest correctness review. - Ported the timeout policy onto the refactored `buzz-db::runtime` pool constructor and kept the shared env overlay for relay, admin, deletion, and audit writers. - Migration/schema-destruction connections now disable `lock_timeout` and `statement_timeout` for their intentional long wait/DDL path. This supersedes the earlier “Behavior change to note”: contended boot migrations wait for the current migration owner rather than crash-looping after five seconds. - The audit worker now preserves and retries the same entry on PostgreSQL `55P03` lock timeouts, using exponential backoff capped at one second. Other database errors retain the existing terminal error behavior, and retries emit `buzz_audit_log_lock_retries_total`. - Added CI-backed PostgreSQL regressions for writer GUC installation/migration exemption, audit-pool lock timeouts, and worker recovery. The worker regression holds the real audit advisory lock past `lock_timeout`, observes a retry, releases the lock, and proves the original entry is appended exactly once. Current verification supersedes the earlier testing notes: workspace Rust clippy passed with warnings denied; all nine infrastructure-free backend unit-test lanes passed; all three focused PostgreSQL regressions passed against PostgreSQL 17; formatting, diff checks, file-size guards, and desktop frontend checks passed. The Linux Blox workstation could not run the unrelated Tauri native lane because `glib-2.0` is absent, so that platform check is left to PR CI. --- Update Aug 31, 11:09 EDT: Rebased onto current `main` at `c3132c3ee9` and reran the requested audit-lock contention scenario on Blox at head `896c3fe9ed`. - `git range-diff` reports both PR commits unchanged by the rebase; the branch remains two commits and the worktree is clean. - `cargo fmt --all -- --check` and clippy with warnings denied passed for `buzz-db`, `buzz-relay`, `buzz-admin`, and `buzz-deletion`. - All three focused PostgreSQL 17 regressions passed: writer session timeout/migration exemption, audit writer timeout bounds, and audit worker recovery of the original entry exactly once. - Live protocol verification used a head-built relay and CLI, native PostgreSQL 17/Redis, `BUZZ_DB_LOCK_TIMEOUT_MS=300`, and an eight-second hold on the community audit advisory lock. The real message was accepted and persisted once while the lock was held; its audit-row count remained zero during contention while retries accumulated. After release, exactly one `event_created` audit row appeared and remained exactly one after an additional two-second duplicate check. The run recorded nine lock-timeout retries, zero audit failures, and event ID `7f8c4ffae28e78555fcf2d56396d6e6c01b3712e5411288dc79e9a54af9d9444`. Generated with Codex --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com>	2026-08-31 11:59:13 -04:00
Wes	e0940927ff	fix(channels): return complete member rosters (#5765 ) ## Summary - return complete channel rosters instead of truncating at 1,000 members - chunk `event_mentions` inserts inside one transaction so large kind `39002` snapshots remain discoverable by every `p` tag - add a targeted `buzz-admin reconcile-channels --channel <uuid>` force-republish path for stale discovery snapshots - cover a 1,501-member roster, 11,000-tag mention index, and kind `39002` tag construction past member 1,000 ## Why The relay builds NIP-29 discovery and several authorization decisions from `get_members()`, but that helper silently returned only the first 1,000 active members. Desktop then counted the truncated kind `39002` event, while late members could be rejected by roster-scanning member actions. Removing the roster cap exposes PostgreSQL's 65,535 bind-parameter ceiling in mention indexing, so the insert is chunked transactionally to preserve all-or-nothing indexing. The existing reconcilers only fill missing discovery events. The targeted admin option bypasses the separately known 1,000-channel reconciliation-list ceiling and replaces an existing channel snapshot using the configured production relay key. ## Attribution This supersedes and builds on #3166 by @LordMelkor. Thank you for identifying the roster boundary and contributing the original complete-roster and mention-index patch. The production roster/query changes and the two PostgreSQL regressions retain that work's shape; this PR rebases it onto current `main`, adds relay coverage, and adds the targeted repair operation requested for rollout. ## Validation Exact pushed head: `24d02e4f3824150ed84913c9d230e675502e5b12` - `cargo check -p buzz-db -p buzz-admin` - `cargo test -p buzz-db channel::tests::get_members_returns_full_roster_beyond_1000 -- --ignored --exact --nocapture` - `cargo test -p buzz-db feed::tests::insert_mentions_indexes_rosters_past_bind_parameter_cap -- --ignored --exact --nocapture` - `cargo test -p buzz-relay --lib handlers::side_effects::tests::group_members_snapshot_keeps_members_past_one_thousand -- --exact` - `cargo run -q -p buzz-admin -- reconcile-channels --help` - mandatory pre-push hook: branch-skew, desktop checks/typecheck/tests, mobile tests, Rust tests, and desktop Tauri checks all passed on the pushed head ## Rollout 1. Deploy the relay/backend build. 2. Run `buzz-admin reconcile-channels --channel <general-channel-uuid>` with `BUZZ_RELAY_PRIVATE_KEY` configured. 3. Verify the replacement kind `39002` roster count matches the active database membership count. No schema migration or desktop release is required. Fixes #3156 Supersedes #3166 --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>	2026-08-13 14:48:04 -07:00
Kalvin C	8a2c9af2db	feat(deletion): add durable whole-community deletion (#4425 ) ## Summary Adds a durable, operator-controlled V1 for deleting an entire Buzz community without deleting another tenant's data. The workflow is exposed through `buzz-admin deletions`: - `sweep` records independent fleet storage-taxonomy observations - `submit`, `list`, `inspect`, and `approve` manage a deletion request - `unblock` resumes a fail-closed request after an operator records remediation identity and reason - `run` and `drain` execute bounded work Requests advance through a PostgreSQL-backed state machine and stop at `retention_pending` after logical deletion has been independently verified across PostgreSQL, object storage, and Redis. This PR ships the engine and CLI, not a continuously running worker or Kubernetes packaging. For V1, a cluster/VM administrator invokes `/usr/local/bin/buzz-admin` from the existing relay image, for example with `kubectl exec` or an equivalent container/VM exec path. ## What whole-community V1 removes For the target community, V1 removes: - rows from the allowlisted community-scoped PostgreSQL catalog, including members, profiles, authored events and bodies, DMs, reactions, mentions, memberships, tokens, workflows, moderation, audit, feedback, and rate-limit state - media sidecars and upload-attribution records under `_meta/<community>/` and `_uploads/<community>/` - Git repository pointers under `repos/<community>/` - Redis keys under `buzz:<community>:` The community row survives as a permanent tombstone, and deletion control-plane records remain as evidence of the request, approval, execution, and result. ## Safety model Deletion is not a broad `DELETE CASCADE` followed by optimistic cleanup. The destructive boundaries are durable and fail closed. ### 1. Inventory and approval - `submit` resolves the target and freezes the schema plus summary-only storage inventory. - Approval is bound to the exact request, community, and frozen inventory digest. - Unsupported manifest versions, malformed keys inside the target's owned prefixes, live scoped-table/write-fence coverage drift, frozen-inventory mismatch, and approval mismatch block execution rather than guessing. Migration and catalog revision numbers are not authorization gates; the executor validates the live safety shape instead. - Storage inventory is server-side prefix scoped to exactly: - `_meta/<community>/` - `_uploads/<community>/` - `repos/<community>/` - The deletion path never lists the whole shared bucket and has no arbitrary per-community object cap. Its listing work is proportional to the target community's bindings, not total fleet storage. - Fleet-wide taxonomy sweeps remain independent observability. They report unknown writer shapes but do not gate deletion submission, fencing, or destructive progress. Maintainers must add deletion taxonomy coverage whenever a new community-owned object-key class is introduced; writer-coverage tests bind the current media and Git writers to that contract. ### 2. Quiesce, fence, and destructive freeze - Writes continue through submission, inventory, and approval. They stop when execution moves the target into `quiescing` and then establishes the durable fence. - Already-admitted external effects finish under heartbeated serving-write leases; the exact admitted lease may renew while the community is quiescing, but new lease acquisition is rejected. The executor drains admitted leases before destructive work. - Invite minting after quiescing begins fails as typed `AccessDenied` (HTTP 503 at the relay boundary) before an invite can be persisted. - Database triggers enforce the community write fence across the complete catalog of community-scoped tables. Startup/readiness and destructive execution validate that catalog so a newly added but unfenced table cannot silently escape. - Named isolation assumption — fresh write snapshot.* Every writer transaction that can reach a community-fenced relation must use PostgreSQL `READ COMMITTED`; each guarded write therefore observes a statement snapshot no older than acquisition of the community deletion lock. `REPEATABLE READ` and `SERIALIZABLE` can retain a pre-fence snapshot and are unsupported for writers. The writer pool refuses non-`READ COMMITTED` sessions at connection setup, and both SQL fence functions reject an explicit per-transaction isolation override with SQLSTATE `25000`. Configuration-delivered bad isolation can surface through SQLx as a pool-acquire timeout because every `after_connect` attempt is rejected; the precise `community writes require READ COMMITTED isolation` reason remains observable when the SQL guard is reached. Read-only replica transactions are outside this assumption. - Holding the shared advisory lock until the guarded write executes is a separate liveness condition: under `READ COMMITTED`, releasing it early does not permit resurrection because the trigger rechecks the fence, but it can turn a fleet sweep into a statement-wide SQLSTATE `55000` abort. - After the fence closes writers, storage is re-enumerated into chunked side-table rows. Per-prefix counts and digests bind those concrete keys to the destructive manifest. - Manifest chunk insertion, update, and deletion are protected after freeze. This closes the race where an unbound key could otherwise appear after the manifest was committed. ### 3. Checkpointed destruction - Target-owned object bindings are deleted from the frozen destructive manifest in bounded batches with durable progress. - The concrete key list lives in chunked side-table rows rather than one request-row JSON value. It supports large communities, resumable execution, and terminal cleanup. - Missing objects are accepted as idempotent crash-window outcomes; malformed ownership, changed evidence, and unexplained target-prefix drift fail closed. - PostgreSQL purging remains scoped by `community_id`, including the guarded NIP-RS hard-delete path discovered with real Desktop kind `30078` read-state data. - Redis cleanup explicitly scans and `UNLINK`s only `buzz:<community_id>:`. Natural expiry is insufficient because some keys, including tunnel generation counters used as fencing state, are deliberately persistent. ### 4. Independent verification - PostgreSQL logical absence is checked after purge. - The three target-owned storage prefixes are freshly inventoried again and must be empty. - Redis requires two complete empty namespace scans. - Only after all three stores pass does the request advance through `logically_verified` to `retention_pending`. ## What V1 deliberately does not erase ### Shared content-addressed storage Per-community deletion removes bindings, metadata, attribution records, and Git pointers. It does not* physically delete fleet-shared CAS bytes that another community may still reference: - media blobs and thumbnails - Git manifests, packs, and indexes (`manifests/`, `packs/`, and `idx/`) Safe reclamation requires a separate fleet-wide reachability and retention GC. Unknown keys elsewhere in the shared bucket do not block one community's deletion; malformed or unrecognized keys inside that community's three owned prefixes still fail closed. ### External retained copies The online logical-deletion proof does not erase object versions/replicas, database backups/WAL, CDN copies, provider retention copies, or observability exports. Those require their own retention and purge controls. ### Member-only erasure This PR erases a whole community. It does not implement the different operation "erase one npub while preserving the community." Removing membership or accepting NIP-09 is not member erasure. A member-only workflow would need to find and selectively remove or redact authored event content and pubkeys, profile data, DMs, reactions, mentions, memberships/roles, tokens, workflows/subscriptions, upload attribution, moderation/audit history, repository attribution, and identity embedded in tags or JSON. It would also need explicit rules for ownership transfer, surviving replies and thread metadata, audit-chain integrity, immutable Git history, and shared-CAS reachability. That requires a pubkey-level fence and selective graph rewrite; it is a separate deletion product, not a safe extension of this whole-tenant worker. ## In scope - migration `0029_community_deletion.sql`: requests, approvals, leases, manifest chunks, checkpoints, tombstones, and the universal write-fence catalog - durable executor leases, generations, heartbeats, retry/block state, and resumable stage transitions - operator-driven `sweep`, `submit`, `list`, `inspect`, `approve`, `unblock`, `run`, and `drain` commands - serving-path fences for database writes and external effects across event ingest, media, Git, workflow, push, invites, mesh/tunnel, and related paths - target-prefix-only storage inventory, summary manifests, post-fence destructive chunks, and bounded batch deletion - exact community Redis namespace purge and two-pass absence verification - cross-community isolation, crash/resume, manifest-integrity, writer-taxonomy, and schema/migration regressions - desired-state `schema/schema.sql` support without requiring a SQLx migration ledger ## Deferred / not covered - dedicated Helm/chart worker Deployment, service account, secrets, probes, resources, and network policy - autonomous `buzz-admin deletions worker` poll loop and worker-only health server - least-privilege separation among migration, relay-serving, and destructive execution roles - fleet-wide shared-CAS physical GC - backup/provider/CDN/observability retention completion - member-only erasure - provider-native conditional-delete improvements - a general force-continue escape hatch; permanent safety failures remain fail closed unless an operator remediates the cause and records an audited `unblock` The removed continuous-worker implementation remains deferred; no remote follow-up branch is claimed by this PR. ## Validation ### Current PR head and repository state Current pushed head: `359d8402ee15f049768f54156f67b953c7a7e2ed`, rebased onto `cc9a2f783375e51a6e8d1f2f9d01d5f7e22813d1` (`origin/main` at push time). The complete PR diff is now 47 files, 9,834 additions, and 517 deletions. The bespoke source-scanner stack was removed to keep this PR scoped to community deletion. Tyler/team requested the underlying fenced-write safety behavior, not `ast-grep`, `crates/buzz-db/tests/community_fenced_writes.rs`, its 27 fixtures, or the new `scripts/lints/community_.yml` rules. Those scanner-specific files, dependencies, Hermit links, and runner wiring are absent from the current tree. The production database write fence, startup/destructive live-catalog validation, and deletion behavior remain. Source validation on this exact SHA passed: - `cargo fmt --all -- --check` - `bash -n scripts/run-tests.sh` - `cargo nextest run -p buzz-db --all-targets`: 102 passed, 173 skipped, 0 failed - `cargo nextest run -p buzz-deletion --all-targets`: 10 passed, 9 skipped, 0 failed - `cargo nextest run -p buzz-admin --all-targets`: 1 passed, 0 failed - affected-package/all-target Clippy with warnings denied - lockfile consistency - Helm 3.16.4 lint and all 44 chart unit tests - Helm region controls using that fixture: default `BUZZ_S3_REGION=us-east-1`, explicit `eu-west-2` override, and blank-region schema rejection The prior Kubernetes battery below was run against `928992237358a3294621ac0280830b77155abc04`. It remains useful evidence for the patch-equivalent production deletion implementation, but it is not* claimed as exact-SHA evidence for current head `359d8402ee15f049768f54156f67b953c7a7e2ed`; the current cleanup removes only scanner/test/tooling infrastructure. CI restarted for the new head after the rebase and is pending. Human review remains `CHANGES_REQUESTED`. ### Prior-head live Kubernetes deletion and safety gates The full program used one immutable image, real PostgreSQL, Redis, MinIO, and a three-relay Kubernetes release: - source: `928992237358a3294621ac0280830b77155abc04` (prior head) - image: `buzz-e2e:sha-928992237358` - immutable image digest: `sha256:a1a204f4618ac22d9e210be5e5290645a15d79831ae30b0e44379357c8e4a895` - evidence root: `/tmp/buzz-e2e/20260807T033025Z-928992237358-full-gates/` - evidence-manifest digest: `82875c5bc9bea7370b796a7aef3457b3a1c8306c84c59e0f7388bbb5ad30e865` Passed gates at that prior head: - Chart/operator region: default `us-east-1`, explicit nondefault propagation, blank-region schema rejection, live in-pod environment, and an in-pod taxonomy sweep over 18 objects with zero unknown. - Fenced writers and lifecycle: open-write/fence ordering; 100-attempt anti-starvation; invite, push matcher, and exhausted-reaper bystander isolation; non-`READ-COMMITTED` rejection; manifest/tombstone contracts; eight-failure stage block and audited `unblock`. - Destructive lifecycle: submit → approve → run → `retention_pending`; PostgreSQL tombstone and Redis/S3 verification true; zero retries/errors; terminal reruns rejected with exit 5. - Fresh 10,001-object crash boundary: exactly two chunks (10,000 + 1). The executor deleted chunk 0 from MinIO while its PostgreSQL stamp was row-lock-blocked, was killed with `SIGKILL`, left one object and both stamps absent, then resumed the same request under generation 2 to zero objects and terminal state. - Independent dead-owner recovery: a dedicated executor claimed generation 1, blocked before effects, and was killed through containerd with `SIGKILL` (no TERM cleanup). The request remained owned and unreclaimable before lease expiry; a successor claimed generation 2 after 60 seconds and completed with two attempts and zero retries. - Three-pod socket isolation: ordinary NIP-42 and joined huddle-audio target witnesses on every replica received exact `1008 / community deleted`; healthy-tenant witnesses on those pods remained live; deleted-host reconnect returned HTTP 404. - Health/provenance: all replicas independently returned ready and retained the exact image digest before/after destructive runs and an audio-enabled rolling restart; PostgreSQL, Redis, and MinIO were healthy at close. Instrument corrections were retained as evidence rather than counted as product failures: a foreground PostgreSQL forward caused an initial `PoolTimedOut`; Kubernetes pod deletion exercised graceful TERM rather than dead-owner recovery; shell-background socket witnesses died with their parent; and the first image build hit the corporate TLS proxy. Detached forwarding/witnesses, containerd `SIGKILL`, and the configured internal CA/Artifactory mirror produced the discriminating runs without weakening product security. ### Prior-head cleanup For the prior-head Kubernetes run, the Helm release was removed, namespace absence was verified, run-owned Screen sessions were absent, and that source worktree remained clean. The evidence manifest was independently recomputed and every indexed artifact passed `shasum -a 256 -c`. The current `359d8402` source worktree is also clean after the scanner-only cleanup and push. --------- Signed-off-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz> Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Signed-off-by: Kalvin Chau <kalvin@block.xyz> Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Signed-off-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz> Co-authored-by: npub122y0pqkertljmedu303rl0aqrj3w8pvu43t6jxm6875lzg6f2pwqegc3xc <5288f082d91aff2de5bc8be23fbfa01ca2e3859cac57a91b7a3fa9f12349505c@buzz.block.builderlab.xyz> Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz>	2026-08-12 09:25:58 -07:00
Wes	af190c93e1	Add private product feedback sidecar (#1857 ) Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Pinky <44b8e82baa6e0e254e0208d68f335c283c94e7b78dd1fa10d5a49d3f13dd0435@sprout-oss.stage.blox.sqprod.co>	2026-07-14 10:41:48 -07:00
Tyler	3292b502aa	fix(relay): enable Redis TLS for rediss:// (ElastiCache) (#1417 ) Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz>	2026-06-30 21:14:43 -04:00
Tyler	14fba21e57	Multi-tenant Buzz relay: community_id as a server-resolved key (comprehensive rewrite) (#1321 ) Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com> Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Dawn <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: Sami <sami@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>	2026-06-29 12:39:02 -04:00
Tyler	73cc31cc52	chore: remove LLM-slop comments across the codebase (#1277 ) Signed-off-by: Tyler Longwell <tlongwell@block.xyz> Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Quinn <96f056ad5f2305c8ddf637dc65d048aa4c12d7daeb8867690e34fca46b0ef64c@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Perci <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Tyler Longwell <tlongwell@block.xyz>	2026-06-25 11:03:05 -04:00
Will Pfleger	0cee0435f7	feat(relay): add buzz-admin member management CLI with NIP-43 roster publish (#1265 ) Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co>	2026-06-24 23:13:31 -04:00
tlongwell-block	2300248d3b	Add automatic database migrations (#988 ) Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Signed-off-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1t2tgm7d8f995uqvmnm8h88sg3wnpp9a5xysjf6dg3tjmgt3ltulqdp8ehr <5a968df9a7494b4e019b9ecf739e088ba61097b4312124e9a88ae5b42e3f5f3e@sprout-oss.stage.blox.sqprod.co>	2026-06-16 08:39:19 -04:00
Will Pfleger	d99ad131f1	refactor: rename sprout backend to buzz (#958 ) Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <wpfleger@block.xyz> Signed-off-by: Will Pfleger <wpfleger@squareup.com> Signed-off-by: Will Pfleger <wpfleger96@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub1fgdl5qqnh3k3f2xkqrvt7cujalhm623x4s7fdjdj5yrtp5fzjl9qrjpucw <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@sprout-oss.stage.blox.sqprod.co> Co-authored-by: npub16v54tttfqacx9ycvc3k0ut0npj564ahcuajzy6qjvh57ntmsf4uq4806j2 <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Will Pfleger <wpfleger96@gmail.com>	2026-06-10 19:29:51 -04:00

24 Commits