mirror of
https://github.com/block/buzz.git
synced 2026-10-27 20:41:57 +00:00
bf8b356f3bf551a01cc24041605ca77d009681dd
147 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Brad Seiler
|
2d0f78c6e1 |
feat(auth): support explicit mesh JWKS transport (#8382)
## Summary Allow an issuer to opt into mesh HTTP for its JSON Web Key Set (JWKS). Public HTTPS remains the default. This supports deployments whose service mesh authenticates and protects the key endpoint. The sealed source contract selects transport and contributes to policy identity. Mesh mode requires a DNS hostname and checks every resolved address against an explicit allow-list before pinning one address. Both modes disable proxies, reject redirects, cap responses, and enforce a whole-fetch deadline. The relay cannot verify mesh security: operators must enforce mutual TLS, endpoint authorization, and egress restrictions. ### Related issue No duplicate mesh-JWKS issue or PR found. Existing NIP-FI relay enforcement: #7224. No deployment change is included. ### Testing - At `eccb8cb19420abda44cf6e5e62b1361e2843bd1d`: full auth package and auth/relay all-target clippy passed. Full relay package passed: 1,457 library tests, 37 binary tests, and 9 boot tests; 446 tests were ignored. - At prior head `113939838cd81d5e8d0b0f8783c42ffaae44a4d3`: the full relay library suite failed the demo echo test (1,455 passed, 1 failed, 440 ignored). Independent checks reproduced that timeout at base `d61fe2071d2b3d8e56dd931947f16bfe0d6173fe`; the receive-cancellation defect is outside this change. A later pass does not establish that defect is fixed. - Three falsifying mutations at `eccb8cb194` were caught: mixed DNS answers, proxy bypass, and whole-fetch deadline. The full locked auth package passed after restoring the exact source. - At `eccb8cb194`, `cargo build --locked --release -p buzz-relay -p buzz-cli -p buzz-admin` passed with command-local `CARGO_PROFILE_RELEASE_STRIP=none`. This avoids a macOS loader error without a source, lockfile, or project-profile change. - Earlier hooks failed the ACP signal test and Tauri login-shell PATH test. The ACP failure also reproduced on unmodified main `93ffc37c`. Main merged without conflict. The full Tauri package then passed with `RUST_TEST_THREADS=1`: 3,422 library tests and 3 integration tests, 19 ignored. All pre-push hooks passed with that same command-local setting; no hook or test was skipped. - At `eccb8cb194`, isolated `just test` passed all 107 runner steps. The monolithic `just ci` run timed out after 1,120 seconds; all recipes passed across that run and separate `just desktop-tauri-test`, `just web-build`, and `just mobile-test` invocations. No monolithic pass is claimed. - Exact-head release live-local checks passed: loopback HTTP mesh JWKS warm; valid assertion 101; expired/wrong issuer/wrong audience 403; valid signed disconnect 200; wrong command audience 403. Redirect-only JWKS returned assertion 503 without a request to its target. Both omitted/default and explicit public transport rejected HTTP at startup. - A separate NIP-FI-off release CLI smoke test created an open channel, sent an accepted message, read it back, and fetched its thread. The assertion probe closes after upgrade; it does not prove a complete NIP-42 session. Local loopback tests do not prove deployment mesh authentication, endpoint authorization, egress controls, or real `240/4` routing. Generated with Codex ## Human Testing valid assertion / configured community audience: HTTP 101 (expected 101) expired assertion: HTTP 403 (expected 403) wrong issuer: HTTP 403 (expected 403) wrong community audience: HTTP 403 (expected 403) valid command JWT / configured command audience: HTTP 200, body=b'{"disconnected": true}' (expected 200) valid command signature / wrong audience: HTTP 403 (expected 403) --------- Signed-off-by: seiler <seiler@squareup.com> Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
c8ea4fb4eb |
feat(mobile): polish Home navigation and Solo workspace (#8393)
## Summary Polish mobile Home and Solo: group Search/Inbox beside a separate profile avatar, move Inbox and Settings to bottom-up pages with Close controls, and place tabs opposite the plus/compose action. Morph plus to compose clockwise when entering Solo and reverse counterclockwise when returning Home, with a directional quarter-turn of the icon inside a level button and a subtle scale pulse, respecting reduced motion. Align Search’s Close control with Inbox, simplify its placeholder, and initialize selected filter contrast correctly. Collapse the plus menu when leaving Home. Center Solo’s native-glass New chat empty state and keep it stationary behind the keyboard. Standardize Tabler icons and neutral backgrounds, retain SF Back/Close on iOS, use native Liquid Glass headers, tab container, and actions (solid grey selected tab, slightly darker on iOS) on iOS and black actions on Android, and smooth the expanded menu corners. Repair native header safe-area/return layout and compose movement after cancellation. ### Related issue None found for this UI refinement; follows #8337. ### Testing - Formatting, analysis, full mobile suite: 3,487 passed, 4 skipped; 3 push-configuration tests passed. File-size checks passed. - Base UI built and installed on Pixel and iPhone in place. Pixel navigation/New chat cancellation checked. Subsequent glass and rotation refinements are previewed on iPhone; motion feel awaits human confirmation. - Draft pending human confirmation of the final Settings entrance/return behavior on iPhone. ### Pixel snapshots | Solo | Expanded actions | | --- | --- | | <img src="https://raw.githubusercontent.com/block/buzz/16dc0f8af971e63d788fee20974997fab925dec9/docs/screenshots/mobile-home-solo/pixel-solo.png" width="280" alt="Solo empty state with New chat" /> | <img src="https://raw.githubusercontent.com/block/buzz/16dc0f8af971e63d788fee20974997fab925dec9/docs/screenshots/mobile-home-solo/pixel-actions.png" width="280" alt="Expanded channel actions with smooth corners" /> | --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
c0ab789499 |
feat(buzz-db): retire the channel-TTL trigger (#8210)
## Summary This drops the channel-TTL event trigger. The application already refreshes ephemeral-channel deadlines in the event transaction (`event_follow_up::refresh_channel_ttls`, from #8135), so the trigger only repeats that work. This PR originally retired both follow-up triggers. #8352 has since retired the push-match trigger in migration 0063, so after the rebase this PR covers only the TTL half. Migration `0065_retire_channel_ttl_trigger.sql`: - `SET LOCAL lock_timeout = '5s'` (the same bound as 0063) - drops `events_refresh_channel_ttl` and `refresh_channel_ttl_after_event_insert()` Dropping the trigger on the partitioned parent also drops its partition clones. Published migrations, including 0022 and 0024, stay checksum-frozen. The same objects are removed from `schema/` (`schema/functions/…`, `schema/triggers/events.sql`, `schema/schema.sql`) and from the pgschema reconcile script. After this, migrated and desired-state databases have no event follow-up triggers. Tests: - `follow_up_retirement_tests.rs` (renamed from `push_retirement_tests.rs`): the push-retirement upgrade test now stops at 0063. A new test upgrades 0064 → 0065 and checks that the TTL trigger is gone from the parent and every partition, that the function is gone, and that the published checksums haven't changed. - `event_follow_up_postgres_tests.rs`: the contract now has two arms. `AppOnly` is the shipped schema, where both triggers are absent. `Dual` reconstructs the historical overlap of both triggers. The `PushRetired` arm (the push trigger dropped, the TTL trigger kept) was the shipped state before this PR. Its tests are removed because `Dual` and `AppOnly` together cover it. - `embedded_migrator_contains_consolidated_initial_schema` pins the 0065 statements and checks that both objects are absent from the desired schema. **Deploy gate:** do not run 0065 against a database still served by a relay older than #8135. After 0065, raw SQL writers no longer refresh TTL deadlines. `docs/push-gateway-deployment.md` is updated to match. ### What got simpler The trigger and the app no longer both refresh TTL deadlines; the app is now the only producer. One plpgsql function and one trigger are gone from both schema paths, and 8 per-partition drops are gone from the reconcile script. The follow-up contract drops from three arms to two. ### Related issue BUZZ-313 (Linear), part of the trigger-retirement work. ### Testing Head is `4f7d0baf0`. Results below are at `678927730` unless noted. `678927730..4f7d0baf0` changes only one test in `crates/buzz-db/src/store/event.rs`, described in the first bullet. - **Fixed test, `permanent_channel_event_commits_do_not_lock_the_channel_row`** (`4f7d0baf0`). - **Why it changed:** after 0065, its raw-pool inserts no longer ran any TTL refresh, so the test passed whatever `refresh_channel_ttls` did. This was raised in review. - **What it does now:** it inserts through the production `AdmittedTx` path and keeps the barrier and the tuple-lock probe. It commits all N transactions while an external transaction holds `FOR SHARE` on the channel row. It also asserts that no `buzz_db_channel_ttl_refresh_failures_total` is counted. - **Mutation check:** with `SELECT … FOR UPDATE` injected into `refresh_channel_ttl`, the new test fails (`must not wait on the channel row`, timeout at 10s). The old test passes under the same mutation. - **Blox:** fmt and `buzz-db` clippy pass. - **GitHub CI at `4f7d0baf0`:** every required check passes (56 pass, 0 fail). CI builds the merge ref with `main` `93ffc37cb`. - **New and renamed tests, checked by name in the CI logs at `4f7d0baf0`.** These pass in PostgreSQL Domain / PostgreSQL Tests ([117033725006](https://github.com/block/buzz/actions/runs/38915998692/job/117033725006)): - `permanent_channel_event_commits_do_not_lock_the_channel_row` - `migration_schema_ttl_retirement_drops_every_partition_trigger_and_keeps_checksums` - `migration_schema_push_retirement_preserves_ttl_and_published_checksums` - `event_insert_ttl_refresh_handles_permanent_ephemeral_duplicate_and_activation_race` - `{desired,migration_schema}_event_follow_up_contract_{dual,app_only}` (four tests) `embedded_migrator_contains_consolidated_initial_schema` passes in Rust / Unit Tests ([117024125423](https://github.com/block/buzz/actions/runs/38915998692/job/117024125423)). No `*_push_retired` test runs anywhere. - **Blox gates at `678927730`** (private PG16 + Redis, on a workstation shared with other sessions): - `cargo fmt --check`, workspace `clippy -D warnings` and `buzz-db --lib` all pass. - The `buzz-admin`, `buzz-audit`, `buzz-deletion`, `buzz-push-gateway`, `buzz-search` and `buzz-workflow` Postgres packages pass. - `buzz-db`: 517 of 519 tests passed. The 2 that hung, both `writer_pool_*` tests, were killed. `buzz-relay`: 413 of 426 passed. The 13 failures are covered below. - **Uncontended reruns at `4f7d0baf0`** on a fresh, idle workstation (PG16.15, `competing=0` for every arm): - **`writer_pool_rejects_non_read_committed_database_default` predates this PR.** It timed out 3/3 at head and 3/3 at `main` `93ffc37cb`, with a 300s timeout per run. - **`writer_pool_metrics_stop_after_session_timeout_setup_failure` also predates this PR.** It timed out 2/3 at head and 2/3 at `main`, and passed in the remaining run on each side. - Both tests pass in CI. The gate accepts them as a named exception for this PR, as a pre-existing hang on the PG16 test environment. - **`buzz-db`:** 518 of 519 tests passed. The single failure is `writer_pool_rejects_non_read_committed_database_default`, killed after it hung (the exception above). - **`buzz-relay`: still open.** 413 of 426 tests passed. The same 13 `handlers::group_state_postgres_tests` failed as on the shared box, each panicking at once in its fixture's `ensure community` step with `Sqlx(PoolTimedOut)`. Contention does not explain them. The fixture's community setup is in `buzz-db`, which this PR changes. A head/`main` comparison of the full `buzz-relay` package on the same idle box is running to find out whether this predates the PR. All 13 pass in CI at this head. - **E2E (Gimli, Blox)** at `050e593af`. `050e593af..678927730` changes only tests and docs; the migration, schema and runtime code are byte-identical. Gimli's audited verdict: **EXERCISED AND WORKS**, for DB and relay runtime only. Gimli ran it at `050e593af`; this head's runtime is byte-identical but was not freshly executed. - **Upgrade:** a real relay startup took the schema from 0064 to 0065. That removed the TTL function and the trigger from the parent table and from all 8 partitions. Checksums for migrations 1–64 are unchanged. - **Lock-timeout rollback:** with a real lock held, the migration failed with `55P03` at 5.0007s and the schema rolled back fully to 64. Once the lock was released, a retry applied 65. - **Fresh desired schema:** after pgschema plus reconcile, the retired objects are absent. - **Real signed kind-9 sends:** on both schema paths, these refreshed ephemeral-channel deadlines to about +600s. Permanent channels stayed NULL. - **Raw SQL insert:** at 64 it refreshed the deadline. After 65 it left the deadline unchanged, on both the upgraded and the fresh schema. - **Not covered:** native or device testing, and deploy. - **Review:** Legolas found 0 P0 and 0 P1. His three optional P2s are fixed in `678927730`. - **No human app test:** this is a DB-only change. - **Migration numbering:** #8237 and #8372 also add a `0065_*` migration. Whichever PR merges second must renumber. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
dff0d16b2a |
docs(nips): define agent-key assertions in NIP-FA and NIP-FI (#8384)
Defines how an enterprise identity adapter issues assertions for agent keys and how a NIP-FI relay admits them. - **NIP-FA:** adapters issue agent-key assertions only under an owner-authorized grant, created through the owner's own authentication; revoking the owner invalidates every earlier grant. The "agent keys" out-of-scope bullet is removed. - **NIP-FI:** new `Agent keys` subsection under the assertion contract (`nostr_owner_pubkey` claim, the verifier returns the owner, owner recording for the session or request, ban/deny checks on both keys, owner-derived authorization), `agent_assertions` discovery field, owner deny-set check in WebSocket and HTTP admission, owner-matching disconnect close scan, and the updated out-of-scope delegation lines. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
7852b1670c |
fix(search): normalize profile name tiers in one place (#8334)
🤖 ## Summary Profile name search (`POST /profiles/search`, added in #8301) sorts results into tiers: an exact name first, then a name that starts with the query, then a name with a later word that starts with it. This PR stacks on #8301 and fixes two review items. **Tier normalization in one place.** Before this change, the query was lower-cased in Rust (`to_lowercase`), and the names were lower-cased in SQL (`lower()`). These can disagree. Under a `C` collation, PostgreSQL's `lower()` changes only ASCII letters, while Rust also lower-cases letters like `É`. So a search for `Émile` did not put the profile named `Émile` in the exact tier. Now the query and both name fields go through the same SQL expression: lower-case, collapse whitespace runs to one space, then trim. The query is bound once and both sides compare equal by construction. The trim also moved after the collapse, so surrounding whitespace that the regex turns into a space is now trimmed too. **Ranking wording.** The doc comment and `docs/profile-search.md` said "ties keep the NIP-50 order". That was wrong. Ties are ordered by `ts_rank_cd` over the two name fields only (not the whole profile, as in NIP-50), then newest first, then event id. Out of scope, not changed here: matching names that contain escaped Unicode or newlines, and any new name index. ## Related issue Review on #8301: https://github.com/block/buzz/pull/8301#pullrequestreview-5524491230 ## Testing - New case in `profile_names_match_name_words_only`: profiles `Émile` and a newer `Émilee`, query `Émile`. The exact match must come first. - Ran the Postgres integration tests against a `postgres:17-alpine` database created with `--locale=C`: - this branch: `profile_names_match_name_words_only` and `profile_names_survive_unparsable_content` pass; - #8301 code with the new case: `profile_names_match_name_words_only` fails (`Émilee` sorts before `Émile`). - Nine other tests in that file fail locally on both this branch and #8301 because they reuse one database (duplicate community hosts). CI gives each test its own database. - `cargo fmt --check`, `cargo clippy -p buzz-search --all-targets -D warnings`, and `cargo test -p buzz-search`: clean. - The pre-push hook's `rust-tests` and `desktop-tauri-checks` passed. Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
f148c0f22c |
feat(relay): add profile search over name and display_name only (#8301)
🤖 > **Review as a pair with block/buzz-app#1084** (the app change that uses this endpoint). Either can merge first: the app falls back to the old search on relays without this endpoint. ## Summary Searching for a person by name over the relay's general search returns profiles that only *mention* the text somewhere else: in an avatar image embedded as text, in the bio, or in the verification address. A search for "wes" could fill every result page with profiles whose avatar data happens to contain "Wes", and never return the person actually named Wes. This adds a relay endpoint, `POST /profiles/search`, that looks only at a profile's **name** and **display name**. Every other profile field is ignored. **Matching and paging are the same as the relay's existing full-text profile search** (NIP-50 search over kind:0 profiles with `search_mode: "prefix"`). A client can switch to this endpoint without a visible change in which people a query finds: - Each query word must match a whole word of the name; the last word can be a prefix. Words can be in any order, so "johnson wes" finds "Wes Johnson". Case is ignored. - Pages work the same way: up to 100 results per page, up to page 1000. - The results are exactly the old results minus profiles that match only outside their names. The order is better for people search: exact name first, then names that start with the query, then names with a later word that starts with it, then other matches. Ties are ordered by `ts_rank_cd` over the two name fields only (not the whole profile, as in NIP-50), then newest first, then event id. The response is a JSON array of the original signed profile events (Nostr kind:0), so clients can verify them as they do today. The buzz-app client change is block/buzz-app#1084. ## Details - Request: `{"query": "wes", "limit": 20, "page": 1}`. `query` is 1–100 characters after trimming; `limit` is 1–100 (default 20); `page` is 1–1000 (default 1); unknown fields are rejected with 400. - The search reuses the full-text search's own query builder (`buzz-search`), so the word rule cannot drift from it. The existing full-text index on events stays the access path; the name check runs only on rows the index already matched. There is no new index. One small migration (`0061`) adds the function `profile_name_fields()`; it reads no table data, so it runs instantly. - Only current profiles count: a replaced profile stops matching. Profiles that are not a JSON object, or whose name fields are not strings, never match. Content that PostgreSQL cannot read as `jsonb` (for example a `\^@` escape or very deep nesting) also never matches, and it cannot make the search fail for everyone else. - The handler loads the matched events through the normal routed event read and applies the same reader visibility check as other reads, in search order. - Access uses the same checks as the relay's GIF search endpoint: a signed HTTP request (NIP-98), identity admission (NIP-FI), rate limit, replay protection, and relay membership. That code moved out of `gifs.rs` into one shared helper in `bridge.rs`; GIF behavior is unchanged. - The route is added to the existing protected-route and NIP-FI shadow/enforce route tables. Docs: `docs/profile-search.md`. - Two test-only lines get `sadscan:disable` comments, as other fixtures already do: an existing public ES256 test key in `router.rs`, and the local Postgres test URL in the search integration test. ### Related issue None found. ### Testing - Postgres test for the search query (`profile_names_match_name_words_only`): only name fields match (avatar data URL, `about`, `nip05`, `banner`, non-string names and non-object content are excluded); words in any order and across both name fields; whole words only for completed words; ranking order; pages split without overlap or gaps; replaced profile; other community; `%` literal. - Postgres end-to-end test through the router: an admitted member gets only the name-matching signed event, and the returned event still verifies. - Request validation unit tests; existing GIF admission tests and router route-table tests pass. - Local end-to-end run with buzz-app#1084 against a relay built from this head, with about 195 seeded profiles. For every query tried, the endpoint's results were a subset of the full-text search's results; the only profiles missing were avatar-only matches. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
fea02d174d |
fix(ios): fetch profiles and avatars in notification extension (#8369)
A notification can arrive from someone whose profile the main app has never loaded. The iOS notification extension now fetches and caches sender profiles, channel details, and avatars itself. The main app supplies account configuration and Keychain credentials only. After resolving the message, the extension allows 1.5 seconds for sender and channel details. It uses available stale data when a refresh is slow or fails, delivers once, and lets late results update its private cache. Profiles, channel details, and avatar refresh checks share a one-hour freshness threshold. A verified avatar removal clears the previous image. Account removal prevents further use of that sign-in's cached data. Profile, channel metadata, and membership use separate bounded downloads, so an oversized roster cannot discard the smaller records. Sender enrichment can finish while channel requests are pending. A successful partial channel refresh atomically removes its old counterpart, preventing a mixed group identity or recipient count. Group-specific notification presentation still requires a verified recipient count. The [specification](https://github.com/block/buzz/blob/native-notification-enrichment/docs/native-notification-enrichment.md) describes the display, caching, and account-removal behavior. General persistent image caching in Flutter remains separate. ### Validation [Hosted CI](https://github.com/block/buzz/actions/runs/38780108487) passed on the final commit, including all Flutter shards, Android, the full iOS release and simulator builds, and native controls/push-navigation tests. The [current-commit security review](https://github.com/block/buzz/pull/8369#issuecomment-6197508385) found no issues. The native package passed 159 tests, including new coverage for cache freshness, avatar replacement and removal, sign-in isolation, concurrent storage, late results, single delivery, bounded downloads, and SVG validation. A regression test exposed identical HTTP authentication event IDs for requests in the same second; each request now includes a unique nonce, matching the Dart and CLI clients. Full local CI passed before that small authentication fix. The complete native suite and signed device build passed again on the latest commit, including response-size limits, overlapping avatar downloads, canonical relay-origin matching, independent profile/channel refreshes, account-key changes before and during requests, and a single initial snapshot read across 64 accounts, signed IPv6 avatar authorities, and partial channel refreshes. Focused Flutter regression tests passed 198 tests with three configuration-specific skips; seven avatar widget tests also passed. Flutter analysis is clean, and the notification extension builds for the simulator. On an iPhone SE, the notification displayed the sender name, avatar, channel, and message before the updated main app was opened. Repeat notifications used a byte-identical private cache. Three notifications delivered together displayed the avatar after the authentication fix. The next notification also displayed correctly after terminating the extension process. The existing generic-push design can select the same newest unread message for several pushes; this change does not alter message selection.  Authenticated HTTP avatar fetching and persistent HTTP cache reuse also passed on the SE. The first notification downloaded a relay thumbnail. After terminating the extension and expiring its separate avatar entry, the next notification displayed the avatar and refreshed that entry without another media request in the relay logs. The main app remained unopened throughout. The original app build and test profile were restored afterward. Automated tests cover slow and failed requests, stale images, removal, and account revocation; those cases have not all been exercised on the phone. ### Before merge - [x] Verify authenticated HTTP avatar fetching and cache reuse on iOS against a working relay media service. <!-- buzz-review-completed --> --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Will Pfleger
|
9092950620 |
docs(nip-ap): narrow relay support and allow offline start (#8368)
🤖 Two corrections to NIP-AP's agent and team config section. **Relay support.** A client now needs only NIP-CL system channels and labels plus NIP-AR artifacts before storing config on the relay. The four other required advertisements are dropped: read-only guests, a retention exemption, complete-history artifact sync, and telling a never-used artifact address apart from a fully redacted one. No relay advertises them, so a conforming client could never store config on the relay. The retention exemption stays in "History" as the relay's obligation. Without a never-used signal an empty lookup is still `unavailable`, never `absent`, so clients fail closed. **Offline start.** When the relay can't be reached (a transport failure, not a read that returned an answer), a client may start the agent from the last revision it confirmed as present. It still must block if the relay's most recent answer showed that the config is invalid, deleted, redacted, archived, or no longer accessible, and it never uses another source. **Consistency.** "Editing" no longer assumes every relay's guest role is read-only. "Artifact state" points its no-substitution rule at the new offline exception. Related: #8295 --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
6766174673 |
chore(push): retire push notification database trigger (#8352)
Buzz must schedule mobile push notifications when it saves an eligible message. The relay, the server that receives and stores messages, now does this in application code. An older database trigger still exists. This trigger is a rule that runs automatically when a message is saved. This PR removes the old trigger and its supporting database function. The relay remains responsible for scheduling notifications. Migration 0063 makes this change in existing databases; updated setup scripts apply it to new databases. The change does not turn on push notifications. Production migration and deployment need separate approval. Before deployment, verify the release images and all pending database migrations. If push needs to be stopped after release, disable it with the feature flag and keep the qualified application version and database schema. Do not downgrade the application image. ### Validation - Local checks passed: 519 isolated database integration tests, 166 database unit tests, 17 source-policy tests, and the relay push-runtime tests. - Tests verify that the migration removes the old trigger, preserves earlier migration records, and lets the application schedule push work correctly. A concurrency test exercises the real application path; deliberately removing its lock makes the test fail. - [Current-head CI passed on retry](https://github.com/block/buzz/actions/runs/38752849383/attempts/2). The first attempt had one database-connection timeout during test setup; the failed jobs passed when rerun on the unchanged commit. - Initial local CI and push checks hit unrelated ACP test failures, which passed when run separately. Only the Rust-test push hook was skipped under the repository's local-check exception; other applicable hooks passed. ### Before merge - [x] Migration 0063 is available and the push trigger remains on [main at |
||
|
Tom Brow
|
5397ea49e2 |
fix(push): verify extended App Attest proofs with a focused owned module (#8329)
## Summary Apple SDK 27.1 device captures include authenticator extension data that the current verifier rejects. Replace the general appattest dependency with a private gateway module that verifies the complete original bytes, frames enrollment and assertions separately, and accepts well-formed optional extensions without SDK-specific length or flag allowlists. The module retains established signature, P-256, CBOR, and certificate-chain libraries. App identity, exact enrollment environment, credential/key binding, nonce, challenge, and increasing-counter checks remain enforced. Bounded parsing rejects ambiguous duplicate keys, truncation, trailing data, and oversized inputs. Fixture helpers are test-only. Adapted code retains both upstream MIT notices and source provenance. Category/version properties remain authenticated observations under the chosen compatibility-release policy. This does not implement Apple's additional distribution-category/version enforcement; introducing those restrictions requires a separately defined deployment policy. The [verification contract](https://github.com/block/buzz/blob/app-attest-owned-verifier/docs/app-attest-compatibility.md) records this and other limits. Related: https://github.com/block/buzz/pull/8020 addresses assertion extension handling. This change also covers enrollment extensions, credential-key parsing, and certificate validation. ### Validation - Gateway suites: 77 default tests and 79 personal-development tests passed, plus startup policy and Clippy. - Added signed enrollment/certificate, generated CBOR mutation, maximum-counter, and PostgreSQL race/replay coverage. All 12 isolated Postgres tests passed. Apple's historical fixture is tested structurally, not accepted as a current enrollment. Positive synthetic leaf certificates are generated at test time. - Protected SDK 26.5/27.1 enrollments and assertions pass the verifier, including the same key across SDK upgrades; development enrollment rejects under production policy. Captures are personal-development identities, not supported production-identity evidence. - Full local CI passed Rust tests and reached 6,828 passing desktop JS tests before an unchanged 10 ms focus-timer test failed; its five-test file passed in isolation. The pre-push native suite had 3,421 passes and an unchanged 300 ms child-PID test failure, also passing in isolation. Prior ACP timing flakes and these unrelated hook exceptions do not waive required CI. - Windows CI exposed a missing system OpenSSL installation for the test fixture generator. The test dependency now builds bundled OpenSSL; all 30 focused App Attest tests pass locally. [Current-head CI](https://github.com/block/buzz/actions/runs/38636062347) passed, including Windows Rust, PostgreSQL, and desktop integration tests. - [Prior-head Codex review](https://github.com/block/buzz/pull/8329#issuecomment-6177515598) found no major issues. Carl subsequently identified a dropped credential-length workaround; fixed in [ |
||
|
Logan Johnson
|
d28f6b2f70 |
feat(relay): let channel-state filters opt in to system channels (#8182)
🤖 ## Summary After part 2, a client can read a system channel only if it already knows the ID. This PR lets a client find system channels on purpose, with an ordinary Nostr filter and no new filter key. A filter on the channel-state kinds (39000–39003) now also returns the system channels the reader can read when it has either: - `#t:["system"]`, the channel type tag that every channel-state event already carries; or - any `#P`, the creator tag from #8174. Asking for the channels of a named creator returns a small result, so it does not fill generic channel lists. The filter then matches as usual. Existing clients send neither on these kinds, so they still never see a system channel. | Reader wants | Filter | |---|---| | My system channels | `{kinds:[39002], #p:[me], #t:["system"]}`, then their kind:39000 by `#d` | | Open system channels | `{kinds:[39000], #t:["system"]}` | | One creator's labeled channel, system or not | `{kinds:[39000], #P:[creator], #t:[label]}` | Private system channels stay hidden from non-members, as for any private channel. Values in one tag filter combine with OR, so `#t:["system", label]` means `system` **or** the label; use `#P` plus the label instead. Part 3 of 3, stacked on part 2. ### Related issue None found. ### Details - The opt-in is decided per filter in `handlers/system_channels.rs` and applies on REQ, COUNT and the HTTP bridge. It applies only when every kind in the filter is 39000–39003. - A label alone is not an opt-in. - `NOSTR.md` documents the system channel query rules. - **Spec.** NIP-CL says how a filter asks for system channels. - The relay information document (NIP-11) now lists `"nip-cl"` in `supported_extensions`. Clients use this to tell if a relay supports labels and system channels, and refuse those calls on a relay that does not. NIP-CL documents this. ### Testing - End-to-end relay test: `#t:["system"]` finds both system channels for a member and only the open one for a non-member; 39002 `#p:[me]` with `#t:["system"]` lists the member's system channels; `#P` includes a creator's system channels; creator plus label returns exactly one channel with `limit: 1` and excludes another creator's channel with the same label; a label alone does not return the system channel; COUNT and HTTP `/query` and `/count` agree. - The test fails when the opt-in is disabled. - Unit tests for which filters opt in. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels ← this PR --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
cdf03f618a |
feat(relay): leave system channels out of reads that do not name them (#8181)
🤖 ## Summary After part 1, a system channel is still returned by every query that returns ordinary channels, so existing clients would show it as a chat. This PR makes the relay leave system channels out of every read that does not name them. Clients do not need to filter, so clients that know nothing about system channels stay correct. | Query | Before | After | |---|---|---| | A query that does not name a channel (no `#h`), COUNT, full-text search, the HTTP `/query` and `/count` routes, artifact queries (`{"artifact": ...}`), the sidebar | Includes system channels | Leaves them out | | "Channels I belong to" (kind:39002 `#p:[me]`) | Includes them | Leaves them out | | A filter that names the channel with `#h`, or with `#d` on kinds 39000–39003 | Ordinary read rules | Unchanged: members can read it, and anyone can read it if it is open | Push notifications and "you were added to a channel" notifications (kinds 44100/44101) skip system channels, because a client that received one would show the channel. A client that is given a system channel's ID, for example in a pasted link, can still open it. Naming the ID is the opt-in. This keeps system channels ordinary NIP-29 groups. Part 2 of 3, stacked on part 1. ### Related issue None found. ### Details - The rule lives in one place: `get_accessible_channel_ids`, the set of channels a query may read when it does not name one. It is the union of the reader's memberships and every open channel; both halves now exclude `channel_type = 'system'`. Because the filter is in SQL before `LIMIT`, pages stay full. - A filter that names a system channel adds it to that filter's scope only (`handlers/system_channels.rs`). A sibling filter in the same request keeps the implicit set, so naming one system channel cannot widen another filter. - Thread windows and the workflows API name their channel, so they follow the same named-channel rule. - Artifact queries have their own read check in SQL. It reads a system channel only when the query's `#h` names it. - When a request re-checks membership for a named channel that its first lookup missed, a confirmed system channel joins only the filters that name it. Before, a reader who joined a system channel during the request could have it added to a sibling filter's scope. - The system-channel lookup runs only when a filter names a channel outside the implicit set, so ordinary queries do not pay for it. - **Spec.** NIP-CL lists the reads that leave system channels out, and says when a filter names one. `AGENTS.md` notes this for `get_channels`. ### Testing - End-to-end relay test (Postgres and Redis) with a private system channel, an open system channel and an ordinary channel, as a member and as a non-member: REQ without `#h`, NIP-50 search, COUNT, HTTP `/query`, HTTP search, HTTP `/count`, the channel list, and 39002 `#p:[me]` return only the ordinary channel. Named reads by `#h` and `#d` follow member-or-open on REQ, COUNT, search and HTTP. - Relay test: adding a member to a system channel sends no kind:44100; an ordinary channel still does. - Both tests fail when the SQL exclusion or the named-channel scope is removed. - Database test: an artifact in an open system channel is missing from `{"artifact":"current"}` and present with `#h:[<system channel>]`, for a member and a non-member. It fails without the channel-type condition. - Relay test: the membership re-check confirms a system channel that the system-channel lookup missed. The channel goes to the filter that names it, and a sibling filter's scope does not include it. It fails without the fix. - Unit tests for the per-filter scope. They now run in `just test-unit` and `scripts/run-tests.sh`. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them ← this PR 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
349f566e93 |
docs(nip-ap): store agent and team config as NIP-AR artifacts (#8295)
🤖 Stores agent and team config as [NIP-AR](https://github.com/block/buzz/blob/main/docs/nips/NIP-AR.md) artifacts in private [NIP-CL](https://github.com/block/buzz/blob/main/docs/nips/NIP-CL.md) system channels, replacing the owner-encrypted `kind:44300` instruction versions. - **Config channels.** Each agent has an agent config channel holding `buzz.agent.settings` and `buzz.agent.instructions`. Each team has a team config channel holding one `buzz.team` artifact with the team name, team instructions and a `team` tag carrying the exact team id, and no member list. The owner creates both kinds as private system channels with no TTL. Each carries a family label (`agent-config` / `team-config`) and a subject label: the agent pubkey, or `team-` plus a hash of the team id, which clients check against the `team` tag. Discovery queries by creator and subject label. Agents find their teams through a `39002` query that opts in to system channels. The oldest matching channel wins, and channels that aren't selected are never deleted. - **Ownership and teams.** An agent's owner is its relay-verified NIP-OA owner, and a team's owner is its channel's creator. Changing an owner isn't supported. A team's members are the owner's agents in the team config channel. Changes to who is on a team follow the channel's role rules, including add policies, owner/admin-only role changes, and last-owner protection against leaving, removal and demotion. Guests are read-only. Every non-guest member can edit config, including humans and agents that aren't on the team. Edits keep unrecognised JSON fields and tags, and a client whose create collides fetches and edits the existing artifact instead of picking a new `d`. - **Artifact states.** A config artifact is present, invalid, absent, deleted, or unavailable. An artifact is absent only when the relay confirms its address was never used. Redacting every revision does not make an address unused. Only an absent artifact lets a client fall back to local instructions or a definition prompt. Invalid, deleted, and unavailable states block the start. - **History and launch.** History is kept until the owner purges it or a moderator removes a revision. Config channels must be exempt from age-based retention, and keeping only the latest revision is not allowed. A revert is an `op=update` with a full snapshot and keeps the team's name. The instruction resolution order and per-team launch sections are unchanged. - **Limits and lifecycle.** Config content is at most 131,072 bytes after encoding (settings: 65,535), and the signed event must fit the relay's limits. Content is never truncated. Settings exclude credentials, local paths and device state. An unsupported setting is an error, never a silent default. Deleting an agent archives its channel and blocks new launches until it's unarchived. Deleting a team archives its channel. - **Relay prerequisites.** Clients keep config local unless the relay supports system channels, artifacts, read-only guests, the retention exemption, sync with complete history, and telling an artifact address that has never been used apart from one whose revisions have all been redacted. - **Removed.** `kind:44300` and its relay rules, deletion rules and owner-encryption text. `30175`, `30177`, `30178` and the team id format are unchanged. Depends on the NIP-CL label and system-channel stack (#8176, #8180–#8182). --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
71d7aa0436 |
feat(relay): add the system channel type (#8180)
🤖 ## Summary Buzz uses channel membership to decide who can read what, and every channel a person belongs to shows up in their client as a chat. So there is no way to make a group that exists only for access control, for example "only this agent and its owner can read the agent's config". Such a channel would appear in both people's sidebars, search results and unread counts. This stack adds a **system channel**: an ordinary channel that queries leave out unless they ask for it. This first PR adds the type itself. - `kind:9007` (create channel) with `["channel_type", "system"]` creates a system channel. - It is private unless the creator sends `["visibility", "open"]`. - While it is private, its channel metadata event (kind:39000) carries the NIP-29 `hidden` marker, which means "hide this group's metadata from non-members". Until now the relay set `hidden` only for DMs. On its own this PR does not yet hide system channels from queries. Part 2 does that. Part 1 of 3, stacked on #8176 (the channel labels and creator lookup stack, which this feature uses). ### Related issue None found. ### Details - Migration 0062 adds `system` to the `channel_type` Postgres enum. No new columns. - `buzz_core::channel::ChannelType::System`, and `system` is now a reserved label name (labels cannot equal a channel type name). - `buzz-admin` reconcile writes `hidden` with the same rule as the relay. Usage metrics count system channels. - **Rollback.** A relay older than this change cannot read `channel_type = 'system'`. Do not roll back past it while system channels exist. - **Spec.** NIP-CL gets a "System channels" section. `ARCHITECTURE.md` lists the `System` type. ### Testing - Relay test: a system channel is private and `hidden` by default; opening it removes `hidden`; its first `t` tag is `system`. - Migration embed and usage-metrics tests. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type ← this PR 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
a3909acc67 |
feat(relay): match label and creator filters before the page limit (#8176)
🤖 ## Summary After parts 1 and 2, a client can ask for "channels with label X" or "channels created by key K". But the relay applied these tag filters after it read one page of the newest channel events. So a request such as `{kinds:[39000], #t:["workspace"], limit:10}` could return fewer than 10 channels, or none, while matching channels existed. This PR moves the label and creator match into the database query, before the page limit. Part 3 of 3, stacked on part 2. ### Related issue None found. ### Details - Applies only when every kind in the filter is a relay-signed channel-state kind (39000–39003). Other kinds keep today's behavior. - The database match uses JSONB containment on `tags`, which is GIN-indexed. Containment can match a little too much (for example, a tag that contains both strings in another position), never too little. So the relay still checks each returned event with the normal Nostr filter rules, and `COUNT` requests keep that check too. - The same match is applied to the query path and the count path. - **Spec.** NIP-CL now says that `#t` and `#P` match before `limit`. - **Choosing one channel.** The relay does not make labels unique, so two creates for one label can both succeed, for example when two processes of one agent start at the same time. NIP-CL gets a "Choosing one channel" section: a client that needs one channel per creator and label looks it up with `#P` + `#t`, creates it only if none exists, and otherwise uses the oldest one (the lowest `created_at` tag, then the lowest channel ID). Every reader that follows this rule picks the same channel, even after an edit. ### Testing - Relay test: many channels, one match, small `limit`. The request returns the match for both `#t` and `#P`. This test fails without the change. - Unit test: `#t` and `#P` reach the database query only for channel-state kinds. - Relay test: one creator sends two creates for the same label at once. Both channels exist, a `#P` + `#t` lookup finds both, and the oldest-channel rule picks the same one in either order, even after the older channel's metadata is re-published by an edit. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel ← this PR **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
2f79374f48 |
feat(relay): channel labels (#8175)
🤖 ## Summary A plugin or agent often needs to tag its own channels and then find them again, for example "my attention-config channel" or "all channels of this workspace". Today a channel has no labels: the relay ignores any `t` tags in create or edit requests. This PR adds channel labels. - A create request can carry up to 8 labels as `["t", "<label>"]` tags. - An edit request with `t` tags replaces the label set. `["t", ""]` alone clears it, the same convention as `["ttl", ""]`. Only owners and admins can change labels, as for the name. - The relay publishes each label as a `t` tag on the channel's state events, after the channel type tag. Part 2 of 3, stacked on part 1. ### Related issue None found. ### Details - **Rules.** A label uses 1–64 characters from `a-z`, `0-9`, `.`, `:` and `-`. The relay refuses a label equal to a channel type name (`stream`, `forum`, `dm`, `workflow`, `system`), so a label cannot make a channel look like another type. - **No trust from labels.** Any admin can set a label. Only the relay-written creator tag (part 1) says who made a channel. - **Storage.** New migration `0061_channel_labels.sql` adds `channels.labels TEXT[] NOT NULL DEFAULT '{}'`. Channels without labels look and behave as before. - **Readers.** The CLI and `buzz-acp` read the channel type from the first `t` tag. Before this PR they used the last one, which would now be a label. - **Spec.** NIP-CL gets a "Channel labels" section with these rules. ### Testing - Relay tests: labels at creation appear after the type tag, edit replaces and clears them, invalid labels are refused, and members who are not admins cannot change them. - Unit tests for the label rules: allowed characters, length, reserved type names, the 8-label cap and the clear marker. - `buzz-acp` test: the channel type comes from the first `t` tag, not a label. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels ← this PR 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
8bb49a1b6a |
feat(relay): publish channel type and creator on every channel-state event (#8174)
🤖 ## Summary Clients cannot tell who created a channel, and only the channel-metadata event says what type of channel it is. This PR makes every relay-signed channel-state event carry both facts, so a client can find "the channel that this key created" with one ordinary Nostr filter. - Channel metadata, admin list and member list events (kinds 39000, 39001 and 39002) now all carry the channel type as their first `t` tag, then the creator as a `P` tag. - The creator comes from the relay's own record of who signed the channel's create request. No client can set it, and it does not change when ownership moves. - Channel metadata (kind:39000) also carries `["created_at", <unix seconds>]`, the time the channel was created. The event's own `created_at` changes on every edit, so before this a client could not tell which of two channels is older. This is part 1 of a 3-part stack (channel labels and creator lookup). The system channels stack (#8180–#8182) builds on it. ### Related issue None found. ### Details - **Why `P`.** Several Nostr specs (NIP-22, NIP-34, NIP-72) use uppercase `P` for the author of the root object, and lowercase `p` for people involved. Lowercase `p` is already used on these events for members and DM participants, and a filter matches tag values, not their meaning, so `#p` would mix them. - **Trust.** A client that trusts a channel because of its creator must also check that the relay key signed the event. - **Order.** The type `t` tag comes first, so clients that read the first `t` tag still get the type after part 2 adds labels as more `t` tags. - **Creation time.** The relay reads it from `channels.created_at`, so an edit does not change it. #8176 uses it to pick one channel when two creates race. - `buzz-admin reconcile` writes the same tags when it rebuilds these events. `NOSTR.md` documents the tags. - **Spec.** New `docs/nips/NIP-CL.md` defines these tags, the trust rule and lookup by creator. Later PRs in both stacks add their parts to it. `NOSTR.md` links to it. ### Testing - Relay tests in `group_state_postgres_tests.rs`: each of the three kinds carries the type first, then the creator; the creator stays the same after an ownership transfer; kind:39000 carries the stored creation time, and still does after an edit. - Unit test for the shared tag builder that both the relay and `buzz-admin reconcile` use. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time ← this PR 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
43b88c3a76 |
docs(nips): allow NIP-FA session token on the adapter origin (#8198)
🤖 NIP-FA limited the adapter session token to the NIP's own endpoints, which stopped a client from reusing the same login for the adapter operator's other APIs on that host. This changes the rule to the adapter's origin (the scheme, host, and port of `{adapter_base}`). The token still never goes to a relay or any other origin, and the rest of the security bullet is unchanged. Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Alia <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@buzz.block.builderlab.xyz> |
||
|
Ian Oberst
|
85b7331784 |
fix(relay): partition audit observability follow-ups (#7994)
> [!IMPORTANT]
> **The alert rules changed in c3a314e71.** If you reviewed the alert
guidance earlier, please look again.
> - **The two range-based rules now apply only to live pods.** A range
selector such as `[30m]` keeps reading a terminated pod's samples until
they leave the window. So `BuzzPartitionAuditNotRunning` could fire for
about 10 minutes after normal pod turnover, and
`BuzzPartitionAuditFailing` could keep firing for up to an hour after
the pod was gone. Each rule now adds `and <counter>`, which requires the
counter to be exported right now:
> ```yaml
> expr: |
> sum without (outcome) (increase(buzz_partition_audit_runs_total[30m]))
== 0
> and sum without (outcome) (buzz_partition_audit_runs_total)
> ```
> ```yaml
> expr: |
> increase(buzz_partition_audit_failures_total[1h]) >= 2
> and buzz_partition_audit_failures_total
> ```
> - **The audit counters start at 0.** When the relay starts, it exports
every `buzz_partition_audit_runs_total{table, outcome}` series and
`buzz_partition_audit_failures_total` at 0, before the first audit.
`increase()` ignores a new series' first sample. Before this change, it
could miss the first periodic failure, or read a table's first
`degraded` run as a stopped loop. This change is also what makes the
`and <counter>` gate work: the counters now exist for a relay's whole
life.
>
> The doc's old claim that pod turnover can't leave an alert firing was
true only of the instant-selector stale rule. The doc now explains the
difference.
## Why
These are the observability follow-ups deferred from #6515 ("fix(db):
audit partition catalog before creation"):
- **Timestamp-age alerts go silent** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4127212141)).
Partition gauges are set only when a table audits successfully, and the
exporter evicts idle gauges after 2,700 s by default. After sustained
failures, `buzz_partition_audit_last_success_timestamp_seconds`
disappears. A table that has never audited successfully never has it. An
alert on `time() - last_success` then has no input.
- **Startup fallback failures are not counted** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4127212158)).
When the startup fallback audit fails, it logs an error but does not
increment `buzz_partition_audit_failures_total`. Periodic failures do.
- **Repaired environments are always `degraded`**.
`PartitionTableAudit::degraded()` counts every legacy leaf, even one
wholly in the past. The `bb-block-staging` audit of the repaired catalog
reports `degraded` only because of `events_p2026_07_08_legacy`
`[2026-07-01, 2026-09-01)`. With this rule, every repaired environment
stays `degraded`, so nobody can alert on the outcome.
- **Partition config tests can see a poisoned env** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4137416297)).
Two partition config tests take only `ENV_MUTEX`. A NIP-FI test running
in the same process can make them see its deliberately invalid
environment.
Partition health is still diagnostic only, through
`/_status.partition_catalog` and metrics. Nothing here affects
`/_readiness`. Handling concurrent creation (#6515 item 4) is out of
scope and stays with the catch-all advancement work.
## What
- **Success timestamp survives failures.** Each failed table audit
refreshes `buzz_partition_audit_last_success_timestamp_seconds` with
`increment(0.0)`, which leaves the value unchanged. It still resets the
exporter's idle tracking, the same mechanism as
`refresh_legacy_active_gauge_recency`. Sustained failures therefore keep
the last success exported and its age rising. A table that has never
succeeded exports `0`. A failed audit still never makes the last success
look fresh.
- **Alert guidance.** New `docs/partition-catalog-monitoring.md` covers
the metrics, how long each series lives, and the audit outcomes. It
notes that `runs_total` has no `unsafe` outcome. It gives three alert
rules:
- timestamp age, which also covers never-successful tables and sustained
failures;
- `sum without (outcome)
(increase(buzz_partition_audit_runs_total[30m])) == 0` for a stopped
audit loop. Counters are never idle-evicted, and this rule fires before
the stopped loop's timestamp is evicted, so there is no false resolve;
- an aggregate failure-counter rule.
Both range-based rules also require their counter to be exported right
now (`and <counter>`), so a terminated pod's leftover samples can't
raise or hold an alert.
- **Zero counter baselines.** `try_install` exports each
`buzz_partition_audit_runs_total{table, outcome}` series (`ok`,
`degraded`, and `error` for both tables) and
`buzz_partition_audit_failures_total` at 0 before the startup audit, the
same way the auth series already start at 0. A failure in the startup
audit itself can still happen before the first scrape. The doc says so;
the stale rule covers that case.
- **Failure counter.** `buzz_partition_audit_failures_total` now counts
failed explicit audit attempts: the startup fallback audit and each
periodic audit. Both paths go through one helper,
`counted_partition_audit`. A failed `ensure_future_partitions` call,
such as a DDL error that the fallback then recovers, is not counted by
itself. A single attempt never counts twice.
- **Historical legacy leaves.** A legacy leaf is historical when its
finite upper bound is at or before the current month start. It is still
reported under `children` as `legacy_leaf`, but it no longer degrades
the table. A legacy leaf that still covers the current month or any
lookahead month stays degraded. All other degradation causes are
unchanged. The verdict now lives in `PartitionAuditReport::outcome()`,
which `buzz-admin partition-audit` uses. The JSON outcome names and exit
codes are the same as before.
- **Test hygiene.** Both partition config tests now use `env_guards()`.
## How it was tested
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay -p buzz-admin --all-targets -- -D warnings`.
- New unit tests:
- The repaired incident layout with an injected clock: `degraded` in
August 2026 and `ok` on 2026-09-01T00:00Z, when the upper bound equals
the current month start, and on 2026-09-15. The legacy leaf is still in
the serialized report. The `outcome` label on the run counter follows
the verdict.
- A table test of the legacy-leaf predicate: the upper bound before, at,
and one second after the month start; lookahead and beyond-horizon
leaves; historical leaves with trigger drift; anomalous and `DEFAULT`
children; no audited month.
- Outcome precedence (error, then unsafe, then degraded) and the
`buzz-admin` exit-code mapping.
- Startup: a failed fallback counts once, a successful fallback after a
DDL failure counts nothing, and a successful ensure never runs the
fallback. Each periodic failure counts once. Against an unreachable
database, startup counts one aggregate failure while each table still
records two `error` runs.
- On the production exporter, never-successful audits leave run counters
and a `0` success timestamp for both tables.
- On the production exporter, all six run-counter series and the failure
counter read 0 before any audit. After one failed audit, `error` reads 1
and `ok`/`degraded` stay at 0. Removing the initialization fails this
test.
- PostgreSQL lane (`scripts/postgres-test-run.sh`) with new or extended
tests:
- The repaired layout is not degraded in September. The same catalog is
degraded in August.
- With a 1 s idle timeout, a failed audit leaves the success timestamp
unchanged, and sustained failures keep it exported past the idle
timeout. When audits stop, the timestamp is evicted but the run counters
remain. The next success exports a fresh timestamp.
- Mutation check: removing the failure-path refresh fails both alert
fixtures.
- These fixtures check the emitted series. They do not evaluate the
rules themselves, including the `and <counter>` gate. `promtool` isn't
available locally or in CI, and the doc points to `promtool test rules`
for that.
- `scripts/test-postgres-test-discovery.sh`.
- `config::` and `nip_fi_config::` tests run together in one process
five times.
## Related
This PR conflicts textually with #7993 (catch-all advancement) in
`crates/buzz-db/src/store/partition.rs` and
`crates/buzz-relay/src/main.rs`. Whichever PR merges second will be
rebased onto the other.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Tom Brow
|
868e2f8f32 |
refactor(push): one app profile per gateway (#8206)
Buzz push currently selects an application profile, such as `dogfood`, during client registration. We want each gateway to serve one Apple app configured by the operator instead. This removes profile selection from the client, relay, and gateway, while allowing internal and public gateways to use their own Apple identities. This PR implements that model in the Buzz source and upstream deployment chart. Discovery, enrollment, grants, and leases no longer carry an application-profile selector. Each gateway has one configured Apple application and APNs transport. The protocol changes are incompatible with the old registration format, so relays advertise `buzz-push-v1` instead of `nip-pl`. Old clients do not discover the new push capability. Updated clients use separate Keychain records and register again. The [push notes](https://github.com/block/buzz/blob/push-capability-v1/docs/buzz-push.md) describe the experimental contract. Landing this PR is the first step. Publish the matching gateway image and upstream chart, then land the [shared chart update](https://github.com/squareup/cash-helm-charts/pull/5740) and [gateway deployment update](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/1085). Before cutover, stop old gateway instances and check legacy registration state. Migrations refuse to proceed if old gateway installations or active relay leases remain; they do not delete that state. Physical-device verification remains necessary before production activation. ### Validation Gateway and focused Flutter push tests pass, as do the Swift package, simulator build, and native notification/enrichment tests. A local relay advertises `buzz-push-v1` without `nip-pl` or `app_profiles`. The PostgreSQL suite passed 868 tests; its two migration-test failures were corrected and pass a focused rerun together with the gateway cutover guard. Chart rendering and release-contract checks pass. Desktop JavaScript tests, desktop and web builds, native desktop compilation, mobile analysis/tests, and desktop native tests pass. The full Rust unit lane and prescribed `just test` command pass, including workspace integration tests against task-isolated local services. The production overlay now requires an explicitly supplied APNs topic, with omission and rendered-value regression coverage. The corrected Swift checksum assertion passes the complete package suite: 78 XCTest tests and 42 Swift Testing tests. No physical-device enrollment or production cutover is claimed. Local merge validation: the embedded migration test and formatting pass after reserving profile migration 0060 behind rollback migration 0059. The local pre-push Rust lane failed ACP timing/steering tests (`idle_resets_on_stdout_activity`, `keepalive_resets_idle_past_deadline`, and the failed/missing-outcome steering tests). ACP and its dependency manifests are unchanged by this branch; all four pass in an isolated single-thread rerun. The documented local-check exception skips only the Rust pre-push lane for publication; required CI remains mandatory. Mobile and desktop pre-push lanes passed. Combined migration validation: after rollback 0059 landed, the main merge preserves profile migration 0060 and both embedded migration assertions. The focused migration test and formatting pass. The next local push again failed only the unchanged ACP idle/keepalive timing tests (1053 passed, 2 failed, 3 ignored); other lanes passed. Publication uses the same documented local Rust-lane exception; required CI remains mandatory. Follow-up review fix: removed the legacy profile column from the copied-lease fixture. All four event-follow-up contract tests pass on isolated PostgreSQL databases, covering desired and migrated schemas in dual and app-only modes. Formatting passes. Publication retains the prior passing local-lane evidence and uses the authorized pre-push exception for the known unrelated ACP timing failures; required CI remains mandatory. --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Tom Brow
|
5b907b838a |
fix(push): isolate push enqueue and stop workers on rollback (#8216)
Push enqueueing previously ran inside the message transaction, so a push lock or database error could block or reject a message. Configured relay writers now submit bounded best-effort work only after commit. A dedicated connection and bounded queue isolate enqueue waits and failures from message-write connections. Overload, errors, process loss, or shutdown may drop a notification; the committed message remains authoritative. `BUZZ_PUSH_ENABLED=false` disables the application producer and workers. Updated writer connections always bypass any surviving enqueue trigger. The overlap migration updates an existing function without resurrecting retired objects. Apply it before relying on isolation with a surviving trigger, and replace every old enabled process and writer connection: missing settings preserve legacy behavior during overlap. SIGTERM/Ctrl-C cancels enqueue, matcher, and delivery work before listener drain. Durable queues, claims, leases, preferences, and delivery configuration remain intact. Claims expire normally; gateway requests already accepted may still deliver. Reactivation requires an explicit operator decision about retained work. There is no automatic purge, startup interlock, or backfill of messages written while disabled. Built on merged [#8135](https://github.com/block/buzz/pull/8135), including [#8178](https://github.com/block/buzz/pull/8178). Trigger retirement [#8210](https://github.com/block/buzz/pull/8210) remains open; this change is tested with the trigger present and after removal. Operational details are in [the rollback and reactivation guide](https://github.com/block/buzz/blob/relay-push-rollback/docs/push-gateway-deployment.md#relay-rollback-and-reactivation). ### Validation [ |
||
|
Bradley Axen
|
1e80066c93 |
feat(relay): reshape read-state API to the sidebar wire format (#8220)
## Summary Changes the private read-state API (`/buzz/v1`) to the sidebar wire format agreed in #buzzin-in-secret. **The counting code is unchanged.** Requests and responses take the new shape, so the client integration can build against it now and later behavior fixes (counting rules, followed threads, another-device pings) ship without another wire change. Nothing ships against v1 yet (no buzz-app use on `main`), so this replaces version 1 directly. Supersedes #8188 (same branch content, rebased onto current `main`, plus the thread-cap and doc changes below). ``` GET /buzz/v1/me/sidebar?limit=&cursor= | ?channel_ids=a,b,… → { channels: [channel], next_cursor } channel: { channel_id, unread: bool, mentions, read_through_id|null, latest_id|null, threads: [thread] } thread: { root_id, unread: bool, mentions, read_through_id|null, latest_id } POST /buzz/v1/me/read-state { intents: [{ type: "mark_through", target: { channel_id [, root_id] }, message_id }] } → { outcomes: [applied|blocked|invalid|unknown], channels: [channel] } ``` What changed and why: - **Channel and thread positions are fully separate.** Channel `unread`/`mentions` count only top-level messages. Thread replies count only on their thread row. `mark_channel_read` and its whole-channel cut are gone. "Mark channel read" sends a channel `mark_through` at `latest_id`, plus one per thread row to clear threads too. - **Channel `latest_id` is always a top-level message.** It used to be able to point at a reply, and a channel `mark_through` rejects a reply as its anchor. It also skips a reply-marked message whose ancestry never resolved, because marking through one would fail on every retry. - **`read_through_id`:** frontiers stored only the anchor's arrival time. Migration 0058 adds `personal_read_frontiers.through_message_id`, which follows the greatest arrival, and drops the now-unused `threads_through_timestamp`. This is the only storage change. - **Counts are plain numbers.** The exact/at_least/unknown wrappers are gone. Evidence the relay can't use (unusable tags, unresolved ancestry, undecided conversation membership, past the scan cap) leaves the message out, so counts can undercount. - **POST returns the updated row** of each distinct channel it applied to, so the client doesn't refetch after every read. If those rows can't be read after the writes commit, `channels` is omitted and the outcomes still stand. The row refresh shares the 8 s intent deadline. - **Up to 25 thread rows per channel** (was 5). Five hid a mention in a sixth unread thread. 25 is the most that keeps a POST's rows for `MAX_INTENTS` (100) channels inside the 1 MiB response limit; 50 would be ~1.4 MB, a 503 that hides already-committed outcomes. A unit test builds that worst-case response and fails if the cap outgrows the limit. - **The doc separates shape from behavior.** `/buzz/v1` promises a stable shape (fields, types, status variants, what `mark_through` does). What the counts include (which messages are unread or directed, which threads are listed, the bounds) may change while v1 is pre-release, so clients replace their rows with each response instead of reconciling against local counts. - **Removed from the wire:** the explicit-contexts read (`GET /me/read-state?targets=`), `account`, channel name/type/archived/hidden (these come from Nostr metadata), `latest_message_at`/`_complete`, and the NIP-11 `max_contexts`/`max_context_messages`/`max_thread_summaries`. Deliberately deferred (behavior or additive, no shape change): - Thread counting rule (relay counts a reply if you wrote or replied to its direct parent; the client uses "posted anywhere in the thread or followed"), the workflow-owner attribution p-tag, and thread-row `mentions` counting every counted reply rather than directed ones. - `follow`/`unfollow` intents, listing followed threads that are already read, and a truncation flag for the thread list. Thread: buzz://channel/6f773b94-34d0-4004-bec9-a100b6ad17d5 (#buzz-bff, root 31020fff) ### Related issue Follows #8157. Supersedes #8188; replaces the wire parts of the closed #8155. ### Testing - `cargo test --no-fail-fast -p buzz-db -p buzz-relay` at |
||
|
Will Pfleger
|
acf1ede202 |
docs(nip-ap): add teams and kind:44300 instructions versions (#8191)
🤖 This PR adds two things to NIP-AP: a Teams section, and `kind:44300` instructions versions, an owner-private history of each agent's and each team's instructions. Today instructions are edited in place, so there's no way to see past edits, tie a behavior change to an edit, or roll back. **Teams.** A team is identified by owner, community, and team id, using the same id grammar as the `kind:30178` `d` tag. The section defines what membership means (scoped by owner and community, not channel membership, follows the team's current instructions), how team instructions combine with an agent's own at launch, and the existing rule that an agent in several teams can't start if their instructions differ. How a client stores its private team records stays client-defined, so this doesn't standardize discovering another client's private teams. **`kind:44300`.** A regular event, so every save is a new event and a subject's events are its history. Each event names exactly one subject: an agent (`p`) or a team (`t`). Content is NIP-44 encrypted to the owner as `v1:` plus the raw text, which keeps empty text representable and caps `content` at 218,548 bytes for the 128 KiB text limit. Current is the newest version, and history pages with the `until` + `before_id` cursor so versions saved in the same second are never skipped. Restore and Reset to template append a new version. Launch order for an agent's own instructions is its current version, then the linked `30175` prompt, then a definition-less `30177`. The relay keeps it author-only on every read surface, excludes it from search, and applies the subject filter before `LIMIT`. Deletion requests for these versions must carry `["k", "44300"]` and get the same author-only treatment, since they reveal version ids and deletion times. A `30178` share is a snapshot: saving a version doesn't change an existing share. **Changes to existing spec text:** - Removed the sentence saying `kind:30176`'s body is untouched "so device sync keeps its contract". The current desktop client doesn't sync teams through `30176`. - Reworded the `30178` `d` tag rationale so it no longer refers to a `30176` head. - The "30175 beats 30177" rule now says a current `44300` version beats both for an agent's instructions. - Replaced the stale search bullet. It claimed no shared-gated kind is in an FTS allowlist; search indexes those kinds, and each result passes the per-event visibility check before delivery. Relay and desktop implementation follow in separate PRs once this is approved. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Paul <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@buzz.block.builderlab.xyz> Co-authored-by: Paul <4a1bfa0013bc6d14a8d600d8bf6392efefbd2a26ac3c96c9b2a106b0d12297ca@buzz.block.builderlab.xyz> |
||
|
Kalvin C
|
8f66f7051c |
feat: raise hosted community limit to fifty (#8186)
## Summary - Raise the stock relay active ownership quota from 5 to 50 for creation and ownership transfer. Raise the absolute lifetime quota from 20 to 50 reserved hosts; completed owner deletions still permanently consume lifetime capacity to bound create/delete host squatting. - Honor the owner-list `can_create` projection across Desktop settings, create flow, and onboarding instead of imposing a client-side count. Display server `quota_used` / `quota_limit` when available; when the projection is missing, let the server's `limit_reached` decide. - Cover 49→50 and 50→51 admissions, deleted-owner lifetime exhaustion, and rendered create gates under 5/50, 0/50 exhausted, and missing projections. ## Verification - Desktop full `pnpm test` and `pnpm typecheck` pass after the 50-cap update; `cargo fmt --all -- --check` passes. Four targeted Playwright cases and `pnpm build:e2e` passed for the original dynamic-quota change at the previous commit, before changing test fixtures from 20 to 50. - `cargo test -p buzz-db --lib` cannot compile here: macOS dyld rejects the `sqlx_macros` dylib with `mis-aligned LINKEDIT string pool` on this host before the crate's tests run. CI must validate the DB package. ## Rollout / manual validation - Confirm `BUZZ_MAX_COMMUNITIES_PER_OWNER` is unset or 50 on **every** production relay replica; this per-process cached override takes precedence over the stock default. - Confirm Builderlab's list/create/transfer proxy passes the relay's `quota_used`, `quota_limit`, `can_create`, and `limit_reached` without its own 5-community gate. Its backing implementation is not in this repo and was not verified here. - Human test: with an owner at 49 live communities, create the fiftieth, verify the fifty-first returns `limit_reached`; transfer into an owner at 49 then at 50 and confirm the same boundary. Check a deleted-owner account at 0 active/50 lifetime displays the generic limit and disables creation. This manual check remains outstanding. --------- Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
5a93e862ee |
fix(chart): own the Datadog version label on every workload Pod (#7876)
## Summary
Every chart-managed Pod now carries a chart-owned
`tags.datadoghq.com/version` label derived from the deployed image,
instead of a value each wrapper restates once per workload.
Covered Pod templates:
- relay Deployment (`templates/deployment.yaml`)
- pairing relay (`templates/pairing-relay.yaml`)
- storage-accounting CronJob
(`templates/storage-accounting-cronjob.yaml`)
- operator jobs, including the deletion drain
(`templates/_operator-jobs.tpl`)
**Source order:** `image.tag`, then `image.digest`, then
`Chart.AppVersion` (`buzz.imageVersionSource`). Tag-first is deliberate:
- The rendered value matches the `sha-<commit>` pins wrappers already
carry, so upgrading to this chart does not roll any Pods.
- An image promotion only has to write `image.tag` + `image.digest`. No
per-workload label edit is needed.
Exact runtime identity stays exact in two places. The image reference is
`repository@digest` when a digest is pinned.
`BUZZ_STORAGE_SNAPSHOT_CODE_SHA` (persisted as `code_sha`) is still
digest-first.
**Label safety:** `buzz.imageVersionLabel` maps any OCI tag or digest to
a valid label value.
- Valid values already in Datadog's normal form (lowercase, no `__`)
pass through byte for byte, so `sha-<hex>` and `1.2.3-rc.4` are
unchanged. Datadog lowercases tag values and collapses `__`, so any
other value is hashed rather than allowed to report the same version as
a different revision.
- A digest keeps 63 hex characters.
- Anything else is replaced by the first 63 hex characters of its
SHA-256.
- Exactly 63 lowercase hex characters is reserved for the hashed and
digest forms, so a rendered label cannot be fed back as a tag to make
two revisions collide.
- When both tag and digest are set, the label names the tag even if they
disagree; the digest still pins what runs.
**Precedence:** a `tags.datadoghq.com/version` supplied through any
workload's `podLabels` is ignored. Every other label, including
`tags.datadoghq.com/env` and `/service`, passes through unchanged.
Overriding instead of failing means a wrapper with a stale pin is
corrected by the upgrade itself. The chart bump and the wrapper cleanup
don't have to land together.
**What got simpler:** one image identity instead of N. Wrappers delete
their per-workload version pins. Image promotion tooling no longer needs
a "mirror the tag into a label path" feature. One shared helper
(`buzz.podLabelsWithVersion`) replaces four `podLabels` passthroughs.
Chart `0.1.10 → 0.1.11`, with an `artifacthub.io/changes` entry.
Documented in the chart README ("Datadog version label") and
`docs/deployment-identity.md`.
Note: a vendor-namespaced label now renders unconditionally in an OSS
chart. This is in tension with
[VISION_SOVEREIGN.md](../blob/main/VISION_SOVEREIGN.md). The label is
inert without the operator-supplied `tags.datadoghq.com/service`/`env`,
and the chart already ships vendor-specific defaults (e.g.
`sidecar.istio.io/inject`).
### Testing
Run on a Blox (Linux) workstation at tree `41634b7ca` (helm v3.16.4,
helm-unittest 0.8.2, the CI versions):
- `helm lint deploy/charts/buzz`: 0 failed
- `helm unittest deploy/charts/buzz`: 12 suites, **85/85 passed**
New `tests/version_label_test.yaml` checks the relay, pairing relay, and
deletion drain: tag wins over a stale `podLabels` pin, the label is
stamped with no `podLabels`, and digest-only images get a label-safe
value. `tests/storage_accounting_test.yaml` covers the same cases for
the CronJob, plus the `Chart.AppVersion` fallback.
Mutation checks:
- Reverting any one template to the plain `podLabels` passthrough fails
tests.
- Switching the source order to digest-first fails tests.
- Restoring the case-insensitive passthrough fails the three
Datadog-normalization tests (uppercase, `__`, uppercased 63-hex).
No Rust, desktop, web, or mobile sources changed.
## Linear tracking
Related work: [BUZZ-215](https://linear.app/squareup/issue/BUZZ-215).
---------
Signed-off-by: tornquist <tornquist@squareup.com>
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
836e7d0d06 |
feat(desktop): browse and moderate every community in the Admin Console (#8027)
The Admin Console only ever saw the community the desktop happened to be connected to. With #8011's community reads, the console can now browse every community on the relay and, for signed staff, moderate any of them by name without switching connections. 🤖 ## What changes - **Communities tab.** Lists every live community on the relay, searchable by host prefix with "Load more" paging. The connected community is pinned at the top by exact host match. Report rows get a community badge that opens that community. - **Community page.** Reports, Restrictions, Members and Actions, each scoped to the community on the page. The Actions form has no host field: it always acts on the page's community, and the TypeScript host normalization is gone from the Actions tab. - **Native reads** (`admin_list_communities`, `admin_search_members`, `admin_get_member`, `admin_get_event`). Every request validates the explicit community host and NIP-98-signs the real method and URL. Failures come back as a structured error (`status`, `bodyComplete`, `bodyEmpty`, `code`), so the UI can tell an older relay's empty 404, a coded `event_not_found` and a transport failure apart, and never says "not found" unless the relay did. - **Explicit-host lifts.** `admin_lift_restriction` sends the page's community host, not the active relay's. It freezes the relay and signer the intent was reviewed under, and refuses to send if either has changed. - **Pending actions survive navigation.** A panel-level `DirectActionsProvider` holds the frozen intent, so switching communities or tabs mid-confirm doesn't lose it. Only Discard drops it. A signer change remounts the controller and drops it. - **Fenced lookups.** Member and event lookups are keyed by origin, signer, community, target and a generation counter, so a late answer for one community can't unlock Review in another. - **Shared member picker** (`MemberSearchResult`, also used by Add Member), showing the truncated npub so members with the same name can be told apart. Delete accepts a note link and previews the message before Review. Relay staff are blocked before Review. - **Disabled admin auth is read-only.** With `BUZZ_ADMIN_AUTH=disabled` the Communities tab and community pages still render, and report badges still open them, but every ban, timeout, delete, add-member and lift control is disabled. The settings badge says the console is read-only and that acting needs `nip98`. Staffing stays hidden. - #7904's protections in the Actions tab are kept: frozen intent, `requestId`, Retry, `notSent` and the key-backup guard. - **Readable load errors.** Admin list and detail loads show the relay's error message (for example "request is not authorized"), never the raw JSON envelope. - **Failed kicks reload the report.** A resolve that fails with `enforcement_failed` reloads the report, so its failed action and "Cancel & reopen" appear immediately. - **Key backups never leave in a search.** Every admin API request URL passes the NIP-49 key-backup guard before it is signed or sent: community and legacy reads, mutations, the connection probe (before its first, unsigned request) and feedback-attachment fetches. The community and member search boxes, including the directory's Load more, refuse a pasted `nsec` or `ncryptsec` anywhere in the input without calling the relay. - **`just admin-seed` for multi-community testing.** Besides the local community, it seeds `beta.localhost:3000`, `gamma.localhost:3000` and 50 filler communities, with named members, a ban and a timeout in each named community, a report with Kick available, and a message for the delete preview. Rerunning it restores all of that to the starting state. It refuses, writing nothing, while a moderation action is unfinished on a seeded report. - The bodyless `GET /events/{id}` preview is added to the events-URL egress inventory. It sends no body, so it carries no key material. ## Tests Native (Rust): - `empty_404_and_405_read_as_complete_and_empty`, `coded_404_carries_the_relay_code`, `truncated_and_over_cap_bodies_are_incomplete`, `transport_failure_has_no_status`: the structured read error tells an older relay, a coded refusal, a cut-off body and a transport failure apart. - `a_401_retry_is_signed_by_the_same_keys`, `lift_signs_the_send_and_401_retry_with_the_checked_keys`: the 401 retry never switches keys. - `read_urls_validate_the_explicit_host_and_encode_the_query`, `lift_url_sends_the_page_host_not_the_active_relay`, `list_url_carries_host_and_cursor_and_refuses_a_changed_relay`, `lift_refuses_before_sending_on_signer_change_or_bad_host`: requests target the explicit host and refuse to send after a relay or signer change. jsdom: - `actions-pending-nav`: a pending intent survives a community change, and only Discard drops it. - `actions-fenced-lookup`, `fenced-load`: a late lookup for an old key never shows under, or unlocks, the new one. - `actions-same-ids`: the same pubkey and event id are read separately in each community. - `actions-community-change-drops-member`, `actions-page-host`, `actions-success`, `actions-identity`: the form acts only in the page's community, and confirm sends the frozen intent. - `actions-preview-errors`, `actions-delete-link`, `actions-staff-target`, `actions-disabled-auth`: preview errors, delete by note link, the staff block, and community pages rendering read-only with every action control disabled under disabled admin auth. - `communities-directory`, `communities-unsupported`, `community-badge`, `community-badge-disabled-auth`: the directory, older relays, and report badges, which open community pages under disabled admin auth too. - `restrictions-lift-signer-change`. The existing restriction tests now run on the community page and use `admin_lift_restriction`. Mutation checks: removing the pending-navigation guard turns `actions-pending-nav` red, and dropping the host from the fence key turns `actions-fenced-lookup`, `actions-same-ids` and `fenced-load` red. Ungating restriction lifts on `canMutate` turns `actions-disabled-auth` red. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
a918e605ca |
docs(nips): add NIP-FA federated identity adapter spec (#8148)
🤖 Moves the enterprise identity adapter's wire contract into a new normative spec, `docs/nips/NIP-FA.md` (Federated Identity Adapter), so third-party adapters have a spec to conform to. The Desktop-only material stays in a guide, renamed from `docs/enterprise-identity-adapter.md` to `docs/federated-identity-adapter.md` to match the NIP's name. - `NIP-FA.md` holds browser login, code exchange, session check, relay assertion (including the denial table), NIP-FI assertion transport, security considerations, and privacy. Agent-key assertions are out of scope. - `docs/federated-identity-adapter.md` is a short Desktop guide for the client in block/buzz-app: build configuration (`BUZZ_BUILD_ENTERPRISE_AUTH_*`), what is implemented today (build config, discovery gate) versus planned (browser login, session check, assertion fetch, profile projection), and the vendor-specific headers adapters must not use as session credentials. Endpoints, fields, headers, and denial codes are unchanged. The NIP converts lowercase requirements to BCP 14 keywords and pins rules the old guide left open: - `/v1/session` reports a bad session as 401 `session_required` / `session_expired`; clients discard the session only on those, and keep it and retry with backoff on 429, 5xx, and network failures, matching the assertion endpoint. - The assertion `aud` is `https://` plus the canonical `relay_url` authority (`wss://c.example.com:8443` → `https://c.example.com:8443`), and adapters issue dedicated `nip-fi+jwt` assertions. Relay authorities on port 80 are not supported. - Codes are single-use and short-lived, only `S256` challenges are accepted, and adapters accept only `127.0.0.1` and `[::1]` callback hosts. - After code exchange the client must call `/v1/session` and compare the RFC 3339 `expires_at` instants before committing the login. - `session_token` is sent only to adapter endpoints, never to a relay, and no credential is logged. - NIP-98 proofs carry exactly one `method` and one `payload` tag; retries honor `Retry-After` and surface failure after bounded attempts; IPv6 relay hosts have a canonical form. - The relay-proof header example is scoped to NIP-98-protected HTTP, with WebSocket (NIP-42 AUTH) and Blossom (kind 24242) deferred to NIP-FI. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Bradley Axen
|
7afdc2aac9 |
feat(relay): record when a read-state account starts (#8157)
## Goal Add `started_at` to read-state accounts from the day the table is created. This prevents users seeing ALL their channels as unread when they first install a client update that starts using these APIs --------- Signed-off-by: hMB2 <0400eb0da32afbbf16cdd1dee6ffd243587e8f3d6a2823fe7dfa95d16b002b7d@buzz.block.builderlab.xyz> Signed-off-by: ART <ea39c1c8acf57180161ef4ba4ed68abf8589f5291956b909a5aa991fc66664ca@buzz.block.builderlab.xyz> Signed-off-by: hMurderbot <219c58afff8ac2c76e4e527a329aa61c2ebbb896786b1916d4f9cca5140ecd04@buzz.block.builderlab.xyz> Co-authored-by: hMB2 <0400eb0da32afbbf16cdd1dee6ffd243587e8f3d6a2823fe7dfa95d16b002b7d@buzz.block.builderlab.xyz> Co-authored-by: hMurderbot <219c58afff8ac2c76e4e527a329aa61c2ebbb896786b1916d4f9cca5140ecd04@buzz.block.builderlab.xyz> Co-authored-by: ART <ea39c1c8acf57180161ef4ba4ed68abf8589f5291956b909a5aa991fc66664ca@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Will Pfleger
|
032198ce43 |
feat(relay): add community reads to the admin API (#8011)
The in-app Admin Console needs to work on any community the relay
serves, not only the one the desktop is connected to. This adds four
read-only routes under `/api/admin/v1` that the stacked desktop PR
builds on, puts every admin route under one explicit access rule, and
binds every admin request's signature to its real method, which fixes
signed `HEAD` on reads.
## Access rule
| Mode | Moderation reads | Writes | Staffing (`/operators`) |
|---|---|---|---|
| `nip98` (default) | signed, any staff role | signed, role-checked |
signed Operator |
| `disabled` | served to anyone who can reach the relay | `403` | `403`
|
"Moderation reads" means every read: reports, feedback and attachments,
restrictions, and the four community routes below. `nip98` behavior is
unchanged, and the Operator API (`/operator/*`) is untouched.
**Existing `BUZZ_ADMIN_AUTH=disabled` deployments start serving the four
new reads** (community directory, member search, member lookup without
staff status, and stored events by ID) with no config change. The
startup `WARN`, `docs/admin/README.md`, `.env.example`,
`deploy/compose/.env.example` and the `just admin` comment list exactly
what disabled mode exposes, say it should be reachable only by people
allowed to see all moderation data in every community, and say the whole
relay port must be private: the admin API is on the relay's main
listener and its hostname check uses the caller's `Host` header. `just
admin` binds the relay to `127.0.0.1` unless `BUZZ_ADMIN_AUTH` is
exactly `nip98`, so an unset, padded or misspelled value stays local.
In code, `authorize()` returns `AdminAccess`, either `NetworkTrusted`
(disabled mode, no identity) or `Staff(AdminPrincipal)` (`nip98`, any
rostered Operator or Moderator). Each route makes one check: **view**
(`authorize()` succeeding), **act** (`AdminAccess::act()`, must be
`Staff`, then the existing role checks) or **operator**
(`AdminAccess::operator()`, must be an Operator). Act and operator
refuse `NetworkTrusted` with `403 this endpoint requires
BUZZ_ADMIN_AUTH=nip98`; no principal is ever created for an anonymous
caller. `every_route_makes_its_check_in_both_auth_modes` lists every
mounted admin route with its check and asserts the result in both modes,
and `every_route_is_listed` fails when `router()` mounts a route that
list does not name.
## Routes
All four live in `api/admin/reads.rs`; `mod.rs` only registers them.
Their queries reject unknown fields, like `ReportQuery`. The three
`?communityHost=` routes share `CommunityQuery`, which now rejects
unknown fields too, so the existing unban, untimeout and direct-action
routes that use it do the same; their desktop callers send only
`communityHost`. Every admin route that takes `limit` validates it with
the module's shared `limit()` helper, so an out-of-range value is `400
invalid_limit`.
| Route | Returns |
|---|---|
| `GET /communities?q=&cursor=&limit=` | `{items: [{id, host,
icon\|null}], nextCursor\|null}` |
| `GET /members/search?communityHost=&q=&limit=` | `{items: [{pubkey,
displayName\|null, nip05\|null, avatarUrl\|null}]}` |
| `GET /members/{pubkey}?communityHost=` | `{pubkey, profile:
{displayName, nip05, avatarUrl, about}\|null, role:
"owner"\|"admin"\|"member"\|null, banned, mutedUntil\|null, isStaff:
bool\|null}` |
| `GET /events/{id}?communityHost=` | `{id, authorPubkey, kind, content,
createdAt, deletedAt\|null, channelId\|null}` |
- **One community per read.** Community-scoped routes bind
`communityHost` through the tenant binder (`400 unknown_community_host`)
and read only that community's `users`, `relay_members`,
`community_bans` and `events` rows.
- **Directory.** It uses the binder's liveness rule (not archived, not
deleted, `deletion_state = 'active'`), so every listed host can be
opened. `q` is an escaped, case-insensitive host prefix of at most 255
characters (`400 invalid_query`). Results are ordered and paged by
`lower(host)`, which is unique, so the opaque base64url cursor carries
only the last host; a malformed cursor is `400 invalid_cursor`. `limit`
is 1 to 100 with a default of 50.
- **Member search.** It reuses `search_users`, which already escapes,
binds and scopes, with no second escaping layer. `q` is trimmed and must
be 1 to 100 characters (`400 invalid_query`), and `limit` is 1 to 50
(default 20). It searches community profiles, so former members appear.
- **Member lookup.** `isStaff` uses the same roster lookup as the
direct-action staff guard and is returned to signed staff only: it is
`null` in disabled mode, where the roster is never read. A roster lookup
failure is a 500, never "not staff".
- **Event preview.** It reads the event by `(community_id, id)` and
never returns a private kind (see below). A malformed id, an id stored
only in another community and a private kind are all `404
event_not_found`, the same answer as an absent event.
## Private events
Admin reads never reveal events the relay keeps private.
`admin_hidden_kinds()` combines the four private kind sets in
`buzz_core::kind`: `AUTHOR_ONLY_KINDS`, `RESULT_GATED_KINDS`,
`P_GATED_KINDS` and `SHARED_GATED_KINDS`. The whole shared-gated set is
hidden, including events tagged `["shared","true"]`; the shared tag is
never evaluated. Every HTTP response applies it the way it treats a
missing event: event preview is `404 event_not_found`, report detail
returns the report with `message: null` and `targetAuthorPubkey: null`,
and the report list returns a null target author.
Moderators can still act on these events, they just cannot read them.
Report detail is one query with the hidden-kind list as a parameter,
exposed two ways: `admin_get_report` hides private kinds and backs every
HTTP response (GET detail, the resolve response, reopen, cancel and the
reporter-notice worker), while `admin_get_report_for_enforcement` reads
the target unfiltered and is used only by the resolve handler's
enforcement load (the staff-target check and ban, timeout and kick
target derivation) and by the recovery worker for interrupted actions.
Its result is never serialized, so a ban from a report on a private
persona bans the real author while GET still shows nothing.
## Method binding
Axum serves `HEAD` through `GET` handlers, but the read handlers
authorized with a literal `"GET"`, so a correctly signed `HEAD` was
rejected. Admin reads now authorize through `authorize_read` and writes
through `authorize_write`, both of which use the request's real method
and full target, so no handler hardcodes a method string. A database
error behind any admin route is now logged before it becomes a 500.
## Directory query plan
`idx_communities_host` is on `lower(host)` in the default collation, so
the prefix `LIKE` is a filter, not an index condition. The plan walks
the index in order and stops once it has `limit` matches. A common
prefix is cheap (on local Postgres 16 with 21,359 communities, a first
page skipped 2,728 index rows in 3.9 ms), but a prefix with few or no
matches walks every index entry, so its cost grows with the number of
communities. A `COLLATE "C"` index would turn the prefix into an index
range; it is left out because the admin directory is low-traffic and the
cost is small at current scale.
The cursor condition is `lower(host) > COALESCE($2, '')`, with no `OR`,
so it remains an index condition under a generic plan:
```text
-> Bitmap Index Scan on idx_communities_host
Index Cond: (lower((host)::text) > COALESCE($2, ''::text))
```
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
|
||
|
Luke Tornquist
|
6d07a29137 |
perf(relay): bound fleet usage metrics collection (#7176)
## Why
The usage poller pays memory and query costs proportional to community
count even when per-community emission is disabled. Every pod loads the
full host map, and the leader materializes all per-community aggregates
before suppressing their labels. That makes telemetry collection itself
a scaling constraint.
## What changed
- Fleet-only usage metrics are now the default. `all` remains available
as an explicit rollback mode.
- In fleet-only mode, the poller skips the community host map and all
community-sized query results.
- Exact community counting is replaced with the PostgreSQL planner's
table-size estimate.
- Stock totals are collected as one fixed row, scanning each stock table
once per hour.
- 1d/7d/30d active-user totals come from one bounded 30-day scan per
day.
- Fleet database snapshots run only on a proved read replica, with a
telemetry-specific freshness budget and server-side statement timeouts.
If no such replica is available, the collection is skipped; it never
falls back to the writer.
- Successful fixed-row snapshots are cached and re-emitted on each poll
tick, so gauges stay scrape-visible without raising query frequency.
- `buzz_usage_snapshot_available{family}` reports whether the latest due
refresh of that family succeeded. After a skipped or failed refresh, the
last values stay visible (with `buzz_usage_snapshot_age_seconds`) but
the family reads unavailable until a 60-second retry succeeds.
- The `all` rollback collector publishes the same availability gauges,
so dashboards gated on availability keep working during a rollback.
- Each due refresh outcome is applied by `apply_stock_refresh` /
`apply_activity_refresh`, the single production mapping from `Ok(Some)`
/ `Ok(None)` / `Err` to cache, availability, skip counter, and retry;
the regression test drives that mapping directly.
- `buzz_usage_snapshot_age_seconds` is emitted in the `all` rollback
too, so age-based alerts keep firing through a rollback.
- Every fleet telemetry attempt that completes lands in
`buzz_db_route_decision`; a query error on a proved reader records
`skipped/replica_error`. An attempt abandoned at the relay deadline
records no route decision and appears only as
`buzz_usage_query_skipped_total{reason="timeout"}`. The shared
`proved_reader` warnings no longer claim "routing to writer", because
the telemetry caller never does (serving callers still record their
writer decision in the route counter).
- The PostgreSQL production-method matrix now exercises
`usage_fleet_stock_snapshot`, so the new `reader/maintenance` pool pair
is proven to be emitted by a real Db method.
- Each fleet collection, reader proof included, runs under a relay-side
`tokio::time::timeout` (10s stock, 20s activity, both above their
server-side `statement_timeout`). A reader that stops answering expires
into the same failure mapping with
`buzz_usage_query_skipped_total{reason="timeout"}`, so it cannot stall
the leader's poller, hold the advisory lock past a liveness check, or
leave a frozen snapshot reporting available.
- Fleet telemetry reader checkouts (`ReaderOperation::Maintenance`) opt
into SQLx `close_on_drop` before their first post-acquire await, so an
abandoned collection's connection is closed within SQLx's 5s
`CLOSE_ON_DROP_TIMEOUT` instead of going through the unbounded
return-to-pool ping while it holds a slot in the reader pool that
serving reads share. Serving reads keep pooled reuse; the cost is one
reader reconnect per fleet refresh.
- On leader demotion, cached snapshots and availability are cleared, and
a fresh collection is forced after the lock is reacquired.
- Fleet-only mode passes no host map to the worker-snapshot storage
emitter. It therefore omits community attribution instead of reporting
every byte as `buzz_storage_unmapped_community_bytes`.
- Lifetime stored-message and active-channel totals are no longer
queried in fleet-only mode. Event throughput remains available through
the existing stored-event counter.
- No test-selector changes: main's `config::tests::` selector in `just
test-unit` and the `run-tests.sh unit` fallback already runs the new
replica-budget config test, and the new fleet schedule, availability,
and storage tests run through the existing bin `tests::` and
`storage_sweep::tests::` selectors.
- Added seeded PostgreSQL contract tests and operator docs covering
rollout, rollback, availability, the no-replica behavior, and leader
election.
No migrations, tables, indexes, or new persistent storage are added.
### What got simpler
The relay-side storage half of the original branch (totals-only S3
listing and `buzz_storage_community_breakdown_available`) was dropped.
Main moved S3 listing out of the relay in #7543 and #7845, so that code
was dead on arrival. The rework adds no storage code path; it only gates
attribution on the presence of a host map.
## Risk
Medium. The default changes which telemetry is emitted:
- In fleet-only mode, `buzz_communities_estimated` replaces the exact
`buzz_communities_total`.
- `buzz_total_messages` and `buzz_total_active_channels` are not
refreshed in fleet-only mode (also called out in the rollout steps of
`docs/usage-metrics.md`).
- Deployments without a verified reader (including local compose and
single-node self-hosts) skip database-derived fleet snapshots instead of
querying the writer, and report both families unavailable.
- Multi-pod dashboards must filter database and storage series to
`buzz_usage_poller_is_leader == 1`.
Setting `BUZZ_USAGE_METRICS_PER_COMMUNITY=all` restores the legacy
collector for temporary rollback.
## Verification
At head `1879ee3aedf4227942120a3b521c8d35fc7839fe` (tree `78477e74`;
merges main `da651ad6`; the PR's diff against main is byte-identical to
`1613823f`'s, minus the Justfile and `run-tests.sh` selector lines that
main's `config::tests::` selector now subsumes):
- On Blox: `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay --all-targets -- -D warnings` pass. Relay bin `tests::`
34/34, `config::tests::` 55/55, `storage_sweep::tests::` 10/10 and
`store::usage::tests` pass, as do the four `fleet_` and lock PostgreSQL
tests that use scratch databases, including
`abandoned_fleet_collection_releases_its_reader_slot`.
- Against a freshly migrated PostgreSQL database:
`store::usage::postgres_tests::` 14/14,
`storage_sweep::postgres_tests::` and `buzz-db --test
observability_source` 14/14 pass. The new reader code passes main's
crate-wide typed-acquisition and serving-table policy checks.
- E2E at `1613823f` is reused for this head. It ran in an isolated relay
against a real PostgreSQL primary and streaming standby. In 2/2 runs a
stock read went dark mid-result; the deadline abandoned it at about 10s,
the reader pool's active and size counts dropped to zero about 1s later,
and a fresh activity read and the scheduled stock retry succeeded on new
backends while the dark session stayed open. The `all`-mode and
healthy-replica controls also passed. Main's merged changes are on the
writer path and in startup step timing, and touch none of the reader
checkout, the `close_on_drop` seam, the fleet poller or the deadlines.
At head `1613823fa8ef1a423bc1a499c57bc40d692fcd4f` (tree `12bc6d1b`;
test/docs-only on top of `57749418`), on a Blox workstation:
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay --all-targets -- -D warnings` pass. Relay bin `tests::` 34/34
pass.
- New PostgreSQL test
`abandoned_fleet_collection_releases_its_reader_slot`: a one-slot reader
pool behind a loopback proxy that goes dark mid-session; the collection
is cut at a deadline, then a later acquire must succeed in under 8s on a
fresh connection. The dark call runs on a fresh `Db` with a cold Aurora
capability cache, so the first post-acquire await is the capability
probe. It passes in 3.4s. With `close_on_drop` removed, or moved below
the probe await, it fails with `PoolTimedOut` after the 15s acquire
timeout. The other three `fleet_` PostgreSQL tests that use scratch
databases pass.
- `each_fleet_family_is_abandoned_at_its_own_deadline` now has an outer
timeout: replacing the stock deadline with a bare `.await` fails it
(`Elapsed`) instead of hanging.
At head `fd194ec79d6c6796d6764564de225ad56485cfc7` (tree `79e94727`,
main merged at `932228936`), on a Blox workstation:
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-relay -p
buzz-db --all-targets -- -D warnings` pass.
- Relay bin `tests::` nextest selection: 27/27 pass (plus
`store::usage::tests`), including
`fleet_refresh_outcomes_drive_availability_through_production_mapping`
and the new `stalled_fleet_collection_times_out_into_failed_refresh`
(paused time: two never-completing collections are abandoned at exactly
10s + 20s; both families read unavailable with cached values kept,
`reason="timeout"` skips, retry scheduled; the next tick runs and
recovers; no route decision is recorded for an abandoned attempt) and
`each_fleet_family_is_abandoned_at_its_own_deadline`. Compile-time
asserts keep each deadline above its `statement_timeout`.
- Falsifiability: replacing the stock deadline with a bare `.await`
makes the new test fail (the tick stalls); swapping the two deadlines
fails the per-family test; deleting `schedule.mark_stock_failure(now)`
from the `Ok(None)` arm fails the mapping test. Trees restored and
re-verified.
- At `970414db`: CI PostgreSQL Tests 809/809 including the
production-method matrix with `reader/maintenance`; Unit Tests ran
`finish_usage_read_records_every_attempt_in_route_decisions`.
- `just ci` / `just test` at `5b18914a` on Blox: not green for
environmental reasons (`mobile-check` could not reach pub.dev;
`_ensure-services` failed so service-backed suites did not run;
`buzz-acp git_bootstrap` fails identically on `origin/main`). PostgreSQL
and integration coverage comes from GitHub CI at this head.
E2E at the previous head `0a01e5f22` (isolated relay, real Postgres
primary plus streaming replica, worker storage snapshot, desktop
frontend in relay mode): with no reader, both families report
skipped/unavailable and route `skipped/disabled` (no writer queries);
fleet-only storage emits totals without `buzz_community_storage_*` or
unmapped bytes; `all` mode reports both availability gauges `1` with
per-community storage series; with a fresh replica, both fleet paths
route `replica/fresh` on `reader/maintenance` acquisitions; desktop
send/receive works. The cached success → due failure → recovery
transition was not reached live (hourly/daily cadence); it is covered by
the seam test above.
The original branch also recorded live verification at `aa3e7023`,
before this rework: reduced-idle cache re-emission, two-relay leader
takeover with demoted availability clearing, the no-reader skip with
60-second retry, `all` rollback restoring per-community series, and a
desktop smoke test. Those runs predate the availability and storage
changes and are not claimed for this head.
## Linear tracking
Related work: BUZZ-167.
---------
Signed-off-by: tornquist <tornquist@squareup.com>
Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
||
|
Tyler
|
1a2ddf9eaa |
feat(relay): private read-state accessory API (#7906)
Buzz BFF: Private read-state accessory API (`/buzz/v1`) |
||
|
Will Pfleger
|
1e67395fd1 |
docs(identity): tighten enterprise adapter contract (#8069)
Tightens `docs/enterprise-identity-adapter.md` to define the enterprise adapter contract implemented by kgoose in [cash-server#132770](https://github.com/squareup/cash-server/pull/132770) and the Buzz Desktop client in [buzz-app#562](https://github.com/block/buzz-app/pull/562). - Requires canonical `wss://host[:port]` relay URLs with a lowercase scheme, the lowercase ASCII (A-label) host form with no trailing dot, the default port omitted, and no trailing slash, path, query, or fragment. Adapters MUST configure relays in canonical form, compare `relay_url` against that exact form, and reject unknown relays with 403 `authorization_denied`. - Requires NIP-98 proof freshness of at most 60 seconds old and at most 5 seconds in the future, with the proof bound to the endpoint, method, and exact request body. - Rejects content-encoded request bodies with 400 `invalid_request`. - Caps assertions at `exp - iat <= 300` seconds and the adapter session expiry, requires `typ` `nip-fi+jwt`, and validates the echoed `nostr_pubkey`. - Defines refusal and retry handling: Clients MUST retry 429 and 503 with bounded backoff whatever the `error` code. Clients MUST treat any other status, a 400/401/403/413 response with a contract-undefined code, or a rejected `200` response including one it cannot parse, as a refusal: keep the session, show it as refused, and not retry automatically. Network failures retry with bounded backoff. This tightens the enterprise adapter contract restored in [#8064](https://github.com/block/buzz/pull/8064). --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Alia <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
e982f70fba |
docs: restore enterprise identity adapter contract with relay assertion (#8064)
🤖 Restores `docs/enterprise-identity-adapter.md`, the vendor-neutral HTTP contract between Buzz Desktop and an enterprise identity adapter, and adds the request that turns an adapter session into a NIP-FI relay assertion. The login, code exchange, session check, transport, and privacy sections are byte-for-byte the text from the head of #7715 (`4ef08e23`). Buzz Desktop's enterprise login client ([buzz-app#525](https://github.com/block/buzz-app/pull/525)) already calls those routes. The new **Relay assertion** section defines `POST /v1/identity/assertions`: - **Request:** a JSON body with `relay_url` and `nostr_pubkey`. The adapter session goes in `Authorization: Bearer`, and a NIP-98 proof signed by that key, bound to the endpoint URL, method, and exact body hash, goes in `Nostr-Authorization`. Clients and adapters must not log either header's value. - **Response:** `assertion`, `nostr_pubkey`, and `expires_at`. Under this contract an assertion lives at most 5 minutes and never outlives the adapter session; that cap is the adapter contract's rule, not a relay constant. The assertion's `typ` and `aud` rules point to NIP-FI. The assertion key must be the key that signs the NIP-42 relay login. - **Denials:** a table of status, error code, meaning, and the client action for each. A missing session is 401 `session_required`; missing or repeated proof headers, a repeated `Authorization` header, multiple session credential kinds, malformed bodies, and unknown fields are 400 `invalid_request`. 401s clear the session and return to login; 403 `authorization_denied` keeps the session and disconnects from that relay without automatic retry; other 403s, 400, and 413 keep the session and show the error without retry; 429, 503, and network failures retry with bounded backoff. Adapters may return `session_required` for any invalid, expired, or revoked credential, and clients handle both 401 codes identically. Agent keys are out of scope and need a separate design. Related: [cash-server#132756](https://github.com/squareup/cash-server/pull/132756) implements this contract in kgoose. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
53a12100b3 |
fix(deletion): require sole owner at admission (#7966)
Follow-up to #7830: reject owner-origin deletion admission for a legacy co-owned community before creating a request. Under the existing community-row lock, admission locks all current owner memberships in pubkey order and requires exactly one matching asserted owner. Existing-request idempotent replay remains unchanged; completion retains its independent post-lock revalidation. **New optional operator field: `community_id`** (`ab0cd713`, `766eb07f`). Owner-origin `POST /operator/communities/delete` now accepts an optional `community_id` that binds the request to the host's community: - **Fresh submission:** checked after sole-owner authority is proven, so a non-owner still gets `404 community_not_found`. A mismatch returns `409 community_id_mismatch` and writes no request row. - **Replay:** checked against the stored request, and only when the stored request is for the same host. A mismatch returns `409 community_id_mismatch`, and the stored request is unchanged. A known UUID sent with a different host goes through the existing convergence check and returns `409 deletion_request_conflict`. - **Omitted:** behaves as before. `admit_owner_request` takes `expected_community_id: Option<Uuid>` rather than a second public entry point, and existing callers pass `None`. - `docs/operator-community-deletion.md` documents the ordering. Also folds in the non-blocking review nits from #7969 (wpfleger96, at `e21151f4`): - **`limit_reached` has a stable `code`.** Create (`provision_community`) and transfer-in now return `409` with `code: "limit_reached"`, matching the other coded lifecycle errors. The `error` message keeps its `limit_reached:` prefix, so clients that match on the message still work. The provision limit test now asserts the code, and a new PostgreSQL test covers a transfer to an owner at the limit (coded 409, no membership change). - **Docs: an ack-version mismatch is a 400, not a 409.** `docs/operator-community-deletion.md` and the `delete_community` handler doc now say a changed host or owner under a known UUID is `409 deletion_request_conflict`, and an unsupported acknowledgement version is rejected first with `400 unsupported_acknowledgement_version`. - **Docs: active-cap overrides above the lifetime cap can't be reached.** The `max_communities_per_owner` doc and the quota section now say `MAX_LIFETIME_COMMUNITIES_PER_OWNER` (20) counts live ownership, so `BUZZ_MAX_COMMUNITIES_PER_OWNER` above 20 can't be reached. - **Legolas P2s:** - The UUID-conflict docs now list every 409 case, including a stored ack version that doesn't match and a UUID held by an operator-origin request. - The runbook now says a legacy co-owned community is rejected as `404 community_not_found` and should be converged with a transfer first. - The preparation drift test now pins `DeletionSafety` plus "sole-owner authority drifted". - **Guard wording:** the `delete_community` handler doc now says "sole current owner", matching the admission check. What got simpler: admission and automatic preparation now enforce the same sole-owner authority, so no accepted-but-unexecutable co-owned request needs a new recovery path. `deletion_api_error` is renamed to `coded_api_error`, since it now carries non-deletion codes too. There's one helper for coded operator errors instead of a second one. Rebased onto main after #7969 merged (`d7a35afa`). The sole-owner commit applied cleanly. Earlier focused Blox verification (at the pre-rebase head `161e9857`): the new admission regression was red before the change and green after it. Five temporary, restored mutants each failed its matching production-seam regression. At the current head `766eb07f`, the relay `api::operator::` suite passes 23/0 on Blox. Each new `community_id` guard, removed one at a time, fails its own regression test. CI owns the full suites. Remaining debt: the separate P2-A purge/completion membership/community lock-order cycle is unchanged. Generated with Codex --------- Signed-off-by: Codex <noreply@openai.com> Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
d7a35afa98 |
feat(buzz-relay): idempotent owner community deletion with quota reservation (#7969)
## Summary This is the relay side of owner community deletion. - **Idempotent delete is the recovery call.** - `POST /operator/communities/delete` checks the request UUID first. - If the request already exists with the same host, owner and acknowledgement version, it returns `202` with the request's current `status`. That holds at any stage, even after membership is purged, and no new work is admitted. - The same UUID with a different host or owner returns `409 deletion_request_conflict`. An unsupported acknowledgement version is rejected first, with `400 unsupported_acknowledgement_version`. - There's **no receipt endpoint**. Clients recover by resending. - **Active quota reservation.** An incomplete owner deletion holds the owner's slot until the deletion logically completes. Migration `0054` adds the supporting index. Migrations 0052 and 0053 are unchanged. - **Lifetime cap.** Deleted communities keep their hosts as permanent tombstones. - On create and on transfer-in, the relay counts live ownership plus every non-aborted owner deletion, completed ones included. - That total is capped at an absolute 20, regardless of the active limit, and going over returns `limit_reached`. - This stops create-then-delete host squatting. - **Stable lifecycle conflict codes,** plus `acknowledgement_version` in the 202. - **Owner-list quota projection:** `quota_used`, `quota_limit` and `can_create`. - `can_create` reflects both caps. - The projection is advisory, and `limit_reached` is authoritative. - **Operator doc** (`docs/operator-community-deletion.md`): the acknowledgement version is a compile-time constant. Admission and the executor's claim/lease both filter on it. A version bump must keep replaying and executing old-version requests. This follows on from #7830. The foundation PRs #7818, #7827 and #7830 are merged. ## Testing New tests: - `owner_delete_resubmission_reports_current_status_without_new_intent` - A replay returns 202 `submitted`, including after membership purge. - Changing any field returns 409 `deletion_request_conflict`. - A non-operator gets 403. - A replay after abort returns 202 `aborted`. - No request rows are added. - `completed_owner_deletions_count_toward_lifetime_cap` - After 20 created-then-deleted communities, the active count is 0 but `can_create` is false. - The next create and a transfer-in both return `LimitReached`. - `owner_quota_admits_only_under_active_and_lifetime_caps`: a unit test. Fellowship gate: PASS at `59375c00`. The code review was clean. The E2E run covered authenticated KGoose → relay → the real drain executor. The later commits only change the operator doc. ## Rollout - Deploy order: this relay first, then KGoose squareup/cash-server#130634, then Web squareup/ext-builderbot-ui#241 and App block/buzz-app#403. - No shipped client ever called a receipt route, so removing it doesn't break any existing client. - The quota changes have no deploy-order requirement. - Keep owner deletion off until this relay and the drain executor are live. - If you need to roll back, turn deletion off before rolling back the relay. - Recreate any database that ran the earlier version of migration 0054. That only applies to disposable preview databases. Never apply this to a database holding real data. The earlier draft 0054 has a different checksum and index predicate. ## Complexity Net simpler: - One route, its handler and a writer-routed `get` are removed. Recovery reuses the existing idempotent admission path. - A single `OwnerQuota::admits()` backs create, transfer and the projection. - The stale "clients fail closed / strict deployment order" doc section is replaced by the advisory quota contract. --------- Signed-off-by: tornquist <tornquist@squareup.com> Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Signed-off-by: Codex <noreply@openai.com> Signed-off-by: Codex <codex@openai.com> Signed-off-by: OpenAI Codex <codex@openai.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Codex <codex@openai.com> |
||
|
Zach Meyer
|
12423f2868 |
docs(nips): fix stray angle brackets in created_at clauses (#4486)
## Summary The `<` in a `created_at<` condition is the comparison operator, but in three places it had been paired with a closing `>` as if it opened a `<placeholder>` bracket: - **NIP-OA** clause list: `created_at<unix-timestamp>` → `created_at<unix-timestamp` - **NIP-OA** satisfaction rule: `created_at<t>` → `created_at<t` - **NIP-GS** "Conditions in Git Context": `created_at<t>` → `created_at<t` (line 437 of the same file already used the correct form) This matches the `created_at>...` forms used alongside them, which never carried a closing bracket. Also normalizes the NIP-OA headings to ATX style and pretty-prints the signed-event JSON example for readability. ## Verification Docs-only change. The NIP-OA test vectors are untouched and were re-verified by recomputing them: the preimage SHA-256, `tag-bytes-hex`, and the signed-event `id` all still match the documented values. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: zmeyer44 <zmmeyer44@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Tom Brow
|
45c288c179 |
docs: specify desktop-driven mobile push suppression (#7809)
## Summary Define when desktop activity suppresses otherwise eligible mobile message notifications: a ten-minute inactivity window, short-lived relay-owned suppression, and per-installation push preferences. Include a shared glossary and renewal timing diagram. The gateway remains unaware of suppression. This is a design draft ready for review, not an implementation-ready protocol. Wire format, synchronization, and several failure-handling details remain explicitly open. Please also review the tension between the agreed relay-confirmed preference changes and VISION_MOBILE.md's requirement that stopping notifications must not depend on a relay response. ### Related issue Related: #3234 covers per-channel notification settings and push filtering; this spec focuses on desktop-activity suppression. No duplicate spec PR found. ### Testing Documentation-only change. Local Markdown links and anchors resolve; whitespace validation and commit hooks passed. The full local CI gate could not complete: dependency compilation exhausted local disk space (OS error 28). GitHub checks currently report no failures. --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Diem Nguyen <ngthuydiem@users.noreply.github.com> |
||
|
Luke Tornquist
|
bba75a71eb |
Automate owner deletion preparation (#7830)
## Why Owner deletion admission stops at `submitted`. Manual inventory and approval still stand between owner intent and the existing deletion executor, preventing self-serve deletion. ## What The existing `buzz-admin deletions drain` process prepares and automatically approves operator-attested owner requests. Operator-origin requests retain manual approval. ## How The drain prioritizes approved work, then claims one owner submission with the existing generation lease. It freezes inventory and records digest-bound `owner_automatic` approval in one transaction before entering the unchanged executor. Before approval, the transaction rechecks archived state and current ownership. It locks the deletion advisory fence, community, owner memberships, and request in that order. A regression test pins privileged abort during a live preparation lease. What got simpler: one drain still owns preparation and execution. There is no second worker, queue, lifecycle, or retry authority. The rebased branch contains only this feature and its review fixes. ## Risk Automatic approval permits irreversible deletion without a human grace period. Operator attestation is not a cryptographic owner signature. Provenance, locked authority checks, generation fences, and digest-bound approval constrain the operation. This remains work in progress. Broad test failures and incomplete mobile checks are disclosed below; publication is not a merge-ready verdict. ## Testing Published head: `e059ae9056b447ca3b5d0c2893da86550ef16f5c`. All seven commit trees match tested source `5605f27eaece1e809f746cc1839b55c7ceef8d80`; only attribution metadata changed. Source-SHA results below are not new-head execution receipts. At source commit `5605f27eaece1e809f746cc1839b55c7ceef8d80`, isolated PostgreSQL tests showed stale archive and ownership fixtures fail closed after the fix. Before the fix both reached automatic approval. Removing abort's generation increment or stage transition made the new live-lease regression fail; restored production passed. `just test` failed in two ACP native-git fixtures and one agent body-timeout test. The failing source files match the pinned base, but there is no exact-base execution comparison proving those failures unrelated. `just test-unit` also failed in the ACP fixtures, leaving 18 tests unrun. `just ci` stopped in Flutter tooling package resolution; separately run non-mobile lanes do not constitute a full `just ci` pass. No live tenant deletion or human testing was performed. A dedicated populated `0052` → `0053` approval-backfill fixture remains a coverage gap. Final-head hosted CI and security review are required. ## Bigger picture Rebased directly onto main `12670bd0f037c66a682272bb81c46c3f254fad74`; prerequisite #7818/#7827 are already merged. Main owns `0052_channel_artifacts`, so this feature's automatic-approval migration is now `0053`. Client/KGoose and quota work are separate PRs and are not part of this diff. Generated with Codex --------- Signed-off-by: Codex <noreply@openai.com> Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Bradley Axen
|
12670bd0f0 |
feat(relay): implement NIP-AR channel artifacts (#7919)
## Summary Implements NIP-AR channel artifacts (spec merged in #7791) on the relay, after a simplicity pass over the earlier two-agent implementation. An artifact is an editable record (kind 45010) with a stable identity (`d`), one home channel (`h`), and full-snapshot revisions chained by `prev`: - **Writes:** one transaction per write. It locks the artifact's head row, requires `prev` to name the current revision, records the revision in a ledger that survives retention, and advances the head. Two competing edits can't both win. A stale edit gets `conflict: artifact head changed`. Resubmitting an accepted event succeeds without applying it twice. - **Permissions:** writes pass the same gates as a kind 9 message in `h`. A move also passes them in the source channel. The move stores the destination snapshot and a relay-signed kind 45011 removal marker for the source in the same transaction, so both channels recover by replay. The marker never names the destination. - **Moderation:** redaction is the ordinary kind 9005 removal. The ledger keeps the ID so `prev` checks still work. Redacting the current revision retires the artifact. - **Reads:** HTTP `/query` and `/count` accept explicit `artifact: current|history` filters with exact multi-character tag matching (e.g. `#assignee`). Generic REQ and search return only current revisions. Kind 5 against artifacts is rejected. NIP-11 advertises the limits. - **Spec:** NIP-AR updated to match: kind 9 permission checks, 9005 redaction and retirement, no creator-only delete rule. A separate small commit routes `insert_channel_head_checked` through the metered writer pool (`acquire_writer`), like every other event write in that module. ### Related issue Follows #7791. No separate issue found. ### Testing - Clippy clean. - Unit lane passes. Failures seen along the way came from agent env leaking into `buzz-acp` tests (they pass in a clean env) and known timing flakes (`keepalive_resets_idle_past_deadline`, two `buzz-agent` databricks tests that passed on retry). This diff doesn't touch those crates. - Postgres lane passes, including new `artifact_postgres_tests` in `buzz-db` and `buzz-relay`. Three presence tests need `REDIS_URL` set. - Docker smoke against a fresh local relay, 29/29: create and idempotent resubmit, update and stale conflict, current, `#assignee` and history queries, generic REQ, kind 5 rejection, delete and restore, move with source marker, private-channel isolation, and 9005 retirement. - A pasteable curl walkthrough of the same flows was shared with the reviewer. Draft until the human hands-on check (AGENTS.md step 3) is confirmed. The pre-push hook was skipped because every lane above had already run on this tree. CI is the gate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Signed-off-by: murderbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz> Co-authored-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: Smartie <fb3185faacfc3760b6b4fe0085a8878d8dd537f7a7671c7d3184fe1aa1b04df4@buzz.block.builderlab.xyz> Co-authored-by: Fizz <400e8babadcee6a7f420103f10a2849d84c4a9c71d5bd04f3948c814216648a3@buzz.block.builderlab.xyz> Co-authored-by: murderbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
6918dd3a83 |
Schedule the deletion drain safely (#7827)
## Why The deletion engine already exposes a one-shot `buzz-admin deletions drain` command, but operators have no safe way to schedule it. A generic workload wrapper could expose arbitrary commands or inherit relay credentials, which would add a second control plane and widen the blast radius. ## What This PR adds an opt-in Kubernetes CronJob for the typed deletion drain command. It is stacked on #7818, which adds owner-request admission. ## How The chart keeps the job disabled by default. The CronJob uses `Forbid`, zero Kubernetes retries, bounded runtime and history, and only the database, Redis, and object-store settings required by the executor. PostgreSQL leases, checkpoints, and retry timing remain the sole durable retry authority. The closed `_operator-jobs.tpl` helper centralizes safe CronJob mechanics without creating an arbitrary command or environment registry. This leaves the touched system simpler than a general job framework would. ## Risk The deployment blast radius is low because the job is disabled by default. When enabled, it can run destructive deletion work that the database has already approved, so the chart limits concurrency, retries, credentials, and execution time. ## Testing Rendered every `deploy/charts/buzz/ci/*.yaml` fixture with `helm template`; all passed at `e68cb753bb6673aa1baa60046969bdb2d4738b62`. Ran the full independent Desktop, Tauri, and web lanes; all passed at `e68cb753bb6673aa1baa60046969bdb2d4738b62`. Flutter package resolution made no progress within three minutes, so the mobile lane is not claimed green. ## Bigger picture A later stack layer must inventory and approve admitted owner requests. Until then, this scheduler cannot make a new owner request destructive. Generated with Codex --------- Signed-off-by: tornquist <tornquist@squareup.com> Signed-off-by: Codex <noreply@openai.com> Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
b37e477211 |
Make relay readiness process-local (#7341)
## Summary Make Kubernetes readiness depend only on the relay process lifecycle, so a shared Postgres or Redis slowdown cannot withdraw every pod at once. Move dependency checks to one fixed-cadence loop per pod. The loop permits only one evaluation at a time, emits metrics after each sample, and caches a timestamped report. `/_status` reads that cache without calling Postgres, Redis, or the deletion catalog. It reports `not_yet_sampled`, `fresh`, or `stale`. Each completed sample also writes a Unix timestamp gauge, so Datadog can compute sample age at query time. Fail closed when the community lifecycle lookup errors. An inconclusive tenant-state lookup no longer admits a socket to AUTH or REQ handling. ### Related issue None found. This addresses the elevated relay HTTP 500 and reconnect incident investigated on 2026-09-03. ### Tradeoff The readiness gauge now reports the latest private-probe observation, not the shutdown transition itself. After shutdown starts, a scrape can still see `1` until the next readiness probe refreshes the gauge to `0`. Kubernetes routing is unaffected because it uses the readiness response from the authoritative process state. A terminal `0` was never guaranteed: even a transition-owned gauge could update in process and then exit before Prometheus scraped it. If no scrape occurs before exit, both designs miss the terminal sample. The new design adds at most the interval until the next readiness probe when a scrape occurs during shutdown. Dependency status is now sampled every 30 seconds rather than on demand. A slow evaluation delays the next sample instead of stacking more work. The status payload exposes the cached report age. The completion timestamp stays unchanged when sampling stops, so `time() - timestamp` crosses the stale threshold without a server aging loop. Outcome counters and duration histograms remain cumulative and cannot provide no-data detection. ### Testing Gimli exercised exact SHA `edff1ec4a5362e132e367bf7449d2b0a68f1e9a9` on an isolated Blox stack. Repeated `/_status` bursts caused no dependency calls, metrics advanced without status traffic, stale status remained visible, and lifecycle lookup failures admitted no AUTH or REQ handling. Follow-up verification at `fdbb394ed598c94cde85ed01cb11e9de0ee8f00d` confirmed sampler-owned outcome and duration emission without status traffic and bound the real startup seam. The final timestamp-gauge change is covered by targeted red/green tests and the same boot regression at `c0f36e22b527c2fd6a63a548d036d006ffa17171`. ### Complexity Readiness remains one process-local sample per probe. Dependency I/O now has one owner and one execution path; request volume cannot increase dependency-check load. One completion timestamp replaces the prior server-side age update path; Datadog computes age without another loop. Community admission also returns to one fail-closed rule for inactive and unknown tenant state. Generated with [Claude Code](https://claude.ai/code) --------- Signed-off-by: tornquist <tornquist@squareup.com> Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Amp <amp@ampcode.com> |
||
|
Will Pfleger
|
447951a19c |
🤖 docs(nip-fi): remove implementation references from the spec (#7912)
This removes implementation references and implementation status from `docs/nips/NIP-FI.md` so the spec holds normative text only. It drops the Blossom compliance note, the known-gap notes, crate/file paths and PR numbers, roadmap "supersedes" sentences, doc-history lines, and the design-rationale note on session-only vs deny-until-TTL (its restart re-push SHOULD remains stated normatively elsewhere). Two wording changes carry normative weight: `PUT /media/upload` is listed as a plain kind-24242 upload route instead of a temporary alias, and the JWKS fetcher's SSRF controls become a MUST instead of a description. It doesn't depend on any enforcement PR and can merge at any time. Related: #7265 --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> |
||
|
Bradley Axen
|
b0d6fb8ad2 |
docs(protocol): propose simplified channel artifacts (#7791)
## Summary Adds a short, proposal-only **NIP-AR: Channel Artifacts** in `docs/nips/NIP-AR.md`, as a simpler alternative to the closed #7771. - Defines complete-snapshot artifact events with stable identity and atomic, `prev`-based revisions. - Uses channel access and posting permissions for reads, edits, moves, and writer-authorized deletion. - Keeps client-defined content opaque to the relay, with exact tag filtering over current accessible state and bounded queries. - Covers durable source removal on moves, terminal deletion, and safe client fallbacks without defining task/project schemas or changing application behavior. ### Remaining design work This is a compact draft, not a complete implementation-ready wire contract. Concrete query request/pagination/error/limit-discovery formats, removal/replay wire details, and numeric query limits remain follow-up work. Prefer extending shared WebSocket `REQ` / HTTP `/query` surfaces rather than adding plugin-specific endpoints. No database layout or scale guarantees are proposed. ### Related issue Closest prior proposal: #7771 (closed). Searched existing PRs and issues for channel artifacts; no matching issue found. ### Testing - Verified the committed document is byte-for-byte identical to the reviewed local draft. - Parsed the example JSON and its embedded JSON content; checked unique two-element example tags. - Reviewed the entire one-file diff and ran `git diff --check`. - Commit and push hooks passed (including DCO sign-off and differential file-size checks). - Full application CI was not run locally: this changes only a Markdown proposal, with no implementation or UI changes. --------- Signed-off-by: Fizz <400e8babadcee6a7f420103f10a2849d84c4a9c71d5bd04f3948c814216648a3@buzz.block.builderlab.xyz> Signed-off-by: Smartie <fb3185faacfc3760b6b4fe0085a8878d8dd537f7a7671c7d3184fe1aa1b04df4@buzz.block.builderlab.xyz> Signed-off-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Co-authored-by: Fizz <400e8babadcee6a7f420103f10a2849d84c4a9c71d5bd04f3948c814216648a3@buzz.block.builderlab.xyz> Co-authored-by: Honey <8e307ae0076a4dab6b94b036ea3edc7e08f823a625269c1e6919e881a048b4d2@buzz.block.builderlab.xyz> Co-authored-by: Smartie <fb3185faacfc3760b6b4fe0085a8878d8dd537f7a7671c7d3184fe1aa1b04df4@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
781d39510c |
feat(push): support configurable HTTP(S) delivery URLs (#7877)
The relay can use a configurable HTTP(S) gateway delivery URL and signs the same URL it sends. The gateway verifies NIP-98 against the received request URL, including authority, port, path and query. App Attest, scoped grants, replay checks and delivery quotas are unchanged. Forwarding proxies must preserve Host and supply the original request scheme through X-Forwarded-Proto. The NIP relay-delivery wording now defines the configured HTTP(S) URL as the signed URL and labels the public URL as an example. <!-- buzz-review-completed --> --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Luke Tornquist
|
9263cda845 |
docs: specify durable data backfills (#7326)
## Why Data backfills need a durable contract outside schema migrations. The contract must survive crashes and support automatic or manual operation without adding version-selection machinery. ## What - Define one PostgreSQL-backed lifecycle for finite data repair - Treat the stable backfill ID as the only durable execution and readiness identity - Use eventually consistent, idempotent registration; compatible workers compete for one exclusive claim - Require bounded batches, atomic mutation and checkpoint commits, generation fencing, and validation before completion - Define all four migration and automatic-backfill configurations - Define the authorized admin API and honest client projection ## How A backfill captures one immutable upper bound, processes finite batches, and commits each target mutation with its checkpoint. PostgreSQL owns lifecycle state, claims, progress, and completion. Registration order and build metadata do not choose a definition. Compatible processes are interchangeable. A material behavior change requires a new stable ID. The orchestrator does not resolve different behavior registered under one active ID. Diagnostic validation on completed rows records only its bounded outcome and audit; it does not create an execution claim. The spec now starts with a six-step mental model and follows the operator flow. It removes repeated requirements and cuts the document from 614 lines and 4,835 words to 474 lines and 3,656 words. ## Risk Low. This PR changes documentation only. It commits Buzz to a design contract, but it does not add runtime, schema, configuration, or protocol behavior. ## Testing No manual runtime testing. A fresh full-spec review checked the lifecycle, safety rules, four configuration cells, admin contract, and production-seam conformance requirements. `git diff --check` and the repository file-size policy pass at `13681664674d4667b181a6211f012090fa40a2cf`. ## Bigger picture The design keeps the relay as orchestrator, a focused backfill crate as domain owner, and `buzz-db` as narrow transaction infrastructure. It removes deployment generations and definition precedence from the implementation plan. Generated with Codex --------- Signed-off-by: tornquist <tornquist@squareup.com> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Will Pfleger
|
8f8c4dfade |
feat(desktop): relay admin console for the /api/admin/v1 operator surface (#4768)
Adds a relay admin console under Settings — the single surface for relay-level operator work — replacing the previously unreachable moderation section. Operators authenticate with their own Nostr key over NIP-98 — no browser extension, no bearer token — to view deployment-wide moderation reports and product feedback, resolve/dismiss/escalate/reopen reports, update feedback status, and manage the operator/moderator staffing roster. ## Nav reachability The old `moderation` section id was defined in `settingsSections` but never wired into any group in `settingsNavGroups`, so the sidebar never rendered it — unreachable since #1617. This PR replaces it with a `relay-admin` section wired into the Communities nav group, pointing at the relay admin console, and drops the separate `admin-console` section id so a single surface owns relay-level trust & safety. The Relay admin nav entry is always visible. Auth gates the panel itself; the nav entry is a door, not a credential — hiding it behind discovery state would strand an operator whose relay transiently fails NIP-11 discovery with no path back to the connection settings. ## Discovery and connection On mount the console reads the connected relay's NIP-11 document for the optional `admin_api` field. When a valid origin is discovered **and its host matches the connected relay's host** (case-insensitive, host identity only), it is automatically saved and probed — the panel renders immediately with no Save step required. A cross-host advertisement is pre-fill only: it populates the manual origin field under Advanced and is never saved or probed without an explicit Save, so a relay cannot direct an unconsented NIP-98 signature at a third-party origin; residual exposure on the same-host path is bounded by NIP-98's URL/method/payload binding. Saving is only required for manual origin changes. The advertised origin is untrusted input: revalidated through `AdminOrigin::parse`, falling back to manual entry when absent or invalid. A saved manual origin always takes precedence over discovery. ## Auth and transport (Rust, `src-tauri/src/commands/admin/`) - `AdminOrigin` value object: validates scheme + host + optional port, rejects credentials/path/query/fragment; `http://` only for loopback. The relay side normalizes its configured admin host at config load and compares inbound `Host`/`Origin` hosts case-insensitively. - `AdminRoute` closed enum: no IPC surface accepts arbitrary URLs, so the NIP-98-signed URL is byte-identical to the fetched URL. - Dedicated no-redirect `reqwest` client as an SSRF guard: a relay `3xx` is an error and the NIP-98 header never crosses origins. - `admin_probe` signs `GET /probe` and returns a typed state (`nip98Authorized` / `nip98Denied` / `disabled` / `notAdminApi` / `networkOrIntercepted`). The response body is validated against the full relay contract — `nip98Authorized` only when the complete NIP-98 invariant holds, `disabled` only on a coherent disabled-mode body; any incoherent or unrelated JSON classifies as `notAdminApi`, so the console never fails open at the network boundary. Unknown fields are tolerated for forward compatibility. - NIP-98 signing via `AppState::signing_keys()`, one retry on `401` with a fresh signed event; response size bounded by a `Content-Length` preflight and a streaming byte counter; per-pubkey origin storage with atomic `0o600` writes. ## Relay admin console The console is gated behind the relay `OPERATOR`/`MODERATOR` roles from #3777; every `/api/admin/v1/*` route returns `403` to anyone else. Community self-administration (`community-members` / Invites) and in-channel enforcement (the 9040–9043 signed commands) are a separate axis and are untouched. The previously-duplicate community report queue (`ModerationQueueCard`, `moderationQueue.ts`, and three dead hooks) was unreachable dead code and is deleted in its own commit; `useBanMemberMutation` and the rest of `features/moderation/hooks.ts` remain in use by the members sidebar and message menu. ## Console UI (TypeScript, `src/features/admin-console/`) - `AdminConsoleSettingsCard`: discovery + probe flow, honest copy for every probe state, manual origin under an `Advanced` disclosure at the bottom. The role (operator/moderator) is rendered in the connection status line as plain text ("Connected as operator") — no badge-shaped elements above the tab row. The origin provenance (relay config / database) appears inside the Advanced card as small muted text. Section title changed to `Admin`. - `AdminConsolePanel` with Reports / Feedback / Staffing tabs (Staffing gated on `role === "operator"`); reports and feedback grouped by community for cross-community triage. - **Reports**: always requests `scope=all` so the existing resolve/cancel/reopen controls can reach every status (`open`, `processing`, `resolved`, `dismissed`, `escalated`). The relay's omitted-scope default is `escalated`-only (the platform-safety backstop); `scope=all` is explicit and scoped to this console. Full action matrix per `target_kind` (event → delete/kick/ban/timeout/dismiss/escalate; pubkey → ban/timeout/dismiss/escalate; blob → dismiss/escalate); kick is suppressed when the report carries no `channelId`. A `processing` row stays navigable — enforcement state lives in the detail view. A `failed` action (always pre-mutation) offers a single **Cancel & reopen** via `POST /reports/{id}/cancel` with `{actionId}` fencing; there is no client-side retry. `pending`/`enforcing` actions belong to the relay recovery worker and offer no button; a `409` reloads detail. Lists refetch on back-navigation after a mutation. - **Report failure UX**: `reportErrorMessage()` strips 4xx/5xx status prefixes and surfaces the relay's own error reason in the toast (e.g. relay returns `"400: you cannot report this content"` → toast shows `"you cannot report this content"`). Self-report blocking UI removed — the relay has no self-report gate, and the failure was purely client-side error surfacing. - **Reason audience disclosure**: the Reason (optional) field shows exact copy under the input based on selected action — Delete discloses that reason is sent verbatim to the affected user and posted publicly in the room; Kick/Ban/Timeout disclose affected-user only; Dismiss/Escalate disclose reporter only. These mirror the actual relay notice paths. - **Enforcement history**: the report detail carries the governing action even after the report leaves `processing` — a report enforced then reopened is `open` yet still shows its `succeeded` action as executed history alongside the resolve form; a reopen does not un-happen a ban. The resolve form gates on `open` status alone. - **Reopen**: terminal reports (`resolved`/`dismissed`/`escalated`) can be returned to `open` for re-triage via `POST /reports/{id}/reopen`. Reopen never reverses applied enforcement, and the copy says so. - **Feedback**: list, detail, and status control (`new`/`reviewed`/`archived`) with a generation-fenced attachment viewer. A purged source community severs provenance to `null` rather than deleting the row; severed rows bucket under a "source community removed" heading. - **Staffing**: Add is create-only with display names resolved via `useUsersBatchQuery` and npub cross-fade on hover (`HoverStaffingIdentity`). An entered key that already appears in the authoritative roster (config, owner-fallback, or DB) is rejected locally with an inline duplicate-key message and no request is issued; a `409 Conflict` from add, role change, or remove surfaces the relay's parsed error body via `adminErrorMessage(e)` — distinguishing config-backed key conflicts from last-operator conflicts without HTTP-status inference. Roster with `config`/`owner_fallback`/`db` source labels; config-backed entries disable remove client-side. In-place role change via `<select>` calling `putAdminOperator`; the relay responds with the effective `OperatorEntry` (`pubkey`, `effectiveRole`, `sources`) in the same shape as the roster list. Read-only badge for config-managed entries. - **Timeout expiry**: `useTimeoutState` clears the store when derived state is `INACTIVE` but the store is still active; the 1s tick is scoped to entries with a known `expiresAtMs` and owned by `ComposerTimeoutBanner` (mounted only while a timeout is active), while `ChannelPane` subscribes to the boolean `useTimeoutActive()` — the pane re-renders when a timeout starts or ends, not every second. `useMembersSidebarModeration` ticks `nowMs` only while the sidebar is open, the viewer can moderate, and some member has a future `mutedUntil`; the tick after the last expiry stops the interval. - Staffing refreshes the principal after a self-demotion or self-removal: `AdminConsolePanel` passes `onSelfMutation` to `StaffingTab`, which re-probes the saved origin so the role badge and tab visibility update without a manual re-probe. - Staffing mutation failures, including both 409 variants (config-backed key vs. last operator), surface the relay's parsed error via `adminErrorMessage`, so the last-operator recovery guidance is shown. - Mutations confirm via `sonner` toasts; failures surface the relay's parsed error `message`, not raw JSON. The `disabled` auth mode renders read-only. ## Mutation idempotency Resolve and reopen carry a per-attempt `requestId`. On first submit the whole command (`{requestId, action, reason, expirationSecs}`) is frozen; after an ambiguous failure (409, 5xx, transport error, truncated response) the snapshot is retained, action/reason/duration controls are locked, and the retry sends the exact same payload byte-for-byte. A definitive pre-commit rejection (non-409 4xx with `bodyComplete: true`) discards the snapshot, unlocks controls, and the corrected submit uses a fresh payload. On success the toast derives from the authoritative `AdminReportResolution` (`activeAction.action` when present; otherwise `status` for decision-only outcomes) — never from the mutable form selection. The native mutation commands — including operator delete — reject with a typed `AdminMutationError` carrying the relay's HTTP status (`relayStatus`, `null` when no verdict exists) and a `bodyComplete` flag that is `true` only when the full response body was read. ## Restriction routes contract change `GET /members/restrictions` and `DELETE /members/{pubkey}/ban|timeout` now take `communityHost` instead of `communityId`; the relay resolves the tenant itself and returns `400 unknown_community_host` for an unmapped host. This is a breaking contract change with no compatibility shim: the relay must run this build before a desktop that uses it, scripts sending `communityId` must switch, and rolling the relay back breaks the desktop Restrictions UI. `docs/admin/README.md` documents the three routes in the route inventory and carries the migration note. Restriction calls also carry the relay the list was loaded from, and the native side rejects a mismatch before building any URL. ## Hardening - Saved admin origins are stored per pubkey and per relay host. The host slug parses the relay URL, keeps the port-drop for DNS names, and encodes IPv6 hosts canonically, so two IPv6 relays never share a file and no IPv6 slug can match a DNS slug. - NIP-11 admin discovery has its own 10s request deadline and reads `/info` success and error bodies through the bounded reader with a 64 KiB cap (`Content-Length` is only a pre-check). - `PUT /operators` builds its `OperatorEntry` response from the validated body (`effectiveRole = role`, `sources = ["db"]`) instead of re-reading the roster, so a concurrent change can't turn a committed write into a `403`. Related: #3777 (relay `OPERATOR`/`MODERATOR` role model + NIP-98 auth + `admin_api` NIP-11 advertisement — provides the runtime and the discovery field this console consumes) Related: [builderbot-platform-core-infrastructure#140](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/140) (prod admin ingress allowlist and NIP-98 rollout) ## Screenshots Captured headless at `197f0ba0c` against the mock bridge (1280×720). ### Setup **Origin setup — Advanced disclosure open, idle state**  **Authorized console — all three tabs**  ### Reports **Reports queue — open, processing, resolved across two relay groups**  **Open report detail — reporter, target, message snapshot, resolve form**  **Processing report — PROCESSING/HARASSMENT, ban enforcing state**  **Resolve form — Delete selected, audience disclosure copy**  **Resolved report — enforcement history block**  ### Feedback **Feedback list — community grouping with status badges**  **Feedback detail — mutable status control (operator mode)**  **Feedback detail — passive status badge (read-only/disabled mode)**  ### Staffing **Staffing roster — config/owner\_fallback/db source labels**  **Add form — new pubkey filled, ready to submit**  **Duplicate rejection — inline error for existing principal**  --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <wpfleger@block.xyz> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Signed-off-by: Alia <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Alia <d32955ad69077062930cc46cfe2df30ca9aaf6f8e76422681265e9e9af704d78@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
dd1c2afc2a |
Discover alternate Buzz ACP commands (#6948)
🤖 I’m Larry. ## Summary Managed-agent definitions can now choose which installed ACP transport will launch the agent. The choice sits beside **Agent harness** while creating or editing an agent, so it is made before first deployment and remains visible afterward. Buzz discovers executable commands named `buzz-*-acp`, keeps **Buzz ACP (default)** as the safe default, and preserves existing custom command values. Discovery and launch share the same resolver, so a command offered in the picker is the command Buzz will execute. ## UX walkthrough ### 1. The ACP command picker appears while creating the agent The default is explicit, and the helper text explains that this selection controls deployment.  ### 2. Installed wrapper commands are offered by name Here Buzz has discovered `buzz-janet-acp` alongside the stock transport.  ### 3. The wrapper is selected before deployment  ### 4. The deployed profile reports the effective command  ### 5. Editing the deployed agent reopens its definition with the choice preserved  ## Details - Definitions carry the selected ACP command through relay publication, snapshots, imports, team adoption, deployment, and later edits. - Existing definitions without this field continue to use `buzz-acp`. - Unknown persisted commands remain visible but unavailable rather than being silently replaced. - Discovery is read-only and never executes candidates. It filters non-files and non-executables, supports Windows `.exe`, `.cmd`, and `.bat` shims, deduplicates aliases, and returns a stable sorted list. - Command names remain portable; Buzz does not persist machine-specific absolute paths. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Signed-off-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz> Signed-off-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@nostr> Co-authored-by: Larry <8cf5a83f590ec0955b11647d1c88f796a98e088c30a492c58e0e46c3026ae7a4@buzz.block.builderlab.xyz> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Kalvin C
|
5a349db91c |
feat(relay): add opt-in newest-first thread windows (#7823)
## Summary Add an opt-in, newest-first thread window to the authenticated HTTP query path. A filter with `thread_window: true` returns: - reply rows ordered by `(created_at DESC, id ASC)` with an exact composite cursor; - optional bounded reactions, edits, deletions, and delete-of-aux closure; - one relay-signed `kind:39007` bounds event bound to the normalized host, authenticated reader, channel, root, request cursor, kinds, depth, limit, and aux choice. Filters without the flag keep the existing oldest-first thread behavior. Channel windows and `kind:39006` are unchanged. No client opts into this mode in this PR. ### Why extend NIP-CW NIP-CW already defines relay-computed, cursor-paged conversation views over extended NIP-01 filters. Thread windows use the same protocol ideas: ordinary signed rows, composite keyset pagination, optional aux closure, and relay-signed bounds. Adding a thread mode keeps those related wire contracts together instead of creating a second NIP with duplicated terminology and compatibility rules. Extending a draft, optional NIP with an additive mode is normal protocol evolution. Existing fields and meanings are not reinterpreted: - `top_level: true` remains channel mode; - `kind:39006` remains channel-window bounds; - `thread_window: true` selects the new mode; - `kind:39007` has a distinct identity and root/request binding; - extension-unaware clients never send the selector; - legacy thread queries remain unchanged. NIP-CW documents only the wire contract and relay semantics. Database plans, replica internals, and rollout procedures remain implementation concerns. ### Why thread windows use independent modules The implementation deliberately keeps `thread_window.rs` separate in `buzz-core`, `buzz-db`, and `buzz-relay` rather than adding branches throughout the existing channel-window path. The modes share pagination concepts, but their invariants differ: - channel mode selects top-level rows; thread mode requires one root and depths `1..=depth_limit`; - thread mode accepts only conversation row kinds and rejects unknown/conflicting fields; - thread bounds bind the root and authenticated reader because cross-channel aux visibility can differ by reader; - thread aux includes the root and requires complete two-hop closure; - thread mode refreshes writer authorization before selection and before signing; - historical cursor pages may use a proved replica transaction, but fallback must restart both aux hops on the writer; - legacy thread traversal is oldest-first and must not inherit descending cursor semantics. Forcing these rules into channel-window code would create a mode flag across parsing, SQL predicates, bounds construction, authorization, replica fallback, and tests. That abstraction would not remove complexity. It would distribute it. Separate modules keep each contract local and leave the established channel and legacy paths untouched. ### Technical design - Strict request parsing normalizes UUIDs, event IDs, kinds, defaults, and cursors before computing the response binding. - SQL applies community, root, channel, depth, kind, deletion, and cursor predicates before `limit + 1`. - Ordered metadata scanning plus exact event-key lookup avoids the stale-statistics plan that previously exceeded the four-second SQL budget on a 10k-reply thread. - Aux work is bounded by target batches, query count, raw candidate count, response bytes, SQL timeouts, and one shared HTTP deadline. - Fresh authorization-set comparison catches both grants and revocations during closure. The relay returns retryable `503` rather than signing an incomplete page. - Replica failure permanently degrades the request to the writer and restarts closure from its original targets without resetting budgets. - Migration `0049_thread_window_index.sql` adds the mixed-direction index used by newest-first scans. It validates a prebuilt index and bounds startup lock/build time. ### Tradeoffs - The stale-statistics-safe plan is intentionally less clever. Measured warm 100k deep-cursor execution increased from roughly 0.30 ms to 4.80 ms, plus about 8 ms planning, in exchange for avoiding a reproduced multi-second timeout after rapid 10k-thread insertion. - Bounds are authoritative only for the served page. There is no cross-page snapshot guarantee, and replica insertion coverage does not imply latest edit/deletion/aux visibility. - Strict parsing rejects unsupported constraints instead of silently serving a different row set. - Required aux closure fails closed. The relay does not omit aux and sign plausible-but-incomplete bounds. - This PR ships relay capability only. Client rendering, retry UX, live-history overlap, and compatibility fallback remain separate work. ### Related issue No existing issue found. This work follows the thread-loading design and review discussion. ### Testing - `cargo test -p buzz-core`: 258 unit tests and 2 doctests passed. - `cargo check --workspace --all-targets`: passed at the rebased implementation head. - `cargo test -p buzz-db`: 131 non-PostgreSQL tests passed; 275 infrastructure tests were ignored as declared. - Focused PostgreSQL migration/runtime regressions: 3/3 passed against an isolated PostgreSQL container, including both corrected migration-ledger tests. - `cargo fmt --all --check`: passed. - Pre-rebase feature validation passed 398/398 PostgreSQL tests, signed-wire multi-tenant/auth/pagination/aux/revocation probes, clean Compose boot/reboot, and full core tests. - Independent HA validation exercised three relay pods and a physical PostgreSQL standby on the same runtime logic before the final documentation-only cleanup. - Independent final-delta review confirmed exactly two changed files, byte-identical executable migration SQL after comments/diagnostic normalization, matching prebuild/catalog/recovery instructions, and `git diff --check`; no runtime receipts were relabeled to the new SHA. - The pre-push hook ran repository checks but its Rust unit lane exited 100 without exposing an individual failed test in the captured multiplexed output. The independently run affected-package checks above passed; current-head repository CI remains required. Known unrelated baseline behavior remains: the mesh-demo echo test can return 504, and same-second archive → unarchive can deduplicate its membership notification. Neither path is changed here. --------- Signed-off-by: Kalvin Chau <kalvin@block.xyz> Signed-off-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz> Signed-off-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: cid <d9f92a72922bf45c17379a47d64dae84b6020397c2d5a52b5317d512068cd9d3@buzz.block.builderlab.xyz> Co-authored-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> |
||
|
Salman Mohammed
|
5621006bcf |
refactor: move agent Git bootstrap into ACP harness (#7819)
discovered in https://github.com/block/buzz/issues/7742#issuecomment-5783512985 ## Summary Goose uses its native shell, so it bypassed the Git identity and signing setup inside `buzz-dev-mcp`. Agent commits could use the host Git identity, while MCP shells had a separate config. This moves Git setup into `buzz-acp`, where both paths receive the same agent identity, Nostr signer, and relay-scoped credential helper. Before: ```text Buzz Desktop └─ buzz-acp ← Desktop supplied relay Git authentication ├─ Goose → native shell ← no agent Git author or signing setup └─ Buzz Agent └─ buzz-dev-mcp └─ shell ← MCP created the keyfile and Git signing config ``` After: ```text Buzz Desktop or Sprig └─ buzz-acp ← owns the keyfile, identity, signing, and scoped credentials ├─ Goose → native shell ← inherits the complete Git environment └─ Buzz Agent └─ buzz-dev-mcp └─ shell ← receives the same Git environment via mcpServers[].env ``` The harness keeps the signing key in a private temporary file and removes it on normal exit, startup failure, and graceful termination. It composes inherited `GIT_CONFIG_*` entries, forwards the complete block to MCP servers, and bundles the helpers through the ACP multicall entrypoint. For standard `buzz-acp` launches, Desktop and Sprig no longer write Git config. A custom Desktop ACP command bypasses the harness, so Desktop preserves its prior relay-scoped credential helper for that supported override. This changes Git settings for every ACP runtime. The startup signal path also changes so a termination during adapter initialization removes the keyfile. The temporary file can remain after SIGKILL or a machine crash. ### Related issue Related: #6177 covers stricter commit identity enforcement and touches the same Git paths. That PR and this one need to be reconciled before either lands. No issue was found for this runtime bootstrap gap. ### Testing - Ran the ignored `git_runtime_tests` with real Goose 1.50.0 and Buzz Agent ACP sessions. `real_goose_native_git_shell` passed again on this PR branch with `BUZZ_TEST_BIN_DIR` pointing to built binaries. Goose made a signed commit and tag through its native shell; Buzz Agent did the same through its MCP shell. Both verified signatures, identity, credential scoping, and keyfile cleanup. - Built the Linux Sprig image and ran `scripts/test-sprig-image.sh buzz-sprig:acp-git-bootstrap`. The container made and verified a signed commit and tag, then confirmed keyfile removal after SIGTERM. - `just ci` passed on the original branch. After rebasing onto `main` and addressing review feedback, the ACP Git bootstrap integration tests, focused desktop smoke test, custom-command regression test, formatting, and ACP/Desktop Clippy passed. The pre-push hook hit two timing-sensitive ACP deadline tests under parallel load; both passed alone and the complete ACP package passed serially (964 library, 2 Git bootstrap, 9 pool lifecycle tests). The push used `--no-verify` after those checks; GitHub CI is rerunning on the new head. - Under injected `BUZZ_ACP_ALLOWED_RESPOND_TO`, `BUZZ_ACP_LAZY_POOL`, and `BUZZ_ACP_IDLE_POOL_SLEEP`, the clean-env serial package command in `crates/buzz-acp/TESTING.md` passed: 964 active library tests, 2 Git bootstrap tests, and 9 pool lifecycle tests. Both named Databricks OAuth tests passed locally; another reviewer reproduced their environment-specific failures on `main`. - No authenticated relay clone or push was run. The local probes used disposable repositories and did not contact a relay. Generated with Codex --------- Signed-off-by: Salman Mohammed <smohammed@squareup.com> |
||
|
ravarora2
|
4142d2b84e |
Use worker snapshots for relay storage metrics (#7845)
## Summary Make the relay read completed storage-accounting snapshots from PostgreSQL by default. An environment with no snapshot emits no storage metrics. When a worker publishes its first result, the relay picks it up on the next successful leader usage tick, without a restart or configuration change. This changes how storage usage is measured and reported. **S3 remains the object store. PostgreSQL stores the completed counts, byte totals, community breakdown, and calculation metadata—not the objects themselves.** ### Why move relay storage accounting entirely to database reads? Today the relay supports two sources: its own S3 scan (`inline`, the default) and the worker's saved snapshot (`external`). That requires operators to deploy a worker and then coordinate a separate relay-mode change in each environment. The worker already owns the S3 listing and calculation. It runs in a separate process with its own object cap, memory limit, and deadline. Keeping a second scan inside the serving relay retains the resource pressure and failure modes that this isolation is meant to remove. PostgreSQL provides a durable handoff. The worker replaces one completed snapshot atomically after a successful calculation. The relay reads that row and exposes its values through the existing metrics endpoint. A relay restart or leadership change can load the last completed result without scanning the bucket again. There is deliberately no S3 fallback when a snapshot is absent or a read fails. Such a fallback would put the expensive scan back in the relay precisely when a worker is absent or unhealthy. It would also leave two calculation paths with different resource limits and freshness behavior. ### What happens if no worker exists? **A worker that has never published is a normal, successful inactive state.** The snapshot table must exist through the normal database schema, but it may contain zero rows. The reader returns success, emits no storage totals or failed-load gauge, and checks again on the next usage tick. It does not exit the relay, start a worker, or list S3. The relay checks for a completed database row; it does not inspect Kubernetes Jobs or CronJobs. Worker presence and snapshot presence are therefore different: | Situation | Storage-reader behavior | |---|---| | No worker and no snapshot row | Return success, emit no storage metrics, and query again next tick. | | Worker starts later and publishes its first row | Read and emit it on the next successful leader usage tick; no restart or mode change. | | Worker is removed or fails, but its last valid row remains | Continue reporting the last completed totals with their increasing age. | | Query succeeds but a previously present row has been removed | Clear the cache and stop refreshing storage series; existing exporter series expire through the idle timeout. | | Database read fails, times out, or returns an invalid snapshot | Log a warning, set `buzz_storage_snapshot_load_ok=0`, retain any last-good cached totals, and retry next tick. | | Snapshot table is missing | Treat this as a schema error, not normal worker absence; follow the failed-read behavior above. | | `BUZZ_STORAGE_METRICS=off` | Skip the snapshot query and all storage metric emission. | No row means “not measured,” not zero bytes. A valid completed snapshot of an empty bucket can report zero. If a read fails before any good snapshot has been cached, only failed-load health is emitted. These reads happen in the existing background metrics task, not the relay startup path. A storage-read failure does not request process exit or change readiness directly. A wider database outage can still affect the relay's existing database-dependent operations and readiness. ### Read, parse, and emit path ```text Worker: S3 listing -> BucketSnapshot -> JSON -> PostgreSQL Relay: PostgreSQL -> JSON -> BucketSnapshot -> existing metric gauges Collector: relay /metrics endpoint -> Datadog ``` 1. `main.rs::run_storage_sweep_tick` calls the reader from the existing leader-only usage task. The default interval remains 300 seconds. 2. `storage_sweep.rs::run_storage_metrics_tick` coordinates the read, cache update, failure health, and metric emission. 3. `refresh_persisted_snapshot` calls the existing `Db::load_storage_accounting_snapshot()` method. A five-second timeout covers pool acquisition and the query. It uses the relay's existing pool. 4. The existing query in `buzz-db/src/store/storage_accounting.rs` reads `snapshot`, `completed_at`, `duration_ms`, `max_objects`, and `code_sha` from the singleton row. 5. `serde_json::from_value(stored.snapshot)` decodes the JSON into the existing `buzz_media::BucketSnapshot` type. Rust infers that type from `cache_persisted_snapshot`'s argument. The generated `Deserialize` implementation handles all fields, including the UUID-keyed `per_community` map. 6. `emit_cached_storage_metrics` sets the existing physical/logical totals and community byte/object gauges. The existing Prometheus exporter exposes them for Datadog collection; this code does not send snapshot JSON to Datadog. The worker's `BucketSnapshot`/`CommunityStorage` structures, JSON field names, SQL publication method, and database schema are unchanged. The old external-mode JSON conversion moves out of `main.rs` into the reader helper. No new format or hand-written field parser is needed. Duration and object-cap metadata are validated before replacing the cache. The five-second relay read timeout is separate from the worker's 30-second startup acquisition budget introduced in #7770. This PR does not change the worker's pool or startup logic. ### Configuration and code changes | Configuration | Before | After | |---|---|---| | Unset, `inline`, or `on` | Relay-local S3 scan | Read completed database snapshots. `inline` logs one migration warning. | | `external` or `snapshot` | Read completed database snapshots | Continue reading snapshots. | | `off` | Disabled | Disabled. | | Unknown or empty value | Disabled with a configuration error | Same. | Existing `inline` settings intentionally become reader aliases. Merely changing the default would leave older deployments on their explicit `inline` setting and preserve the need for coordinated configuration PRs. Remove relay-local scan scheduling, in-flight scan state, scan configuration, and obsolete scan-attempt tests and health metrics. Preserve the physical/logical total and per-community metric names, leader-only emission, community scope filtering, and cleanup of old community labels. The shared relay-state field remains in place; its comment now describes a cached worker result. The Helm chart defaults to `relayMode: external` while leaving `storageAccounting.enabled: false`. Its rendering test verifies that the reader is configured even when no worker is created. The reader regression tests cover missing rows, later publication, updates, invalid data, timeout, removal, reactivation, and the explicit off switch. `docs/storage-accounting.md` describes this operating model. ### Freshness and rollout impact `buzz_storage_snapshot_age_seconds` uses the worker's original completion timestamp. Re-reading an old row never makes it fresh. `buzz_storage_snapshot_load_ok=1` means the row was read and decoded; it does not prove that the worker's latest attempt succeeded. Monitor snapshot age against the worker's schedule and allowed runtime, and monitor Job failures separately. The old relay scan-attempt health gauges are retired because the relay no longer runs those scans. Deploy this relay version through the usual release process. Once it reaches an environment, workers can be enabled independently. Already-released charts that still set `inline` work with the new binary; future chart releases default explicitly to `external`. An intentional `off` override still requires an operator to enable the reader. **Environments with no completed snapshot lose relay-calculated storage totals after this upgrade.** That is the intended tradeoff for removing scans from the serving process. Environments with a prior snapshot continue showing its last completed totals and age. This PR does not install workers in additional environments or deploy the new relay image. ### Related issue Related to #4601. Follows the worker startup fix in #7770. The separate fleet-usage collector work in #7176 overlaps this code and may need reconciliation when it merges. ### Testing Built and ran the relay locally against isolated PostgreSQL and Redis with the legacy `inline` setting, five-second usage ticks, and a 15-second metric idle timeout. The same process was exercised through these steps: 1. Start with an empty snapshot table: relay becomes ready and exposes no storage series. 2. Publish a snapshot containing 123 bytes: the community gauge becomes 123. 3. Replace it with 456 bytes: the gauge becomes 456. 4. Remove the row: storage series expire without fabricating a zero-byte measurement. 5. Publish 789 bytes: metrics reactivate without restarting the relay; readiness remains healthy. The local S3 endpoint recorded zero requests. The unrelated Git conformance startup probe was disabled for this isolated reader check; the test covers storage accounting, not every other use of S3 in the relay. No new relay image was deployed to staging or production during this test. The broad local integration run also hit an existing `p0_pool_acquisitions_use_typed_operation_pairs_without_other` failure in `buzz-db/tests/observability_source.rs`. The same source-only test fails on unmodified `main` at `0ef7a2222`; this PR changes neither the test nor its failing source. The full local `just ci` run stopped at mobile native-asset setup: the `objective_c` build hook received no SDK path from `xcrun`. Mobile tests did not run. Generated with Codex Signed-off-by: Ravneet Arora <rarora@squareup.com> |
