mirror of
https://github.com/block/buzz.git
synced 2026-10-28 04:51:40 +00:00
d0c5b7d1878b1e2fef9bf7fe2ba6f8e2f6f388ef
211 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
seiler
|
355783470d |
Merge updated invitation core into historical repair candidate
Retain the separate five-file maintenance delta above the pinned-main update. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: seiler <seiler@squareup.com> |
||
|
seiler
|
1a9d70f698 |
Merge pinned main primary-key repair into invitation candidate
Preserve invitation migration 0069 and admission semantics while integrating main events_pkey reconciliation and schema-parity coverage. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: seiler <seiler@squareup.com> |
||
|
Ian Oberst
|
20ba95ed89 |
fix(db): rebuild events_pkey in declared order after pgschema apply (#8525)
## Summary Re-applying `schema/schema.sql` to a database that `pgschema apply` and `scripts/reconcile-schema-after-pgschema.sql` already bootstrapped fails with `cannot drop inherited constraint "events_p2026_01_pkey" of relation "events_p2026_01" (SQLSTATE 42P16)`. Because of that, `scripts/start-relay-for-tests.sh` breaks against a reused database. **Cause:** pgschema 1.7.4 moves a partitioned table's partition key to the front of its primary key when it creates the table. A pgschema-bootstrapped `events` therefore gets `PRIMARY KEY (created_at, community_id, id)`. `schema/tables/public/events.sql` and `migrations/0001` both declare `(community_id, created_at, id)`. Every later plan drops and re-adds `events_pkey` and each declared child's key. This happens even on a database that was never reconciled. The plan only becomes unexecutable after the reconcile script attaches the children, because their keys are then inherited. As a side effect, pgschema-bootstrapped databases also had a differently ordered PK index than migration-managed ones. **Fix:** after the partition ATTACH block, the reconcile script now compares `events_pkey` with the declared `PRIMARY KEY (community_id, created_at, id)`. If they differ, it rebuilds the key on the parent. It then asserts that `events` and every partition carry the declared key, which is the "convergence plus live assertion" shape `docs/database.md` requires. PostgreSQL recreates each partition's key with the same `<child>_pkey` names, so a reconciled re-apply now plans nothing. The block does nothing when the key is already correct. `migrations/0001` is untouched. Only pgschema-bootstrapped dev and test databases run this script, so it does not touch the out-of-band partitioned-table procedure. `delivery_log` is not affected. Its key `(delivered_at, id)` already starts with the partition key. **Existing local databases:** a database reconciled before this change would still fail its next `pgschema apply`, because the reconcile script only runs after apply. The repair therefore also lives in `scripts/repair-events-pkey.sql`. That file does nothing when `events` doesn't exist yet, and `scripts/start-relay-for-tests.sh` runs it before apply; it's the only launcher that reuses its database. The reconcile script keeps a verbatim copy for fresh bootstraps, and a unit test keeps the two identical. The rebuild drops the key with `IF EXISTS`, so a database that has lost its key still converges. It also takes its locks with a 10s `lock_timeout`. If a relay still holds `events`, it fails with "Stop any relay using this database, then rerun" instead of hanging. `start-relay-for-tests.sh` now builds the relay's `DATABASE_URL` from the `PG*` variables it already exports. The secret scanner flags the literal URI once the file is touched. The value is unchanged. pgschema 1.13.3 keeps the declared order and creates children as native `PARTITION OF`. Upgrading would make this block and the ATTACH blocks unnecessary, but it changes behavior for every caller, so it is left for a separate change. ### Related issue Follow-up to block/buzz#8487, which added the probe test and `.pgschemaignore`. Now rebased onto `main` after that PR merged. ### Testing - `pgschema_reapply_keeps_runtime_partitions` now asserts the plan after the maintenance pass is **empty**, which replaces the `DROP TABLE` and per-monthly plan checks. It then applies and reconciles a second time. Without the new block it fails with `ALTER TABLE events DROP CONSTRAINT events_pkey; ...`. With the block it passes. - It also includes the optional notes from the #8487 review: - After maintenance, it checks that both catch-alls are still present and attached, not only that the zipped bounds differ. - The first runtime monthly comes from the declared catch-all's lower bound instead of a hard-coded `p2026_07`. - The doc comment now states only what the test pins: bounds are not compared, and the plan is empty. - I broke the rebuild on purpose by re-adding the key in the wrong order. The new assertion fails the reconcile script with `events and its partitions must have PRIMARY KEY (community_id, created_at, id)`. - CLI runs against local Postgres 17: - Fresh DB: apply → reconcile → apply → reconcile → plan. The second apply reports no changes and the plan is empty. - DB reconciled with the old script: the next apply fails with SQLSTATE 42P16. After running the new reconcile script once, apply reports no changes and the plan is empty. - `events_pkey_repair_runs_before_reused_apply_and_matches_reconcile` checks two things: the reconcile script carries the repair block verbatim, and `start-relay-for-tests.sh` runs the repair before `pgschema apply`. It fails if the two copies drift. - I ran the repair script and then the bootstrap against four local probe databases: one reconciled with main's script, an empty one, one applied but never reconciled, and one with no `events` key. All four end with an empty plan. With another session holding a lock on `events`, the repair fails after 10s with the hint and leaves the key untouched. It succeeds once the lock is released. - `admin_schema_parity_between_desired_state_and_migrations` and the `runtime::migration` unit tests pass. fmt and clippy are clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
seiler
|
b9c3840135 |
Merge repaired invitation core into historical maintenance candidate
Retain the standalone repair and its approval, writer-pause and repair-twice requirements. Update its upgrade boundary to main schema 0068 and invitation expansion 0069 without changing executable repair SQL. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: seiler <seiler@squareup.com> |
||
|
seiler
|
2fabeed17e |
Merge main into invitation revocation candidate
Preserve entity deletion migration 0068 and move the schema-only invitation expansion to 0069. Keep both migration inventory contracts and exercise populated invitation upgrades from the complete main schema through 0068. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: seiler <seiler@squareup.com> |
||
|
ravarora2
|
2d289f720d |
BUZZ-301: Add user deletion request intake (#8496)
## Summary
Add the first testable piece of user deletion: submit a synthetic
request, list requests, and inspect one by UUID. Operators can verify
that intake reaches PostgreSQL before we add inventory, approval, and
execution.
Add one operator-global `entity_deletion_requests` table with an
additive migration and matching desired-state schema. Identical
submissions return the same row. Conflicting submissions fail without
changing it. Only `demo:` targets are accepted, and requests stay
`submitted`.
Keep the commands in `buzz-deletion`, reached through a thin `buzz-admin
user-deletions` adapter. Reuse the existing database connection setup.
This change adds no worker, stage history, community gates, or deletion
operations.
### Rollout plan
This is an expand-only schema change plus an operator CLI. Existing
relay code does not use the new request table. No backfill or real
deletion is part of this release.
1. **Order the PRs.** Land #8496 first, using 0068; main now owns 0067
for channel join roles. Rework #8489 to build on this table with the
next available migration number. Renumbering its existing CREATE TABLE
migration alone is insufficient; both PRs currently create the same
request table.
2. **Apply on staging.** For deployments that manage schema separately,
apply the reviewed additive migration before using the new CLI. Where
auto-migration is enabled, the new relay binary applies it at startup.
Keep existing tables and records unchanged.
3. **Verify on staging.** Use the new `buzz-admin` binary with
`DATABASE_URL` pointing to staging. Submit a unique synthetic label,
list requests, and inspect the returned UUID:
```bash
buzz-admin user-deletions submit --target demo:staging-intake-1
--requested-by <operator>
buzz-admin user-deletions list --limit 5
buzz-admin user-deletions inspect <returned-uuid>
```
Verify one `entity_deletion_requests` row with status `submitted`.
Repeat the identical submission and verify the same UUID and timestamps.
Use a new label if this label was used by an earlier staging test.
4. **Apply beyond staging.** After verification, use the deployment's
reviewed schema/release workflow to add the table in the remaining
environments. Requests run only through the operator CLI; this release
starts no deletion worker. No feature flag is needed for this synthetic
intake: the database accepts only `demo:` targets, and real deletion is
absent.
5. **Rollback.** Keep the additive table and saved requests; stop using
the new commands and roll back the binary if needed. After 0068 is
applied, an older binary with auto-migration enabled can fail at startup
with sqlx `VersionMissing`. A binary rollback therefore requires
migration execution to be disabled on the older relay
(`BUZZ_AUTO_MIGRATE=false`, including the corresponding deployment
configuration).
### Related work
This is a smaller first slice of the full-workflow draft
[#8489](https://github.com/block/buzz/pull/8489), which is preserved for
later extraction. The two PRs overlap and should not both be merged as
written.
### Testing
The author submitted `demo:rarora-manual-1` to a separate local
PostgreSQL 17 database. Repeating the same submission returned its
original UUID and timestamps. List, inspect, and a direct SQL query all
returned the same saved request.
<details>
<summary>Actual command inputs and outputs</summary>
Commands ran from the new worktree with `DATABASE_URL` set to the local
`buzz_user_deletion_submit` database. Outputs below are stdout;
connection diagnostics on stderr are omitted.
**Submit again — returns the existing request without creating a
duplicate**
```bash
./target/debug/buzz-admin user-deletions submit \
--target demo:rarora-manual-1 --requested-by rarora
```
```json
{
"id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89",
"entity_type": "user",
"target": "demo:rarora-manual-1",
"status": "submitted",
"requested_by": "rarora",
"reason": null,
"created_at": "2026-10-20T19:57:44.516698Z",
"updated_at": "2026-10-20T19:57:44.516698Z"
}
```
**List saved requests**
```bash
./target/debug/buzz-admin user-deletions list --limit 5
```
```json
[
{
"id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89",
"entity_type": "user",
"target": "demo:rarora-manual-1",
"status": "submitted",
"requested_by": "rarora",
"reason": null,
"created_at": "2026-10-20T19:57:44.516698Z",
"updated_at": "2026-10-20T19:57:44.516698Z"
}
]
```
**Inspect one request**
```bash
./target/debug/buzz-admin user-deletions inspect 2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89
```
```json
{
"id": "2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89",
"entity_type": "user",
"target": "demo:rarora-manual-1",
"status": "submitted",
"requested_by": "rarora",
"reason": null,
"created_at": "2026-10-20T19:57:44.516698Z",
"updated_at": "2026-10-20T19:57:44.516698Z"
}
```
**Read the table directly**
```bash
docker exec buzz-user-deletion-dev-20261020 \
psql -X -U buzz -d buzz_user_deletion_submit -c \
"SELECT id, entity_type, target, status, requested_by
FROM entity_deletion_requests ORDER BY created_at DESC;"
```
```text
id | entity_type | target | status | requested_by
--------------------------------------+-------------+----------------------+-----------+--------------
2e5bcdec-ab68-44d2-9743-1dd9fa7d2f89 | user | demo:rarora-manual-1 | submitted | rarora
(1 row)
```
</details>
A separate CLI smoke test also verified that a conflicting operator and
a non-demo target fail without changing the saved request. The existing
community-deletion `list --limit 1` command passed against the smoke
database.
See [the intake
walkthrough](
|
||
|
Brad Seiler
|
6ad196c997 |
Add invitation history repair as a separate maintenance change
Revoke historical legacy candidates and clear examined admission markers under the deletion/admission locks. Require separate approval before core deployment and a controlled rollout that prevents new timeout/mute history. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Brad Seiler <seiler@squareup.com> Signed-off-by: seiler <seiler@squareup.com> |
||
|
seiler
|
9017435270 |
Separate invite history repair from schema rollout
Signed-off-by: seiler <seiler@squareup.com> Co-authored-by: Codex <noreply@openai.com> |
||
|
seiler
|
0b18470638 |
Merge latest pinned main into invitation repair candidate
Signed-off-by: seiler <seiler@squareup.com> |
||
|
Luke Tornquist
|
039e1c8d59 |
feat(relay): write the NIP-43 member list in the membership transaction (#8482)
## Summary The relay-signed NIP-43 member list (kind:13534) was published after each membership change had already committed. If that publish failed, or if a change never republished at all (startup `bootstrap_owner`), the list stayed wrong until the background reconcile sweep found and fixed it. #8467 moved that sweep off the startup path. This PR removes the reason the sweep exists. Now every `buzz-db` function that changes `relay_members` writes the new list **in the same transaction**, as its last statement before commit: - It takes the per-community list lock last, reads the members, signs the list, retires the old one and inserts the new one. - If the list write fails, the membership change rolls back and the caller gets an error. - A call that changes nothing (already a member, role mismatch, host exists, limit reached) writes no list. The functions return `MembershipWrite<T> { outcome, snapshot }`. Callers push `snapshot` to live subscribers after commit. That push is the only thing left outside the transaction; if it fails, live clients see the change on their next fetch, and the stored list is already correct. Write paths covered (RPR #2, every write path): `add_relay_member`, `claim_relay_membership`, `claim_relay_invite`, `remove_relay_member`, `remove_relay_member_if_role`, `update_relay_member_role`, `bootstrap_owner` / `provision_owner`, `transfer_ownership`, `backfill_from_allowlist`, and the owner insert in `create_community_with_owner[_and_nip_fi_issuers]`. Community deletion removes the members together with the community, so it writes no list. The relay keypair is passed in explicitly instead of being stored on `Db`, so the compiler finds every caller. ### Behavior changes - **Open relays get an accurate list too.** Provisioning and ownership transfer used to publish only when `require_relay_membership` was on. The write is now unconditional. - **Same-second changes can't reuse an id.** The list's `created_at` is one second past the newest earlier list for that community, including retired ones, or the current time if that is later. Before, adding and then removing a member in the same second could produce an event with the same id as an earlier list. The insert then failed, and the stale list stayed live. A burst of changes can push `created_at` ahead of the clock by up to one second per change; it returns to the clock once the burst ends. - **`buzz-admin add-member` / `remove-member`** no longer build their own list (`publish_membership_list_with_bump` is deleted). They push the list the database wrote. Concurrent CLI runs now serialize on the database lock rather than depending on `run.sh` running one at a time. - **Owner mutations lock the community row `FOR NO KEY UPDATE`.** `transfer_ownership` and `bootstrap_owner` used `FOR UPDATE`. The list insert takes `FOR KEY SHARE` on that row through `events.community_id`'s foreign key, so a repair holding the list lock, or a removal or role update holding a member row, deadlocked with a concurrent owner change. `FOR NO KEY UPDATE` still conflicts with archive, unarchive and deletion admission, so the lifecycle ordering is unchanged. - **Startup delivers its list to other replicas.** When backfill or `bootstrap_owner` changes members at startup, the relay fans the newest list out through pub/sub once its state is ready, so clients connected to other replicas during a rolling deploy see an owner rotation without a refetch. - **New `buzz-admin republish-members`.** Rewrites the community's list from the stored members in one community-write transaction and pushes it live. This replaces the sweep as the repair for a list that is missing or stale: a `relay_members` change made directly in SQL, a restore, or a deployment that turns on `BUZZ_REQUIRE_RELAY_MEMBERSHIP` after running without it. ## Rollout Nothing rewrites a list automatically once the sweep is gone. A community whose list drifted before this deploys stays wrong until its next membership change or until an operator runs `buzz-admin republish-members`. Either deploy #8467 and let its sweep complete one pass first, or run `republish-members` for each community after this deploys. The command repairs only the community that `RELAY_URL` maps to, so on a relay hosting many communities, run it once per community host. ## What got simpler Deleted: - `reconcile_nip43_membership_snapshots`, `spawn_nip43_membership_reconciler`, its leader lock key and #8467's standby retry - the three `nip43_membership_snapshot_needs_reconciliation*` drift reads (one already deprecated) - `publish_nip43_membership_list` / `publish_nip43_membership_locked`, the separate after-commit republish at every call site, and `buzz-admin`'s duplicate publisher - the reconcile metrics `buzz_nip43_membership_reconciliations_total` and `buzz_nip43_membership_reconciliation_failures_total` - the after-commit publish metrics `buzz_nip43_membership_publications_total` and `buzz_nip43_membership_publication_seconds`, and the `publish_nip43_membership_locked` transaction metric label. A failed list write is now a failed membership write and shows up in the caller's existing error handling and the transaction metrics of its write path. - the `BUZZ_NIP43_RECONCILE_INTERVAL_SECS` setting There is now one way the list gets written, and it can't disagree with `relay_members` after commit. **Unused deploy config (outside this repo):** hosted deployments that set `BUZZ_NIP43_RECONCILE_INTERVAL_SECS` can drop it. It is now ignored. Dashboards or alerts on the four deleted metrics will stop receiving data. ## Testing New Postgres tests (`buzz-db`): - `relay_members::every_membership_write_commits_its_list`: for each write path, the returned list is the single live kind:13534 and names exactly the committed members. No-op calls return no list and leave the live list alone. - `relay_members::a_failed_list_write_rolls_back_the_membership_change`: holds the list lock, waits for the join to block on it, cancels that statement, and checks that the member was not added and the previous list is still live. - `relay_members::concurrent_joins_all_land_in_the_list`: 8 concurrent joins; the final list names all 8. - `relay_members::same_second_return_to_an_earlier_membership_writes_a_new_list`: add, remove, add, remove within a second; every list is newer than the last and no id repeats. - `relay_members::owner_transfer_and_list_repair_both_commit` and `relay_members::owner_transfer_and_member_removal_both_commit`: hold a repair (list lock) or removal (member row) open, wait until a transfer blocks behind it, then finish it. Both transactions commit; with `FOR UPDATE` back in place, one of them fails. - `community::create_community_with_owner_is_atomic_and_create_only` and `relay_invite::bounded_claim_exhausts_and_existing_member_retry_does_not_consume` now also check the list. Each new test was checked to fail against a mutation of the new code that brings back the old behavior (details in the thread). Stacked on #8467. Merge order: #8467, then #8468, then this. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
c9e35f8ac7 |
fix(mobile): restore Solo chats from channel labels (#8466)
Restore Solo chats and project placement on mobile using `buzz.solo`
labels and creator metadata. Invalidate stale channel details when
identity changes so opening and sending preserve refreshed Solo
identity; shared chats retain ordinary sending. New mobile Solo chats
publish the discovery label, and the personal send guard enforces
ownership even when route hints or legacy placement suggest Solo.
The current desktop producer lives in `block/buzz-app`. [Contract
documentation](
|
||
|
Wes
|
edbfbf08db |
test(mobile): journey-first testing guidance and suite cleanup (#8397)
## Summary This PR does two things. It updates the mobile testing conventions to favor a few journey tests over many small logic tests. It also applies those conventions to the existing mobile suite, one commit per step. ### Testing conventions (`AGENTS.md`, `mobile/README.md`) - Start with one widget journey per user-visible flow, driven through the real widgets, notifiers and coordinators. Assert what the user sees and what is durably saved. - Cover retry, cancel and resume as steps inside the journey, not as separate tests. - Fake only external boundaries (network, relay, OS, browser, storage). - Use focused, table-driven tests only for security and data-loss invariants, protocol or crypto vectors, and pure functions with many input combinations. - When review finds a defect, extend an existing journey or table instead of adding a test per finding. - Delete a lower-level test only once a journey demonstrably catches the same failures. - Every test must fail when the production code it guards is broken. These conventions keep the Review-Proven Rules' safety bar and change only how coverage is organized. ### Suite inventory and plan The suite has 234 Dart test files, about 110k lines and about 2,600 declared tests. Native tests are separate: BuzzPushKit, RunnerTests, NativeControlTests and Android. Line-saving estimates are hypotheses, not a deletion budget. Each deletion needs a replacement that rejects a representative production mutation. 1. **CI and reliability.** Run the native tests CI skipped, make time-dependent tests deterministic, and make tests that could not fail falsifiable. *(done, see below)* 2. **Shared harness.** Add one fake relay (socket plus HTTP boundary, with the real `RelaySessionNotifier`) in `test/helpers/`. Today 38 files each write their own `RelaySessionNotifier` fake. *(done, see below)* 3. **Per-area journeys.** Convert page tests that fake the flow's own notifiers (for example, `channel_detail_page_test` overrides about 60 providers). Fold near-duplicates into tables. Remove constant-pinning, pixel-geometry and call-count tests once a journey covers the outcome. Split `channel_detail_page_test.dart` (17.7k lines). Each area commit reports before/after counts and the mutation it rejects. 4. **CI speed.** Shard by file instead of by test, since every shard currently compiles the whole suite. *(done, see below)* Keep as-is: relay protocol and NIP handling, crypto vectors, auth, push security, persistence and journal invariants, recipient routing, and real-socket transport and cancellation tests. ### Step 1 changes - **Android CI** runs the 23 JVM unit tests, which were never run before. It retries the APK build once, only on SDK or plugin download failures (the cause of 4 recent `main` failures). - **iOS CI** runs all RunnerTests (previously 2 of 99) and the previously uncalled Me-agent accessibility script. The Swift Native lane builds the simulator app once, in the background while the simulator boots and runs the native-control scripts, instead of twice. The lane stays non-required. - **Stale RunnerTests.** Two reaction-details tests still expected the filter list that #8280 replaced; they now cover the toggle list. #8393 moved navigation bar controls onto a shared glass surface and made unknown Tabler symbols assert. The avatar and members-activity tests now find controls on that surface, three tests use real symbol names, and a 240 pt title width pinned to the old control sizes is removed (its gap assertions remain). - **Known bug, not fixed here.** Since #8393 the DM title avatar image is nil, because `makeItem` returns avatar data as a custom-view item. No screen passes a title avatar yet. The test marks it `XCTExpectFailure`. - **Typing retry** and **sections backoff** tests use fake time instead of about 37 s of real sleeps and a `Stopwatch < 100ms` threshold. - **Manage-channel sheet tests** previously asserted inside callbacks that the sheet's `catch (_)` swallows. A wrong saved description made the test loop until killed. The tests now record arguments, assert outside the caught code, and cancel instead of re-presenting endlessly. - **`video_viewer_transport_test` removed.** It exercised only `package:http`. Removing the production `request.sink.close()` already fails `video_viewer_test`. ### Step 2 changes - **`test/helpers/fake_relay.dart`** fakes only the websocket (through the session's socket factory) and the HTTP bridge (through its client). The real `RelaySessionNotifier` still connects from `build()`, handles OK and CLOSED, batches events, backs off and replays subscriptions. - The fake serves the NIP-CW channel window. It can hold OK and live echo independently, reject a chosen publish, and drop the connection. Any request it cannot answer faithfully fails the test at teardown, because the app often catches relay errors and falls back. - **`channel_messages_journey_test`**: history through the channel window, a live message, a send that is pending until OK and stays one row after the echo, a rejected send rolled back, then drop and reconnect with missed and later messages. - **`channel_send_journey_test`**: type and send on `ChannelDetailPage` with no feature-notifier overrides. The row and sending spinner show at once, OK settles them, and a rejected send puts the text back in the composer and shows a snackbar. - Existing per-file relay fakes and `SidebarRelay` are unchanged. Step 3 migrates them area by area. ### Step 4 changes - `scripts/mobile-test-shard.sh` splits the 235 test files across the 3 shards, balanced by line count. `flutter test --total-shards` split tests, so each shard compiled and loaded every file. Locally at 4 workers, one shard took 63 s instead of 73 s and used 40% less CPU. `channel_detail_page_test.dart` (17.7k lines) is now the long pole; splitting it in step 3 shortens the slowest shard. Out of scope here: requiring branches to be up to date before merge (a repository setting), and the onboarding tests in #8375. ## Validation - Mutation checks: - Typing retry bound: test fails. - Sections backoff reset: test fails. - Wrong saved description: fails in about 11 s. The old tests hung instead. - Removed sink close: `video_viewer_test` fails. - Changed Dart files pass locally, and `flutter analyze` is clean. - Android `testDebugUnitTest` passes locally (23 tests). The Me-agent accessibility script passes on a local simulator. - Step 2 mutation checks, each made a journey fail: accepted OK ignored, rejected row not rolled back, optimistic row not added, draft not restored, local message left pending after OK, channel window replaced by the fallback, live subscriptions not replayed after reconnect. - Both journeys run in about 1 s on fake time. - All 99 RunnerTests pass on an iPhone 16 iOS 26.5 simulator, the CI destination, after rebasing on #8398. - The shard script partitions all 235 test files exactly once. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
seiler
|
28fdf29d61 |
Merge pinned main into invite revocation candidate
Signed-off-by: seiler <seiler@squareup.com> |
||
|
Will Pfleger
|
91e7b469b8 |
fix(relay): wait for the peer's Close before dropping any WebSocket (#8426)
🤖 A NIP-42 login refused for a NIP-FI key mismatch could reach the client as an abnormal close (`1006`, no reason) instead of the `NOTICE "restricted: authorization denied"` followed by close `1008` that every other authorization denial produces. The relay did write the NOTICE and the `1008` Close, but the connection task dropped TCP as soon as the writer finished, before the peer finished the close handshake. The client could then lose frames it hadn't read yet. It depends on timing, so it showed up intermittently and is not specific to key pairing. ## Relay route - `recv_loop` now hands back the read half even while a frame handler is still running. Frames are handled one at a time while the connection is live. Once it is cancelled, an in-flight handler (for example a Nostr `CLOSE` waiting on the subscription lock behind a REQ stuck in Redis) is spawned to finish instead of holding the reader. It is never dropped partway, so subscription retirement stays atomic, and cleanup joins it before the expiry barrier and application cleanup. - As soon as the writer finishes, `await_client_close` keeps the transport open until the client's Close arrives or the stream ends. It runs concurrently with joining the heartbeat, auth-timeout, expiry and auth-cancel tasks and the connection's application cleanup (subscriptions released, registry entry removed, presence cleared). Cleanup never waits on the peer, so a nonresponsive client cannot delay registry counts, presence or `has_admitted_connection`. The socket never waits on cleanup, so a stalled Redis (the pool has no wait timeout) or an expiry join blocked on its quiescence barrier cannot hold the socket and its file descriptor open past the bounded wait. The active-connection gauge and the connection permit are released after both finish. - The connection permit (`BUZZ_MAX_CONNECTIONS`) is now taken at the start of `handle_connection`, before the NIP-FI gate and the community-active bootstrap check, and held through the active handler or the refusal drain and its bounded wait. Previously a socket in bootstrap, or refused there, held no permit, so a burst of refusals could open sockets beyond the limit. Over the limit, the socket is still dropped with no frames. `buzz_ws_connections_active` still counts only connections that reached the active handler. ## Huddle-audio route - The relay and audio bootstrap drains share one bounded helper. A socket refused before admission waits after its community registry entry is released. - Every exit of the audio handler gets the bounded peer wait. Exits whose cleanup follows their terminal frames (`room_full`, `room_ended`, version mismatch, archived channel, owner rejection or unreachable, failed commit confirmation) and the normal teardown hand the socket to a background task that waits for the client's Close, so lease releases, `48102` writes and room archiving on Postgres and Redis never hold the socket open. The remaining exits send their frames last, and `handle_active_audio_connection` waits once they return. The wrapper joins the background task before returning, so the audio connection permit is held until the socket ends, as on the relay route. - Audio error exits now follow their existing JSON error with a Close frame: | Exit | Close code | | --- | --- | | `huddle_relay_draining` | `1012` service restart | | `join_rejected` (owner resolution failed), `huddle_owner_unreachable`, failed channel lookup | `1011` internal error | | `huddle_audio_unavailable`, `unsupported_version`, `upgrade_required`, `room_full`, `room_ended`, archived channel, owner rejection, NIP-FI authorization denial | `1008` policy | Audio login failures and authorization refusals outside NIP-FI `authorization_denied` still send only their existing error frame, then get the bounded wait. - The audio read half is an enum, so the hand-off has no `expect()` that can panic. ## Timing - The peer wait is bounded by `WS_TERMINAL_FLUSH_TIMEOUT` (1s), and drains to end-of-stream so a client-initiated Close gets its reply. The terminal writer flush has its own 1s budget, so the worst case for terminal transport work is about 2s per refused or closed socket. - Terminal frame writes (`flush_terminal_frames`, `send_exit_frames_bounded`) still give up on the first failure, but a failed or timed-out send is now logged at `debug` instead of being discarded. - The graceful-shutdown comment in `main.rs` now describes per-stage budgets (restart jitter, restart-Close acknowledgement, terminal flush, peer wait) rather than a per-socket total. The 30s hard drain bounds every socket, and the 35s process shutdown budget fits within the chart's 60s `terminationGracePeriodSeconds`. ## Tests Real-socket tests in the infra-free relay lane of `just test-unit` and `scripts/run-tests.sh` unless marked PostgreSQL: - The key-mismatch login and a session-expiry denial (as a control) both assert NOTICE then `1008` / `authorization denied` on the wire. - A raw-TCP client that never answers Close: the relay holds the socket for at least the bounded wait, has deregistered the connection by the time the socket ends, and ends it on its own under a 10s watchdog. - A raw-TCP client that withholds its Close until the connection has deregistered: the relay ends the socket in under the 1s cap, which fails if cleanup waits on the peer. - A raw-TCP client that never answers while final cleanup is stalled at a test pause point: the socket still ends after the bounded wait. - A raw-TCP client that never answers Close on the relay bootstrap drain, the audio bootstrap drain and an early audio-handler exit: the relay holds each socket for the bounded wait. - On the relay route with a one-slot budget through the real `handle_connection`: a refused bootstrap holds the slot until its socket ends, and a bootstrap stalled on the community check already holds the slot before the AUTH challenge is sent. - An established relay session whose reader is blocked in a Nostr `CLOSE` handler on a held subscription lock when the session expires: the socket ends within the bounded wait while the lock is held, and cleanup completes with the subscription retired once it is released. - An established relay session whose expiry task is stalled behind a held session effect permit: the socket still ends within the bounded wait. - PostgreSQL: an established audio session whose teardown is stalled after its Close, and a stalled archived-channel exit, both end within the bounded wait from the hand-off. - PostgreSQL: a fast refusal (unsupported protocol version) through the real `handle_audio_connection` with a one-slot permit budget keeps the permit taken until that socket ends. - PostgreSQL: `huddle_audio_unavailable` sends its error and then Close `1008` as the very next frame. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
5730c83f7d |
fix(release): keep mobile candidates pinned as branches advance (#8454)
Mobile candidate publication currently fails if the source branch advances after selection, and the operator can report failure even after the correct immutable tag was created. It also prevents releasing an explicitly selected historical baseline. Add `--commit FULL_SHA` and independently verify in the operator and publisher that the selected commit belongs to the selected source branch's history. Compare immutable SHAs, keep the candidate pinned through publication, and verify the resulting annotated tag instead of requiring the branch to remain stationary. Stable-branch review/check protection and immutable-tag permissions remain enforced. Without `--commit`, selection still defaults to the current source tip. ### Validation - Updated operator and publisher contracts pass: historical main/stable selection, branch advancement during publication, unrelated or missing commits, ancestry lookup failure, and unchanged tag/sequence checks. - Removing ancestry enforcement causes the unrelated-history regression to fail. - Live read-only ancestry verification accepts the original 0.24 baseline. Local publication stops at the existing App-permission check; no tag or build was created. - Full local `just ci` passed. Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Will Pfleger
|
5b433d9292 |
feat(relay): resolve NIP-FI issuer bindings from the database (#8377)
Stack: #8372 (merged) → this PR → #8399 → #8401 Makes relay login checks read the per-community NIP-FI issuer bindings that #8372 stores. Without this, the relay can't keep up with bb-public, where new communities are created at every signup and a static `BUZZ_NIP_FI_COMMUNITIES` list won't do. **Source switch.** `BUZZ_NIP_FI_COMMUNITY_SOURCE=database` resolves each community's trusted issuers from `community_nip_fi_issuers`, filtered by the issuer registry, instead of the static list. Each community gets its own policy ID: the base ID no longer hashes every community audience, and the final ID hashes the base plus this community's audience, so policy IDs change once (see `docs/operator-nip-fi-bindings.md`). `Config::from_env` refuses to start, in every mode, on an unknown source, on `database` together with `BUZZ_NIP_FI_COMMUNITIES`, and on `database` without `BUZZ_NIP_FI_PROVISION_BINDING`. `BUZZ_NIP_FI_RESOLVER_*` and `BUZZ_NIP_FI_SNAPSHOT_*` are validated in every mode, so a bad value fails the deploy that sets it. In `off` mode no resolver is built and no snapshot is loaded. **Resolver (`nip_fi_resolver.rs`).** - **Snapshot:** each pod loads every binding into memory, capped at `BUZZ_NIP_FI_SNAPSHOT_MAX_ROWS` (going over is a failed load). It refreshes every 30 s, start to start, with jitter that only shortens the delay and backoff after failures. Each attempt is limited by `BUZZ_NIP_FI_SNAPSHOT_REFRESH_TIMEOUT_MS`, which must be shorter than the interval. A refresh whose deadline passes before it publishes publishes nothing. The old snapshot is dropped off the runtime. - **Freshness:** a host in a snapshot loaded under two minutes ago is answered from memory. Any other host, and every host once the snapshot is stale, goes to a live lookup. Every answer carries a deadline two minutes after the read behind it started, and WS, HTTP and audio recheck it just before admitting. - **Live lookups:** bounded and single-flight. In-flight lookups are capped by `BUZZ_NIP_FI_RESOLVER_MAX_INFLIGHT` (default 8, clamped below the database pool size), and the pending hosts and their waiters have fixed caps. Positive answers are used for that admission only. A "not found" is cached for 5 s, up to `BUZZ_NIP_FI_RESOLVER_NEGATIVE_CACHE_MAX` hosts, and only when no snapshot was published and no binding was created while it ran. A lookup that panics still leaves the pending map. - **Removal:** a removed binding stops admitting new connections within 30 s plus the refresh timeout (40 s by default) while refreshes succeed, and within two minutes of the last good load while they fail. Open WebSocket sessions are not rechecked when a binding is removed; they end at their own session deadline. **Shadow mode** resolves through the same path as enforce, including the database lookup and the freshness rules. It keeps the resolved binding's deadline and checks it at the same point as enforce; a binding that goes stale during the tenant lookup is recorded as `stage=deadline, outcome=unavailable`, without changing the legacy response or closing the socket. With `source=database` its `community` metric label is only `bound` or `unmapped`; with `source=env` it keeps the existing per-community audience label, so current shadow dashboards are unchanged. Each request resolves its Host once: the router guard's resolution, failures included, is reused by shadow recording and admission. **Provisioning.** Both operator provisioning paths drop this pod's cached "not found" for the new host after a commit that bound issuers, so the creator's first login looks it up. **Startup and readiness.** With `source=database` and a mode other than `off`, the first snapshot loads before any listener binds, under its own startup phase `nip_fi_snapshot_load`. A failed load exits non-zero, so a pod is never ready without a snapshot. **Operational consequence:** boot already fails when the database is unreachable; what is new is that a snapshot over `BUZZ_NIP_FI_SNAPSHOT_MAX_ROWS` or a load past its timeout also crash-loops new pods. After two minutes the old pods' snapshots go stale and most logins fall through to the 8-slot live lookup, which sheds the rest with 503. The way out is raising the cap or timeout, or `mode=off`. **Metrics.** `buzz_nip_fi_binding_snapshot_*` covers loads, failures, rows and freshness. `buzz_nip_fi_binding_snapshot_loaded_timestamp_seconds` is written on every refresh attempt, so alert on `time() - x > 120` together with `absent(x)`. `buzz_nip_fi_binding_snapshot_rows` against `buzz_nip_fi_binding_snapshot_max_rows` gives the headroom to the row cap. `buzz_nip_fi_resolver_*` covers live lookups and the negative cache. The env vars and alerts are documented in `.env.example`, the chart README and `docs/operator-nip-fi-bindings.md`, which also has the staged off → shadow → enforce rollout for `source=database` with go/no-go signals at each step, and the over-cap and load-timeout failure modes with their ways out. **Regression coverage.** Router-level tests run the database resolver through the real `build_router` for root WS, audio and protected HTTP, in enforce and shadow mode. Over PostgreSQL and Redis, two pods using the env community source (two exact bindings) replay assertions retained from before a disconnect through a real root NIP-42 session, the audio deny-set check and signed NIP-98 requests: they are denied on both pods, and a root session upgraded before the disconnect is refused by the post-registration deny-set check. Postgres tests in `buzz-db` check that archived and quiescing communities, and issuers missing from the registry, never reach the snapshot or the live lookup. Process-level `boot_lifecycle` tests spawn the real relay with `source=database`: an over-cap or timed-out startup load exits non-zero with no health listener bound, and a loadable snapshot lets readiness turn ready. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
30ca980b49 |
feat(release): support version-matched mobile stable branches (#8430)
Mobile candidates currently require the current `main` commit, so a stable release branch cannot publish through the existing protected tag workflow. Add optional `--source-branch mobile-vMAJOR.MINOR` selection while preserving `main` as the default. Both the command and App-backed publisher reject a stable branch whose major/minor differs from the requested version. Candidates retain exact-tip checks and immutable annotated tags; the private build pipeline is unchanged. Document the temporary stable-only distribution policy and explicit per-fix backport decisions. No candidate, build, or upload is triggered by this change. No duplicate implementation was found among open release PRs. Stable publication now checks active review and status-check protection in both the operator and publisher. The process permits approved backport PRs and stable-only corrections for regressions caused by earlier backports. Administrator activation and verification of the no-bypass ruleset remain outstanding; non-admin API responses hide the bypass actor list. ### Validation - Full local `just ci` passed during preparation; the final changes affect shell release tooling, its workflow inputs, CI path detection, and documentation. - Publisher contract passed, including version-matched stable sources and mismatched branch rejection. - Operator contract covers default main, a divergent stable commit, missing or mismatched sources, immutable sequencing, and publication races. The final operator contract run passed. - Updated publisher and operator contracts pass, including 17 weakened-policy cases and rejection of unavailable protection. The live, currently unprotected stable branch is rejected. --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
seiler
|
22b5282a64 |
Repair strict legacy invite revocation
Signed-off-by: seiler <seiler@squareup.com> Co-authored-by: Codex <noreply@openai.com> |
||
|
Local Builder
|
a3b7b181a8 |
fix(db): reconcile invitation admission with main
Adapt invitation helpers to the community-admitted transaction API and move the unreleased revocation migration after the current schema changes. Exercise upgrades from the latest pre-revocation schema. Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Local Builder
|
2ab99e9b7e |
fix(db): own invite revocation in writer transactions
Acquire admission before domain row locks in restriction and owner writers. Revoke invitations in the same commit without SQL triggers, while keeping existing owner links and ordinary user inserts lock-free. Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Brad Seiler
|
4432ca2308 |
test(relay): select mesh receive regression in both CI paths
Co-authored-by: Codex <codex@openai.com>
Signed-off-by: Brad Seiler <seiler@squareup.com>
(cherry picked from commit
|
||
|
Brad Seiler
|
e6a811ab03 |
test(relay): include the v1 verifier cutoff boundary in CI
Co-authored-by: Codex <codex@openai.com> Signed-off-by: Brad Seiler <seiler@squareup.com> Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Brad Seiler
|
d0441db919 |
test(relay): select invite boundaries and seed fenced upgrade fixtures
Co-authored-by: Codex <codex@openai.com> Signed-off-by: Brad Seiler <seiler@squareup.com> Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Brad Seiler
|
d2c2a74943 |
fix(relay): revoke invitations durably and fence restricted claims
Keep v1 natural drain with an optional UTC cutoff. Serialize claim and mint policy reads with restriction and owner writes. Share historical revocation repair across migration and desired-state upgrades while retaining deletion fences. Co-authored-by: Codex <codex@openai.com> Signed-off-by: Brad Seiler <seiler@squareup.com> Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
6286a54321 |
feat(relay): store per-community NIP-FI issuer bindings (#8372)
Stack: this PR → #8377 → #8399 → #8401 bb-public creates communities all the time, so a hand-edited `BUZZ_NIP_FI_COMMUNITIES` map can't list them. This PR stores an explicit NIP-FI issuer allowlist for each community in the database and writes it when the community is created. Login checks don't read the table yet; that comes in #8377 together with the database-backed resolver. Operator steps are in [`docs/operator-nip-fi-bindings.md`](docs/operator-nip-fi-bindings.md). **Storage.** Migration `0066_community_nip_fi_issuers` adds `community_nip_fi_issuers (community_id, issuer, created_at)`, with primary key `(community_id, issuer)` and the community write fence attached. The assertion audience is `https://<communities.host>`, so the table has no `aud` column. Zero rows means no binding. The table is included in whole-community deletion and purge. **One bindable rule.** `BUZZ_NIP_FI_PROVISION_BINDING` (`{"host_suffix": ..., "authorized_issuers": [...]}`) is parsed in `buzz-auth` with `BUZZ_NIP_FI_ISSUERS`, whatever `BUZZ_NIP_FI_MODE` is set to, and both the relay and `buzz-admin` use the same parser. Each authorized issuer must be a configured issuer (1 to 2048 bytes), and the suffix must not cover any host declared in `BUZZ_NIP_FI_COMMUNITIES`; an unreadable declaration refuses startup even in mode `off`. `ProvisionBindingPolicy::is_bindable(host)` is true for a host exactly one label under the suffix that is not reserved. It decides creation-time binding, explicit `bind`, and what the lookup route reports. **Binding on creation.** Rows are written in the same transaction as the community insert on all three creation paths (`POST /operator/communities` `create_only`, its legacy convergence branch, and startup bootstrap), and only when the call actually created the community, so existing rows are never rewritten. **Reserved host.** A host whose `https://<host>` equals a configured `command_audiences` value never holds a binding. Operator provisioning of it is rejected with a 400 and the `reserved_host` error code, bootstrap creates it unbound, and the availability route reports it unavailable with an error code. `main.rs` calls `verify_reserved_hosts_unbound` at startup; if a reserved host has rows the relay refuses to start and names the fix (`buzz-admin nip-fi unbind --host <h> --issuer <i>`, or revert the command audience). A database failure during that check is a separate `Check` error. **Operator commands.** - `buzz-admin nip-fi backfill` pages unbound communities by host (1000 per page) and binds each page with one `INSERT ... SELECT` limited to active communities. It skips communities being deleted, always prints `{rows_bound, unbound, unbound_hosts, truncated}`, and exits non-zero while any bindable community is still unbound. Archived communities are bound on purpose. Both bind paths take the shared deletion lock and then lock the community row (`FOR NO KEY UPDATE`, in id order for a batch), the same order deletion uses, and the backfill insert also requires the community to still have no binding, so an operator `bind` that lands after the backfill listed a community wins and the default issuer is not added beside it. - `buzz-admin nip-fi bind --host --issuer` refuses a reserved host, a host outside the suffix, or an unconfigured issuer. - `buzz-admin nip-fi unbind --host --issuer` refuses to remove a community's last issuer (archive or delete the community instead), with one exception: when a valid provisioning policy shows that backfill can't bind the host (it's reserved, or outside the suffix), the last issuer can be removed. That is the reserved-host recovery path in the runbook. A missing or malformed policy doesn't block `unbind`; it just means the exception never applies. **Lookup route.** `GET /operator/communities/nip-fi-binding?host=` (NIP-98, `RELAY_OPERATOR_PUBKEYS`) reads the database and returns `{host, active, issuers[]}`. `host` is the stored host. It returns 404 for an unknown host, 200 with `active:false` for an archived or quiescing community, and `issuers:[]` when no provisioning policy is set. It lists only issuers that are configured and on a bindable host. kgoose depends on this contract; it is also documented in the runbook. **NIP-FA.** The `relay_url` bullet now also says: "An adapter MAY obtain its exact relay list from a configured, authenticated source; it MUST still compare `relay_url` exactly." **Deployment dependency.** The kgoose config side is already handled by [cash-server#137671](https://github.com/squareup/cash-server/pull/137671). In public-community mode it requires the command audience not to be a community audience. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
d79446316b |
fix(mobile): support DeviceHub in mobile-dev (#8404)
`just mobile-dev` fails on Xcode 27 machines that have DeviceHub instead of Simulator. Resolve the simulator frontend inside the selected Xcode installation: DeviceHub for Xcode 27, or Simulator for Xcode 26. Launch by full path so an older installed or running app cannot determine the choice. This extracts the still-needed DeviceHub portion of Wes's [#8330](https://github.com/block/buzz/pull/8330). The CocoaPods deployment-target work already landed in [#8379](https://github.com/block/buzz/pull/8379); this PR does not change Podfile or its lockfile. CI and pre-push mobile selection include the shared Justfile and mobile recipe regression script, so edits to either run the regression in the mobile gate. ### Validation - Nine production-recipe checks pass, covering side-by-side Xcode 26/27 selection with spaces in paths, missing apps, launch failure, and gateway argument forwarding. Mutations reject unqualified Launch Services lookup and removal of DeviceHub selection. - All 39 tests of the pinned paths-filter action pass, including the Justfile and recipe-test selection cases. Mobile formatting, analysis, and the full mobile suite pass. - Native macOS smoke checks exercised launch from the selected Xcode installation and reached the Flutter boundary. Flutter and identity setup were stubbed, so no app was built or installed. - The initial full local CI gate stopped on an unchanged ACP signal-cancellation test; that exact test passed in isolation. The initial pre-push run also hit missing dependencies and two unchanged desktop agent-discovery tests. Desktop lint and TypeScript checks passed after installing pinned dependencies. Publishing used the authorized local-hook exception. - Current-head [CI](https://github.com/block/buzz/actions/runs/38963831198) passed, including all nine recipe cases and native Swift validation. Owner approval and Codex code/security reviews are complete with no unresolved findings. buzz-review-completed --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Brad Seiler
|
2d0f78c6e1 |
feat(auth): support explicit mesh JWKS transport (#8382)
## Summary Allow an issuer to opt into mesh HTTP for its JSON Web Key Set (JWKS). Public HTTPS remains the default. This supports deployments whose service mesh authenticates and protects the key endpoint. The sealed source contract selects transport and contributes to policy identity. Mesh mode requires a DNS hostname and checks every resolved address against an explicit allow-list before pinning one address. Both modes disable proxies, reject redirects, cap responses, and enforce a whole-fetch deadline. The relay cannot verify mesh security: operators must enforce mutual TLS, endpoint authorization, and egress restrictions. ### Related issue No duplicate mesh-JWKS issue or PR found. Existing NIP-FI relay enforcement: #7224. No deployment change is included. ### Testing - At `eccb8cb19420abda44cf6e5e62b1361e2843bd1d`: full auth package and auth/relay all-target clippy passed. Full relay package passed: 1,457 library tests, 37 binary tests, and 9 boot tests; 446 tests were ignored. - At prior head `113939838cd81d5e8d0b0f8783c42ffaae44a4d3`: the full relay library suite failed the demo echo test (1,455 passed, 1 failed, 440 ignored). Independent checks reproduced that timeout at base `d61fe2071d2b3d8e56dd931947f16bfe0d6173fe`; the receive-cancellation defect is outside this change. A later pass does not establish that defect is fixed. - Three falsifying mutations at `eccb8cb194` were caught: mixed DNS answers, proxy bypass, and whole-fetch deadline. The full locked auth package passed after restoring the exact source. - At `eccb8cb194`, `cargo build --locked --release -p buzz-relay -p buzz-cli -p buzz-admin` passed with command-local `CARGO_PROFILE_RELEASE_STRIP=none`. This avoids a macOS loader error without a source, lockfile, or project-profile change. - Earlier hooks failed the ACP signal test and Tauri login-shell PATH test. The ACP failure also reproduced on unmodified main `93ffc37c`. Main merged without conflict. The full Tauri package then passed with `RUST_TEST_THREADS=1`: 3,422 library tests and 3 integration tests, 19 ignored. All pre-push hooks passed with that same command-local setting; no hook or test was skipped. - At `eccb8cb194`, isolated `just test` passed all 107 runner steps. The monolithic `just ci` run timed out after 1,120 seconds; all recipes passed across that run and separate `just desktop-tauri-test`, `just web-build`, and `just mobile-test` invocations. No monolithic pass is claimed. - Exact-head release live-local checks passed: loopback HTTP mesh JWKS warm; valid assertion 101; expired/wrong issuer/wrong audience 403; valid signed disconnect 200; wrong command audience 403. Redirect-only JWKS returned assertion 503 without a request to its target. Both omitted/default and explicit public transport rejected HTTP at startup. - A separate NIP-FI-off release CLI smoke test created an open channel, sent an accepted message, read it back, and fetched its thread. The assertion probe closes after upgrade; it does not prove a complete NIP-42 session. Local loopback tests do not prove deployment mesh authentication, endpoint authorization, egress controls, or real `240/4` routing. Generated with Codex ## Human Testing valid assertion / configured community audience: HTTP 101 (expected 101) expired assertion: HTTP 403 (expected 403) wrong issuer: HTTP 403 (expected 403) wrong community audience: HTTP 403 (expected 403) valid command JWT / configured command audience: HTTP 200, body=b'{"disconnected": true}' (expected 200) valid command signature / wrong audience: HTTP 403 (expected 403) --------- Signed-off-by: seiler <seiler@squareup.com> Signed-off-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> Co-authored-by: Codex <noreply@openai.com> Co-authored-by: Local Builder <1c476cb1127ea270cb900bdc3891060140bdd1173f91498030ee9b4db01c2f20@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
c0ab789499 |
feat(buzz-db): retire the channel-TTL trigger (#8210)
## Summary This drops the channel-TTL event trigger. The application already refreshes ephemeral-channel deadlines in the event transaction (`event_follow_up::refresh_channel_ttls`, from #8135), so the trigger only repeats that work. This PR originally retired both follow-up triggers. #8352 has since retired the push-match trigger in migration 0063, so after the rebase this PR covers only the TTL half. Migration `0065_retire_channel_ttl_trigger.sql`: - `SET LOCAL lock_timeout = '5s'` (the same bound as 0063) - drops `events_refresh_channel_ttl` and `refresh_channel_ttl_after_event_insert()` Dropping the trigger on the partitioned parent also drops its partition clones. Published migrations, including 0022 and 0024, stay checksum-frozen. The same objects are removed from `schema/` (`schema/functions/…`, `schema/triggers/events.sql`, `schema/schema.sql`) and from the pgschema reconcile script. After this, migrated and desired-state databases have no event follow-up triggers. Tests: - `follow_up_retirement_tests.rs` (renamed from `push_retirement_tests.rs`): the push-retirement upgrade test now stops at 0063. A new test upgrades 0064 → 0065 and checks that the TTL trigger is gone from the parent and every partition, that the function is gone, and that the published checksums haven't changed. - `event_follow_up_postgres_tests.rs`: the contract now has two arms. `AppOnly` is the shipped schema, where both triggers are absent. `Dual` reconstructs the historical overlap of both triggers. The `PushRetired` arm (the push trigger dropped, the TTL trigger kept) was the shipped state before this PR. Its tests are removed because `Dual` and `AppOnly` together cover it. - `embedded_migrator_contains_consolidated_initial_schema` pins the 0065 statements and checks that both objects are absent from the desired schema. **Deploy gate:** do not run 0065 against a database still served by a relay older than #8135. After 0065, raw SQL writers no longer refresh TTL deadlines. `docs/push-gateway-deployment.md` is updated to match. ### What got simpler The trigger and the app no longer both refresh TTL deadlines; the app is now the only producer. One plpgsql function and one trigger are gone from both schema paths, and 8 per-partition drops are gone from the reconcile script. The follow-up contract drops from three arms to two. ### Related issue BUZZ-313 (Linear), part of the trigger-retirement work. ### Testing Head is `4f7d0baf0`. Results below are at `678927730` unless noted. `678927730..4f7d0baf0` changes only one test in `crates/buzz-db/src/store/event.rs`, described in the first bullet. - **Fixed test, `permanent_channel_event_commits_do_not_lock_the_channel_row`** (`4f7d0baf0`). - **Why it changed:** after 0065, its raw-pool inserts no longer ran any TTL refresh, so the test passed whatever `refresh_channel_ttls` did. This was raised in review. - **What it does now:** it inserts through the production `AdmittedTx` path and keeps the barrier and the tuple-lock probe. It commits all N transactions while an external transaction holds `FOR SHARE` on the channel row. It also asserts that no `buzz_db_channel_ttl_refresh_failures_total` is counted. - **Mutation check:** with `SELECT … FOR UPDATE` injected into `refresh_channel_ttl`, the new test fails (`must not wait on the channel row`, timeout at 10s). The old test passes under the same mutation. - **Blox:** fmt and `buzz-db` clippy pass. - **GitHub CI at `4f7d0baf0`:** every required check passes (56 pass, 0 fail). CI builds the merge ref with `main` `93ffc37cb`. - **New and renamed tests, checked by name in the CI logs at `4f7d0baf0`.** These pass in PostgreSQL Domain / PostgreSQL Tests ([117033725006](https://github.com/block/buzz/actions/runs/38915998692/job/117033725006)): - `permanent_channel_event_commits_do_not_lock_the_channel_row` - `migration_schema_ttl_retirement_drops_every_partition_trigger_and_keeps_checksums` - `migration_schema_push_retirement_preserves_ttl_and_published_checksums` - `event_insert_ttl_refresh_handles_permanent_ephemeral_duplicate_and_activation_race` - `{desired,migration_schema}_event_follow_up_contract_{dual,app_only}` (four tests) `embedded_migrator_contains_consolidated_initial_schema` passes in Rust / Unit Tests ([117024125423](https://github.com/block/buzz/actions/runs/38915998692/job/117024125423)). No `*_push_retired` test runs anywhere. - **Blox gates at `678927730`** (private PG16 + Redis, on a workstation shared with other sessions): - `cargo fmt --check`, workspace `clippy -D warnings` and `buzz-db --lib` all pass. - The `buzz-admin`, `buzz-audit`, `buzz-deletion`, `buzz-push-gateway`, `buzz-search` and `buzz-workflow` Postgres packages pass. - `buzz-db`: 517 of 519 tests passed. The 2 that hung, both `writer_pool_*` tests, were killed. `buzz-relay`: 413 of 426 passed. The 13 failures are covered below. - **Uncontended reruns at `4f7d0baf0`** on a fresh, idle workstation (PG16.15, `competing=0` for every arm): - **`writer_pool_rejects_non_read_committed_database_default` predates this PR.** It timed out 3/3 at head and 3/3 at `main` `93ffc37cb`, with a 300s timeout per run. - **`writer_pool_metrics_stop_after_session_timeout_setup_failure` also predates this PR.** It timed out 2/3 at head and 2/3 at `main`, and passed in the remaining run on each side. - Both tests pass in CI. The gate accepts them as a named exception for this PR, as a pre-existing hang on the PG16 test environment. - **`buzz-db`:** 518 of 519 tests passed. The single failure is `writer_pool_rejects_non_read_committed_database_default`, killed after it hung (the exception above). - **`buzz-relay`: still open.** 413 of 426 tests passed. The same 13 `handlers::group_state_postgres_tests` failed as on the shared box, each panicking at once in its fixture's `ensure community` step with `Sqlx(PoolTimedOut)`. Contention does not explain them. The fixture's community setup is in `buzz-db`, which this PR changes. A head/`main` comparison of the full `buzz-relay` package on the same idle box is running to find out whether this predates the PR. All 13 pass in CI at this head. - **E2E (Gimli, Blox)** at `050e593af`. `050e593af..678927730` changes only tests and docs; the migration, schema and runtime code are byte-identical. Gimli's audited verdict: **EXERCISED AND WORKS**, for DB and relay runtime only. Gimli ran it at `050e593af`; this head's runtime is byte-identical but was not freshly executed. - **Upgrade:** a real relay startup took the schema from 0064 to 0065. That removed the TTL function and the trigger from the parent table and from all 8 partitions. Checksums for migrations 1–64 are unchanged. - **Lock-timeout rollback:** with a real lock held, the migration failed with `55P03` at 5.0007s and the schema rolled back fully to 64. Once the lock was released, a retry applied 65. - **Fresh desired schema:** after pgschema plus reconcile, the retired objects are absent. - **Real signed kind-9 sends:** on both schema paths, these refreshed ephemeral-channel deadlines to about +600s. Permanent channels stayed NULL. - **Raw SQL insert:** at 64 it refreshed the deadline. After 65 it left the deadline unchanged, on both the upgraded and the fresh schema. - **Not covered:** native or device testing, and deploy. - **Review:** Legolas found 0 P0 and 0 P1. His three optional P2s are fixed in `678927730`. - **No human app test:** this is a DB-only change. - **Migration numbering:** #8237 and #8372 also add a `0065_*` migration. Whichever PR merges second must renumber. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
f148c0f22c |
feat(relay): add profile search over name and display_name only (#8301)
🤖 > **Review as a pair with block/buzz-app#1084** (the app change that uses this endpoint). Either can merge first: the app falls back to the old search on relays without this endpoint. ## Summary Searching for a person by name over the relay's general search returns profiles that only *mention* the text somewhere else: in an avatar image embedded as text, in the bio, or in the verification address. A search for "wes" could fill every result page with profiles whose avatar data happens to contain "Wes", and never return the person actually named Wes. This adds a relay endpoint, `POST /profiles/search`, that looks only at a profile's **name** and **display name**. Every other profile field is ignored. **Matching and paging are the same as the relay's existing full-text profile search** (NIP-50 search over kind:0 profiles with `search_mode: "prefix"`). A client can switch to this endpoint without a visible change in which people a query finds: - Each query word must match a whole word of the name; the last word can be a prefix. Words can be in any order, so "johnson wes" finds "Wes Johnson". Case is ignored. - Pages work the same way: up to 100 results per page, up to page 1000. - The results are exactly the old results minus profiles that match only outside their names. The order is better for people search: exact name first, then names that start with the query, then names with a later word that starts with it, then other matches. Ties are ordered by `ts_rank_cd` over the two name fields only (not the whole profile, as in NIP-50), then newest first, then event id. The response is a JSON array of the original signed profile events (Nostr kind:0), so clients can verify them as they do today. The buzz-app client change is block/buzz-app#1084. ## Details - Request: `{"query": "wes", "limit": 20, "page": 1}`. `query` is 1–100 characters after trimming; `limit` is 1–100 (default 20); `page` is 1–1000 (default 1); unknown fields are rejected with 400. - The search reuses the full-text search's own query builder (`buzz-search`), so the word rule cannot drift from it. The existing full-text index on events stays the access path; the name check runs only on rows the index already matched. There is no new index. One small migration (`0061`) adds the function `profile_name_fields()`; it reads no table data, so it runs instantly. - Only current profiles count: a replaced profile stops matching. Profiles that are not a JSON object, or whose name fields are not strings, never match. Content that PostgreSQL cannot read as `jsonb` (for example a `\^@` escape or very deep nesting) also never matches, and it cannot make the search fail for everyone else. - The handler loads the matched events through the normal routed event read and applies the same reader visibility check as other reads, in search order. - Access uses the same checks as the relay's GIF search endpoint: a signed HTTP request (NIP-98), identity admission (NIP-FI), rate limit, replay protection, and relay membership. That code moved out of `gifs.rs` into one shared helper in `bridge.rs`; GIF behavior is unchanged. - The route is added to the existing protected-route and NIP-FI shadow/enforce route tables. Docs: `docs/profile-search.md`. - Two test-only lines get `sadscan:disable` comments, as other fixtures already do: an existing public ES256 test key in `router.rs`, and the local Postgres test URL in the search integration test. ### Related issue None found. ### Testing - Postgres test for the search query (`profile_names_match_name_words_only`): only name fields match (avatar data URL, `about`, `nip05`, `banner`, non-string names and non-object content are excluded); words in any order and across both name fields; whole words only for completed words; ranking order; pages split without overlap or gaps; replaced profile; other community; `%` literal. - Postgres end-to-end test through the router: an admitted member gets only the name-matching signed event, and the returned event still verifies. - Request validation unit tests; existing GIF admission tests and router route-table tests pass. - Local end-to-end run with buzz-app#1084 against a relay built from this head, with about 195 seeded profiles. For every query tried, the endpoint's results were a subset of the full-text search's results; the only profiles missing were avatar-only matches. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
b1d60f98e2 |
Bring synced Solo chats and projects to mobile (#8337)
## Summary Bring desktop Solo chats and project organization to mobile, with synced collapsible sections, shared unread highlights and tab badges, agent selection, shared search, native iOS creation sheets, and inline typing bubbles that transition into replies. Resolve Solo placement for every chat entry route, retain encrypted organization snapshots for offline reading, and let users resume or discard unfinished chat and project setup without deleting existing messages. ### Related issue Related: #6565; expandable agent transcripts remain separate work. ### Testing - Fresh Solo read state seeds each active chat from its captured initial last-message timestamp before exposing row/section/tab unread state. Baselines survive intervening channel updates and reset across identity/community boundaries while surviving read-state readiness changes. Regressions cover multiple chats, events before read-state readiness and during/after seeding, and identity changes; removing either the seed or the display gate fails them. - Chat completion reconciles a retained Solo-save journal even when placement is already confirmed. False/throwing cleanup regressions ensure the outer intent remains until reconciliation and only one placement is published. Typing terminal-close loops remain bounded; the retry budget resets after an uninterrupted eight-second health interval or transport reconnect. - Typing retry budgets reset after sustained recovery; invalid Solo updates retain cached rows with a retry warning that survives failed refreshes. Project names use the editors’ 120-grapheme limit, including emoji and combining characters. Regression coverage exercises repeated recovery, invalid live/refresh heads, and character-limit boundaries. - Rebased onto current main, preserving the newer Search controls, Huddle timeline filtering, and message-media layout. - Failed project journal writes reload storage before claiming recovery and reopen the entered name for retry. Typing subscription rejections are caught; terminal closures clear stale indicators and retry with a bounded backoff, canceled on disposal. Regression tests cover both false/throwing storage writes and initial/late subscription rejection. - Explicit ordinary destinations skip optional Solo lookup waits. Generic destinations perform one bounded lookup in the destination (search no longer duplicates it); an unavailable classification falls back to ordinary sending per the product owner's explicit choice. Positively identified Solo destinations use Solo providers. - Terminal Solo subscription closure preserves cached rows and exposes live-update Retry. Page coverage verifies warning visibility, usable rows during recovery, and a subsequent terminal close. Removing the warning or the immediate ordinary-route guard fails its regression. - Generic channel entries share an ordinary default when Solo classification is unavailable, per the requested product behavior. Known Solo placement takes precedence; callers can explicitly require unresolved placement to remain blocked. Bounded initial lookup preserves known Solo deep-link routing. - Dirty newest snapshots and offline-save warnings survive transport reconnects; Retry persists that same event even if the relay returns an older head. Community changes replace the service, and agent-directory reconnect refresh remains intact. - Pending-save discard reloads failed removals. Repeated reconnect testing confirms typing subscriptions are released on provider rebuild and disposal; removing the existing cleanup fails the regression. - Ordinary Home/create routes explicitly preserve private-channel sending during Solo failures; identified Solo sends use the Solo sender and explicit unresolved routes remain fail-closed. Solo unread replies beyond the initial 200-event page are loaded through the oldest-unread boundary. - Chat/project/agent cleanup reloads failed removals, retains same-process recovery, and protects replaced records. False/throw regressions verify retries do not duplicate creation or membership. Route, unread, and cleanup guard-removal mutations each fail their regression. - Ordinary channels, DMs, and forums remain sendable when optional Solo sync fails; explicit unresolved placement stays blocked. Controlled cache races retain the newest verified snapshot, keep usable chats visible, and retry offline persistence without stale writes. Both protection-removal regressions fail as expected. - Confirmed journal cleanup checks false/throwing removal and exact-record identity. Interrupted agent choices are visible immediately, identify the saved agent, and cover Resume/Later/Discard with accessible name/key-fallback controls. - Notification deep links preserve the contiguous history cursor; same-second messages infer the newest agent consistently. Failed offline-cache writes surface Retry, with false/throwing storage failures and offline restart covered. - Composite history cursors cover 350 same-second events plus older history; removing the ID cursor makes the regression fail. Agent changes now journal before granting access, with restart/failure and Resume/Later/Discard widget coverage. Signed malformed profiles remain isolated. - Native iOS avatar tests passed on the simulator: visible-row loading, three-request concurrency, streamed byte limits, response validation, downsampling/pixel limits, cancellation, and visible avatar updates. - Solo and Home share observed-event/read-marker logic. Widget coverage verifies unread rows, collapsed-section indicators, the tab badge before first Solo visit, message/thread reads, forced unread, and filtering archived/nonmember/unplaced chats. - Chat intent is saved before ownership and organization preflight; tests cover interrupted ownership lookup and a disconnect while the native creation form is open. - Mobile analysis, formatting, file-size checks, and full mobile tests passed at `1d4c41354`. - Project creation persists its encrypted name/UUID before connectivity checks or relay reads. Restart tests cover Resume/Later/Discard, scope isolation, stale-intent rejection, and lost acknowledgements. Removing the pre-network save makes all four production-flow regressions fail. - Regression coverage includes notification/Activity routing, inline replies, unknown-placement send protection, restart recovery/discard, and encrypted offline cache isolation/refresh. - Earlier feature build exercised on a physical iPhone with existing account/community data; review follow-ups have not been reinstalled on the device. UIKit agent-row accessibility checks passed on an iOS simulator. - Earlier cross-platform `just ci` stopped on the existing `buzz-acp` test `signals_cancel_startup_turn_and_pending_stdin_without_hanging` (process exit-code assertion); no Rust production changes are included in this PR. ### Remaining review limitation Agent discovery is bounded to 10 pages / 1,000 profiles with automatic retries disabled. Owner-scoped discovery remains open because the current relay ignores `#auth` filters; supporting that query requires a relay change. ### Snapshots Sample data, shared Flutter/Android surface. Native iOS uses platform sheets and controls. | Solo: projects and chats | New chat | Choose an agent | | --- | --- | --- | | <img src="https://raw.githubusercontent.com/block/buzz/d9b29aa079a3b9a88e8ed89b70df8d6af06fa41b/pr-8337--01-solo.png" width="240" alt="Solo with collapsible projects and chats" /> | <img src="https://raw.githubusercontent.com/block/buzz/d9b29aa079a3b9a88e8ed89b70df8d6af06fa41b/pr-8337--02-new-chat.png" width="240" alt="New chat form" /> | <img src="https://raw.githubusercontent.com/block/buzz/d9b29aa079a3b9a88e8ed89b70df8d6af06fa41b/pr-8337--03-agent-picker.png" width="240" alt="Agent selection list" /> | --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> Co-authored-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Kalvin C
|
cdfc38ecbd |
feat(search): add newest-first sort to bridge search (#8371)
## Summary Adds an opt-in newest-first order to bridge search, so clients can offer "Most recent" alongside relevance. - `buzz-search`: `SearchQuery` gains `sort: SearchSort`. `Relevance` keeps the existing `ts_rank_cd` ordering. `Newest` orders hits by `created_at DESC, id`, and `page` steps through that order. Newest skips the exact-profile-lexeme prioritization, which only applies to relevance ranking. - Bridge `POST /query`: search filters accept `search_sort: "newest"`. When the field is missing or has another value, results stay in relevance order. Community scope, channel intersection and per-hit authorization are unchanged. - NIP-11 lists `buzz-search-sort` in `supported_extensions`. Clients only send the field to relays that advertise it, so they never get relevance results labelled as newest. - WebSocket NIP-50 search stays relevance-only. `NOSTR.md` documents the bridge extension. ### Related issue N/A ### Testing - New Postgres integration test `newest_sort_pages_by_time_then_id_within_scope`. It pages a channel in newest order, including an equal-timestamp tie split across a page boundary and a newer hit outside the channel scope. It checks that the pages join up in `(created_at desc, id asc)` order and that relevance still ranks the strongest match first. - Unit tests: - `bridge_search_sort_extension_selects_newest_only_when_named` - `advertises_nip_cl_and_search_sort_extensions` - Ran the buzz-search Postgres integration suite, the relay `nip11` and `bridge_search` unit tests, `cargo fmt --check`, and `cargo clippy -p buzz-search -p buzz-relay --all-targets -D warnings` on Linux. - Local relay exercised through signed `POST /query`: - NIP-11 lists `buzz-search-sort`. - Relevance and newest return different orders. - Newest pages 1–3 return 25 hits with no gaps or duplicates, including same-second ties across page boundaries. - A non-member gets no hits from a private channel; a member gets all of them. - Client integration is in a companion block/buzz-app change. It was exercised end to end against this relay through the production broker: switching sort, paging, scope changes and capability fallback. Deployment note: clients only offer newest sorting after a relay with this change advertises `buzz-search-sort`. --------- Signed-off-by: Kalvin Chau <kalvin@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
klopez4212
|
d61fe2071d |
Use native UIKit scrolling for iOS photo carousels (#8361)
Builds on the mobile polish merged in #8359; this PR now targets `main`. On iOS, inline photo galleries now use a UIKit collection view for dragging, deceleration, and snapping. Photo taps retain the existing full-screen viewer and reply/more actions; gallery scrolling continues to fade the message avatar. The final photo stops at the full message gutter rather than the smaller gap between photos. Android keeps the current Flutter carousel. Visible cells subscribe to continuing frames from Flutter's existing authenticated image providers, preserving animated GIF/APNG/WebP media. Request IDs fence late frames and cancel subscriptions on reuse/disappearance/disposal. A four-slot bound includes retired unresolved loads until their transport terminates. Shared media fetches now use abortable HTTP requests with a 30-second deadline through response-body completion, so stalled requests release their slots without leaving live transports behind; native eager precaching is disabled so it cannot bypass that bound. Frame conversion coalesces to the newest pending frame. Native cells expose photo labels and positions to VoiceOver, support RTL, and preserve the selected photo during resizing. Validation: - Mobile formatting and analysis passed. - Native simulator regression checks passed for geometry, RTL, snapping, taps, frame replacement, cancellation, and resizing. The native CI lane now runs the carousel executable. - Full mobile suite and pre-push gates passed, including two-frame GIF delivery, delayed-provider bounds/cancellation, real HTTP stalled-header/body recovery with four requests maximum, disposal, and unconfigured-push tests. - Signed iPhone Release build of the frame/subscription implementation passed. Physical iPhone testing covered the earlier carousel/gutter revision; the new animation/cancellation changes have local Flutter and native simulator coverage, not a new physical-device journey. Preview-only signing overrides preserve its bundle/Keychain group; production entitlements are unchanged. Local provisioning excludes production push/App Attest capabilities. - Full `just ci` was started but stopped after unrelated Rust/desktop compilation consumed about 5 GB; broad CI is incomplete. Device build required local Xcode 27 workarounds: iOS 16 deployment target and a generated swift-secp256k1 cache compatibility edit. Neither changes repository source. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
6766174673 |
chore(push): retire push notification database trigger (#8352)
Buzz must schedule mobile push notifications when it saves an eligible message. The relay, the server that receives and stores messages, now does this in application code. An older database trigger still exists. This trigger is a rule that runs automatically when a message is saved. This PR removes the old trigger and its supporting database function. The relay remains responsible for scheduling notifications. Migration 0063 makes this change in existing databases; updated setup scripts apply it to new databases. The change does not turn on push notifications. Production migration and deployment need separate approval. Before deployment, verify the release images and all pending database migrations. If push needs to be stopped after release, disable it with the feature flag and keep the qualified application version and database schema. Do not downgrade the application image. ### Validation - Local checks passed: 519 isolated database integration tests, 166 database unit tests, 17 source-policy tests, and the relay push-runtime tests. - Tests verify that the migration removes the old trigger, preserves earlier migration records, and lets the application schedule push work correctly. A concurrency test exercises the real application path; deliberately removing its lock makes the test fail. - [Current-head CI passed on retry](https://github.com/block/buzz/actions/runs/38752849383/attempts/2). The first attempt had one database-connection timeout during test setup; the failed jobs passed when rerun on the unchanged commit. - Initial local CI and push checks hit unrelated ACP test failures, which passed when run separately. Only the Rust-test push hook was skipped under the repository's local-check exception; other applicable hooks passed. ### Before merge - [x] Migration 0063 is available and the push trigger remains on [main at |
||
|
Logan Johnson
|
d28f6b2f70 |
feat(relay): let channel-state filters opt in to system channels (#8182)
🤖 ## Summary After part 2, a client can read a system channel only if it already knows the ID. This PR lets a client find system channels on purpose, with an ordinary Nostr filter and no new filter key. A filter on the channel-state kinds (39000–39003) now also returns the system channels the reader can read when it has either: - `#t:["system"]`, the channel type tag that every channel-state event already carries; or - any `#P`, the creator tag from #8174. Asking for the channels of a named creator returns a small result, so it does not fill generic channel lists. The filter then matches as usual. Existing clients send neither on these kinds, so they still never see a system channel. | Reader wants | Filter | |---|---| | My system channels | `{kinds:[39002], #p:[me], #t:["system"]}`, then their kind:39000 by `#d` | | Open system channels | `{kinds:[39000], #t:["system"]}` | | One creator's labeled channel, system or not | `{kinds:[39000], #P:[creator], #t:[label]}` | Private system channels stay hidden from non-members, as for any private channel. Values in one tag filter combine with OR, so `#t:["system", label]` means `system` **or** the label; use `#P` plus the label instead. Part 3 of 3, stacked on part 2. ### Related issue None found. ### Details - The opt-in is decided per filter in `handlers/system_channels.rs` and applies on REQ, COUNT and the HTTP bridge. It applies only when every kind in the filter is 39000–39003. - A label alone is not an opt-in. - `NOSTR.md` documents the system channel query rules. - **Spec.** NIP-CL says how a filter asks for system channels. - The relay information document (NIP-11) now lists `"nip-cl"` in `supported_extensions`. Clients use this to tell if a relay supports labels and system channels, and refuse those calls on a relay that does not. NIP-CL documents this. ### Testing - End-to-end relay test: `#t:["system"]` finds both system channels for a member and only the open one for a non-member; 39002 `#p:[me]` with `#t:["system"]` lists the member's system channels; `#P` includes a creator's system channels; creator plus label returns exactly one channel with `limit: 1` and excludes another creator's channel with the same label; a label alone does not return the system channel; COUNT and HTTP `/query` and `/count` agree. - The test fails when the opt-in is disabled. - Unit tests for which filters opt in. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels ← this PR --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
cdf03f618a |
feat(relay): leave system channels out of reads that do not name them (#8181)
🤖 ## Summary After part 1, a system channel is still returned by every query that returns ordinary channels, so existing clients would show it as a chat. This PR makes the relay leave system channels out of every read that does not name them. Clients do not need to filter, so clients that know nothing about system channels stay correct. | Query | Before | After | |---|---|---| | A query that does not name a channel (no `#h`), COUNT, full-text search, the HTTP `/query` and `/count` routes, artifact queries (`{"artifact": ...}`), the sidebar | Includes system channels | Leaves them out | | "Channels I belong to" (kind:39002 `#p:[me]`) | Includes them | Leaves them out | | A filter that names the channel with `#h`, or with `#d` on kinds 39000–39003 | Ordinary read rules | Unchanged: members can read it, and anyone can read it if it is open | Push notifications and "you were added to a channel" notifications (kinds 44100/44101) skip system channels, because a client that received one would show the channel. A client that is given a system channel's ID, for example in a pasted link, can still open it. Naming the ID is the opt-in. This keeps system channels ordinary NIP-29 groups. Part 2 of 3, stacked on part 1. ### Related issue None found. ### Details - The rule lives in one place: `get_accessible_channel_ids`, the set of channels a query may read when it does not name one. It is the union of the reader's memberships and every open channel; both halves now exclude `channel_type = 'system'`. Because the filter is in SQL before `LIMIT`, pages stay full. - A filter that names a system channel adds it to that filter's scope only (`handlers/system_channels.rs`). A sibling filter in the same request keeps the implicit set, so naming one system channel cannot widen another filter. - Thread windows and the workflows API name their channel, so they follow the same named-channel rule. - Artifact queries have their own read check in SQL. It reads a system channel only when the query's `#h` names it. - When a request re-checks membership for a named channel that its first lookup missed, a confirmed system channel joins only the filters that name it. Before, a reader who joined a system channel during the request could have it added to a sibling filter's scope. - The system-channel lookup runs only when a filter names a channel outside the implicit set, so ordinary queries do not pay for it. - **Spec.** NIP-CL lists the reads that leave system channels out, and says when a filter names one. `AGENTS.md` notes this for `get_channels`. ### Testing - End-to-end relay test (Postgres and Redis) with a private system channel, an open system channel and an ordinary channel, as a member and as a non-member: REQ without `#h`, NIP-50 search, COUNT, HTTP `/query`, HTTP search, HTTP `/count`, the channel list, and 39002 `#p:[me]` return only the ordinary channel. Named reads by `#h` and `#d` follow member-or-open on REQ, COUNT, search and HTTP. - Relay test: adding a member to a system channel sends no kind:44100; an ordinary channel still does. - Both tests fail when the SQL exclusion or the named-channel scope is removed. - Database test: an artifact in an open system channel is missing from `{"artifact":"current"}` and present with `#h:[<system channel>]`, for a member and a non-member. It fails without the channel-type condition. - Relay test: the membership re-check confirms a system channel that the system-channel lookup missed. The channel goes to the filter that names it, and a sibling filter's scope does not include it. It fails without the fix. - Unit tests for the per-filter scope. They now run in `just test-unit` and `scripts/run-tests.sh`. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them ← this PR 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Logan Johnson
|
a3909acc67 |
feat(relay): match label and creator filters before the page limit (#8176)
🤖 ## Summary After parts 1 and 2, a client can ask for "channels with label X" or "channels created by key K". But the relay applied these tag filters after it read one page of the newest channel events. So a request such as `{kinds:[39000], #t:["workspace"], limit:10}` could return fewer than 10 channels, or none, while matching channels existed. This PR moves the label and creator match into the database query, before the page limit. Part 3 of 3, stacked on part 2. ### Related issue None found. ### Details - Applies only when every kind in the filter is a relay-signed channel-state kind (39000–39003). Other kinds keep today's behavior. - The database match uses JSONB containment on `tags`, which is GIN-indexed. Containment can match a little too much (for example, a tag that contains both strings in another position), never too little. So the relay still checks each returned event with the normal Nostr filter rules, and `COUNT` requests keep that check too. - The same match is applied to the query path and the count path. - **Spec.** NIP-CL now says that `#t` and `#P` match before `limit`. - **Choosing one channel.** The relay does not make labels unique, so two creates for one label can both succeed, for example when two processes of one agent start at the same time. NIP-CL gets a "Choosing one channel" section: a client that needs one channel per creator and label looks it up with `#P` + `#t`, creates it only if none exists, and otherwise uses the oldest one (the lowest `created_at` tag, then the lowest channel ID). Every reader that follows this rule picks the same channel, even after an edit. ### Testing - Relay test: many channels, one match, small `limit`. The request returns the match for both `#t` and `#P`. This test fails without the change. - Unit test: `#t` and `#P` reach the database query only for channel-state kinds. - Relay test: one creator sends two creates for the same label at once. Both channels exist, a `#P` + `#t` lookup finds both, and the oldest-channel rule picks the same one in either order, even after the older channel's metadata is re-published by an edit. ### Stack These are two stacks. Merge them in this order. Client support (creating, labeling and finding these channels) will be in block/buzz-app, not the buzz SDK and CLI. **Stack 1: channel labels and creator lookup (#8174–#8176)** 1. #8174 channel type and creator tags, and the channel creation time 2. #8175 channel labels 3. #8176 database-side label and creator match, and the rule for choosing one channel ← this PR **Stack 2: system channels (#8180–#8182)**, stacked on stack 1 because it uses labels and the creator tag 4. #8180 the system channel type 5. #8181 leave system channels out of reads that do not name them 6. #8182 channel-state filters opt in to system channels --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
99c5fc8bba |
ci(docker): publish qualified, sortable push gateway main tags (#8305)
## Summary This PR makes two changes to the push gateway image job (`push-gateway-merge` in `docker.yml`). Together they let staging auto-promote the gateway the same way it already auto-promotes the relay: 1. **Publish only after same-SHA CI qualification.** `push-gateway-merge` now needs `qualify` as well as `push-gateway-build`. This is the same gate the relay's `merge` job already uses: a successful `ci.yml` push run on the exact source SHA. Before this change, the gateway image was published even when main CI was red. That was safe while staging pinned the gateway by hand. It isn't safe once the newest main image deploys automatically. 2. **Publish the sortable `:main-<YYYYMMDDHHmmss>-<sha7>` tag on main pushes.** It's the same `type=raw` line the relay matrix gained in #8240. It uses the same `enable=` guard, so it applies only to `push` on `refs/heads/main`. A Lexical selector on `^main-\d{14}-[0-9a-f]{7}$` can then pick the newest gateway build from the tag list alone. `:sha-<7>` has no ordering. `:main`, `:sha-<7>`, and the release semver tags don't change. The gateway has no debug variant, so no `debug-main-*` tag is published. **Tradeoff:** a gateway image now appears only after `qualify` finishes, which takes as long as main CI. If main CI fails, no gateway image is published for that commit, and that includes `:main` and `:sha-<7>`. The relay already behaves this way, and it's the behavior we want for anything that auto-deploys. **What got simpler:** the relay and gateway images now share the same publication gate and the same tag set. Downstream tooling no longer needs a special case for the gateway's unqualified, unordered tags. **What stays different:** the relay's `merge` job also attests the `deployment-eligibility/v1` predicate (`docs/deployment-identity.md`); the gateway publishes no such attestation. That's intentional for now: the staging promotion guard selects the gateway by package and `main-*` tag and does not verify attestations, so the gateway warehouse must not be wired to require one. If promotion ever starts verifying eligibility, the gateway needs the same attestation first. ### Related issue N/A (staging rollout tooling). The downstream promotion-guard and warehouse changes land separately. The warehouse switches on only after this tag has been published. ### Testing - `scripts/test-relay-image-eligibility-workflow.sh` now pins both properties of the `push-gateway-merge` job (it runs in CI under "Relay image eligibility contract"). Both checks match whole active lines, so a commented-out copy can't satisfy them. - At `9b8c49600` it passes. - With the live `needs` line replaced by `needs: [push-gateway-build]` and a commented-out `# needs: [push-gateway-build, qualify]` left behind, it fails. - With the gateway `main-*` line commented out, it fails. - With `needs` reverted to `push-gateway-build`, it fails with `push gateway image must publish only after same-SHA CI qualification`. - With the `main-*` tag line removed, it fails with `push gateway image must publish the sortable main-<commit UTC>-<7> tag on main pushes`. - `scripts/test-release-ref-contract.sh` now counts the sortable `main-*` tag line on all three image legs (relay, relay-debug, gateway) instead of two. It passes at `9b8c49600`. - `bin/actionlint .github/workflows/docker.yml` is clean at `f6c03b6b5`. - PR runs don't push images. The first real `main-*` gateway tag appears on the first main push after merge, and I'll check the registry for it on that manifest. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Jarrod Sibbison
|
8599efb6a3 |
feat(relay): allow HTTP operator listener delivery URLs (#8287)
## Summary Operator listener configuration currently rejects HTTP URLs, preventing delivery to sibling services over internal HTTP connections. Accept HTTP and HTTPS delivery URLs while retaining the existing pubkey, duplicate-route, and URL restrictions. Add configuration coverage for internal hostnames and custom ports, verify signed delivery through the real HTTP client, and include the listener suite in both unit-test runners. Shared NIP-98 signing and push gateway code are unchanged. ### Related issue Related: #7934 refactors operator listener delivery tests. No matching open issue found. ### Testing - `cargo test -p buzz-relay --lib operator_listener`: 13 passed, including signed HTTP delivery and redirect rejection. - `cargo fmt --all -- --check`, shell syntax, Justfile parsing, and diff whitespace checks passed. - `just test`: passed. - `just ci`: stopped at mobile analysis with an existing undefined `buzzDevPushAppProfile` in `mobile/test/features/settings/notification_toggle_test.dart:81`. Workspace and native desktop formatting/Clippy, desktop frontend checks, and web checks passed. The mobile file is unchanged by this PR. Locally verified with a spiked listener client Signed-off-by: jsibbison-square <jsibbison@squareup.com> |
||
|
Ian Oberst
|
85b7331784 |
fix(relay): partition audit observability follow-ups (#7994)
> [!IMPORTANT]
> **The alert rules changed in c3a314e71.** If you reviewed the alert
guidance earlier, please look again.
> - **The two range-based rules now apply only to live pods.** A range
selector such as `[30m]` keeps reading a terminated pod's samples until
they leave the window. So `BuzzPartitionAuditNotRunning` could fire for
about 10 minutes after normal pod turnover, and
`BuzzPartitionAuditFailing` could keep firing for up to an hour after
the pod was gone. Each rule now adds `and <counter>`, which requires the
counter to be exported right now:
> ```yaml
> expr: |
> sum without (outcome) (increase(buzz_partition_audit_runs_total[30m]))
== 0
> and sum without (outcome) (buzz_partition_audit_runs_total)
> ```
> ```yaml
> expr: |
> increase(buzz_partition_audit_failures_total[1h]) >= 2
> and buzz_partition_audit_failures_total
> ```
> - **The audit counters start at 0.** When the relay starts, it exports
every `buzz_partition_audit_runs_total{table, outcome}` series and
`buzz_partition_audit_failures_total` at 0, before the first audit.
`increase()` ignores a new series' first sample. Before this change, it
could miss the first periodic failure, or read a table's first
`degraded` run as a stopped loop. This change is also what makes the
`and <counter>` gate work: the counters now exist for a relay's whole
life.
>
> The doc's old claim that pod turnover can't leave an alert firing was
true only of the instant-selector stale rule. The doc now explains the
difference.
## Why
These are the observability follow-ups deferred from #6515 ("fix(db):
audit partition catalog before creation"):
- **Timestamp-age alerts go silent** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4127212141)).
Partition gauges are set only when a table audits successfully, and the
exporter evicts idle gauges after 2,700 s by default. After sustained
failures, `buzz_partition_audit_last_success_timestamp_seconds`
disappears. A table that has never audited successfully never has it. An
alert on `time() - last_success` then has no input.
- **Startup fallback failures are not counted** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4127212158)).
When the startup fallback audit fails, it logs an error but does not
increment `buzz_partition_audit_failures_total`. Periodic failures do.
- **Repaired environments are always `degraded`**.
`PartitionTableAudit::degraded()` counts every legacy leaf, even one
wholly in the past. The `bb-block-staging` audit of the repaired catalog
reports `degraded` only because of `events_p2026_07_08_legacy`
`[2026-07-01, 2026-09-01)`. With this rule, every repaired environment
stays `degraded`, so nobody can alert on the outcome.
- **Partition config tests can see a poisoned env** ([review
comment](https://github.com/block/buzz/pull/6515#discussion_r4137416297)).
Two partition config tests take only `ENV_MUTEX`. A NIP-FI test running
in the same process can make them see its deliberately invalid
environment.
Partition health is still diagnostic only, through
`/_status.partition_catalog` and metrics. Nothing here affects
`/_readiness`. Handling concurrent creation (#6515 item 4) is out of
scope and stays with the catch-all advancement work.
## What
- **Success timestamp survives failures.** Each failed table audit
refreshes `buzz_partition_audit_last_success_timestamp_seconds` with
`increment(0.0)`, which leaves the value unchanged. It still resets the
exporter's idle tracking, the same mechanism as
`refresh_legacy_active_gauge_recency`. Sustained failures therefore keep
the last success exported and its age rising. A table that has never
succeeded exports `0`. A failed audit still never makes the last success
look fresh.
- **Alert guidance.** New `docs/partition-catalog-monitoring.md` covers
the metrics, how long each series lives, and the audit outcomes. It
notes that `runs_total` has no `unsafe` outcome. It gives three alert
rules:
- timestamp age, which also covers never-successful tables and sustained
failures;
- `sum without (outcome)
(increase(buzz_partition_audit_runs_total[30m])) == 0` for a stopped
audit loop. Counters are never idle-evicted, and this rule fires before
the stopped loop's timestamp is evicted, so there is no false resolve;
- an aggregate failure-counter rule.
Both range-based rules also require their counter to be exported right
now (`and <counter>`), so a terminated pod's leftover samples can't
raise or hold an alert.
- **Zero counter baselines.** `try_install` exports each
`buzz_partition_audit_runs_total{table, outcome}` series (`ok`,
`degraded`, and `error` for both tables) and
`buzz_partition_audit_failures_total` at 0 before the startup audit, the
same way the auth series already start at 0. A failure in the startup
audit itself can still happen before the first scrape. The doc says so;
the stale rule covers that case.
- **Failure counter.** `buzz_partition_audit_failures_total` now counts
failed explicit audit attempts: the startup fallback audit and each
periodic audit. Both paths go through one helper,
`counted_partition_audit`. A failed `ensure_future_partitions` call,
such as a DDL error that the fallback then recovers, is not counted by
itself. A single attempt never counts twice.
- **Historical legacy leaves.** A legacy leaf is historical when its
finite upper bound is at or before the current month start. It is still
reported under `children` as `legacy_leaf`, but it no longer degrades
the table. A legacy leaf that still covers the current month or any
lookahead month stays degraded. All other degradation causes are
unchanged. The verdict now lives in `PartitionAuditReport::outcome()`,
which `buzz-admin partition-audit` uses. The JSON outcome names and exit
codes are the same as before.
- **Test hygiene.** Both partition config tests now use `env_guards()`.
## How it was tested
- `cargo fmt --all -- --check` and `cargo clippy -p buzz-db -p
buzz-relay -p buzz-admin --all-targets -- -D warnings`.
- New unit tests:
- The repaired incident layout with an injected clock: `degraded` in
August 2026 and `ok` on 2026-09-01T00:00Z, when the upper bound equals
the current month start, and on 2026-09-15. The legacy leaf is still in
the serialized report. The `outcome` label on the run counter follows
the verdict.
- A table test of the legacy-leaf predicate: the upper bound before, at,
and one second after the month start; lookahead and beyond-horizon
leaves; historical leaves with trigger drift; anomalous and `DEFAULT`
children; no audited month.
- Outcome precedence (error, then unsafe, then degraded) and the
`buzz-admin` exit-code mapping.
- Startup: a failed fallback counts once, a successful fallback after a
DDL failure counts nothing, and a successful ensure never runs the
fallback. Each periodic failure counts once. Against an unreachable
database, startup counts one aggregate failure while each table still
records two `error` runs.
- On the production exporter, never-successful audits leave run counters
and a `0` success timestamp for both tables.
- On the production exporter, all six run-counter series and the failure
counter read 0 before any audit. After one failed audit, `error` reads 1
and `ok`/`degraded` stay at 0. Removing the initialization fails this
test.
- PostgreSQL lane (`scripts/postgres-test-run.sh`) with new or extended
tests:
- The repaired layout is not degraded in September. The same catalog is
degraded in August.
- With a 1 s idle timeout, a failed audit leaves the success timestamp
unchanged, and sustained failures keep it exported past the idle
timeout. When audits stop, the timestamp is evicted but the run counters
remain. The next success exports a fresh timestamp.
- Mutation check: removing the failure-path refresh fails both alert
fixtures.
- These fixtures check the emitted series. They do not evaluate the
rules themselves, including the `and <counter>` gate. `promtool` isn't
available locally or in CI, and the doc points to `promtool test rules`
for that.
- `scripts/test-postgres-test-discovery.sh`.
- `config::` and `nip_fi_config::` tests run together in one process
five times.
## Related
This PR conflicts textually with #7993 (catch-all advancement) in
`crates/buzz-db/src/store/partition.rs` and
`crates/buzz-relay/src/main.rs`. Whichever PR merges second will be
rebased onto the other.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Luke Tornquist
|
d6dc813290 |
ci(docker): publish a sortable main-<commit UTC>-<sha7> relay tag (#8240)
## Summary
Each push to main adds one more tag to the relay image manifest that
`docker.yml` already publishes:
- `:main-<YYYYMMDDHHmmss>-<sha7>` (UTC commit time)
- `:debug-main-<YYYYMMDDHHmmss>-<sha7>` on the debug manifest
`:main`, `:sha-<7>`, and the release tags don't change.
**Why:** `:sha-<7>` has no ordering, so a deploy tool that wants "the
newest main build" has to fetch metadata for every tag before it can
sort. Our staging promotion warehouse (Kargo, `NewestBuild`) does that
on every discovery cycle, against about 1,700 `sha-*` tags (4,800 total)
in `ghcr.io/block/buzz`. That's slow, it hits registry rate limits, and
one failed fetch aborts the whole cycle. The timestamp prefix makes the
tag sort lexically in build order. A `Lexical` selector with
`^main-\d{14}-[0-9a-f]{7}$` can then pick the newest build from the tag
list alone and fetch only the tags it keeps.
Details, checked against the pinned `docker/metadata-action` v6.1.0
(`80c7e94d`):
- `{{sha}}` is the 7-char short sha (`defaultShortShaLength = 7`), the
same value as `:sha-<7>`.
- `{{commit_date}}` formats in UTC by default. On a push it uses the
event's `commits[].timestamp`, so rerunning a commit produces the same
tag.
- `type=raw` inherits the flavor prefix, so the merge job's debug leg
emits `debug-main-…`. That's the same mechanism that produces
`debug-main` today.
- `enable=` limits the tag to `push` on `refs/heads/main`. Release tags,
rescue dispatches, and PRs never get it.
On main's last 40 commits, author time and committer time are the same
instant, and both increase monotonically. A commit pushed with a
backdated timestamp would sort below newer builds and just wouldn't be
auto-picked; the next push supersedes it.
**What got simpler:** this adds one tag line in each of the two places
that already mirror the tag matrix, plus header docs. In return,
downstream tooling no longer needs per-tag metadata fetches to find the
newest build.
### Related issue
N/A (staging rollout tooling). The downstream warehouse and tag-guard
changes land separately and don't switch over until this tag has been
published.
### Testing
- `scripts/test-relay-image-eligibility-workflow.sh` and
`scripts/test-release-ref-contract.sh` pass at `9ba1d8b7f`.
- PR runs don't push images (`push: false`), so the first real tag
appears on the first main push after merge. I'll check the registry for
`main-<14 digits>-<sha7>` and `debug-main-…` on that manifest.
---------
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
40c1c474e7 |
fix(relay): hide deletions of author-only events from other readers (#8197)
🤖 Opened by an agent (Hayt) for Will Pfleger. The relay keeps `AUTHOR_ONLY_KINDS` (event reminders `30300`, push leases `30350`, private managed agents `30179`) readable only by their author, but a NIP-09 deletion request (`kind:5`) for one of those events was readable by any community reader. The deletion carries no `h` tag, so ingest stores it with its target's channel (none), and every read gate classified it by its own kind, `5`. Another reader could read and count it by author, by id, live, or over HTTP, learning the deleted ids, how many, and when. A deletion is now exactly as private as the event it deletes: - **Classification by the deletion's own `k` tag.** `buzz_core::kind::is_author_only_event_kind` treats a `kind:5` that names an author-only kind in a `k` tag as author-only. Only the canonical decimal spelling counts (`"30300"`, not `"030300"` or `"+30300"`), so Rust classification, ingest and the admin SQL (which matches the tag text) always agree. Because it reads the deletion, not a target lookup, the deletion stays private after its target is soft-deleted or gone. It is generic over `AUTHOR_ONLY_KINDS`, so a kind added to the list later (for example `44300`) is covered automatically. - **Every read surface.** The shared per-event gate (`is_author_only_event`, used by WS REQ historical and lookup by id, COUNT fallbacks, HTTP `/query`, `/count`, search hydration and the thread window), live fan-out in `handlers/event.rs`, the COUNT fast-path guard (`filter_can_match_author_only_kinds` now treats any filter that can match `kind:5`, including unfiltered and id-only filters, as needing per-event filtering), the channel-window aux closure in the bridge, and admin event preview plus report list and detail (SQL suppression of `kind:5` rows with a tag whose first two elements are `k` and an author-only kind, matching the Rust classifier; soft-deleted rows included). - **Enforcement still reaches the author.** The admin report query takes the private-deletion `k` list from its caller next to the hidden kinds. HTTP-facing reads pass both; `admin_get_report_for_enforcement` passes neither, so report-directed bans, timeouts and action recovery still resolve the stored author of a reported private deletion. - **Ingest agreement.** `validate_standard_deletion_event` rejects a deletion of a stored (including soft-deleted) author-only event, by `e` or by `a` coordinate, unless it carries the matching canonical `["k","<kind>"]`. It also rejects an author-only `k` on a deletion of any other kind, so a public deletion can't hide itself from the readers who need it. A deletion whose target is itself a private deletion is rejected with `invalid: cannot delete a private deletion request`: it would otherwise be stored as a public event revealing the private deletion's id and timing, and NIP-09 gives a deletion of a deletion no effect. A deletion of a public deletion is unchanged. The `k` tag only narrows who can read the request; it grants no one new deletion authority. - **Unauthorized deletions learn nothing about the target.** On every delete-by-id path, no target-dependent error (its kind, the artifact and `k`-marker errors, the artifact-removal-marker error, or where it lives) is returned until every access check that could block the deletion has passed. A sender who fails any of them gets `invalid: deletion target not found or not deletable by you`, the same text and status as for a missing target. Errors that depend only on the request, never on the target, are unchanged: for example a `kind:9005` whose own `h` channel is archived still gets `channel is archived`, for a missing target and a real one alike. "May not delete" includes an author who has since been removed from the target's private channel, an author whose token is scoped to a different channel, and anyone deleting in an archived channel. The single-target rule (exactly one `e` or `a` tag) now runs before any target lookup, so every target faces the gates. For `kind:5` with an `e` target, ingest decides authority (effective author or the owning human of the authoring agent) first, then takes the channel from that same lookup, soft-deleted targets included, so tombstones face the same gates; an authorized re-deletion of a soft-deleted target now keeps the target's stored channel instead of being treated as global, so historical access checks, `#h` filtering and live routing all use that channel: readers outside a private channel no longer see it, and that channel's members and subscribers do. The token channel scope, scoped-token global-event, membership and archived checks then reject with the shared denial, all before `validate_standard_deletion_event` returns its artifact and `k`-marker errors. Other kinds, and `kind:5` with only `a` coordinates, keep their existing errors. For NIP-29 `kind:9005`, a target outside the `h` channel is denied like a missing one, so the relay no longer returns "different channel" or "no channel"; `may_delete_channel_event` (the author while still a member or the channel is open, a channel owner/admin, or the owning human of the author agent) is then checked in the `h` channel before the artifact-removal-marker error. No access restriction is removed and nothing becomes newly deletable. An authorized sender who names the wrong `h` channel for a `kind:9005` now gets the shared denial. Whether kind:5 content is full-text indexed depends on how the database was built. The desired-state schema (`schema/schema.sql`, which CI applies) and databases created before migration `0008` index it; an empty database initialized through `0008` gets a positive allowlist that doesn't. Where it is indexed, the per-event gate on search hydration is what keeps a private deletion out of other readers' search results. No current client is affected by the new rejection. `buzz-app`'s signer only allows `kind:5` for channel messages (with `h`) and managed-agent `30177` coordinates, and neither `buzz-app` nor this repo's desktop or CLI publishes deletions for `30300`, `30350` or `30179` today. NIP-ER already requires `k=30300` on reminder deletions. Coverage: - `crates/buzz-test-client/tests/e2e_author_only_deletion.rs` (Backend Integration job): a non-author gets nothing from WS REQ, WS COUNT, HTTP `/query` and `/count` (by author, mixed kinds, and id), search, or live delivery, while the author gets the deletion. Search runs WS and HTTP against a private deletion and a public control deletion sharing a reason word; with `BUZZ_TEST_SEARCH_INDEXES_DELETIONS=1` (set by the CI step) the author must get exactly both and another reader exactly the control; ingest rejects a missing, wrong, non-canonical, or public-target marker, and an untagged deletion of an already soft-deleted target. - `e2e_author_only_deletion.rs` also sends one non-owner's deletions over HTTP and WS for a missing, live, soft-deleted, private-channel and global target, plus the `kind:9005` equivalents including a target in another channel and one with no channel, and requires the identical rejection text and HTTP status in every case; another member's deletion of a private event never names its kind. - `e2e_author_only_deletion.rs` `removed_author_deletion_does_not_reveal_target`: an author removed from a private channel gets the missing-target rejection over HTTP and WS for `kind:5` and `kind:9005` (with `h` set to a channel they can write to) against their retained message; the channel owner can still delete the removed author's message, and the author can still delete in a channel they can write to. - `e2e_author_only_deletion.rs` `archived_target_deletion_does_not_reveal_target`: over HTTP and WS, `kind:5` against a message, the same message with a wrong `k`, and an artifact in an archived channel, and `kind:9005` against the archived-channel message with `h` set to an active channel, all match the missing-target rejection; a `kind:9005` for an artifact-removal marker in the archived channel matches a missing target with the same `h`; in an active channel the owner still gets the `k`-mismatch and artifact errors. - `e2e_author_only_deletion.rs` `deletion_of_private_deletion_rejected`: over HTTP and WS, the author's deletion of their private deletion, with no `k` and with `k=5`, gets the rejection and stores nothing; a non-author's gets the missing-target rejection; a `kind:9005` against it gets the missing-target rejection; the private deletion stays hidden from another reader in WS REQ and COUNT, HTTP `/query` and `/count`, and live delivery; a deletion of a public deletion is still accepted. - `e2e_author_only_deletion.rs` `soft_deleted_target_keeps_its_channel`: an author removed from a private channel, and an author in an archived channel, re-deleting their own soft-deleted message get the missing-target rejection; a distinct successful re-deletion is readable by a channel member and not by an outsider, by id and by `#h`, and is delivered live to the member's `#h` subscription but not to the outsider's. - `e2e_author_only_deletion.rs` `multi_target_deletion_error_ignores_targets`: deletions with two `e` tags, or an `e` and an `a` tag, get the identical single-target error whether the first target is missing, live, in another private channel, a private reminder or an artifact. - `handlers::ingest::postgres_tests::scoped_token_deletion_rejection_does_not_reveal_target`: a non-owner and the target's author, each with a token scoped to another channel, get the same rejection for a missing, live, soft-deleted and global target, and the author's `kind:9005` naming the token's channel gets it for a missing target, a live target in another channel and a global target; a token scoped to the target's channel still deletes, and its re-deletion of the soft-deleted target is stored with that target's channel. - `thread_window::postgres_tests`: stored private deletions never reach a non-signer through the thread-window or channel-window aux closures, or by id once their target is gone. - `api::admin::postgres_tests`: ban and stranded-ban recovery on a reported private deletion reach the stored author while report detail hides it; `admin_moderation` tests treat non-canonical markers as public. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
868e2f8f32 |
refactor(push): one app profile per gateway (#8206)
Buzz push currently selects an application profile, such as `dogfood`, during client registration. We want each gateway to serve one Apple app configured by the operator instead. This removes profile selection from the client, relay, and gateway, while allowing internal and public gateways to use their own Apple identities. This PR implements that model in the Buzz source and upstream deployment chart. Discovery, enrollment, grants, and leases no longer carry an application-profile selector. Each gateway has one configured Apple application and APNs transport. The protocol changes are incompatible with the old registration format, so relays advertise `buzz-push-v1` instead of `nip-pl`. Old clients do not discover the new push capability. Updated clients use separate Keychain records and register again. The [push notes](https://github.com/block/buzz/blob/push-capability-v1/docs/buzz-push.md) describe the experimental contract. Landing this PR is the first step. Publish the matching gateway image and upstream chart, then land the [shared chart update](https://github.com/squareup/cash-helm-charts/pull/5740) and [gateway deployment update](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/1085). Before cutover, stop old gateway instances and check legacy registration state. Migrations refuse to proceed if old gateway installations or active relay leases remain; they do not delete that state. Physical-device verification remains necessary before production activation. ### Validation Gateway and focused Flutter push tests pass, as do the Swift package, simulator build, and native notification/enrichment tests. A local relay advertises `buzz-push-v1` without `nip-pl` or `app_profiles`. The PostgreSQL suite passed 868 tests; its two migration-test failures were corrected and pass a focused rerun together with the gateway cutover guard. Chart rendering and release-contract checks pass. Desktop JavaScript tests, desktop and web builds, native desktop compilation, mobile analysis/tests, and desktop native tests pass. The full Rust unit lane and prescribed `just test` command pass, including workspace integration tests against task-isolated local services. The production overlay now requires an explicitly supplied APNs topic, with omission and rendered-value regression coverage. The corrected Swift checksum assertion passes the complete package suite: 78 XCTest tests and 42 Swift Testing tests. No physical-device enrollment or production cutover is claimed. Local merge validation: the embedded migration test and formatting pass after reserving profile migration 0060 behind rollback migration 0059. The local pre-push Rust lane failed ACP timing/steering tests (`idle_resets_on_stdout_activity`, `keepalive_resets_idle_past_deadline`, and the failed/missing-outcome steering tests). ACP and its dependency manifests are unchanged by this branch; all four pass in an isolated single-thread rerun. The documented local-check exception skips only the Rust pre-push lane for publication; required CI remains mandatory. Mobile and desktop pre-push lanes passed. Combined migration validation: after rollback 0059 landed, the main merge preserves profile migration 0060 and both embedded migration assertions. The focused migration test and formatting pass. The next local push again failed only the unchanged ACP idle/keepalive timing tests (1053 passed, 2 failed, 3 ignored); other lanes passed. Publication uses the same documented local Rust-lane exception; required CI remains mandatory. Follow-up review fix: removed the legacy profile column from the copied-lease fixture. All four event-follow-up contract tests pass on isolated PostgreSQL databases, covering desired and migrated schemas in dual and app-only modes. Formatting passes. Publication retains the prior passing local-lane evidence and uses the authorized pre-push exception for the known unrelated ACP timing failures; required CI remains mandatory. --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Tom Brow
|
5b907b838a |
fix(push): isolate push enqueue and stop workers on rollback (#8216)
Push enqueueing previously ran inside the message transaction, so a push lock or database error could block or reject a message. Configured relay writers now submit bounded best-effort work only after commit. A dedicated connection and bounded queue isolate enqueue waits and failures from message-write connections. Overload, errors, process loss, or shutdown may drop a notification; the committed message remains authoritative. `BUZZ_PUSH_ENABLED=false` disables the application producer and workers. Updated writer connections always bypass any surviving enqueue trigger. The overlap migration updates an existing function without resurrecting retired objects. Apply it before relying on isolation with a surviving trigger, and replace every old enabled process and writer connection: missing settings preserve legacy behavior during overlap. SIGTERM/Ctrl-C cancels enqueue, matcher, and delivery work before listener drain. Durable queues, claims, leases, preferences, and delivery configuration remain intact. Claims expire normally; gateway requests already accepted may still deliver. Reactivation requires an explicit operator decision about retained work. There is no automatic purge, startup interlock, or backfill of messages written while disabled. Built on merged [#8135](https://github.com/block/buzz/pull/8135), including [#8178](https://github.com/block/buzz/pull/8178). Trigger retirement [#8210](https://github.com/block/buzz/pull/8210) remains open; this change is tested with the trigger present and after removal. Operational details are in [the rollback and reactivation guide](https://github.com/block/buzz/blob/relay-push-rollback/docs/push-gateway-deployment.md#relay-rollback-and-reactivation). ### Validation [ |
||
|
klopez4212
|
7e8aac4403 |
fix(mobile): polish channel management and iOS Settings (#8171)
## Summary Use native iOS sheets for channel editing and Add Members, with profile avatars, matching search, and destructive removal dialogs. Canvas content becomes a tappable preview with Add/Remove actions. Fix the Settings availability menu and clipped copy confirmation. Align channel action messages with chat styling and keep agent identities distinguishable across selection, attribution, and profile surfaces. ### Related issue No duplicate found in the existing mobile Settings/channel issues and PRs. ### Testing - Flutter analysis, formatting, file-size checks, focused collision regressions, and the full mobile test suite passed. - Production UIKit picker simulator check passed: distinct visible/accessibility labels, avatar updates, selected rows, and pubkey callbacks. Wired into Mobile Swift CI. - Original UI was built and tested on the physical iPhone with author confirmation; profile/community data preserved. - Review fixes await renewed author device confirmation before restoring `buzz-review-completed` and ready-for-review status. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> |
||
|
Will Pfleger
|
836e7d0d06 |
feat(desktop): browse and moderate every community in the Admin Console (#8027)
The Admin Console only ever saw the community the desktop happened to be connected to. With #8011's community reads, the console can now browse every community on the relay and, for signed staff, moderate any of them by name without switching connections. 🤖 ## What changes - **Communities tab.** Lists every live community on the relay, searchable by host prefix with "Load more" paging. The connected community is pinned at the top by exact host match. Report rows get a community badge that opens that community. - **Community page.** Reports, Restrictions, Members and Actions, each scoped to the community on the page. The Actions form has no host field: it always acts on the page's community, and the TypeScript host normalization is gone from the Actions tab. - **Native reads** (`admin_list_communities`, `admin_search_members`, `admin_get_member`, `admin_get_event`). Every request validates the explicit community host and NIP-98-signs the real method and URL. Failures come back as a structured error (`status`, `bodyComplete`, `bodyEmpty`, `code`), so the UI can tell an older relay's empty 404, a coded `event_not_found` and a transport failure apart, and never says "not found" unless the relay did. - **Explicit-host lifts.** `admin_lift_restriction` sends the page's community host, not the active relay's. It freezes the relay and signer the intent was reviewed under, and refuses to send if either has changed. - **Pending actions survive navigation.** A panel-level `DirectActionsProvider` holds the frozen intent, so switching communities or tabs mid-confirm doesn't lose it. Only Discard drops it. A signer change remounts the controller and drops it. - **Fenced lookups.** Member and event lookups are keyed by origin, signer, community, target and a generation counter, so a late answer for one community can't unlock Review in another. - **Shared member picker** (`MemberSearchResult`, also used by Add Member), showing the truncated npub so members with the same name can be told apart. Delete accepts a note link and previews the message before Review. Relay staff are blocked before Review. - **Disabled admin auth is read-only.** With `BUZZ_ADMIN_AUTH=disabled` the Communities tab and community pages still render, and report badges still open them, but every ban, timeout, delete, add-member and lift control is disabled. The settings badge says the console is read-only and that acting needs `nip98`. Staffing stays hidden. - #7904's protections in the Actions tab are kept: frozen intent, `requestId`, Retry, `notSent` and the key-backup guard. - **Readable load errors.** Admin list and detail loads show the relay's error message (for example "request is not authorized"), never the raw JSON envelope. - **Failed kicks reload the report.** A resolve that fails with `enforcement_failed` reloads the report, so its failed action and "Cancel & reopen" appear immediately. - **Key backups never leave in a search.** Every admin API request URL passes the NIP-49 key-backup guard before it is signed or sent: community and legacy reads, mutations, the connection probe (before its first, unsigned request) and feedback-attachment fetches. The community and member search boxes, including the directory's Load more, refuse a pasted `nsec` or `ncryptsec` anywhere in the input without calling the relay. - **`just admin-seed` for multi-community testing.** Besides the local community, it seeds `beta.localhost:3000`, `gamma.localhost:3000` and 50 filler communities, with named members, a ban and a timeout in each named community, a report with Kick available, and a message for the delete preview. Rerunning it restores all of that to the starting state. It refuses, writing nothing, while a moderation action is unfinished on a seeded report. - The bodyless `GET /events/{id}` preview is added to the events-URL egress inventory. It sends no body, so it carries no key material. ## Tests Native (Rust): - `empty_404_and_405_read_as_complete_and_empty`, `coded_404_carries_the_relay_code`, `truncated_and_over_cap_bodies_are_incomplete`, `transport_failure_has_no_status`: the structured read error tells an older relay, a coded refusal, a cut-off body and a transport failure apart. - `a_401_retry_is_signed_by_the_same_keys`, `lift_signs_the_send_and_401_retry_with_the_checked_keys`: the 401 retry never switches keys. - `read_urls_validate_the_explicit_host_and_encode_the_query`, `lift_url_sends_the_page_host_not_the_active_relay`, `list_url_carries_host_and_cursor_and_refuses_a_changed_relay`, `lift_refuses_before_sending_on_signer_change_or_bad_host`: requests target the explicit host and refuse to send after a relay or signer change. jsdom: - `actions-pending-nav`: a pending intent survives a community change, and only Discard drops it. - `actions-fenced-lookup`, `fenced-load`: a late lookup for an old key never shows under, or unlocks, the new one. - `actions-same-ids`: the same pubkey and event id are read separately in each community. - `actions-community-change-drops-member`, `actions-page-host`, `actions-success`, `actions-identity`: the form acts only in the page's community, and confirm sends the frozen intent. - `actions-preview-errors`, `actions-delete-link`, `actions-staff-target`, `actions-disabled-auth`: preview errors, delete by note link, the staff block, and community pages rendering read-only with every action control disabled under disabled admin auth. - `communities-directory`, `communities-unsupported`, `community-badge`, `community-badge-disabled-auth`: the directory, older relays, and report badges, which open community pages under disabled admin auth too. - `restrictions-lift-signer-change`. The existing restriction tests now run on the community page and use `admin_lift_restriction`. Mutation checks: removing the pending-navigation guard turns `actions-pending-nav` red, and dropping the host from the fence key turns `actions-fenced-lookup`, `actions-same-ids` and `fenced-load` red. Ungating restriction lifts on `canMutate` turns `actions-disabled-auth` red. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
d34c9c2c9b |
ci: shorten the PR critical path (desktop lanes, smoke shards, parallel relay artifacts) (#8156)
## Summary The PR critical path on main push [run 37635224099](https://github.com/block/buzz/actions/runs/37635224099) (`e097c4299`) was about 42 minutes: - **Desktop Core** took 41 min. It ran frontend checks, two clippy graphs, the Tauri suite, the mesh-llm suite and every compiled-flag state one after another. The compiled-flag step alone took 16m51. - **Relay artifact producer → Desktop E2E Integration** took about 31 min. The producer built the relay binaries (5m07), the backend integration archive (4m41) and the PostgreSQL archive (8m18) one after another. - **Desktop Smoke E2E** took 19–24 min per shard across 4 shards, with 82 min of tests in total. This PR attacks all three. ### 1. Run each compiled-flag state once (`Justfile`) | Previous step | Cost | Why it's redundant | |---|---|---| | Clean state: full `buzz_lib` suite (`--lib`) | 3m27 run | `desktop-tauri-test` runs the same suite in the same compile state one step earlier. When `BUZZ_TEST_EXPECTED_AGENT_ACCESS_OWNER_ONLY` is unset, the expected value already defaults to `false` in non-owner-only builds (`runtime/test_fixtures.rs`, `agents_tests.rs`). | | Internal state: auto-connect and owner-only built separately | 1m22 + 1m06 + 56s across three recompiles | Internal release packaging always sets both capabilities together, so building owner-only without auto-connect tests a configuration that never ships. The extra 56s happens because a bare `cargo test <filter>` builds every target, not just the `--lib` harness. | | Demo state: `cargo test --workspace` | ~30s more than `--lib` | `buzz-terminal` and the `tests/*.rs` integration tests don't read any `BUZZ_BUILD_*` variable or `buzz_lib`, so they ran the same binaries a second time. | A fourth cost was hidden: every recipe invocation spent ~1m20 recompiling `buzz-desktop` to the *same* unit hash `desktop-tauri-test` had just built. `_ensure-sidecar-stubs` ran `touch` on every sidecar stub on each run. `tauri-build` emits `rerun-if-changed` for each `externalBin`, so the new mtimes reran `build.rs` and recompiled the crate. Stubs are now created only when they're missing. This also speeds up any local `desktop-tauri-*` recipe run back-to-back. After this change: - **Clean state:** runs only the two compiled-flag assertions (`compiled_flag_matches_expected`, `compiled_policy_matches_expected`) with `expected=false`. No recompile. - **Internal state:** one compile with `BUZZ_BUILD_AUTO_CONNECT_DEFAULT_RELAY=1` and `BUZZ_BUILD_AGENT_ACCESS_OWNER_ONLY=1`. It runs both assertions with `expected=true`, then the full `buzz_lib` suite. The suite now runs with auto-connect enabled as well, which matches the shipped internal build more closely than before. - **Auto-connect-only state (assertions only):** auto-connect `true`, owner-only `false`. The two capabilities are documented as independent. Without this state, a `build.rs` block that read the other capability's variable would pass, because every other state sets both or neither. - **Demo state:** one compile with only `BUZZ_BUILD_DEMO_SLUG`. It asserts the slug and asserts both capabilities stay `false`, then runs the full `buzz_lib` suite. - The recipe clears the three toggled `BUZZ_BUILD_*` capability variables and any caller `BUZZ_TEST_EXPECTED_*` values up front. Each state exports its own values in a subshell. Every invocation uses `--lib`, so each state compiles one test harness. - Each ignored-assertion run requires exactly one passing test per name filter. Previously, a renamed or un-ignored assertion matched nothing and passed silently. - The 32-character demo-name rejection check is unchanged. The full-suite reruns in the internal and demo states are kept on purpose. They are the only coverage that the whole suite holds up in those two shipped build shapes. Estimate: the compiled-flag work goes from about 17 min to about 11. ### 2. Split Desktop Core into parallel, path-gated lanes (`_ci-desktop.yml`) - **Desktop JS:** `desktop-check`, `desktop-test` and `desktop-build`. These compile no Rust, but some node:test cases read relay and DB crate sources to pin shared constants (for example `relayReconnectReplay.test.mjs`). So the job runs on desktop *and* Rust changes, in parallel with the Tauri lanes. `lefthook.yml` notes this as a deliberate local deviation. - **Desktop Tauri (core | mesh | compiled-flags):** a matrix of parallel Rust lanes. - `core` and `compiled-flags` share the existing `desktop-core` Rust cache key, so they start warm. `compiled-flags` restores it read-only. - `mesh` keeps its own cache because `mesh-llm` pulls a different dependency graph. - **Smoke E2E and the Windows frontend build** compile no Rust and read no crate sources, so they no longer run for Rust-only changes. - **Smoke E2E goes from 4 to 10 shards.** Playwright shards by test count, not duration. A simulation from the per-test durations in run 37635224099 reproduces the observed 4-shard split exactly. With 10 shards, the slowest shard runs about 11 min of tests. - **The `Desktop` gate is stricter.** Each selected job must succeed and each unselected job must be skipped, so a mis-gated job can't pass silently. The required check context (`Desktop`) is unchanged. `scripts/ci-required-gates.test.mjs` pins each job's `if:` to the gate's selection expression, and checks which jobs a Rust-only, desktop-only, Tauri, push, and empty selection pick. - **Path-filter fixes:** - The `desktop` filter now includes `_ci-desktop.yml`, `ci.yml` and `Justfile` (which holds the desktop recipes), so edits to them run the frontend lanes. - The `rust` filter now matches `Justfile`. It still named the pre-rename `justfile`, so a `Justfile`-only change selected no Rust suites. - `scripts/ci-selection.test.mjs` gains `Justfile` and desktop-workflow scenarios, and the shared-workflow scenario now expects `desktop`. - **Dead step removed:** the failure-only Playwright artifact upload in the old Core job never had anything to upload, because that job never ran Playwright. ### 3. Build relay artifacts in parallel (`_ci-relay.yml`) - **The producer is now a 3-way matrix** (`relay-binaries`, `backend-integration-tests`, `postgres-tests`). - Each lane has its own exact-source cache entry and uploads `desktop-e2e-relay-<artifact>`. - Consumers download `desktop-e2e-relay-*` into `target/ci`, so their paths are unchanged. - The required `Desktop E2E Relay` context still reads the producer job's aggregate result. - **Rust cache saving:** all lanes restore the existing Rust cache key, but only the widest lane (`postgres-tests`) saves it. Otherwise the fastest lane could overwrite it with a narrower target dir. - **Desktop E2E Integration goes from 2 to 3 shards.** Shard labels now use `strategy.job-total`. ### 4. Bound degraded apt steps (`scripts/ci-apt-retry.sh`) - With more shards, more runs hit the occasional degraded Ubuntu mirror. On this PR, a Tauri lane ran `apt-get install` for 27 minutes: apt's `Acquire` timeouts kept firing, but apt kept re-queuing the requests and never failed. Separately, three Smoke shards stalled silently in `playwright install-deps` (no output after `InRelease`) until the job timeout. Healthy apt steps across four recent runs finished in at most 238s. - Every apt-backed step in Desktop and Integration now runs through `scripts/ci-apt-retry.sh`. The script writes the apt options to `apt.conf.d`, so Playwright's own apt-get calls get them too. It kills each attempt after 7 minutes (90s for `apt-get update`). Between attempts it stops any `apt-get` or `dpkg` the attempt left running, repairs dpkg under its own 60s deadline, and retries up to 3 times. The stop matters for `playwright install-deps`, which runs `sudo sh -c "apt-get update && apt-get install ..."`: sudo puts its command in a new process group, so the deadline's signal stops `sh` and apt-get survives holding the dpkg lock (seen on Smoke shard 8 of run 37696815557, and reproduced with real sudo). - Downloaded `.deb`s are kept between attempts, so a retry resumes where the last one stopped. Against a mirror that stays degraded, this bounds each wrapped command at about 25 minutes rather than recovering it. The Tauri lanes wrap two commands, `apt-get update` (about 8.5 minutes worst case) and `apt-get install` (about 25), so about 34 minutes together. Job timeouts are sized to fit that bound plus the job's own work: Tauri 55m (lanes take up to ~15m), Integration 40m, and Smoke 45m. A degraded mirror therefore ends with the wrapper's error instead of a silent job timeout. - **The retry has already fired on this PR.** In [run 37703795342](https://github.com/block/buzz/actions/runs/37703795342), `playwright install-deps` on Integration shard 1/3 hit the 420s attempt deadline. The second attempt started about 1.4s later, downloaded and configured the real packages, and succeeded. The step took 526s (8m46), and that shard took 17m24 while the other two took about 5m. Healthy installs have taken at most 238s, so 420s leaves headroom, and a slow mirror costs one attempt instead of the job. - Between attempts the script stops **every** `apt-get`/`dpkg` on the machine, not only the ones it started. That's intentional: on a hosted runner any other one would hold the same dpkg lock. It assumes an ephemeral runner, and the script says not to use it on a self-hosted runner or a developer machine. - `scripts/ci-apt-retry.test.mjs` covers the following with a fake `sudo`: success, retry, the attempt budget, a stalled attempt, a stalled dpkg repair, an apt-get that escapes the attempt's process group (on a retry and on the final attempt), and usage. It needs Linux (GNU `timeout`, `flock`, `setsid`, procps), so it skips on other platforms. CI runs it on Ubuntu. ### What got simpler - **Compiled-flag recipe:** two full-suite runs instead of three, six cargo invocations instead of eight, and sidecar stubs no longer invalidate the desktop build cache. - **Path selection:** each desktop job declares only the paths that actually affect it. The stale `justfile` filter and a dead artifact step are gone. - **Apt options:** set once in `apt.conf.d` by one script, instead of as repeated `-o Acquire...` flags on each apt call. - **Shard counts:** the shard count is written once per matrix, not repeated in names and commands. ### Cost - **More runners:** a full run uses more runner jobs. Smoke E2E goes from 4 to 10 jobs, Desktop Core from 1 to 4, the relay producer from 1 to 3, and Integration from 2 to 3. - **Fewer runners on single-area PRs:** desktop-only PRs skip the Rust lanes, and Rust-only PRs skip Smoke E2E and the Windows build. - **One extra Rust cache entry:** `desktop-mesh`. ### Related issue None found. ### Testing Blox, compiled-flag recipe at `77b9874b`: - [x] Recipe passes with output redirected to a file. The log keeps all 4 state headers, the ignored assertions ran 2/2/2/3, and both full suites passed (`3423 passed`). - [x] A second back-to-back run shows no clean-state `Compiling buzz-desktop`. - [x] Mutation tests all fail as required: - `build.rs` auto-connect reads the owner-only var: rc=101. - The reverse swap: rc=101. - Rename `compiled_policy_matches_expected`: rc=1. - Wrong owner-only expectation: rc=101. - [ ] `just ci` Workflows: - [x] `actionlint` on `_ci-desktop.yml`, `_ci-relay.yml` and `ci.yml`: clean. - [x] `scripts/test-ci-required-context-isolation.sh`: passes. - [x] GitHub Actions: runs [37701003713](https://github.com/block/buzz/actions/runs/37701003713) and [37703795342](https://github.com/block/buzz/actions/runs/37703795342) took 27m09 and 28m51. Before this PR, PR runs had a median of 40.0m (n=21) and main pushes took about 42m. These are observations across mixed runner images, not a measured causal speedup. - [x] Selection, gate and apt suites (74/74 on Linux), `actionlint` and `bash -n` on Blox at tree `db9725fe`. --------- Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
||
|
Luke Tornquist
|
745501962e | refactor(schema): fan desired-state schema out to one file per table (#8131) | ||
|
Luke Tornquist
|
1972b7d256 | feat(admin): add community archive commands (#7107) | ||
|
Luke Tornquist
|
e097c42996 |
feat(relay): time post-metrics-bind startup steps (#8132)
## Summary
Relay pods on large deployments can take over 5 minutes to open their
health port, and right now nothing shows which startup step is slow.
Startup timing ends at `metrics_bind`, and the slow work all happens
after it.
This PR times every startup step from Postgres connect through public
listener bind. It does not change startup behavior.
- Each step logs `Startup phase started` and `Startup phase finished`,
with `phase`, `elapsed_ms`, and `status` (`succeeded` / `degraded` /
`failed`, the same values `buzz_process_lifecycle` records use).
- Gauge `buzz_startup_phase_current{phase}` is `1` while a step runs and
`0` after it ends. A pod that is still starting shows exactly which step
it is on.
- Gauge `buzz_startup_phase_seconds{phase}` records each finished step's
duration.
- The startup NIP-43 membership reconcile logs `NIP-43 startup
reconciliation progress` every 5,000 communities, with `done`, `total`,
`republished`, `failed`, and `elapsed_ms`.
These are gauges rather than counters on purpose. Each value is written
once per boot, and a monotonic-counter scrape drops a series' first
sample, so a one-shot counter never shows up.
## Contract change
`deploy/charts/buzz/README.md` ("Early-startup telemetry contract") said
the early lifecycle phases are log-only. That still holds: `crypto_init`
through `metrics_bind` are unchanged and emit no metrics.
The README now also documents the post-bind steps. The guard in
`boot_lifecycle.rs` now allows exactly the two `buzz_startup_phase_*`
families. It still rejects any other startup, boot, or lifecycle family,
and it also rejects any exported label value that names an early phase.
## Design
- `crates/buzz-relay/src/startup_steps.rs` adds a closed `StartupStep`
vocabulary (18 steps) and a `StepTimer` guard.
- A step ends exactly once:
- `finish()` → `succeeded`
- `degrade()` then `finish()` → `degraded` (a non-fatal error branch)
- dropped without finishing → `failed` (covers `?` early returns and
panics)
- The NIP-43 and large-roster reconciles now return `ReconcileSummary {
repaired, failed }`. A sweep where any item fails ends its step
`degraded`, and the final log line includes the `failed` count. NIP-FI
warm ends `degraded` when any issuer did not warm. The deprecated
`reconcile_nip43_membership_snapshots` wrapper still returns `repaired`.
- Steps that are disabled by config (migrate, audit connect, NIP-FI
warm, git probe, NIP-43 reconcile) are not timed, so their series never
appear.
- The gauges are set once, so they age out after the existing gauge idle
timeout (2700 s with default intervals; logged as `idle_timeout_secs`).
A step that runs longer than that loses its `_current` series. The
README says so.
- Progress logging runs only for the `Bootstrap` reconcile. The 60 s
maintenance sweep stays quiet.
## What got simpler
Little got simpler in the code: this mostly adds instrumentation. One CI
gap closed along the way: the 54 `config` tests and the 9 non-ignored
`boot_lifecycle` tests, which ran in no CI lane, now run in `test-unit`.
The instrumentation is scoped as small as I could make it, and it is the
evidence needed for the follow-up that should delete code: moving the
per-community NIP-43 reconcile off the pre-bind path, after which the
900 s startup-probe override can go back to the chart default.
## Testing
- `startup_steps` unit tests (`DebuggingRecorder`):
- a running step is current and has no duration
- a finished step clears current and records its elapsed time
- dropped and degraded steps clear current and record a duration
- log capture: `finish`, `degrade`+`finish`, drop, and panic each emit
exactly one `Startup phase finished` record with the right `status`.
Mutating `degrade()` to a no-op and removing the ends-once guard makes
these tests fail.
- the vocabulary is unique and shares no names with the early lifecycle
phases
- Tests that capture logs or check callsite interest now share
`test_support::tracing_dispatch_lock()`. These are the `startup_steps`
helpers, two `telemetry` tests, and the `config` and `bridge`
log-capture helpers. tracing-core caches each callsite's interest across
the whole process. Without the lock, the new panic test lost its record
in 60 of 150 parallel runs of `telemetry:: startup_steps:: config::`.
With the lock it failed 0 of 300. Removing the lock from `gauges()`
brings the failures back (53 of 100 runs).
- `nip43_progress_logs_every_interval_inside_the_startup_sweep_only`
checks the cadence at 79,942 communities (15 progress lines, none at the
start or end) and table-tests that the maintenance sweep never logs
progress.
-
`boot_lifecycle::successful_main_emits_complete_lifecycle_and_reports_current_startup_step`
boots the real relay binary against a TCP peer that never speaks
Postgres. It asserts that `/metrics` shows
`buzz_startup_phase_current{phase="db_connect"} 1` and no finished-step
durations, and that no early phase leaks into metrics.
- **CI routing.** `just test-unit` picks relay tests by hand-maintained
filter. Before this PR, CI's filters selected none of the
`startup_steps`, `telemetry`, `config` or non-ignored `boot_lifecycle`
tests above; the NIP-43 cadence test is picked up by the existing
`handlers::side_effects::tests::` clause. Its infra-free relay stanza
now selects `startup_steps::tests::`, `telemetry::tests::` and
`config::tests::` (anchored so `nip_fi_config` is not selected twice),
and a new `cargo nextest run -p buzz-relay --test boot_lifecycle` line
runs the non-ignored `boot_lifecycle` tests (they spawn the real binary
and need no Postgres or Redis; the Postgres cases stay `#[ignore]`d).
`scripts/run-tests.sh` mirrors both, running `telemetry` and `config`
single-threaded because they mutate process env.
### Manual check
1. `just relay` (or run `buzz-relay` against local Postgres and Redis).
2. Once the relay is up: `curl -s localhost:9102/metrics | grep
buzz_startup_phase`. Every step that ran has a `_seconds` value, and
every `_current` value is 0.
3. In the relay logs, `"Startup phase finished"` lines show each step's
`elapsed_ms`.
---------
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
||
|
Luke Tornquist
|
d0fd96fca6 |
BUZZ-175: Route serving event writes through the tenant lock (#7828)
## Summary
Route every serving event write through one community-admitted
transaction entry point. `Db::begin_event_write_transaction(community)`
is now the only `Db` constructor for event-write transactions: it takes
the shared community admission lock before returning, so callers that
use it cannot take domain or row locks ahead of tenant admission, and a
quiescing community rejects the write at entry. Database triggers and
commit-time fences remain the authoritative backstop.
This migrates normal, replaceable, roster, relay membership, reaction,
reminder, admin-delete, push-owned, artifact-accept, huddle-join
(48101), workflow-deletion, and command-executor event writes. Coupled
mention writes stay in the same transaction; pool-level
`insert_mentions` now opens its own admitted transaction.
**Review fixes (since `950d5f6`):**
- The unguarded event-write constructor is gone. Removed with it: the
deprecated `begin_transaction` alias, the test-only replica-floor
constructor, and `begin_community_write_transaction` (now identical to
the single constructor). Artifact accept, the huddle join, and both
workflow-deletion paths now admit before any domain or row lock;
`persist_command_event` no longer hand-rolls `guard_transaction`;
read-only `query_artifacts` no longer opens an event-write transaction.
- `insert_channel_head_checked` is now `insert_canvas_head_checked`: it
rejects non-canvas kinds before opening a transaction and always takes
the coordinate lock.
- Source policy now also scans `buzz-relay`: any function that hands a
transaction to an event-write helper must have opened it through the
admitted constructor. The scan drops each `#[cfg(test)]` item instead of
stopping at the first one: it tracks brace depth and ends the item on a
line ending in `;` or `}` (ignoring a trailing `//` comment, even after
a string that contains `//`), with a column-0 `}` as fallback. Fixtures
pin a unit struct, a multi-line static, a `const { … }` block, a struct
with fields, a module, and one-line items with trailing comments,
including after `"https://…"` and `"ws://…"` strings; a raw opener after
each must still be flagged. Known limit: braces inside string or char
literals or comments are still counted, and only `//` trailing comments
are recognized, so a test-only item with unbalanced braces in a literal
or comment, or with a trailing `/* */`, would hide what follows; none
exists today. The scan checks that admission is present; the per-path
PostgreSQL tests pin that it comes before domain locks.
- `just test-unit` (and the non-nextest fallback in
`scripts/run-tests.sh`) now runs `buzz-db --test observability_source`.
Before this, no CI job ran these source-policy tests: the unit lane ran
`buzz-db --lib` only and the PostgreSQL lane runs only ignored tests.
- Renamed or removed since `950d5f6`: `begin_transaction`,
`begin_community_write_transaction`, and the replica-floor test
constructor are gone; `insert_channel_head_checked` →
`insert_canvas_head_checked`; test helper `quiesce_community_for_tests`
now delegates to `main`'s `set_deletion_state`, which reads the real
`deletion_fence_generation` instead of hard-coding `'0'`;
`runtime::insert_mentions` is pinned to the admitted constructor by
`pool_level_insert_mentions_opens_the_tenant_local_chokepoint`.
**What got simpler:** four transaction constructors collapse to one, and
the hand-rolled admission guard is deleted. The compiler rejects callers
of the removed constructors, but a transaction opened from `Db::pool()`
can still reach the public `*_in_transaction` helpers; only the
source-policy scan and the database fences catch that until BUZZ-254.
**Behavior note:** #7932 made `insert_event` skip the transaction for
kinds that are not listener mentions. Every serving event write now has
to run in a transaction that holds the community admission lock, so
`insert_event` always opens the admitted transaction again. `main`'s
`soft_delete_event_and_update_thread` / `_in_tx` split and its
removal-marker guard are kept; the outer function now opens the admitted
transaction.
### Related issue
[BUZZ-175](https://linear.app/squareup/issue/BUZZ-175). Caller-migration
stage following #7706 (merged); rebased onto `main` at `5173fad66`.
Follow-up: [BUZZ-254](https://linear.app/squareup/issue/BUZZ-254) makes
event-write helpers require a type-level admitted transaction.
**Known debt, out of scope:** `create_channel`, `update_channel`, and
`insert_thread_metadata` write channel and thread rows (not events)
outside the admitted constructor.
### Testing
At `efe4a94623dd69973780a3c39f16dd972cf7cb76` on a cloud workstation:
fmt; `buzz-db` clippy (all targets, `-D warnings`);
`observability_source` 14/14; `just --dry-run test-unit` includes the
new step. Reverting just the end-of-item check to an earlier rule makes
a trailing-comment fixture fail.
At `eaf190ae59d1595741c3d19dc4c23ce492b5f4d4` (the commit after it only
changes this test file, `Justfile`, and `scripts/run-tests.sh`): full
GitHub CI green; on the workstation, clippy for `buzz-db` and
`buzz-relay`, ordinary `buzz-db` 154/154, `just test-unit` green, and
the PostgreSQL lane 805 of 806 (exception below). An end-to-end run
against a local relay and the built desktop app covered artifact accept,
huddle join, and workflow deletion for both a healthy and a quiescing
community, plus p-tag reaction rollback and replay of the same signed
event.
New tests: one fenced-community test per migrated path (artifact accept,
huddle join, both workflow deletions) puts the community into
`quiescing`, holds the lock the path takes next, and asserts rejection
at entry with nothing stored; a DB-free test pins the non-canvas
rejection.
Exceptions:
-
`runtime::postgres_tests::writer_pool_rejects_non_read_committed_database_default`
hangs on that workstation's local Postgres 16 at both this head and an
earlier `main` base, `f0eb5575f` (2/2 runs each). The test, `Db::new`,
and the pool setup are unchanged by this PR, and the test passes in
GitHub CI. Live traces put the hang in the client, not the database: the
1 s acquire timeout stops firing after sqlx's `after_connect` rejection
path under tokio's `current_thread` runtime. No database lock waits were
observed.
- Two `observability_source` assertions were already failing on `main`,
unnoticed because no CI job ran that test binary: a stale
`fetch_optional` pin after #8005, and lease-seam attribution that now
lives in the `bounded_serving_lease_sql` helper. Both are corrected
here, and the binary now runs in `just test-unit`.
Generated with Codex
---------
Signed-off-by: tornquist <tornquist@squareup.com>
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
|
