mirror of
https://github.com/block/buzz.git
synced 2026-10-31 14:31:02 +00:00
ae689ab54d716d6f6df6bf18c67f12a3a810ecfe
3070 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Luke Tornquist
|
ae689ab54d |
fix(desktop): resume reconnect replay from the last live point (#8652)
## Summary A relay rolling deploy closes every socket with 1012. On reconnect, desktop replayed each live subscription's **original filter** (startup `since`, full `limit`) unless that subscription had received an event. Quiet subscriptions therefore re-read history back to app start every time. Paged channels did this twice: they resent the original filter *and* ran the HTTP repair pages over the same window. A 1012 close also reset the backoff to the 1s minimum, so a client bounced from pod to pod during a deploy came back at roughly the same moment as every other client. **This PR makes three changes:** 1. **Resume cursor.** The subscription records `liveUntil`, when the connection closes, if it reached EOSE on that connection. That time is the last frame received from the relay, not the close itself. The session tracks that time per peer frame, independent of the stall watchdog, so a close delivered late (after a sleep, or while replay is still running) doesn't overstate how long the subscription was live. Close and error notices don't count as liveness. A frame processed more than 70s after the previous one (the stall watchdog's idle timeout plus its check interval) may have been queued through a sleep, so from then on it doesn't extend liveness for the rest of that connection. Liveness also starts fresh on every connection. If the time is unknown, the previous `liveUntil` is kept rather than guessed from the clock. The replay then uses `since = max(original since, lastSeen − 5s, min(liveUntil − 960s, now − 3600s))`. - **Why the one-hour window:** the relay can drop a fan-out frame for a client whose send buffer is full without closing the socket (`ConnectionState::send`), so continued pings don't prove a subscription missed nothing. The liveness term therefore never resumes later than an hour ago, which is the same window as the agent harness's silent-drop repair (#8653). The saving applies to history older than an hour, so it matters most for long-running clients, which is the deploy-bounce case. Older drops still reach the timeline, because auto-heal refetches the open channel's newest window on reconnect. What they miss are live-only side effects such as unread and mention badges. - **Why 960s:** the relay accepts `created_at` up to ±900s from its clock (`MAX_TIMESTAMP_DRIFT_SECS` in `crates/buzz-relay/src/handlers/ingest.rs`). An event stored after the close can therefore be backdated by that much, and the extra 60s covers client clock skew. - **Gap floors win:** a pinned failed-backfill floor still takes precedence. - **No cursor:** a subscription without one replays as before. - **Paged channels** keep their 1865s event-time lookback. 2. **Live-only REQ for paged channels.** When the HTTP repair covers the gap, the live REQ goes out as the original filter with `limit: 0`. The relay serves no stored events for it (`req.rs` passes the limit through to a SQL `LIMIT 0`). - **Backfill/live overlap:** the repair waits for that REQ's EOSE before it starts. EOSE arrives only after the subscription is registered, so live and repair always overlap. - **If EOSE never arrives**, a single 25s deadline, measured from when the REQ was sent, leads to the bounded history fallback below. The stall watchdog runs during replay, so a half-open socket that never sends EOSE or a close is reset by the watchdog rather than holding recovery. - **A reset, disposal or supersession** wakes that EOSE wait immediately, and the wait never starts for a retired connection, so a dead attempt never holds `connect()` (and the next reconnect) behind it. A `connect()` whose connection was replaced during replay settles as superseded instead of arming the watchdog. - **If EOSE never arrives, or all three repair attempts fail**, the live REQ is re-issued under the same ID with the window's bounded history (`since` = floor, `limit` ≤ 500): the pre-change behavior. The missed window still fills on this connection, and the floor stays pinned for anything the cap left out. Duplicate suppression stays on until that request's own EOSE, so events live already delivered aren't dispatched twice. If the original EOSE was never seen, a late one can't be told apart from the fallback's, so only the deadline or teardown ends suppression. Teardown also ends it, and so does a 25s deadline in case the relay drops the EOSE. A subscription disposed while the fallback is in flight still ends with a CLOSE. 3. **Reconnect spreading.** After a 1012 or 1001 close, the first attempt waits a uniform random delay in [0, 10s] instead of resetting to 1s. Ordinary backoff jitter is a symmetric ±25%, capped at 30s. The backoff resets only after the existing 60s stable-connection check, never just because a socket opened. **Simpler?** The backoff state, including its 60s stable-connection reset timer, moved out of `relayClientSession.ts` into a small `RelayReconnectBackoff` class in `relayReconnectPolicy.ts`. That makes the jitter math testable on its own. It also keeps the session file at 1199 lines, the same as `main` (its cap is 1200), even with the new liveness and replay logic; the liveness gap rule lives in `relayResumeCursor.ts`. The stall watchdog's API is unchanged from `main`. **Identified debt (not changed here):** - `relayClosedRecovery.ts` retries a retryable live CLOSED with the original filter (1s → 30s backoff), with no terminal state. - `useRelayAutoHeal.ts` sends an invalidation wave after reconnect. - The `lastSeen − 5s` term keeps its existing gap for events backdated after a close. - `liveUntil` means the subscription was registered, not that every event was delivered. The one-hour window bounds this: a dropped fan-out frame older than an hour on a quiet subscription isn't re-read over the websocket. The fix is a relay-side drop signal, the same follow-up as #8653's. - An HTTP repair page that is already in flight when the socket resets still holds `connect()` until it returns, as on `main`. The native HTTP client sets no request timeout. - A `limit: 0` REQ still makes the relay acquire a reader for an empty query. ### Related issue N/A. This is part of a set of reconnect-load reductions. Companion PRs: relay read-pool warm (#8651), agent harness (#8653) and mobile (#8659). ### Testing - **Full desktop gate** on a Linux workstation at `045030c4`: - `pnpm typecheck`, `pnpm check` and the file-size check pass. - `pnpm test`: 6901/6901, plus 165/165 jsdom. - `pnpm build:e2e` passes. - The `relay-reconnect` Playwright spec passes, including the restart spread draw, which passed 10/10 repeats. - **New or extended tests** (`node:test`): - `relayResumeCursor.test.mjs`: a table test over the cursor inputs, including the one-hour window, plus the liveness gap rule. - `relayClientResumeCursor.test.mjs`: drives the real `RelayClient`. It covers a dropped fan-out followed by pings, a frame queued through a sleep, fresh credit per connection, a half-open socket reset by the watchdog during replay, and a reset or disposal while parked on EOSE. - Extended: `relayReconnectReplay` (deadline from send, a single timeout → fallback, a late original EOSE, fallback dedupe and CLOSE convergence), `relayReconnectPolicy` and `relayClosedRecovery`. - **Mutations:** each guard was reverted and failed at least one test. That covers every cursor term, the 900 + 60 skew, the one-hour window, `limit: 0`, the EOSE wait, its deadline and its timeout, floor precedence, EOSE generation stamping, the last-inbound cap and gap rule, the per-connection reset, the watchdog starting before replay, fallback dedupe, the late-EOSE guard, the 1001 match, the backoff reset, the restart spread and jitter symmetry. - **Exercised against a real relay** (PG16 and Redis) in a relay-backed desktop build, with Chromium on a Linux workstation, base `b310fea0` vs head `045030c4`. Each channel was seeded with 200 reactions and 3 messages, all two hours old: | | Base | Head | |---|---:|---:| | Quiet channel, stored events replayed over the websocket | 204 | 1 | | Paged channel, stored events replayed over the websocket | 50 | 0 (`limit:0`) | | Paged HTTP repair, events returned | 204 | 1 | | Event-query calls / rows returned | 68 / 895 | 73 / 441 (−50.7% rows) | - **21-minute process freeze:** the watchdog fired first. The quiet cursor matched `max(lastSeen − 5, min(preFreeze − 960, now − 3600))` exactly, and 3/3 messages posted during the freeze were recovered once each. - **Drop before the paged EOSE:** `connect()` released in 13ms. The next dial followed on the normal backoff, and the message arrived once. - **Half-open socket** (all relay frames withheld, no close): the watchdog reset it at 60.0s, the next dial came 1.2s later, and both messages were recovered once each. - **Earlier run at `180dc9ae`, before the one-hour window, relay restart (1012):** the quiet channel resumed from last inbound − 960s. At this head that cursor is also capped at now − 3600s. The paged channel sent `limit:0`, with repair starting about 45ms after EOSE. Both messages posted during the bounce were delivered, including one backdated 600s. The first reconnect came about 8s after the close. buzz-review-completed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Luke Tornquist
|
aed4be8592 |
Add a scenario-driven relay load generator (#8636)
## Summary Buzz has no way to load-test the relay as a whole. `wamp_bench` sends kind:9 messages from one identity to one channel, and `perf/relay_bus_scaling.py` only measures Redis fan-out. Neither can show what happens to a relay with many more users, channels, or deep threads. This PR adds `buzz-loadgen`, a load generator in `crates/buzz-test-client`: - `buzz-loadgen seed <scenario.toml>` creates communities (through `POST /operator/communities`), channels, channel history, and deep threads. - `buzz-loadgen run <scenario.toml>` drives simulated humans and agents through ramp steps (1x, 2x, ...) and prints p50/p90/p99 latency, errors, and rate-limit rejections per operation for each step. - Humans hold a WebSocket with live subscriptions, fetch history on connect, and send presence, typing, messages, thread replies, and reactions. Agents authenticate with a NIP-OA tag, poll HTTP `/query`, post through HTTP `/events`, and send encrypted observer frames to their owner. - Per-client rates come from the scenario. Set them to model a deployment, and a step of `users = 10` means 10x that deployment's population. - Every identity, host, and channel id comes from the scenario's `run_id`, so `--shard i/n` splits a population across machines with no coordination. - Each step measures from ramp start to hold end, so connect failures during the ramp count. A step aborts the run when an operation's error rate or p99 (delivery included) crosses the scenario's `[abort]` limits, when a relay `CLOSED`s a live subscription often enough, or when a client exits unexpectedly. The report names the limit that tripped. - Each connect plus authentication has a deadline (`target.connect_timeout_secs`), and shutdown is bounded, so a relay that stalls connections still produces a report and a non-zero exit. - `seed` exits non-zero unless the relay accepted every planned post. - `NostrWsConnection::connect_to_addr` opens the TCP connection to a given address while the URL still names the host. This lets a client inside the relay's cluster reach a community host through the relay's service address. - `wamp_bench` is deleted. Its paced single-channel load is a subset of a scenario. What got simpler: one load tool instead of two partial ones. The change touches only test tooling, the unit-test lane, and one additive method in `buzz-ws-client`. Relay code is unchanged. ### Related issue None found. ### Testing Smoke run on a Linux workstation against a local relay built from this branch, with its own Postgres, Redis, and MinIO containers: ```bash cargo run --release -p buzz-test-client --bin buzz-loadgen -- seed perf/scenarios/local-smoke.toml cargo run --release -p buzz-test-client --bin buzz-loadgen -- run perf/scenarios/local-smoke.toml ``` - `seed`: 710 of 710 posts accepted (10 channels, 200 history messages, 10 thread roots, 500 thread replies). - `run`: both steps (1x = 50 humans + 5 agents, then 2x) completed with 0 errors and 0 rate-limit rejections. Every operation recorded traffic: 55/55 connect/auth, 205/205 live subscriptions, 150 history fetches, presence/typing/observer frames, messages and reactions, HTTP `/query`, HTTP `/events`, and publish→delivery. Delivery p99 was 45 ms at 1x and 42 ms at 2x. - Failure paths are covered by unit tests: a listener that accepts TCP but never answers (connect deadline, then `run` aborts and returns), a fake `/events` server that rejects posts (`seed` fails), `CLOSED` subscriptions, and invalid scenarios. `just test-unit` (and its `scripts/run-tests.sh` fallback) now runs the new `loadgen::` unit tests. Before this change, no CI lane ran `buzz-test-client` lib tests. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ravarora2
|
8caf8351c1 |
Add standalone owned-community deletion handlers (#8645)
## Summary
User deletion needs to delegate owned communities to the existing
community deletion process and report what is still waiting. This adds
the standalone components for that flow; wiring them into the user
workflow is a later change.
The owned-community handler checks for an existing community deletion
request first. If none exists, it calls `submit_and_prepare`, the same
function used by the community CLI. The handler saves the child request
ID, last observed status, and operator instructions on the existing user
stage. Community approval and execution remain separate. Repeated or
concurrent calls reuse the existing child without resetting its approval
or execution state. The user stage follows the current request for its
approved community ID. If an operator aborts A and submits B for that
community, the next invocation saves B on the same stage row.
An unfinished child leaves the user stage waiting. A later handler
invocation refreshes its observation and completes the stage when the
child reaches logical completion (`retention_pending` with a completion
timestamp). Retained-copy obligations remain visible in the saved
result. Lease owner and generation checks protect stage updates.
The separate summary handler reads only the user request, approval, and
stage tables. It lists unfinished stages, saved external waits and
operator actions, completed results, and inventory communities not yet
reached. It never queries or refreshes community deletion state, so
child statuses are explicitly labeled as last observed.
### Scope and rollout
- No migrations, new HTTP endpoints, CLI commands, or worker dispatch
changes. Owned-community intake remains disabled.
- Waiting uses the existing stage status plus `metadata_json.waiting`.
This slice does not advance past waiting stages. Integration must add an
explicit resume or bounded revisit cadence before enabling automatic
scheduling.
- The CLI and user handler share the existing database submission,
inventory building, and inventory freezing operations. There is no API
wrapper class or second insertion path. Preparation failures leave a
durable child request and a retry record on the user stage; subsequent
lookup exposes instructions for the operator to finish preparation.
Guidance distinguishes operator-origin preparation, automatic
owner-origin preparation, blocked requests that need remediation and
`unblock`, and aborted requests that can be replaced.
- Existing submission accepts a host for the CLI or an immutable
community ID for the parent. A transaction protects target resolution
and admission. ID-pinned calls never replace an existing child, even an
aborted one. Inventory helpers take their store and media dependencies
directly. Successful stage persistence clears any saved waiting marker.
- These components become available to callers after release but are not
scheduled by the relay or user worker. No data migration is needed to
roll back this change.
See [the implementation and testing
guide](
|
||
|
klopez4212
|
e922dc08ff |
Unify mobile onboarding and community entry screens (#8687)
## Summary Use consistent fullscreen screens for identity setup, community selection, joining, and Add Community. Restore all memberships and open the selected community; share the community grid with the switcher. Add QR-first choices, animated link entry with 12px keyboard clearance, a two-finger double-tap pairing toggle, and the centered reversing bee loader. Preserve cancellation, Back, and reduced-motion behavior. Add intentionally omits community creation and Builderlab sign-in: this route follows sign-in and the existing-community picker. ### Related issue Related: #8628 (native profile setup). No duplicate fullscreen onboarding PR found. ### Testing - Full standard and hosted-authentication mobile suites, static analysis, and required push checks passed. - iPhone preview iterated with Kenny; final simulator walkthrough below uses synthetic data and does not authenticate or save account changes. - Independent review completed; regressions cover cancellation during loading, Add recovery after failed/conflicting memberships, retrying skipped communities, and explicit picker stages. Walkthrough recorded before the copy and recovery followups: https://github.com/user-attachments/assets/bfea3ad3-1dde-40c4-8748-44c84226c292 --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
8abcf2ea2a |
fix(mobile): keep read state across reconnects and resume (#8672)
### Context Every app resume and relay reconnect tore down `ReadStateManager` and built a new one. `ReadStateNotifier.build()` watched the whole `relaySessionProvider` state, so each status change (disconnected, connecting, connected) rebuilt the notifier. On an iPhone 12 mini each resume re-ran the full read-state fetch and decrypt (about 19 to 24 decrypts, 60 to 95 ms). The rebuild also cleared `_forcedUnreadContexts`, so a message or channel the user marked unread lost that flag on any connection change. The watch dates from #422, when the manager was built with `remoteEnabled` tied to the connection status. #802 made `remoteEnabled` constant and added the `ref.listen` that calls `reinitializeRemote()` on connect, but left the watch behind. ### Change - Watch `relaySessionProvider.notifier` instead of its state. Config, nsec, community, pubkey, crypto and a replaced session still rebuild. Status changes don't. - Reconnects re-sync through the existing listener (`reinitializeRemote()`: re-fetch, then re-subscribe, preserving the history and live overlap). - Re-syncs are serialized. One runs at a time after any pending `initialize()`, and reconnects that arrive while one waits share it. Overlapping passes could otherwise each install a live subscription and leak one. - A connect that lands while the first `initialize()` is running (the cold-start connect) skips the re-sync when that initialize fetched and subscribed successfully. Without this, cold start did the read-state work twice. ### Validation - Provider journey: disconnected, connecting and reconnecting cause no refetch, no re-subscribe and no readiness drop, and the forced-unread flag and read mark survive. Connected runs exactly one refetch and one live-subscription swap. A second reconnect behaves the same. Account and community switches still rebuild and clear forced flags. - Manager tests: a connect during a successful `initialize()` does not sync again; one during a failed `initialize()` does, and leaves one live subscription; three reconnects during a running re-sync produce one more shared pass and one live subscription. - Falsified: restoring the old watch fails the reconnect journey (`isReady` false, flag lost). The previous version without serialized re-syncs fails the overlap test (5 passes instead of 3). - `just mobile-check` and `just mobile-test` pass. iPhone 12 mini (Profile builds of base and this branch, real community over the production relay, instrumented manager lifecycle and live-subscription counts): | | base | this branch | | --- | --- | --- | | Resume after 3 s, 10 s (x3), 60 s background | 3 new managers and 3 initializes per resume | no new manager, 1 re-sync | | Live read-state subscriptions after resume | 1 | 1 | | Channel marked unread, then 10 s / 60 s background | flag lost after the first resume | still unread after both | | Read-state work per resume (median) | ~24 decrypts, ~176 ms main isolate | ~18 decrypts, ~125 ms | | Unread badges | same bold set | same bold set | The first device run also found the doubled cold-start sync, fixed above. A re-check on this head: | Cold launch | base | first version | this head | | --- | --- | --- | --- | | Managers built | 3 | 1 | 1 | | Read-state syncs | 2 initializes | 1 initialize + 1 re-sync | 1 initialize | | Decrypts | 19 | 30 | 15 | | Main-isolate ms in read-state code | 124 | 185 to 197 | 94 to 98 | Four resumes on this head: no new manager, 1 re-sync and 1 live subscription each, 17 to 20 decrypts. A foreground socket drop was not exercised on the device; every resume there was a full disconnect and reconnect. buzz-review-completed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Logan Johnson
|
5ea5ae1727 |
fix(desktop): always flash a linked message in the channel timeline (#8660)
## Problem When you follow a link to a message, the channel timeline should scroll to it and flash the row for about 2 seconds. On `main` the flash often does not appear: | How you reach the message | Flash on `main` | |---|---| | Same channel, row already on screen | yes | | Same channel, row loaded but off screen | **no** | | Same channel, row in older history not loaded yet | yes | | From another channel, row near the bottom | **no** | | From another channel, row further up | **no**, and the timeline stays at the bottom | The main timeline is always virtualized. `useAnchoredScroll` had three ways to scroll to a target there. Only one of them, "the row is already fully on screen", fired the flash. In the other cases the scroll returned without the flash, and the follow-up target effect marked the target handled without checking the result, so the flash never fired. Opening a link from another channel had a second problem. The row could look on screen while the channel's first bottom pin was still in flight in Virtua. The shortcut used a direct DOM `scrollTo`, which does not cancel that pin, so Virtua kept going to the bottom and took the target with it. Top-level message links also open the thread panel, and that resize settled the timeline at the bottom again. A third problem also exists on `main`. Centering a row near the top of the loaded history loads an older page. Opening a channel also starts a refresh of the newest channel window, and that refresh then drops the older pages from the head of the list. A head trim was not passed to Virtua as a `shift`, so the rows above the reader vanished with no compensation and the view fell to the bottom about 0.3–0.8 s after the flash started. The deep-link e2e test stopped checking the flash in #5306, so CI did not catch this. ## Change - In virtualized mode, every target scroll now goes through Virtua's `scrollToIndex` (centered). That replaces any bottom pin still in flight. Smooth scrolling is kept only when the row is already in the viewport. - Every target scroll that Virtua accepts now fires the flash. The highlight is keyed by message id, so a row that Virtua mounts after the scroll still shows it. - The scroll returns `true` once Virtua accepts the target and `false` while the target is not loaded yet. The follow-up target effect and the search retry now just read that value. This removes the "phase 1" special case and the duplicate DOM lookup. - A target scroll also clears the virtualizer's at-bottom state, so a resize right after it (for example the thread panel opening) does not pin back to the bottom. - Only an accepted target scroll retires bottom intent. It also cancels the hook's queued second mount-to-bottom frame. A target that is not loaded yet leaves the opening bottom pin alone while it retries. - `didPrependVirtualizedTimeline` now also admits a pure head trim (the new keys are an exact suffix of the previous keys), so Virtua's `shift` keeps the reading row in place when a window refresh drops older pages. The oldest loaded day has no divider while more history exists, so a head trim stays an exact suffix even when it cuts inside a day. The non-virtualized path (thread panel) is unchanged. ## Screenshots Before and after, from another channel, for a link to a row further up: https://github.com/block/buzz/pull/8660#issuecomment-6481877304 ## Testing Evidence at head `653d89766`: - New `desktop/tests/e2e/message-target-highlight.spec.ts`: a 5-row table with the cases above. Each case asserts that the target row is on screen **and** has the flash class at the same time, and that the row is still on screen 2.2 s later, after the flash ends. 15/15 passed (3 runs each). - On unfixed `main`, 3 of the 5 cases fail the flash check. - With only the head-trim change reverted, "same channel, loaded off screen" and "other channel, further up" fail the 2.2 s check. - `useAnchoredScroll.lifecycle.test.mjs`: an on-screen virtual target scrolls through the virtualizer, makes no direct DOM write, retires bottom intent and does not re-pin later. New: a target that is not loaded yet keeps the bottom pin. It fails with the old cancel order. - `virtualizedTimelineItems.test.mjs`: new rows for a head trim inside the oldest day and at its boundary (admitted, with the size cache aligned), and for a trim plus append and a middle removal (rejected). - `pnpm test` (6837 + 161 pass), `pnpm check` (tsc via pre-push, biome, px-text, pubkey truncation). - These e2e specs, run together: message-target-highlight, scroll-history, virtualization, timeline-no-shift, unread-pill, overscroll-boundary, dm-history-live-regression, live-broadcast-reply-timeline, message-copy-link, thread-focus-mode, thread-follow, thread-reply-anchor-roleplay, thread-unread, channels, messaging. 257 passed, 0 failed. Later commits at `b8e172672` add no production behavior. They move one helper above `computeAnchor`'s doc comment, and add a lifecycle test (from Cerveau's probe): a target that loads before the queued second mount frame is not scrolled back to the bottom. That test fails without the mount-frame cancellation. Pre-push `pnpm test` passed at `b8e172672`. Adversarial review by Auto (virtualizer and callers) and Cerveau (lifecycle, tests and thread panel). Their must-fix findings are fixed above, and both approved `653d89766`. Follow-up, not in this PR: after a jump to a row at the very bottom, centering is clamped, so no scroll event fires and the at-bottom state stays false until the next scroll. `main` has the same gap for `isAtBottom`. Not tested yet in the native app. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Cerveau <fc91c5708d48aecea993d9219505dc841bc7b0222b7ed325b8edfd026bfb43b5@buzz.block.builderlab.xyz> |
||
|
Wes
|
0d19ca69d3 |
fix(mobile): wait for app activation before browser auth (#8647)
## Summary Opening an invite from Safari can deliver the login request while iOS still considers Buzz inactive. `ASWebAuthenticationSession.start()` then returns false, showing a connection error until the user retries. Wait for app activation before starting the retained session, with a ten-second timeout and cancellation cleanup. Extract the existing browser-auth bridge from the size-limited AppDelegate and add one native journey test through its Flutter method channel. URL validation, callback fencing, and normal active-app login retain their existing behavior. Temporary diagnostic logging and generated dependency changes are excluded. ### Related issue No duplicate found in the open handoff/authentication search. Related invite-consent work: #8601; this fixes the subsequent iOS authentication presentation race. ### Testing - Focused `RunnerTests/BrowserAuthBridgeTests` passed on an iPhone 18 Pro / iOS 27 simulator. Covers inactive handoff, cancellation before activation, retry, duplicate activation notifications, and immediate active-app start. - A local build reproduced the original inactive-app `start=false` failure; the fixed build opened authentication on the first background-to-app attempt. The same fixed build subsequently completed an existing-account staging flow. - Mutation check passed in isolated source, Git metadata, and build outputs: clean control passed; bypassing the production activation wait failed the inactive-handoff assertion; restored source passed again, with its SHA-256 verified against the committed file. - Self-review found no blocking issue. CI and final review remain pending. Mandatory pre-commit and pre-push hooks passed, including mobile analysis and the hook-run Flutter suite. No full local suite was manually invoked. - Human acceptance: from Safari, tap **Accept invite in Buzz** and confirm sign-in opens on the first attempt without **Try again**. Final acceptance of this PR remains pending. The separately reported callback failure may have involved production rather than staging. That cause is unconfirmed and is outside this change. Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz> |
||
|
Tom Brow
|
8161a7cad0 |
perf(mobile): derive public keys with native libsecp256k1 (#8655)
Follows #8654. Deriving an x-only public key from a secret key with `nostr.Keys(secret).public` costs about 5.6 ms in pure Dart on an iPhone SE (2nd gen, Profile). The app does it in 10 places, including relay config, the signed-event relay's `pubkey` getter, push lease checks, status updates, and the settings identity row's `build` method. This routes all of them through `derivePublicKeyHex` from #8654, which takes 16.5 µs on the same device. Behavior is unchanged for valid keys: output is the same lowercase hex. For a malformed or out-of-range key the thrown type changes from nostr's `InvalidKeyException` to `ArgumentError`; every affected site either catches all errors or rethrows its own `FormatException`/`StateError`, and nothing in `lib` or `test` matches on the old type. Sites that need a full `nostr.Keys` object (key generation, `AccountIdentityStore`) are unchanged. ### Validation - `derivePublicKeyHex` is checked against `nostr.Keys` for 64 random keys in #8654's `event_signing_test.dart`. - Full mobile analyze and test suite pass on this branch after rebasing onto main, including the package's own tests. - iPhone 12 mini, Profile build of this stack (with #8656 and #8657 on top): paired to the internal Buzz community through Desktop; relay auth, identity row, avatar, and channel/DM/inbox loading all worked under the derived key. buzz-review-completed (agent review, Codex and security review clean; device validation above; human app test not applicable under the mobile testing exception) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: george <973be878edd6d4f79692a3827f20a6f7fe0ed0362f269ff6b6ba8867033d7d8e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
770ab7ff82 |
perf(mobile): decode relay query responses off the UI isolate (#8669)
### Context Profiling on an iPhone 12 mini (Profile build, real community) showed `RelaySessionNotifier._postForEvents` producing contiguous main-isolate busy runs with p90 110 ms and max 124 ms (27 runs of 50 ms or longer). Stacks were dominated by `jsonDecode` of `response.body`, with `NostrEvent.fromJson` smaller. The whole HTTP response decode ran on the UI isolate: gzip inflate (done by `dart:io`), the UTF-8 decode behind `response.body`, JSON parsing, and event parsing. ### What changes - Successful `/query` and `/profiles/search` responses (the only two callers of `_postForEvents`) are now decoded in a single `Isolate.run` from `response.bodyBytes`: inflate, UTF-8, JSON, validation, `NostrEvent.fromJson`. The parsed events come back by ownership transfer (`Isolate.exit`), not a copy. - Gzip: the query transport now sends `Accept-Encoding: gzip` and sets `autoUncompress = false`, so compressed bodies reach the isolate still compressed. Whether bytes are gzip is decided by the gzip magic bytes, so a transport that already inflated (or an injected test client) still works. Error bodies are inflated inline (small). - Unchanged: `FormatException` messages, the malformed-event wrapping, `RelayException.body`, and the rate-limit gate reading the error body. - Fencing (Review-Proven Rule 2): the `_disposed` / `_ageRestricted` / `_contextGeneration` check still runs before the decode and now runs again after it. If the session retires during the decode, the query reports `StateError` even when the decode itself failed, matching the old ordering where a retired session never decoded. ### Design notes - `Isolate.run` per response instead of a long-lived worker: no lifecycle or disposal to manage, no idle isolate heap on a small phone, easy to fake in tests, and the same shape as `parseProfileEventBatch`. The main-isolate side of a spawn measured at microseconds, so there is no size threshold; small responses take the same path. - Event id and signature verification does not happen on these events on the main isolate today (nothing in the mobile app verifies queried events; NIP-OA verification in profile parsing is already in an isolate), so nothing was moved. - Other HTTP decode sites found and left alone: invite create/join, hosted join policy, BuilderLab account, media upload response, community icon lookup, dev push lease NIP-11, archived-agents. These decode small, rare responses (one small JSON object each), where an isolate adds complexity without a measurable gain. Media and avatar byte downloads do no JSON decoding. The mobile app has no `/count` caller. ### Validation Added tests (`relay_session_test.dart`, through the production `queryRelay` / `searchProfiles` path with the real isolate decoder): parsing with non-ASCII content; profile search; gzip success body and `Accept-Encoding`; gzip error body feeding the rate-limit gate and `RelayException.body`; malformed JSON; non-list body; non-object element; malformed event; invalid UTF-8; error status skips decoding; session disposed, age-restricted, or retired-before-decode while a decode is outstanding. The fencing tests are falsifiable: removing the post-decode check fails the disposed and age-restricted tests; removing the retired-wins-over-decode-error branch fails its test; removing the pre-decode check fails the never-decodes test. `FakeRelay` (test helper) now injects an inline decoder, because `fakeAsync` cannot deliver isolate results; it calls the same parse function. The default transport still honors `http.runWithClient`, which the hosted journey tests use to fake the network (`relay_http_query_client_test.dart`; fails without the fix). `just mobile-check` and `just mobile-test` pass. Host benchmark (AOT executable on an Apple silicon Mac, median of 25, copy of the parse logic, main-isolate stall measured with a 0.5 ms heartbeat timer): | Response | Main-isolate blocked before | after | | --- | --- | --- | | 500 events, ~630 KB | 2.4 ms | 0 ms (max gap 0.05 ms) | | 500 events, ~630 KB gzip | 3.2 ms | 0 ms (max gap 0.19 ms) | | 500 events with 50 20 KB avatars, ~1.6 MB | 5.1 ms | 0 ms (max gap 0.28 ms) | | 500 events, ~4.5 MB | 14.0 ms | 0 ms (max gap 0.27 ms) | | 500 events, ~4.5 MB (827 KB gzip) | 17.4 ms | 0 ms (max gap 0.35 ms) | Total latency is unchanged within noise (for example 14.5 ms to 14.9 ms). The host is much faster than the 12 mini, so absolute device savings should be several times larger. On the iPhone 12 mini (Profile builds of base and this branch, real community over the production relay, two runs per scenario; main-isolate stalls measured with a 4 ms heartbeat timer, since the wireless device tunnel has no VM service): | Scenario | Main-isolate `_postForEvents` time, base | this branch | Max stall, base | this branch | | --- | --- | --- | --- | --- | | Cold start + 60 s (~16k events, 7.8 MB gzip) | 282 to 295 ms (max 116 ms) | 0.01 ms | 147 to 150 ms | 100 to 102 ms | | Inbox + search (~7k events) | 122 to 123 ms (max 111 ms) | 0.01 ms | 138 to 142 ms | 97 to 99 ms | | Open 6 channels (558 events) | 5 to 6 ms | 0.01 ms | 88 ms | 86 to 87 ms | | Background 10 s + resume (~7k events) | 118 to 124 ms | 0.01 ms | n/a (suspension) | n/a | - The production relay serves gzip (about 4.4:1; the largest catch-up response is 3.15 MB on the wire, 13.9 MB inflated, 7k events). Inside the isolate that response takes 30 to 47 ms to inflate, 39 to 55 ms for UTF-8, 65 to 105 ms for `jsonDecode` and 7 to 32 ms for `NostrEvent.fromJson`. - Many 16 to 90 ms stalls remain on both builds; they come from other work (mostly markdown build and layout), not response decoding. - Home, channels, inbox, search, profile and a thread render the same on both builds. buzz-review-completed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
a7e65c91da |
feat(mobile): choose remote agent models and gate agents on Builderlab sign-in (#8650)
Follow-up to [#8626](https://github.com/block/buzz/pull/8626). Mobile remote-agent creation now matches desktop. - **Model and reasoning effort:** Create agent has Model and Reasoning effort menus. Their options come from the remote service's catalog, and nothing is preselected (same as desktop). The server checks the selection against the catalog again before registering, and saves it on the agent. - **Builderlab gate:** creating a remote agent requires a hosted community and Builderlab sign-in. The Agents directory, community bots and the Bestie chooser stay available without it. Signing out closes an open Create agent sheet; the Agents sheet stays open. - **Enrollment:** `attest-agent` enrolls the agent in the requested community, as on desktop, so the separate `add-buzz-subscription` call is gone. - **Quieter progress:** spinners replace the progress bars and the "Working…" label. Status and success text ("is ready", "Remote agent deleted.", setup steps) is gone, and only failures show text. Refreshing an already-loaded agent list no longer blanks it. - **Shared setting menu:** `BuzzSettingMenu` is a bordered setting row that opens a Material 3 menu spanning the row's width. Opening it dismisses the keyboard. The channel-TTL setting in Create channel uses it too. Agent avatars and QR/link-based creation follow in stacked PRs. Editing existing agents comes later. | Agents | Create | Model menu | | --- | --- | --- | | <img src="https://raw.githubusercontent.com/block/buzz/d0bee87c45d30b9bdd2163a75966ccd5d64f322a/pr-8650--1-agents.png" width="240"> | <img src="https://raw.githubusercontent.com/block/buzz/d0bee87c45d30b9bdd2163a75966ccd5d64f322a/pr-8650--2-create.png" width="240"> | <img src="https://raw.githubusercontent.com/block/buzz/d0bee87c45d30b9bdd2163a75966ccd5d64f322a/pr-8650--3-model-menu.png" width="240"> | ### Validation - Checked on the staging iOS simulator with a Builderlab account: the Agents list, the Create sheet, and the model and effort menus (including with the keyboard open). - Added tests: catalog-gated model selection (including switching models clears an unsupported effort), the Builderlab sign-in gate, controller creation and resume with a model, a failed listing reported instead of an empty registry, and create sheet busy and failure states (including the catalog reload after a rejected create). - The hosted-only agent tests are skipped without Builderlab defines, which `mobile-test` doesn't pass. They pass locally with the staging defines (20/20). - Not covered: Android. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: george <973be878edd6d4f79692a3827f20a6f7fe0ed0362f269ff6b6ba8867033d7d8e@sprout-oss.stage.blox.sqprod.co> |
||
|
klopez4212
|
39b8b2b6a6 |
feat(mobile): unify Messages navigation with Solo and Team filters (#8649)
## Summary Unify mobile conversations in Messages with All / Solo / Team filters, a collapsing header, and separate Search, New, and default-agent shortcuts. Move workspace actions into the community menu and keep collapsed sections intact across filters. Align Inbox and list typography, spacing, and muted controls. Preserve native glass layering through navigation and give every filtered list enough clearance above the footer. The default-agent shortcut reads the shared encrypted preference and opens an existing chat immediately. When unset, it opens Agents; a star action selects the default through a production signed, self-encrypted preference writer. Save failures remain visible and retryable. Android now uses rounded navigation controls and filters, optically balanced search icons, and expanding New/community menus with leading Tabler icons matching iOS. Preset avatars retain transparent WebM animation through an Android WebView renderer, with lifecycle pausing and poster fallback. ### Related issue None found in searches for mobile navigation issues and open PRs. ### Testing - Android follow-up: ARM64 debug build, Dart analysis, and emulator checks passed for menu placement, selection, icon alignment, search, and avatar playback/pause/resume. The avatar implementation passed 3,840 Flutter tests and native Android tests before subsequent styling iterations. Automated tests for the later menu/icon iterations were skipped at the author’s request; those earlier results do not cover the final styling changes. - Mobile formatting/analysis and full test suite (3,838 passed, 23 skipped, plus 4 push-configuration tests); regression journey covers filtering, collapse retention, and scrolling the last Solo row above the footer. - iPhone 18 Pro simulator (Honeycomb): filters, menus, Inbox, Invite/Back, Switch Community, and default-agent chat/Back exercised. iOS release build and Pairing Preview install/launch passed during iteration, preserving the saved profile. - P1 follow-up: production writer-to-reader widget journey covers selection, publish failure/retry, and the unset footer opening Agents. Simulator Agents open/close passes; its saved community has no selectable agents, so live selection was not verified there. - Independent agent review completed; its Solo footer-clearance finding is fixed. The final clearance adjustment is covered by widget tests; physical-device acceptance of that adjustment and hosted CI remain pending. Final Messages layout:  Switcher layering during iteration (before → after): | Before: filter edge showing through | After: Home hidden behind switcher | | --- | --- | |  |  | --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
7c165c3718 |
Smooth profile avatar editor transitions (#8640)
Smooths profile avatar editing with concurrent 200 ms entry/return transitions, fixed-size circle movement between tabs, and a crossfade to the dotted placeholder on cancel. New empty avatars open on Emoji, ready to save. Review fixes: keep the suggested emoji separate from the Image fallback, and preserve native editor transparency across appearance changes. Regression journeys cover empty Image previews, retained Emoji selection, and light/dark updates. Validation: focused Flutter journeys pass; restoring the old baseline fails both fresh-editor regressions. At `bffb83f5e`: full Flutter suite and push checks passed; all 107 native tests passed on the iOS simulator. Prior iPhone visual approval covers the transition work; these review fixes still need a device check. Follow-up to #8628. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Matt Toohey
|
6e332f46c7 |
chore(desktop): point just staging at the new Block staging relay (#8623)
bb-block staging is moving from `test.blockstaging.build` to `block.builderlabstaging.xyz` ([infra #1570](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/1570), merged). Its relay moves to `buzz.block.builderlabstaging.xyz`, so `just staging` now points there instead of `buzz.test.blockstaging.build`. That was the only reference to the old host on `main`. `mobile/` has none. The cutover has landed: [#2028](https://github.com/squareup/builderbot-platform-core-infrastructure/pull/2028), which removed the cutover DENYs, merged on 2026-10-27. `buzz.block.builderlabstaging.xyz` now serves NIP-11, and the old `buzz.test.blockstaging.build` returns 404, so `just staging` on `main` currently points at a dead relay. Validation: every local pre-push hook passed except `mobile-checks`, which I excluded. Changing the Justfile triggers it, and it can't run here because local Flutter ships Dart 3.9.2 while `mobile/` requires ^3.11.4. CI runs it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Jitter <2b2e6415e748c35180847a37aadae06a11e30dddc76b784e7fd9354c4eb42e7a@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Matt Toohey
|
eda26b9423 |
ci: repeat changed tests ten times on pull requests (#8613)
This ports the advisory flake check from block/buzz-app (https://github.com/block/buzz-app/pull/890) to this repository: pull requests repeat the tests they change ten times, with retries off, so a test that passes CI by luck is caught before it merges. ## What it does `flake-select` diffs the PR merge commit against its first parent (`HEAD^1..HEAD`, `fetch-depth: 2`) and maps changed hunks to tests per runner: a hunk inside a test body selects that test; a change anywhere else in a test file, or a new file, selects the whole file. One lane per runner then repeats only that selection 10× with retries disabled and writes a test × iteration table to its step summary. Lanes skip when nothing was selected, run only on `pull_request`, and are not read by any required gate in `ci.yml` — their own statuses carry the result. ## Per runner | Runner | Selection granularity | Repeat mechanism | |---|---|---| | Rust (cargo-nextest) | `crates/<crate>/<path>.rs::<module::test>` from `#[test]`-style attribute spans (comments and literals blanked first). A hunk outside any test body selects the file's tests. `#[ignore]`d and Postgres / `external_infra` tests are excluded. | One `cargo nextest run --profile flake` per package per iteration, with target flags derived from the path (`src/**` → `--lib --bins`, `tests/<stem>.rs` → `--test <stem>`), a filterset of exact `binary_id & test` pairs, and JUnit parsed per iteration. New `[profile.flake]`: `retries = 0`, `fail-fast = false`. | | node:test | Whole changed `desktop/src/**/*.test.mjs`, `*.jsdom-test.mjs`, `desktop/scripts/*.test.mjs` | 10× loop, each file alone, with the flag sets from `desktop/package.json` `test` | | Playwright `smoke` | `file:line` journeys from `test()`/`it()` spans (TypeScript AST) intersected with `playwright test --list --project=smoke`, deduplicated | Native `--repeat-each=10 --retries=0`, sharded `min(10, ceil(journeys / 5))` ways; `playwright-summary` tabulates the JSON report | | Flutter | Whole changed `mobile/test/**/*_test.dart` | 10× `flutter test --file-reporter=json` loop with the mobile lane's `--dart-define` | A runner whose selection exceeds 64 KiB of entries, or more than 250 Playwright journeys, is skipped with a note rather than overrunning its timeout. ## Differences from the buzz-app version - Lanes live in a reusable `_ci-flake.yml` called from `ci.yml`, matching this repo's `_ci-*.yml` convention, instead of inline jobs. - Playwright specs are parsed with the desktop app's own `typescript` package rather than `rolldown/parseAst`. - No Playwright `integration` project lane: it needs the relay artifact chain. Integration-only specs list nothing for `smoke` and the selection stays empty. - Playwright shards are capped at 10, the same bound as the regular smoke lane. - Rust entries carry the file and in-file module path, and the filterset is anchored to exact `(binary, test)` pairs, so a same-named test in another module is not repeated. - nextest target flags are derived from the changed file's path, so a `buzz-relay` change does not compile every test target of the crate. - A journey-count cap (250) skips the Playwright lane for selections ten repetitions could not finish within its 45-minute timeout. ## Heads-up - The Unit Tests `rust-cache` moves from `key: sherpa-cache-v1` to `shared-key: unit-tests-sherpa-cache-v1` so the flake lane can restore it read-only. This changes the cache key once: the first Unit Tests run after merge builds cold. - This PR itself changes no file the selector recognises (`scripts/flake-check.test.mjs` is a root script, run by the Desktop lane), so its own Flake Check lanes are expected to skip. ## Verification - `node --test scripts/flake-check.test.mjs` — all unit tests pass (also wired into the Desktop lane). - `actionlint` on the changed workflows; `biome check` on the two scripts. - `scripts/test-rust-cache-contract.sh` and its regression harness pass with the new shared key. - Synthetic-diff selection runs for each runner, plus `select` against this PR's own merge-base (empty, as expected). - 2-iteration repeats of a real selection on all four runners locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Tom Brow
|
82b2781376 |
fix(mobile): rebuild buzz_secp256k1 when vendored headers change (#8664)
Follow-up to #8654, from [Kalvin's review note](https://github.com/block/buzz/pull/8654#discussion_r4407146414). `native_toolchain_c`'s `CBuilder` registers only the listed C sources (and files under `includes`) as build-hook dependencies. The `buzz_secp256k1` hook lists four sources and no headers, so after an initial build a header-only vendor update (for example `src/modules/schnorrsig/main_impl.h`) could reuse the old library in incremental app and test builds. The hook now also registers every `.h` file under the vendored tree. It does not add include paths, so compilation is unchanged. ### Validation - With the change, editing the content of `third_party/secp256k1/src/modules/schnorrsig/main_impl.h` makes the next `dart test` rebuild `libbuzz_secp256k1.dylib`. Without it, the library is reused unchanged. - The hook output now lists all 62 vendored headers as dependencies. - Package tests pass (27). buzz-review-completed (Codex and security review clean; agent verification above; build-only change, no app behavior to test) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
713b864e2f |
fix(push-gateway): log which App Attest certificate check failed (#8671)
[#8630](https://github.com/block/buzz/pull/8630) made App Attest rejections log which check failed. In production, an internal iPhone's fresh enrollments now log `reason: certificate` with `detail: invalid certificate path or structure`. That detail still covers about eight distinct failures: a non-array or empty `x5c`, a bad configured root, a malformed or duplicate-extension certificate, any webpki path rejection (untrusted issuer, expiry, unsupported critical extension and so on), and a key-usage mismatch. So the logs still cannot say why this device's attestation is rejected while other devices enroll normally. Each certificate failure now carries a `CertificateCheck`, and the existing warning's `detail` names it: - `x5c is not a non-empty array` - `invalid configured root` - `invalid encoding at x5c[i]` - `invalid key usage at x5c[i]` - `path rejected: <webpki error>`, for example `CertExpired { time: .., not_after: .. }` or `UnknownIssuer` The webpki error includes only its variant and, for validity errors, the timestamps it compared. It includes no certificate bytes or device identifiers. The `reason` stays `certificate`, and the HTTP response stays a uniform `invalid_attestation`. ### Validation - Added tests: a table test runs each signed bad-chain case through the production `verify_chain` and asserts the exact check: wrong usage, expired, unknown critical extension, duplicate extension, trailing bytes, empty chain, and a bad root. Swapping the key-usage result for another check fails it. The shared certificate constraint corpus now builds its chains from the same case list. - The full push gateway suite passes locally. Clippy is clean. The local pre-push `rust-tests` lane failed only in `buzz-relay` tests, which do not depend on this crate, so I skipped it for the push and am relying on CI for it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Luke Tornquist
|
974495fd4a |
fix(mobile): resume reconnect replay from the last live point and spread reconnects (#8659)
## Summary A relay rolling deploy closes every socket with 1012. On reconnect, mobile replayed each live subscription's **original filter** (startup `since`, full `limit`) unless that subscription had received an event, so quiet subscriptions re-read history back to app start every time. Reconnects used a fixed 1s delay with no jitter. The delay also reset to 1s on every successful connect, so a client bounced from pod to pod during a deploy kept coming back at the same moment as every other client. **This PR makes two changes:** 1. **Resume cursor.** If a subscription reached EOSE on a connection, it records `liveUntil`: the time of the last frame received from the relay on that connection, capped at the clock at close. The replay then uses `since = max(original since, lastSeen − 5s, liveUntil − 960s)`, and never goes past the filter's `until`. - **Why 960s:** the relay accepts `created_at` up to ±900s from its clock (`MAX_TIMESTAMP_DRIFT_SECS` in `crates/buzz-relay/src/handlers/ingest.rs`). An event stored after the close can therefore be backdated by that much, and the extra 60s covers client clock skew. - **Only relay frames count as liveness.** Close and error paths never extend `liveUntil`, so a close noticed after the device sleeps doesn't overstate how long the subscription was live. - **Ambiguous EOSE is ignored.** If a subscription has a second REQ on the same socket, that subscription doesn't advance `liveUntil` for the rest of the socket. A CLOSED ends liveness for the rest of that socket, because the retry is a second REQ. Subscription IDs are never reused, so an old EOSE can't mark a re-subscribe as live. - **No cursor:** a subscription that never reached EOSE keeps its previous cursor, or replays as before. 2. **Reconnect spreading.** - After a 1012 or 1001 close, the first attempt waits a uniform random delay in [0, 10s]. A client already backing off waits the longer of that delay and its backoff. - Ordinary backoff gets symmetric ±25% jitter, capped at 30s. - The backoff resets only after a connection stays up for 60s, not on connect. This matches desktop. - Foreground resume and a manual `reconnect()` still reset to the base delay, as before. **Simpler?** The pure reconnect math (cursor, jitter, spread, stability) lives in a new `relay_reconnect_policy.dart` and is table-tested on its own. `relay_session.dart` was near its 1200-line cap, so its five small private bookkeeping classes moved verbatim to `relay_session_subscriptions.dart`. Net concept count is unchanged. **Identified debt (not changed here):** - Dart's WebSocket ping/pong isn't visible to the session, so a long-idle but healthy connection that drops replays back to its last application frame − 960s. This is still bounded, and much cheaper than replaying from app start. - The unread catch-up in `channels_provider.dart` runs one REQ per active channel (`since = readAt + 1`, `limit: 1000`) on reconnect and every 60s. - A retryable CLOSED is retried with a 30s cap, no jitter and no attempt limit. - `lastSeenCreatedAt` advances on stored events received before EOSE. If a relay returns stored events newest-first and the socket dies before EOSE, the next replay can skip older stored events. - The `lastSeen − 5s` term keeps its existing gap for events backdated after a close. ### Related issue N/A. This is part of a set of reconnect-load reductions. Companion PRs: relay read-pool warm (#8651), desktop (#8652), agent harness (#8653). ### Testing - Mobile gate on a Linux workstation, at the head SHA: - `dart format --set-exit-if-changed`: pass. - `flutter analyze`: no issues. - Gateway recipe tests: pass. - `flutter test`: 3881 pass, 1 fail. `push_gateway_origin_validator_test` fails the same way on `main` on that host, due to a host git-config error. - New `relay_reconnect_resume_test.dart`: - pure table tests for the cursor and the delay - a real-socket test that reads the 1012 close code - journeys through `RelaySessionNotifier`. They check that a close or error after an hour asleep doesn't extend liveness, that a second REQ or a delayed EOSE for an unsubscribed ID can't advance the cursor, and that a backdated event inside the 960s window is still replayed. - 22 production mutations, each failing the new tests. They cover every cursor term, the 960s window, the `until` clamp, the liveness cap, the ambiguous-EOSE guard, CLOSED handling, jitter, the 1012 spread and floor, the backoff reset, and close-code capture. - `fake_relay.dart` now draws a fixed jitter midpoint, so existing journey tests stay deterministic. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>mobile-v0.27.0-rc.4 mobile-v0.26.0-rc.5 |
||
|
Tom Brow
|
bad9d26142 |
fix(mobile): push every message in your own Solo chats (#8615)
## Summary Mobile push only covered Solo chats through explicit mentions. Solo chats are private streams rather than DMs, and the push lease subscribed channel-wide only to DMs. Agent replies in your own Solo chats therefore never woke the phone. The lease now subscribes channel-wide to the viewer's own joined, unarchived Solo chats, using the same predicate as the Solo tab, including older unlabelled Solo chats listed in the encrypted Solo record. Self-authored messages stay excluded through the existing lease ignores. Muted Solo chats stay out of the lease, like muted channels, so they push only on mention. The lease waits while the Solo record loads so an unresolved record never drops legacy Solo chats. If that record fails with no cached copy, DM and mention updates proceed without the legacy chats, which the Solo tab cannot list either, and they return when it recovers. Solo chats created by someone else still push only on mention, matching how they appear in the channel list. A lease is bounded by its channel count (750), by the relay's advertised plaintext and content sizes, and by NIP-44's 65535-byte plaintext ceiling. DMs always stay, as before. Solo chats fill the remaining room most recent first, sized with the publisher's own lease encoding, the exact NIP-44 payload length and the largest endpoint the relay accepts. Sizes are computed rather than built, so a relay's advertised limits never drive an allocation. Solo chats beyond the room push only on mention instead of failing the lease update or publication, and Notifications settings says how many. A relay too small even for DMs and mentions fails push setup visibly instead of saving a lease it would reject. Publication accepts only endpoint grants in the gateway's JSON-safe alphabet, so the sizing assumption holds. Thread replies outside DMs still do not push unless they mention you. That is a separate gap. ### Related issue None found. ### Validation - Added test: the desired lease includes joined DMs and the viewer's own Solo chats, and includes legacy Solo chats from the Solo record, and excludes archived and muted Solo chats, other people's Solo chats, and unlabelled private streams. It fails without the fix. - Added test: the sync provider holds the desired lease while the Solo record loads, proceeds without legacy chats if the record fails, and adds them once it arrives. It fails if loading is treated as empty or if a failure blocks updates. - Added tests: over the channel count, and over each size limit (relay plaintext, relay content, NIP-44 plaintext), the lease keeps every DM and exactly the newest Solo chats, reports the rest as omitted, and is accepted by `publishBuzzDevPushLease` with a grant at the relay's endpoint limit, while a max-length grant that JSON would escape is rejected as not a gateway grant. A relay advertising a 2^50-byte endpoint limit is sized without allocating. Each row fails if its guard is removed. - Added test: a relay whose content or plaintext limit cannot fit even the lease without Solo chats rejects that lease at publication, and the provider fails setup instead of saving it. It fails if the guard is removed. - Added test: the NIP-44 payload length matches real encryption at every padding boundary up to 65535 bytes. - Added test: Notifications settings shows "Older Solo chats notify only on mention" with the omitted count. It fails without the row. - Mobile lint and the full mobile test suite pass. - Not exercised end to end on a device. A signed dev build was installed on a test iPhone against a personal dev relay, but the phone's dev identity owns no assistant there, so the device could not create a Solo chat without exporting its key. This will be validated in dogfood. The relay matcher already handles channel-wide `#h` leases for DMs, and Solo messages use the same kind and scoping. buzz-review-completed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
4ff02bf03b |
perf(mobile): sign events with native libsecp256k1 on iOS and Android (#8654)
Mobile signs every Nostr event (sends, relay AUTH, media auth headers, preference writes) in pure Dart on the UI isolate. On an iPhone SE (2nd gen, Profile build) one BIP-340 signature costs about 10.7 ms and deriving the public key another 5.6 ms, so a burst of signed writes drops frames. This adds `mobile/packages/buzz_secp256k1`, a small FFI package that builds vendored bitcoin-core/libsecp256k1 v0.8.0 (ecdh, extrakeys, schnorrsig modules) through a Dart build hook, so the same C and the same Dart API ship on iOS and Android with no platform bridge code. All 13 `nostr.Event.from` signing sites now go through `signNostrEvent`, which keeps the `nostr` package's event-id serialization and moves only key derivation and signing to native code. Calls stay synchronous. The package also exposes `schnorrVerify` and `ecdhX`; follow-up PRs move verification and the NIP-44 conversation key over. Incoming relay events are still not verified, unchanged by this PR. Safety: the signing context is randomized once per isolate, secret keys and keypairs are wiped from the native scratch buffer after each call, aux randomness comes from `Random.secure()`, and the static verify context runs libsecp256k1's self test before first use. ### Validation - Added tests: BIP-340 official vectors and native-vs-pure-Dart differential tests in the package; `event_signing_test.dart` signs with 64 random keys and checks pubkey, event id and signature against the existing `nostr` builder and verifier, plus malformed-key rejection. - Full mobile analyze and test suite pass locally. - Builds: iOS release (framework ~1.3 MB) and Android debug APK with the library for arm64-v8a, armeabi-v7a and x86_64 (~1.2–1.3 MB each). - On-device (iPhone SE 2nd gen, Profile, median per op): sign 10,736 µs → 41 µs, derive pubkey 5,580 µs → 16.5 µs, full signed event 10,744 µs → 65 µs. Event-id JSON and SHA-256 are 23 µs, so they stay in Dart. - Device, iOS (iPhone SE 2nd gen, Profile): `buzz_secp256k1.framework` embedded; the signed-in app authenticated to a real community relay, loaded channels and history, and paged older messages. No native-asset or secp256k1 errors in the console. - Device, Android (emulator, debug): APK contains `libbuzz_secp256k1.so` and loads it. Against a local mock relay that verifies with the pure-Dart `nostr` package, the app's NIP-42 AUTH, NIP-98 `POST /query` headers, and kind 20001/30078 events all verified. - Blossom GET auth, iOS: the dev relay returns 401 for unauthenticated `/media/` GETs. From inside the running app, the production `MediaGetAuthService.headersFor()` header for a nonexistent hash got 404 (accepted, file missing), compared with 401 without it. The community has no relay-hosted image, so none was rendered. buzz-review-completed (agent review, Codex and security review clean; device validation above; human app test not applicable under the mobile testing exception) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: george <973be878edd6d4f79692a3827f20a6f7fe0ed0362f269ff6b6ba8867033d7d8e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Brad Seiler
|
d6196a16c1 | fix(relay): revoke invitations and restrict claims (#8237) | ||
|
Luke Tornquist
|
0a362eef15 |
perf(db,relay): bound thread participant scans and coalesce live summaries (#8635)
## Summary A thread summary's `participants` (up to 10 recent authors, used for the facepile) was computed by aggregating **every** reply in the thread. The live `kind:39005` emit runs that aggregate on the writer after every reply insert and every threaded delete. Each summary therefore cost O(thread), recomputed after every reply, so one very large thread could saturate the writer. ### Scale and impact A single long-running thread can grow to hundreds of thousands of replies and take several new replies per second. Each of those replies then pays a whole-thread aggregate on the writer, so writer CPU follows the reply rate of the largest active threads, and the read-side participant queries (channel window, thread root summaries) slow down the same way. Synthetic reproduction (PostgreSQL 17, the repo's `schema/schema.sql`, default `shared_buffers`): one channel with 200k background messages and three threads of 450k, 6k and 300 replies from a few authors with ~800-byte bodies. `EXPLAIN (ANALYZE, BUFFERS)` of the participant query, warm cache (runs 2–3 of 3): | Thread | Before (whole-thread aggregate) | After (newest 1000 replies) | |---|---|---| | 450k replies | ~7.6 s, ~117k buffers read, ~16k temp blocks spilled | ~6.8 ms, ~4k buffers | | 6k replies | ~25 ms, ~24k buffers | ~7 ms, ~4k buffers | | 300 replies | ~2.6 ms, ~1.2k buffers | ~2.6 ms, ~1.2k buffers | Cost now stops growing at 1000 replies, and small threads cost the same as before. With coalescing, a busy thread runs this query at most about once per second per pod instead of once per reply. ### Change 1. **Bound the participant scan to the newest 1000 replies.** All three participant readers now share one SQL fragment (`recent_thread_participants_sql!`): - `get_thread_summary` (the live 39005 emit) - the channel-window batch - `resolve_thread_root_summaries` The fragment walks `idx_thread_metadata_window` newest-first with `LIMIT 1000`, then joins `events` only for those rows, to get `pubkey` and drop deleted replies. Deleted replies keep their `thread_metadata` row, so they count toward the 1000 but contribute no author. It aggregates by author and returns the 10 most recently active. Cost no longer grows with thread size. Desktop and mobile show the 3 most recent participants and the forum view shows up to 10, so an author who hasn't posted in the last 1000 replies is not visible in any client today. `NIP-CW`, `docs/bridge-channel-window.md` and the `ThreadSummary::participants` doc now state the bound, deleted replies included. 2. **Coalesce live summary emits per thread root.** `emit_live_thread_summary` goes through a new `CoalescedRefresh` keyed by `(community, root)`: - A quiet thread emits immediately, as before. - On a busy thread, emits are spaced at least 1s apart per pod. A trailing emit covers the rest of the burst and reads state committed after its last mutation. - Because emits are at least 1s apart, consecutive overlays from one pod always get strictly increasing `created_at`, so the client's newest-wins replacement is never tied. - A panicking or cancelled refresh releases its key, including a task dropped before its first poll: the release guard is built before the spawn. Each run is bounded by a 30s timeout so a hung refresh can't wedge it. ### What got simpler - Three hand-written copies of the participant aggregate are now one fragment. - `get_thread_summary` is one statement instead of two round trips. - The channel-window batch dropped its `ROW_NUMBER()` window function and the `GROUP BY root, pubkey` over every reply of every root on the page. ### Related issue None found. ### Testing - `participant_summaries_read_only_the_newest_replies` (buzz-db, Postgres) seeds a thread with 1001 replies: - an early author, pushed just outside the window - a filler author - a deleted reply - a latest author All three readers must return `[latest, filler]`. Mutant check: removing the `LIMIT` (whole-thread aggregate) turns it red. - `coalesced_refresh` unit tests (paused tokio clock): - A 100-request burst runs once immediately and once trailing, then frees the key. - Separate keys run independently. - A panicking refresh doesn't wedge its key. - A hung refresh times out and releases its key. Mutant check: removing the timeout turns it red. - A request that arrives while a run is in flight gets one trailing run, and that run reads the state written mid-run. Mutant check: dropping the rerun flag turns it red. - A task dropped before its first poll (runtime shut down) still releases its key. Mutant check: building the guard inside the task turns it red. These run in `just test-unit` (added to the buzz-relay `--lib` filter and `scripts/run-tests.sh`). - Blox gate: `cargo fmt --check`, clippy `-D warnings` (buzz-db, buzz-relay), and the full buzz-db and buzz-relay Postgres nextest lanes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
b310fea05b |
mobile: find DMs opened ahead of the list everywhere (#8646)
## Summary This follows up [#8638](https://github.com/block/buzz/pull/8638) and covers the optional review notes from that PR. - **Shared lookup.** A new `channelByIdProvider` returns the channel from the list, or the copy remembered from a fast DM open when the list doesn't have it yet. The mention picker, mention search failure state, compose bar DM check, thread page, and message unread action all use it. Before this change, those surfaces only searched the list, so a DM opened before the list loaded was treated as an unknown channel. For example, the mention picker skipped directory search and never labeled anyone "not in DM". - **Eviction and scope.** Opened-ahead DMs live in a notifier scoped to one relay and identity, and `channelByIdProvider` watches it. Once the list includes a remembered DM, its opened copy is dropped. Remembering also skips a DM the list already has, since the open's background refresh can list it first. A DM hidden or left later, or one from another community or identity, is no longer found through a stale copy. - **Call-site tests.** Search, the profile Message button, and the New message sheet now assert that they open the DM with `waitForChannelList: false`. ## Validation - `just mobile-check` and `just mobile-test` pass. - The journey test "a DM opened before the channel list loads sends mentions" now also checks the outside-person picker in a DM that isn't in the list yet: a failed directory search shows Retry, then the result appears labeled "not in DM". Mutation checks: reverting the lookup in `mentionCandidatesProvider`, `mentionSearchFailedProvider`, or the compose bar fails the test. - New tests "a DM opened ahead of the list is not found after it is left" (including the refresh-before-remember order) and "a DM opened ahead of the list stays in its community" (a mounted lookup across community and identity switches). Mutation checks: removing the eviction, the listed check in `rememberOpened`, the relay or identity watch, or the lookup's reactive watch each fails a test with `Expected: null, Actual: <Instance of 'Channel'>`. - Call-site assertions. Mutation check: flipping any of the three call sites to `waitForChannelList: true` fails its test with `Expected: [false], Actual: [true]`. - Simulator (iPhone 17 Pro, local relay): I started a fresh DM with Sim Tester from the New message sheet. It opened at once, and typing `@Pat` showed Pat Peer from the directory, labeled "not in DM". <img src="https://github.com/user-attachments/assets/0fe68178-00b5-4533-99f8-fe5d03e8ed88" width="300" alt="Fresh DM with Sim Tester: typing @Pat shows Pat Peer labeled not in DM"> buzz-review-completed 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: george <973be878edd6d4f79692a3827f20a6f7fe0ed0362f269ff6b6ba8867033d7d8e@sprout-oss.stage.blox.sqprod.co> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
ravarora2
|
2c28e07133 |
BUZZ-413: Add user deletion fence storage (#8632)
## Summary Add storage and lock helpers for the first user-deletion fencing PR. No serving handler or deletion worker calls these helpers yet, so this does not block users or enable real deletion. The operator-global `user_deletion_fences` table stores global user restrictions and community-scoped agent restrictions. Community and profile deletion retain these rows unchanged. `scope_community_id` records a restriction's scope; it does not make the row community-owned. The helpers use the caller's transaction, stable identity lock keys, ordered shared/exclusive locks, and indexed lookups. Inserts preserve the original request and owner on retry and reject conflicting provenance. Inserts validate the current worker lease and approved inventory through the existing entity-deletion guards. Callers must still verify that the target identities and agent ownership belong to the request. This is an additive schema expansion with a five-second migration lock timeout. It adds no database triggers, functions, foreign keys, or backfills. Both schema sources and operator-global registrations are updated. The unused request-ID index was removed. The existing community-deletion catalog and purge logic are unchanged. Tests cover live schema parity, concurrency, rollback, and retention through the existing purge and verification functions. ### Related issue Part of [BUZZ-413](https://linear.app/squareup/issue/BUZZ-413/gate-user-and-owned-agent-activity). This is the storage foundation; serving checks and real workflow handlers follow separately. No duplicate fence-storage PR found. ### Testing No manual app testing: these utilities have no application callers yet. Local PostgreSQL validation uses an isolated test container and the repository's per-test database runner. Passed 84 selected PostgreSQL tests covering fence authorization, concurrent inserts, schema parity, retention, community deletion, entity-deletion storage, and user discovery. The authorization regression also failed as expected when the insertion guard was temporarily removed; the guard was restored before the final run. Command: `scripts/postgres-test-run.sh -p buzz-db --lib -E 'test(/user_deletion_fences/) | test(/admin_schema_parity_between/) | test(/^store::deletion::postgres_tests::/) | test(/^store::entity_deletion::postgres_tests::/) | test(/^store::user_deletion::/)'` with the local test PostgreSQL connection and `PG_BIN_DIR` configured. The full `just test` run had one failure: `composition_tests::composition_log_does_not_leak_issuer_url` missed an expected log message. The same compiled test passed when rerun alone. No relay logging code is changed by this PR. The full local `just ci` run and all normal pre-push hooks passed on the updated branch. Generated with Codex --------- Signed-off-by: Ravneet Arora <rarora@squareup.com> |
||
|
Tom Brow
|
9e18d17a36 |
fix(mobile): open new DMs without waiting for a full channel reload (#8638)
## Summary
Starting a DM from the mobile **New message** sheet could look broken:
after picking a person and pressing Done, the sheet kept spinning, so it
seemed like the picker could only add people. The DM was being created,
but `openDm` reloaded the whole channel list (memberships, metadata,
hidden DMs, huddles, names, latest messages) before returning. On a
large account or slow connection that can take many seconds.
Interactive opens (New message sheet, Search, profile sheet) now open
the DM from its own metadata and roster, the same targeted read Solo
chat creation uses, and refresh the channel list in the background. The
sidebar still picks up the DM once that refresh lands. When the open
reopens a hidden DM, the fast path also reads the caller's visibility
snapshot in the same round trip and fails visibly if the relay accepted
the open but didn't unhide it, so Done can retry instead of opening a DM
the sidebar keeps hidden. Search and the profile sheet now push the DM
as an ordinary route, as New message already did, so it doesn't sit on a
spinner while Solo groups load. Hidden-DM resurfacing from Activity
keeps the old behavior (wait for the full list), because that reload is
what clears its hidden state and its retry coalescing depends on it.
The fast open also records the DM it read on the channel list notifier,
which falls back to it until the list catches up. Both composers
(channel and thread) look the DM up through it, so a selected mention in
a just-opened DM doesn't wait on, or fail against, the list, and thread
replies there keep their DM recipients.
`findJoinedChannel` now also carries the DM's participant pubkeys so the
opened DM gets its proper title ("Pat Peer", not "DM") before the list
catches up. It also queries metadata and roster separately instead of
under one shared limit, so a lingering pre-backfill metadata revision on
an older DM can't crowd out the roster.
### Related issue
N/A (reported in Buzz #buzz-mobile).
### Validation
- Added tests: `dm_open_latency_test.dart`, a regression test with the
full channel-list load stalled. The interactive open finishes within 2s
with the right channel id, DM flag, membership, and participants. It
publishes exactly one kind 41010, then the background list refresh runs
once the stall clears. The fake serves events like a relay (newest
first, one limit across kinds) and keeps a stale metadata revision
around; the shared-limit read fails it. The test also covers: a DM the
caller can't read fails visibly, a reopen whose unhide snapshot failed
fails visibly, the default (resurfacing) path still waits for the list,
and a community switch mid-read cancels the open.
- Added test (compose bar journey): with the channel list still loading,
a fast-opened DM sends a selected participant mention as a recipient
instead of failing "could not check who is in this channel". It fails if
the composer ignores the opened DM.
- Added test (thread): a thread reply in a DM opened ahead of the list
sends with the DM channel, so it keeps DM recipient semantics. It fails
if the thread page ignores the opened DM. The latency test also checks
the open records the DM before the list has it.
- Mutation checks: the test fails if the background refresh is removed,
if the default path stops waiting, if the interactive path waits, if
participant pubkeys are dropped, or if the still-hidden check is
removed.
- Added tests (Search and profile journeys): with Solo groups still
loading, tapping a person result or the profile's Message button opens
the DM with its composer, not a placement spinner. Each fails if its
call site drops the ordinary placement.
- Full mobile suite: `just mobile-check` clean, 3847 tests passed (23
skipped), plus the unconfigured-push suite.
- iOS simulator (iPhone 17 Pro) against a local relay: New message, pick
Pat Peer, Done. The sheet closed and the DM opened titled "Pat Peer" in
about 1s. Reopening the existing DM took about 0.8s. The DM appeared in
the sidebar after the background refresh. With 400 ms of added one-way
latency (800 ms RTT) through the local proxy, the same flow opened the
DM in 2.5s.
- Re-ran on the simulator at the final head (
mobile-v0.27.0-rc.3
mobile-v0.26.0-rc.4
|
||
|
Logan Johnson
|
646ef8abf9 |
Stop requiring the 0032 roster trigger at relay startup (#8611)
🤖 ## Summary When a relay starts, it checks that a database trigger from migration 0032 is installed and works. If the trigger is missing, the relay refuses to start. This PR removes that startup check, so a later migration can drop the trigger. The trigger blocks a stale channel member list from being saved. Since #8316, the relay runs the same check in Rust before every save of a member list. The trigger is now only a duplicate backstop. Removing it lets #8494 compute the `member`/`bot` label when it reads members, instead of storing `bot` in the database. This PR does not drop the trigger. The trigger stays installed and keeps working. Only the requirement that it exists at startup is removed. ### Related issue Prepares the trigger retirement that #8494 needs (review thread on #8494 about storing `bot`). No duplicate PRs found. ### Rollout `docs/database.md` requires this sequence for retiring a trigger: 1. Ship the replacement in the app. This was #8316. 2. Fully deploy a release in which no relay requires the trigger. **This PR.** 3. Drop the trigger in a separate migration, and name the step-2 release as a required stop in its release notes. A relay from before this PR refuses to start after the trigger is dropped. So every environment, staging included, must run this release before the drop ships. ## Details - Removed the startup step `channel_roster_fence`, which ran the catalog check and a behavior probe (a test insert that must be rejected, then rolled back). - Removed the catalog check that ran after migrations in `run_migrations_locked`. - The catalog check function stays, but only tests use it. - Removed four tests about the startup gate. Added `migrations_run_without_the_roster_trigger`: it drops the trigger in a scratch database and runs the migrations. If the post-migration check comes back, the test fails. - `observability_source` lists the event writers that may skip the guard. The probe was one of them, so it is removed from those lists, and the expected writer count goes from 9 to 8. - The Helm chart README now says that newer relays do not require the trigger. ### Testing - `cargo clippy -p buzz-db -p buzz-relay --all-targets -D warnings` and `cargo fmt --check` pass. - `observability_source` (21) and `startup_steps` (6) pass. - `migrations_run_without_the_roster_trigger` passes. With the post-migration catalog check restored, it fails with "channel roster fence trigger missing". 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Luke Tornquist
|
78b5dae1f6 |
fix(relay): store a kind:9000 and its membership writes in one transaction (#8606)
Follow-up to #8575. ## Problem Today a kind:9000 is stored first and the membership write runs afterwards, on its own. If that write fails, the relay logs it and still returns an accepted OK. A resend of the same event then returns `duplicate:`, so the person is never added. This is the catch-log-and-accept failure that Review-Proven Rules #1 and #5 describe. ## Change kind:9000 now uses the same atomic path as kind:9002 (`persisted_atomically` in ingest). - **`buzz-db`:** new `put_members_with_event`, which runs one transaction: 1. Take the channel's membership lock, before the event insert, the TTL lock and the channel row. This matches the order the other membership writers use, so it adds no new lock cycle. 2. Store the event. If it's already stored, roll back and return `Duplicate`. 3. Run the existing `put_members` decisions and upsert. These now live in a shared `put_members_on` step on the transaction. 4. Commit. - **Failure:** nothing is stored and the client gets an `error:` rejection, so it can resend the same event and have it applied. - **Everyone refused:** if the decisions under the lock refuse every person, nothing is stored and the event is rejected with the same text validation gives. Before, a single-person add refused at that point was accepted with an empty message. - **After commit:** the live effects stay as they were. They're best-effort and read the committed state: cache invalidation, `member_joined` notices, member-added notifications and 39002 discovery. - **`NOSTR.md`:** now says a failed kind:9000 can be resent as is. ## What got simpler - kind:9000 no longer has its own store-then-apply path in ingest. It shares the atomic branch with 9002 and workflow deletion. - `apply_put_user`'s "write failed, report it as a per-person rejection inside an accepted OK" branch is gone. - The `Db::put_members` wrapper is deleted. Its only caller was the kind:9000 path. ## Identified debt (not in this PR) kind:9001 (remove), 9021 (join) and 9022 (leave) have the same gap. Each stores the event, then runs the membership write in `handle_side_effects`, and a failure is only logged. kind:9008 and 9005 follow the same store-then-apply pattern. These can move onto the same atomic shape in a follow-up. ## Tests - `failed_put_user_stores_nothing` (relay, Postgres): a test trigger fails the second person's row in a two-person add, after the event row and the first person were already written in the same transaction. It checks: - Both the send and a resend are rejected with `error:`. - No kind:9000 is stored, nobody is added, and no `member_joined` notice is posted. - With the trigger dropped, the same signed event adds both people. - One more resend returns `duplicate:` and posts no second notice. - `put_members_with_event_stores_the_event_with_its_writes` (buzz-db): checks the three outcomes: - Refused: nothing is stored. - Applied: the event is stored and the person is written. - Duplicate: the roster is left alone. - `put_user_refused_under_the_lock_stores_nothing` (relay, Postgres): a test trigger removes the sender from a private channel inside the write transaction, so validation passes and the decision under the lock refuses. For a single add and a batch, it checks the `invalid:` rejection, that nothing is stored and no notice is posted, and that the same event applies once the trigger is gone. - Mutation checks, run in Blox gates: - At `d18e88fd`: committing the refused event fails the buzz-db test; applying a duplicate fails the first two tests; storing the event in its own transaction fails the first two tests. - At `3b106623`: turning the refusal under the lock into `error:` fails the new refusal test, and so does committing the refused event. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Matt Toohey
|
3ed07e5c75 |
fix(deps): bump pulldown-cmark to 0.13.5 for RUSTSEC-2026-0414 (#8644)
## Summary Bumps `pulldown-cmark` 0.13.4 → 0.13.5 in `Cargo.lock` to clear [RUSTSEC-2026-0414](https://rustsec.org/advisories/RUSTSEC-2026-0414), which `cargo-deny` started flagging this morning. The advisory landed in the RustSec database at 08:50 UTC on 2026-10-28; main's last Security run passed at 00:24 UTC, and every PR touching a Rust path since (e.g. #8613, `jitter/block-staging-relay-domain`) fails the required **Security** check on it. Main will go red on its next Rust-path push until this merges. ## Exposure Not exploitable in Buzz: the advisory requires `Options::ENABLE_WIKILINKS`, which is off by default. The relay's only use is `render_policy_document` in `crates/buzz-relay/src/api/invites.rs`, which parses operator-authored policy Markdown via `Parser::new` with default options. This is a dependency-policy fix. ## Change - `cargo update -p pulldown-cmark --precise 0.13.5` under the pinned 1.98.1 toolchain; `crates/buzz-relay/Cargo.toml` already requires `0.13.4` (`^0.13.4`), so no manifest edit. - The same re-resolution moved `data-encoding-macro-internal`'s `syn` edge from 2.0.117 to the 1.0.109 already in the lock (declared requirement `>=1, <3`). No crate is added or removed. - `desktop/src-tauri/Cargo.lock` does not contain `pulldown-cmark`, and `cargo-deny` only checks the root workspace. Not touched here: `deny.toml:50`'s `RUSTSEC-2026-0381` ignore now warns `advisory-not-detected` (the crate is gone). Harmless; can be removed separately. ## Verification - `cargo update --dry-run` showed `pulldown-cmark` as the only version change before applying. - CI's Security lane on this PR is the real check. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Will Pfleger
|
24a3706060 |
feat(relay): log who and why for NIP-FI shadow would-deny (#8634)
In shadow mode, the relay's `nip-fi shadow would-deny` line now says which client enforce would refuse and why, not just the coarse public outcome. - The line moves from `debug` to `info`. Admits still write no line. - It gains `reason`, one value from a fixed snake_case set naming the step's private condition: `community_unresolved` (community resolution failed or was unavailable), `community_stale`, `session_deadline`, `duplicate_authorization`, `missing_proof`, `invalid_proof`, `missing_assertion`, `malformed_assertion`, `verifier_unavailable`, the verifier's own error code without its `nip_fi_` prefix (for example `expired`, `unknown_issuer`, `invalid_signature_or_claims`, `token_type_rejected`), `key_mismatch`, `no_asserted_key`, `deny_set`, `owner_banned`, `not_stored_owner`, `owner_ban_unavailable`, `owner_link_unavailable` and `tenant_unavailable`. - It gains `pubkey`, the hex Nostr key of the would-be-refused client, or `absent`. On HTTP it is the verified NIP-98 proof key, for every step including community and cardinality. On WebSocket and audio it is the key NIP-42 AUTH proved, or for deny-set and deadline the key a verified token asserts. A router-guard or upgrade would-deny precedes any proof, so its line names the key the client later proves: the request's verified NIP-98 key, or the connection's first verified NIP-42 AUTH key. Each such line is written exactly once. On HTTP it is written when the handler verifies its NIP-98 proof, or as `absent` when the request ends without one. On WebSocket and audio it is written at the first verified NIP-42 AUTH, while the connection stays open, or as `absent` when an unproven connection's read loop ends (close or auth timeout), before any cleanup that can block. Its counter still increments at once. An unsigned `X-Pubkey` never counts. - `buzz_nip_fi_shadow_total` keeps exactly its `route`, `stage`, `outcome` and `community` labels, so cardinality stays bounded. No token, assertion, issuer, subject or other claim reaches the line; verifier reasons are static strings. The denial classes and response bodies clients see are unchanged. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
5c2ea2e507 |
feat(mobile): add full-screen profile setup with inline avatar editing (#8628)
Replace the iOS profile setup sheet with a full-screen name → avatar flow. Require a display name, enable avatar Continue after Done, and let Skip save the name without an avatar. After a failed save, Not now continues without saving the draft. Embed photo, emoji, and animated avatar editing with native Back/Done, matching 220-point previews, a rounded-notch pencil edit badge, an entry haptic, equal emoji-preview spacing, and 200 ms transitions that move the avatar into its editor position without resizing. Apply appearance changes live across setup and editing while preserving the draft. Keep the camera exit rail stationary, use a solid shutter, and present photo positioning full-screen with Back/Done. Preserve the main Flutter view during presentation and hide underlying controls from VoiceOver while editing. Validation: Flutter analysis, 3,796 Flutter tests, and 106 native tests passed. Built, installed, and launched the isolated iPhone preview; inspected the native emoji/badge rendering. The local preview entry point is excluded. Draft pending real login/join → profile-save verification and owner verification of the updated device flow. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Wes
|
5a8102ee5b |
fix(mobile): preserve invite onboarding and offer identity setup (#8617)
## Summary Cold-start iOS invite links could open the welcome screen without continuing sign-in, and accounts without a stored backup had no explicit identity setup choice. Forward scene launch URLs to `app_links`, offer creation for an unbound account with no backups, and offer matching-key import for an existing binding without a backup. Existing key storage, binding, backup, and admission checks remain in use. Discover saved hosted destinations using `known-communities/list` and remember successfully admitted destinations with `known-communities/add`. Saved destinations are not membership proof; normal admission and the existing failed-claim membership reconciliation remain separate. A failed save preserves the intent and local community for idempotent retry. ### Related issue Related to #8599 (QR-first mobile onboarding) and #8601 (invite policy consent; overlapping setup UI). No exact duplicate found in the open onboarding PR / identity-backup issue search. ### Validation before the latest main integration - 48 tests passed across five focused files: `hosted_journeys_test`, `hosted_communities_test`, `hosted_communities_flow_test`, `hosted_environment_flow_test`, and `hosted_multi_backup_recovery_test`. Test defines enable hosted domains and the push gateway. - Four additional pairing handoff tests passed with hosted domains enabled after updating their shared HTTP fixture for community saving. - Targeted Dart analysis: no issues. Required pre-commit and pre-push hooks passed, including the broader mobile analysis/test lane and file-size checks. - Native iOS build passed during iteration. On an isolated iOS simulator, a terminated app opened from an invite automatically entered browser login; human recording confirmed login continued into identity setup and new identity creation. - Human staging testing on October 27 passed invite entry with an existing email/password and new identity, community entry, messaging, cold restart, and recovery on two separate clean iOS simulators. These results apply to build `ff2138fc325790e0af4d9a64b2ed033fae3a8295`; the backend backup-upload blocker described previously is no longer blocking that tested flow. - Fresh-account signup through an invite is deferred. Physical-camera QR scanning, native regression/mutation coverage, independent review, and native retesting after the latest main integration remain outstanding. - Known limitation: failed-claim membership reconciliation still calls `communities/joined`; its endpoint/response contract needs alignment with the supported backend before that fallback is considered ready. The successful primary recovery path does not prove this fallback. - No account reset tooling, environment-specific credentials, generated dependency files, or recording artifacts are included. ### Current review fixes and validation Head `9902a98d42904a4a7d44a79541d81ceb94d8eac3` addresses the two review blockers and both small import races: - Hosted imports explain a linked-account key mismatch; the shared self-hosted/occupied-device message remains unchanged. The linked-account journey enters the wrong key through the actual sheet, checks the visible error and empty storage, then imports the correct key. - Closing the import sheet while its operation is pending now waits for that operation and resumes successful setup. The new-account journey covers creation and import, including dismissal while awaiting the status response, binding, backup receipt, and retry. - An unbound account with newly appeared backups refuses direct import, preserving recovery's receipt-before-adoption rule. The existing multiple-backup invariant covers matching and different imported keys. - The configured route journey uses host labels and its existing fake account now supports destination saving; the real-account journeys separately exercise persistence. 53 tests passed across six focused files with CI's `BUZZ_AUTH_DOMAINS=buzz.xyz,builderlab.xyz,builderlabstaging.xyz` and push-gateway define. In a separate clone with independent Git metadata/build output, restoring the old production import paths failed the three intended regressions; restoring the fixes passed all 43 tests in that control. Required pre-commit and pre-push hooks passed, including mobile analysis/tests and file-size checks. Hosted CI and native retesting of this head are still pending. The earlier independent review was at `10815e5d9`; these fixes still need reviewer confirmation. Optional accessibility and import-navigation refinements are deferred. Fresh signup, authenticated production add/list and the membership fallback remain open as described above. ### Staging testing and production configuration Staging selection is build configuration, not a changed production default. `HostedEnvironment` is unchanged by this PR: `BUZZ_BUILDERLAB_URL` defaults to `https://app.builderlab.xyz`; `BUZZ_AUTH_DOMAINS` is empty by default (hosted account features disabled). Builds that enable production hosted domains receive these onboarding behavior fixes after merge. This PR does not deploy anything or redirect production builds to staging. Check out this PR (`gh pr checkout 8617` in a dedicated `block/buzz` checkout/worktree), install dependencies per `mobile/README.md`, and boot a clean iOS simulator. With the repository-pinned Flutter toolchain: ```sh . ./bin/activate-hermit ./scripts/mobile-worktree-overrides.sh cd mobile flutter run -d <clean-simulator-udid> \ --dart-define=BUZZ_BUILDERLAB_URL=https://app.builderlabstaging.xyz \ --dart-define=BUZZ_AUTH_DOMAINS=communities.public.builderlabstaging.xyz \ --dart-define=BUZZ_DEBUG_PRIVATE_LOGIN=true ``` Private login requests an ephemeral browser session only in debug builds. Hosted relay assertions remain off by default; this test does not validate assertion-enabled admission. Generated local Xcode settings, dependency metadata, credentials, and simulator state are not included in the PR. - Invite test: open a valid public-staging invite in the simulator browser, choose **Accept invite in Buzz**, sign in or sign up, complete identity setup, enter the community, and send a message. - Recovery test: preserve that simulator, start another clean simulator with the same build, choose **or sign in with email**, and sign into the same account. Verify the recovered public key matches, the saved community and earlier message return, and a new message sends. - Fresh signup: repeat invite entry using a new account. This still needs human verification. Keep this PR draft while the listed validation gaps remain. --------- Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz> |
||
|
ngoodger-block
|
f33ebaafa1 |
refactor(schema): stop auto-fencing new tables in desired-state bootstrap (#8620)
### Why?
This is a cleanup that brings the schema bootstrap in line with how the
codebase works today, as asked for by
[`docs/database.md`](
|
||
|
Tom Brow
|
d4a07063c8 |
feat(mobile): manage Builderlab remote agents (#8626)
Adds Builderlab remote agents to mobile: a member signed in to Builderlab can list, create and delete their own hosted agents from the channels header, and active remote agents show as Online. Everything is gated by `HostedEnvironment.enabled` (the build-time `BUZZ_AUTH_DOMAINS` define). App Store and internal release builds do not set it, so they render and fetch nothing new. ### Behavior - **Agents** button in the channels header (native iOS bar and Material header) for hosted communities only. It opens a sheet listing the community's agents; tapping one pushes its profile inside the same sheet. - **Create**: for the member's own agents (the NIP-OA owner is the member, not the community). Registers the agent with Beekeeper, attests it to the community, saves optional private instructions, subscribes it to the relay, then publishes the kind 30177 registration with a NIP-OA owner proof. A failed step resumes with the same agent identity on retry; instructions stay in secure storage, never in preferences. - **Delete**: removes the relay registration, confirms it is gone, then deletes the Beekeeper agent. A partial failure keeps a retry row in the list. - **Presence**: an ACTIVE agent with a live kind 30177 registration on the active relay shows Online in profiles, channel rows, DMs and mentions, since remote agents wake on messages. Local presence data is unchanged. - Sign-in reuses the shared hosted flow. The account's saved Buzz key must match the server binding; nothing is adopted or rewritten. - `BuilderlabAccountClient` gains an allowlist for the six Beekeeper routes desktop already uses. Not included from the prototype: the separate entrypoint and second sign-in, test-community creation, staging-only hosts, QR-code agent links, and the USB proxy. ### Validation - Added tests: build-time gate, account/key binding before listing, setup resume with private instructions, deletion contract and retry, owner-proof verification, status parsing, presented presence requiring a live registration in the active community and following registration publish and removal (mutation-checked), account-change clearing, load retry, recovery of a pending setup whose instructions were lost, a lost registration response that can only resume with the same idempotency key (from any community), an HTTP-rejected registration that can be discarded, deletion of other agents while a setup is unfinished, rejection of owner proofs whose signed conditions or profile signature fail, registration publish and removal over a loopback WebSocket relay for an `http://`-saved community, and widget tests for in-sheet detail navigation, creation overlay with failure then resume, and delete confirmation. - The new tests also run in the CI lane with `BUZZ_AUTH_DOMAINS` configured, so the enabled path is covered. - Full mobile format, analyze and test suite pass locally. - Independent agent review: no gate leak or key-handling issue found; its robustness findings (discarding a setup that never got a server identity, clearing registry state on account change, retrying a skipped load, guarded notifications) are fixed; discard and account change have added tests. - Simulator: a `BUZZ_AUTH_DOMAINS` debug build launches to Builderlab sign-in. The signed-in flow was not re-run on this branch. - Codex Security Review: an earlier head received a published review, and its two findings were fixed. Onmobile-v0.27.0-rc.2 mobile-v0.26.0-rc.3 |
||
|
Tom Brow
|
b929b2cf84 |
fix(mobile): replace a push enrollment the gateway rejected (#8629)
An iPhone could stop receiving push permanently after the gateway rejected its enrollment, even after a reinstall. Before sending an enrollment, iOS saves the exact signed request in the Keychain and replays those bytes until the gateway answers, so a lost response cannot orphan a committed installation. The saved request was replaced only on `404`. When the gateway rejected it with `401 invalid_attestation` (or `400 invalid_request`), each later attempt replayed the same rejected bytes for the saved request's whole lifetime (up to 30 days). The Keychain survives reinstall, so reinstalling did not help. We saw this on an internal device: in production, its gateway requests failed with repeated `401`s and it never got an installation. The gateway returns `401` before it looks up a committed installation, so replaying the same bytes can never recover one. iOS now replaces a saved request rejected this way with a fresh challenge and App Attest key, once the request is old enough. The first replacement waits an hour, and the wait doubles with each further rejection. Every replacement creates and attests a new key, so without the wait a device the gateway keeps rejecting would create a key on every retry. With it, such a device creates about ten keys over the saved request's 30-day lifetime, yet still recovers on its own once the cause is fixed. Requests saved by earlier builds carry no preparation time and are replaced right away, which recovers devices that are already stuck. Each replacement attempt is saved before any key is generated, and the rejected request stays saved until the fresh request overwrites it. A failed challenge fetch, a failed attestation or an app restart therefore waits instead of creating another key. If the fresh request also fails, the error still propagates and that request is saved for the next attempt. A `404` (the challenge is gone and nothing committed) still replaces right away, as before, so ordinary lost requests recover at once. A `409` or `404` answer to a request marks it as rejected. A conflict with a live installation for the same token consumes the challenge, and a fresh request answered `404` has outlived its own, so either replay then returns `404`. That `404` waits like a rejection, so neither case can create a key on every retry. This covers a request that did commit, whose response was lost and whose replay arrived after its attestation certificate expired. Enrollment then succeeds once the old installation expires. Recovering that installation itself would need the gateway to match an exact replay before checking certificate validity. That is out of scope here. `400` keeps the saved request, as before: it can be a temporary policy rejection (for example a lowered lifetime limit) of a request that did commit. Transient failures (for example `5xx` or network loss) also keep it. Companion gateway PR [#8630](https://github.com/block/buzz/pull/8630) logs which App Attest check rejected a device, so a `401` like this can be diagnosed from gateway logs. ### Validation - Added tests: one table-driven driver test. It loses the first response, then has the replay answered with `400`, `401`, `404` or `503`, at various ages, and the fresh request answered with `201`, `401`, `404` or `409`, or lost. Each row retries twice and checks the saved request and the number of App Attest keys generated. Rows that keep being rejected then check that the wait doubles, and that a replacement whose attestation fails after generating its key records the attempt, so the next retry waits even with a new driver instance. An ordinary `404` replacement whose attestation fails is checked the same way. With the source change reverted, the `401` rows fail. Removing any of these guards also fails the test: the age limit, the doubling, the immediate replacement for older builds' requests, saving each attempt, the `404` and `409` marks, keeping `404` out of the count, or keeping `400`. - The full BuzzPushKit suite passes locally (97 XCTest and 65 Swift Testing tests). - Not exercised on a device. The replay path needs a real device whose saved enrollment the gateway rejects, and the simulator does not support App Attest. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Logan Johnson
|
403855d788 |
test(relay): give group-state Postgres tests a normal acquire timeout (#8564)
🤖 ## Summary The relay's PostgreSQL CI job sometimes fails in the channel group-state tests with `ensure community: Sqlx(PoolTimedOut)`. The tests themselves are fine. They connect to the test database through a test helper that waits only 100 ms for a connection, and on a busy CI runner the first connection can take longer than that. One example is `group_state_events_carry_type_then_creator` in [this #8522 run](https://github.com/block/buzz/actions/runs/40013074144/job/120623293420): it failed after 0.57 s, while its other 1110 tests passed. This PR gives these tests a pool with sqlx's normal acquire timeout. No product code changes. ## Details - `test_state_with_database_url` keeps its 100 ms timeout on purpose. Its other callers point at a closed port (`127.0.0.1:1`) to test fail-closed paths, and the short timeout makes them fail fast. - `group_state_postgres_tests::state()` is the only caller that passes it the real test database. It now builds a normal lazy pool and passes it to `test_state_with_database_pool`. The same `state()` also serves `create_with_members_postgres_tests`, `edit_metadata_postgres_tests`, and `system_channel_postgres_tests`. ### Related issue N/A. I searched open PRs for `PoolTimedOut` and `acquire_timeout` and found none for this. ### Testing To reproduce a slow first connection, I ran the tests through a local TCP proxy that waits 250 ms before it connects to Postgres: - Before this change: `group_state_postgres_tests` had 13 of 14 tests fail with `PoolTimedOut`. - After this change: all 14 pass. `create_with_members_postgres_tests` (3), `edit_metadata_postgres_tests` (7), and `system_channel_postgres_tests` (9) also pass through the same proxy. - Without the proxy, all 14 pass before and after. - `cargo clippy -p buzz-relay --all-targets -D warnings` and `just test-unit` pass. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Tom Brow
|
c477663211 |
fix(push-gateway): log which App Attest check rejected a device (#8630)
When the push gateway rejected a device's App Attest attestation or assertion, it logged the reason only at debug level. In production that left a bare `401 invalid_attestation` in the access logs, with no sign of which check failed. We hit this diagnosing an internal iPhone that never received push. Its enrollments failed with repeated `401`s, and the gateway logs could not say why. Rejections now log at warn level with: - `operation`: `attestation` or `assertion` - `reason`: the existing category, for example `certificate` or `identity` - `detail`: the specific failed check, for example `invalid AAGUID` or `invalid App ID` Malformed base64 now has its own `encoding` reason. The size, base64 and challenge checks that run before full verification now take the same path. Previously they rejected without logging anything. Without `RUST_LOG`, the gateway now defaults to `error,buzz_push_gateway=warn`, which keeps every target's errors plus the gateway's own warnings. Previously the default showed only errors, and the Helm chart sets no `RUST_LOG`, so a default deployment would have dropped these warnings. The HTTP response is unchanged and stays a uniform `invalid_attestation`. The logged fields are fixed strings, with no device or request data. Companion iOS PR [#8629](https://github.com/block/buzz/pull/8629) stops a device from replaying the same rejected enrollment indefinitely. ### Validation - Added tests: a single-test binary checks the emitted JSON warning (level, operation, reason, detail) through `AppAttestVerifier::verify_attestation`, for malformed base64, an empty proof, and malformed CBOR. Another checks that the default filter keeps gateway warnings and dependency errors, drops dependency warnings, and still honors an explicit `RUST_LOG`. Both fail when the warning is downgraded to debug or the default filter loses its `error` fallback. - The existing push gateway suite passes locally, including the App Attest rejection tests. Clippy is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Tom Brow <tomb@block.xyz>mobile-v0.27.0-rc.1 mobile-v0.26.0-rc.2 |
||
|
Kalvin C
|
3ce3a2e2df |
docs(nip-ap): define agenttype on agent settings (#8631)
## Summary
`buzz.agent.settings` content already carries `agentType` in practice
(block/buzz-app writes it and refuses to launch with another type's
settings), but NIP-AP defines the content as `{"version": 1, "settings":
{...}}` and never mentions it. block/buzz-app#1226 makes `agentType`
drive cross-device behaviour (following a type change, and starting
settings fresh on a retype), so the spec should define it.
## Changes
- Config artifacts table: settings content is `{"version": 1,
"agentType": "<string>", "settings": {...}}`.
- New **Agent type** rule:
- `agentType` is a client-defined string, compared exactly.
- Writers MUST include it; a record without it counts as the agent's
current type (keeps older records working).
- A client MUST NOT launch an agent with settings for a different type.
- A type change is an edit that writes the new `agentType`; a client
reading a current revision for another type MAY switch the agent to that
type when it supports it.
- **Editing**: one explicit exception to keeping unrecognised fields: an
edit that changes `agentType` MAY drop the fields inside `settings`,
since they belong to the old type. Top-level fields and tags are still
kept.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Co-authored-by: peon <9ac6794b000690b7e814eb1805ad32405d0bec7d52838de3a86cf967565dacc0@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Tom Brow
|
f167c4c3e1 |
fix(push): accept canonical channel UUIDs across versions (#8609)
Starter channels use UUIDv5 IDs, but mobile push accepted only UUIDv4 channel IDs, so building a subscription that included a starter channel threw locally. After #8608, no non-v4 ID reaches mobile push today: positive `#h` filters cover joined DMs only, and mutes are limited to subscribed channels. This PR is therefore a safety net for future starter-channel subscriptions, and it prepares mobile for a relay that accepts them. Mobile now accepts canonical lowercase, hyphenated UUIDs of any version in subscriptions. It recognizes both the current `uuid-v4-lowercase` relay descriptor and a future `uuid-lowercase` descriptor. Against a `uuid-v4-lowercase` relay, mobile checks positive and mute channel filters before enrollment or publication, keeps the desired filters, and rechecks relay capabilities every five minutes instead of sending a lease the relay would reject. The relay is unchanged. It keeps validating and advertising `uuid-v4-lowercase`. Installed mobile builds reject any other `h_grammar`, and they parse the descriptor before renewal, opt-out and revocation, so changing the advertisement now would leave their leases active until expiry. The relay validation and advertisement change will follow in a separate PR once those builds have drained. `docs/buzz-push.md` records this sequencing. ### Validation - Regression tests reproduced the UUIDv5 rejection in Dart before the fix. Added coverage checks positive and mute filters, persisted subscription round trips, encrypted lease content, malformed IDs, retained bounds, parsing of both descriptor grammars, and pre-publication rejection of non-v4 IDs against a `uuid-v4-lowercase` relay. - Mobile push tests on the current head: 104 passed, 4 skipped. `flutter analyze` on the push code and tests is clean. - Before the split, exercised actual mobile lease construction, NIP44 encryption and NIP42 publication against an isolated local relay. The relay half of that exercise is no longer part of this PR. - Relay source is identical to `main`, so the relay serves the same descriptor that installed builds parse today. The standing mobile test exception applies. Agent review and changed-behavior exercise are complete. buzz-review-completed --------- Signed-off-by: Tom Brow <tomb@block.xyz> Co-authored-by: george <973be878edd6d4f79692a3827f20a6f7fe0ed0362f269ff6b6ba8867033d7d8e@sprout-oss.stage.blox.sqprod.co> |
||
|
Tom Brow
|
b77ad2ed10 |
fix(mobile): align Solo agent policy and labels with the owner's inventory (#8616)
## Summary
`main` now discovers Solo agents from the owner's kind 30177
managed-agent inventory. This PR aligns two remaining parts of that path
with desktop's agent library.
- **Unsupported policies fail closed.** Desktop parses each agent's
latest 30177 record as `ManagedAgentEventContent` and drops records it
can't parse. Mobile only checked `name`. It now also requires an integer
`parallelism` in `u32` range, a `respond_to` of `owner-only`,
`allowlist` or `anyone`, an allowlist that is absent or a list of
strings, and string-or-null optional text fields. A newer unsupported
record hides the agent instead of falling back to an older one. Records
written by the apps always carry these fields.
- **The owner's name labels the agent in Solo's selection, header and
creation surfaces.** The directory already used the 30177 `name`.
Message authors and the typing indicator still use the agent's profile
name. The existing-chat header, pending-change label, fallback avatar
initials (header, native header action, new-chat row) and the resumed
native creation's saved agent preferred the agent's self-published
kind:0 name. They now use the owner's name first and fall back to the
profile. Profile artwork is unchanged.
### Validation
- Added tests: a table of 11 unsupported policy shapes that must hide
the agent, accepted policy shapes, and journeys where the owner's name
("Owner Name", "Alpha") differs from the cached profile name
("Confusable …"). The journeys cover the new-chat row's title and
initial and the existing-chat header tooltip, initial and pending label.
They also assert the native new-chat payload and picker row, which
already used the inventory name.
- Mutation checks: removing the policy check fails 10 tests; preferring
the profile for the header name, header initial or new-chat initial
fails the matching journey. The saved-agent name on resumed native
creation has no dedicated test.
- The full mobile suite passes (3,806 tests), and `flutter analyze` is
clean.
- The human-testing requirement does not apply under the mobile client
testing exception.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Tom Brow <tomb@block.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Logan Johnson
|
a4e83dfb50 |
fix(mobile): open message links to public channels the user has not joined (#8518)
🤖 ## Summary - **Problem:** On mobile, tapping a link to a message in a public channel did nothing if you had not joined that channel. The app showed a brief "Channel not found in this workspace" notice and dropped the link, although you are allowed to read that channel. - **Now:** The app asks the relay for that one channel. If the channel is public, it opens as a channel you can read but have not joined, the same as opening it from Browse channels. Private channels and DMs still open only when you are a member. ## Details - Before, the link handler looked for the channel only in the app's loaded channel list. That list holds the channels you joined. It holds other public channels only after Browse channels has loaded them. - When the channel is not in that list, the app now reads that channel's metadata from the relay (kind:39000, the event that names and describes a channel). The relay returns it only if you may see it. The app then refuses private channels and DMs, as Browse already does. - If the lookup fails, the app shows "Could not open the channel". If the relay returns no channel, it still shows "Channel not found in this workspace". - Links are handled in arrival order. While one link is being looked up, later links wait, including links to channels you already joined. When the lookup ends, the app handles the next link. A slow relay can therefore delay later links by up to the lookup timeout (8 s). The app shows no loading indicator during the wait. - A link to a channel already in the list opens at once, even while the app reconnects to the relay. A link that needs the relay lookup waits until the relay connection is signed in, then opens without a second tap. If the connection drops during the lookup, the app keeps the link and tries again after it reconnects. - During hosted setup the app turns link navigation off. If a lookup ends during setup, the link stays queued and opens when setup closes. It does not open a channel over the setup screen. ### Related issue None found. #8292 covers a different part of message links. ### Testing - New dispatcher tests cover a public channel you have not joined, a private channel you have not joined, and a failed lookup. All three fail on main and pass here. - Delayed-lookup tests hold the relay response open. They cover navigation turning off during the lookup (the link opens only after navigation turns back on; this test fails without the fix), a queued link behind a slow lookup, and the dispatcher closing during the lookup. - Reconnect tests start with cached channels and a reconnecting session. An unlisted-channel link sends no lookup and stays queued, then opens after the session connects. A listed-channel link opens at once while reconnecting. A session drop during the lookup keeps the link and retries it, both when the session returns after the cancelled lookup ends and before it ends. The three reconnect cases fail on the previous head. - `flutter test test/features/channels test/features/onboarding` (1990 tests) and `flutter analyze` pass on the rebased head. - End-to-end on an Android emulator (API 36) against a local relay. Alice made a public channel `#launch-notes` and posted in it. In a private channel `#team`, she posted a link to that message. The phone user is a member of `#team` but not of `#launch-notes`. The same install was upgraded in place from a main build to this branch's build. | Before (main): tap the link | After: tap the same link | After: link to a private channel you are not in | | --- | --- | --- | | <img width="300" alt="Tapping the message link shows 'Channel not found in this workspace' and stays in #team" src="https://github.com/user-attachments/assets/aa4341aa-8d87-4a78-a805-ef71f4bd6cc5" /> | <img width="300" alt="Tapping the message link opens #launch-notes with Alice's message and a 'Join this channel from Manage to participate' footer" src="https://github.com/user-attachments/assets/267930ab-b723-4d97-bb7f-960fe1497e63" /> | <img width="300" alt="Tapping a link to a private channel still shows 'Channel not found in this workspace'" src="https://github.com/user-attachments/assets/10780121-c3a4-4cf4-a8d1-a6994cac0101" /> | --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
ff9eb8f4dc |
Make mobile onboarding QR-first and synchronize scanner motion (#8599)
Make QR scanning the primary onboarding action, with full-width contrasting buttons inside the existing gutters. The landing offers **Scan a QR code** and **or sign in with email**. A two-finger double tap on the open welcome area reveals **Enter a community link** and **Use a pairing code**; a screen-reader action provides the same access. Action buttons remain outside double-tap detection so scanner taps respond immediately. Pending sign-in keeps its cancellation action. The Dynamic Island scanner uses 230 ms opening and closing springs. Welcome content fades and scales with the same progress, camera startup begins on the first frame, the scanner prompt fades with the camera reveal, the live preview adds a 230 ms fade when camera startup completes, and opening gives selection haptics. Native iOS status indicators request a matching 230 ms fade, returning when dismissal begins. Reduced motion is respected. Validation: full mobile suites and analysis run through local checks and pre-push hooks. The 230 ms iOS Release build was signed, installed, and launched in the separate Onboarding app on iPhone Air. The owner reviewed the simplified landing and immediate tap response; the latest 230 ms timing is installed for comparison. The latest 230 ms build, including the prompt fade, is installed and launched for comparison. Native status-bar timing has not been independently measured on-device. Product intent confirmed by Kenny: the supported visible onboarding choices are email sign-in or QR scanning. Camera permission/startup failure intentionally does not expose manual pairing or community-link recovery. Those remain hidden developer options, with no keyboard path by design. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
klopez4212
|
b14884980e |
Speed up mobile Solo chat creation and agent discovery (#8547)
## Summary Solo compose now opens before agent discovery finishes, and chat creation avoids full channel-list reloads while overlapping independent reads. Durable retries, ownership checks, and relay confirmation remain intact. The agent picker uses the owner's saved inventory, excludes known archived identities, verifies profiles off the UI thread, and reuses avatar artwork. ### Related issue No matching issue found. Related: #8546 adds project selection to the same compose surface. ### Testing - Full mobile suite: 3,755 passed, 19 skipped; four additional artifact tests passed. - Analysis, formatting, file-size checks, and iOS Release build passed. - Regression tests fail when the old blocking paths are restored. - Installed on a physical iPhone with existing profile/community settings preserved. Owner tested the earlier creation/picker improvements; final compose timing awaits device feedback. No timing recording captured. --------- Signed-off-by: kenny lopez <klopez4212@gmail.com> Co-authored-by: Terp <5856f6900e8e0a4e0ea1f4f8d2ee33e4753d2119bc122f94ecafc660b46c0511@buzz.block.builderlab.xyz> |
||
|
Matt Toohey
|
6861e18a86 |
fix(desktop): keep FileCard inside narrow message rows (#8612)
## Summary Non-image attachment cards ran past their message row in the thread side pane, and the download icon was clipped off. `FileCard` is an `inline-flex` box, so it shrinks to fit its content and is never narrower than its min-content width. That width comes from the `truncate` (nowrap) filename. The only width cap was `max-w-sm` (24rem), so a long filename pushed the card out to the full 24rem even when the pane body was only about 300px wide (the default 380px thread pane). - Add `max-w-full`, so the cap becomes `min(24rem, 100%)`. The flex column ancestors already have `min-w-0`, so the percentage resolves against the pane width. The card shrinks, the filename gets an ellipsis, and the size and download icon stay visible. Wider surfaces (forum posts, DMs, wide timelines) look the same because 100% is wider than 24rem there. - Add a Playwright regression test in `file-attachment.spec.ts`. It seeds a text attachment with a long filename and opens its thread. It then checks that the card and its trailing icon stay inside the row and that the filename is actually truncated. Without the fix, the card's right edge lands about 82px past the row. ## Before / After | Before | After | | --- | --- | |  |  | Measured in the default 380px thread pane: before, the card's right edge sits at 1342px against a row edge of 1260px (+82px overflow); after, it sits at 1252px (8px inside the row). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Matt Toohey <contact@matttoohey.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Tom Brow
|
5996e5dd8b |
fix(mobile): isolate push setup failures from app rendering (#8610)
Push subscription setup ran synchronously inside the root widget build, so an invalid subscription could replace the entire app with Flutter's error surface. Push initialization and the cold-start native notification read could also prevent the app from rendering. Keep push work independent of the app subtree, observe asynchronous persistence failures, and retain the original errors and stack traces. Setup retries use bounded backoff; Settings shows a failure and offers manual retry without resetting navigation. Retry preserves a healthy revocation worker, and community-removal callbacks follow a recovered worker so lease cleanup continues after setup recovery. Related: [#8608](https://github.com/block/buzz/pull/8608) fixes the specific unrelated-mute input that exposed this startup coupling. ### Validation Added Flutter journeys cover invalid subscriptions, asynchronous storage failure, dependency initialization failure, retry exhaustion and manual recovery, navigation preservation, accessible retry actions, and pending/failed native startup reads with notification-target recovery. Both original rendering regressions failed before the fix. Community-removal coverage also verifies cleanup after a worker initialization failure and after retrying healthy setup; that regression failed before the worker fix. The full mobile suite passed (3,782 tests; 19 existing skips), as did the four unconfigured-build tests and static analysis. The corrected head also passed the full pre-push mobile checks and tests, plus 42 focused push and community tests. Rendered widget evidence uses mocked native platform channels; no device APNs delivery test was performed. Rendered Flutter widget evidence with synthetic data and mocked native platform views. ### Setup failure The app remains usable and Settings offers a retry after push initialization fails.  ### Recovery Retry clears the error without replacing the navigator or leaving Settings.  <!-- buzz-review-completed --> --------- Signed-off-by: Tom Brow <tomb@block.xyz> |
||
|
Matt Toohey
|
8216db1c71 |
chore(toolchain): bump Rust to 1.98.1 for macOS 27 proc-macro loading (#8598)
## Summary
Bump the pinned Rust toolchain from 1.95.0 to 1.98.1 so release builds
work on macOS 27 hosts.
Rust ≤ 1.97 strips Apple-target artifacts with its bundled
`llvm-objcopy`, which leaves the Mach-O `LC_SYMTAB` string pool
(`__LINKEDIT`) only 4-byte aligned. macOS 27's dyld refuses to `dlopen`
such images (`mis-aligned LINKEDIT string pool`). Because cargo applies
the release profile's default `strip = "debuginfo"` to proc-macro dylibs
too, `cargo build --release` of any crate that uses sqlx macros
(`buzz-db`, `buzz-relay`, `buzz-admin`, …) fails on macOS 27: rustc
cannot load `libsqlx_macros-*.dylib`. Dev builds pass because the dev
profile does not strip, which is why this only shows up as a
release/packaging failure.
The fix (llvm/llvm-project#203680, pulled into rustc by
rust-lang/rust#158410 for rust-lang/rust#157750) first shipped in Rust
1.98.0 and was never backported to 1.97.x. `block/buzz-app` pinned
1.98.1 for the same failure in block/buzz-app#529. With this bump the
`CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_STRIP=none` / custom-linker
workarounds people have been using locally are no longer needed.
### What changes
Two commits: the toolchain bump (with the desktop clippy fixes), then
the workspace clippy fixes.
| Location | Change |
|---|---|
| `rust-toolchain.toml` | `channel = "1.95.0"` → `"1.98.1"` (profile
unchanged) |
| `Dockerfile`, `Dockerfile.push-gateway` | `ARG RUST_VERSION=1.95` →
`1.98` |
| `Dockerfile.sprig` | `rust:1.98-alpine3.22` with the manifest-list
digest verified on Docker Hub (shared by the `1.98` and `1.98.1` tags) |
| `.github/workflows/release.yml` | `dtolnay/rust-toolchain` pinned to
the tip of its `1.98.1` branch (`18f04874`) for the Windows desktop
release job |
| `benchmarks/harbor-buzz-orchestra` | `RUST_IMAGE = rust:1.98-alpine` |
| `_ci-rust.yml`, `windows-canary.yml`, `app_state_tests.rs` | stale
"1.95" comments reworded; the `unknown_lints` allow stays with a
version-agnostic rationale |
| `desktop/src-tauri` | lints clippy 1.98 newly raises under `-D
warnings`: `chunks_exact_to_as_chunks` at five fixed-width PCM/UTF-16
decoders, `needless_borrows_for_generic_args` on the `Copy` `norm`
closure in `config_bridge/reader.rs`, and a redundant `Engine as _`
import in `hpke_key_backup_tests.rs` that `use super::*` already
provides |
| Workspace crates (9) | the 30 lints clippy 1.98 newly raises in `cargo
clippy --workspace --all-targets -- -D warnings`, listed below |
Workspace lint fixes:
| Lint | Sites | Fix |
|---|---|---|
| `result_large_err` (20) | `buzz-relay` `api/git/transport.rs` (8),
`api/git/settings.rs` (6), `api/gifs.rs`, `api/workflows.rs`,
`api/profiles.rs`; `buzz-push-gateway` `verify_installation_assertion` |
Clippy 1.98 now checks `async fn`s, so every `async fn … -> Result<_,
axum::response::Response>` trips it. These follow the existing allow
pattern in `api/media.rs` and `nip_fi_http.rs` ("Response is the natural
error type for axum handlers"): module-level in the two git files,
per-function elsewhere. Boxing `Response` would touch every handler for
no runtime benefit. |
| `chunks_exact_to_as_chunks` (3) | `buzz-media` MP4 brand scan;
`buzz-voice` npy f32 reader and stereo test fixture | `chunks_exact(N)`
→ `as_chunks::<N>().0` |
| `question_mark` (3) | `buzz-acp` queue flush fallback and
slash-command mention skip; `buzz-agent` `find_git_root` | `match … {
None => return None }` → `?` |
| `useless_format` (2) | `buzz-auth` nip_fi startup test policies |
`format!("…")` → `"…".to_string()` |
| `useless_borrows_in_formatting` (1) | `buzz-pairing-cli` secret print
| drop the borrow |
| `manual_noop_waker` (1) | `buzz-sdk` broker test `block_on` |
`Waker::noop()` (stable since 1.85) |
All `as_chunks` rewrites keep `chunks_exact`'s remainder-dropping
semantics, and `as_chunks` is stable since 1.88, which is the MSRV.
One change in `api/git/transport.rs` is not a lint fix. Staging that
file trips the pre-commit secret scanner on its existing test-only ES256
fixture (`GIT_TEST_EC_PEM`), and the scanner only honours a suppression
marker on the flagged line. So the `\`-continued string literal is
rewritten as a byte-identical `concat!` whose `BEGIN` line carries
`sadscan:disable kingfisher.privkey.2`.
Left as is on purpose: `Cargo.toml` `rust-version = "1.88.0"` (MSRV
floor, a separate concern), the README/CONTRIBUTING "1.88+" text, Hermit
(no Rust packages there), the lockfiles, and the `(1.95.0)` comment in
`scripts/start-isolated-test-relay.sh` (the pre-commit secret scanner
flags that file's pre-existing dev-only `DATABASE_URL` whenever it is
staged).
### Related issue
Fixes #8074
### Testing
All under rustc/clippy 1.98.1 on a macOS 27 host (Darwin 27.0.0,
aarch64), with no `RUSTFLAGS` / `SDKROOT` / strip overrides in the
environment:
- `cargo build --release -p buzz-db` (the #8074 repro: the release
profile strips the proc-macro dylib): **builds**. The resulting
`libsqlx_macros-*.dylib` (14 MB) has `LC_SYMTAB stroff % 8 == 0` per
`otool -l`. The same build on 1.95.0 died with `mis-aligned LINKEDIT
string pool`.
- `just clippy` (workspace, `-D warnings`) and `cargo fmt --check`
(workspace and `desktop/src-tauri`): pass
- `just desktop-tauri-clippy` (default and `mesh-llm` feature sets):
pass
- `just test-unit`: every crate passes. The `buzz-acp` binary failed
only under pre-push lane contention; rerun solo it is 1060 passed, 0
failed.
- `just desktop-tauri-test`: 3422 passed, 0 failed
- After the lint fixes: lib/bin unit tests for `buzz-acp`, `buzz-agent`,
`buzz-auth`, `buzz-media`, `buzz-voice`, `buzz-sdk`, `buzz-pairing-cli`,
and `buzz-relay`'s `api:
|
||
|
Logan Johnson
|
a38fedb234 |
fix(desktop): recognize agents without NIP-OA in Huddles (#8544)
🤖 ## Summary In a Huddle (a voice call in a channel), Buzz Desktop decides which participants are agents. It uses that answer to choose whose text it speaks aloud, which agents can publish voice, and when the last person leaves and the Huddle ends. Desktop currently treats a participant as an agent only when its channel role is `bot`. #8494 changes the relay so that it gives `bot` only to agents with a verified owner attestation (NIP-OA). After that change, agents without an attestation join as ordinary members, and Desktop treats them as people: - their messages are not spoken, - their voice publisher is refused, - the Huddle stays open after the last person leaves while such an agent is still connected. With this PR, Desktop also counts a member as an agent when a trusted person has published a managed-agent record for it. Desktop already publishes such a record for every agent it runs, on every machine. Trusted people are: - **you**, so your own agents count as agents on all of your machines; - **the Huddle's owner**, usually the person who started it, so the agents they added count as agents for everyone who joins. Some agents without an attestation still count as people: an agent added by someone who does not own the Huddle, as seen from any machine other than that person's. The only way to recognize those agents is an owner attestation. This PR does not sign one. ### Details - `huddle/roster.rs` (`fetch_huddle_roster`) is now the only place that decides agent or person for Huddles. A member is an agent if its role is `bot`, or if the current user or a channel `owner` published a kind:30177 managed-agent record whose `d` tag is that member's key. Records from other authors are ignored. Records whose signature does not verify are ignored. - All Huddle checks use the roster: voice-publisher authorization, `get_huddle_agent_pubkeys` (spoken-text gating and the participant list), the periodic roster refresh, adding an agent, cleanup at the end of the Huddle, and the last-person check. - Trusting the owner's records is a compatibility policy for Huddles, not proof of identity. A record creates no owner. It lets the channel owner mark a member as an agent for every Huddle behavior that reads the roster: spoken text, voice publishing, remote transcription and turn-taking (both skip agent audio), cleanup, and the last-person check. Before #8494 the client chose the role, so an owner could already do this by labeling a member `bot`. - If the managed-agent records cannot be read, Desktop does not drop agents that have the `bot` role. Spoken text, voice publishing, adding agents and cleanup keep working for those agents. The Huddle does not end automatically, because an agent without an attestation could be counted as a person. When you join, and on each periodic refresh, Desktop adds the `bot` agents to its stored list and keeps stored agents that are still members. ### Related issue Follow-up to #8494 (the relay stores `bot` only for attested agents). No existing issue found. ### Testing - New helper-level unit tests in `huddle/roster.rs`. They do not run the relay query, the pipeline writers or the leave path. - the `bot` role, the user's own record and the owner's record each mark a member as an agent; - records from other members are ignored; - a record with a forged signature is ignored; - when the record read fails, `bot` agents stay agents, the Huddle cannot end automatically, and the next successful read restores the full roster. - when the record read fails, a joiner with no stored agents still stores the `bot` agents, an agent known only by its record stays stored, and a stored agent that left drops out. - New tests in `huddle/roster_hydration_tests.rs` run the real `post_connect_setup` and hot-start entrypoints against a fake relay whose managed-agent record read fails. The voice-model, audio and pipeline steps are passed in, so the tests replace them. Post-connect stores the `bot` agent before media setup starts; the hot-start refresh also keeps a still-member agent known from an earlier record and drops one that left; the same incomplete roster gives no human count for the last-person check. The tests fail when either entrypoint skips its roster read, and when the roster code accepts only complete rosters. - `cargo test --lib huddle::` (Huddle tests only), `cargo clippy --tests -D warnings` and `cargo fmt --check` pass in `desktop/src-tauri`. - Not yet tested in a live Huddle. --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz> |
||
|
ravarora2
|
834dca26bf |
Generate user community inventory for the dummy deletion workflow (#8605)
## Summary User deletion can now discover current communities from a hex public key or npub, save that inventory, and run it through the existing approval and dummy workflow. `--pubkey demo` remains the default. Execution only prints messages; it does not fence users or delete data. Discovery combines explicit relay memberships with active channel memberships, including users who have no relay membership row. It deduplicates communities and preserves owner relationships. The saved manifest contains only community IDs, labels and relationships. Historical discovery and owned-community execution remain later work. Submission rejects owner inventories atomically with a specific unsupported-execution error; preview still includes owners. This is an expand-schema change after #8520. Migrations 0072–0073 replace the demo-only target check with a bounded identifier check, with 5-second lock and 30-second statement timeouts. The database keeps the user-only entity-type check while allowing different identity formats in the target field. Workflow-specific commands reject entity types they cannot execute. The application currently resolves demo and hex/npub inputs; account-ID lookup is future work. The desired-state constraint matches. No backfill, trigger, function, or foreign key is added. Existing demo rows remain valid. Old binaries fail closed on new manifests; use the new CLI for their approval and execution. Older auto-migrating binaries need the documented rollback procedure. Identity parsing and demo fixture selection stay in intake. Approval and resume use the saved target without reparsing its format. The user workflow defines its stage names and handlers; CLI stage validation reads those rules instead of a separate three-name list. Shared persistence accepts caller-defined stages. Permanent target uniqueness is unchanged. Dummy submissions and execution belong in isolated, disposable development databases; production preview uses read-only access. CLI operator names remain self-asserted. ### Related issue Follows #8520. Searched open PRs; no duplicate inventory change found. #8489 is earlier workflow scaffolding. ### Testing Built `buzz-admin` locally and applied all migrations to a fresh disposable PostgreSQL database. Ran the actual CLI through preview → submit → inspect → approve → `run --next-step` → resume → completed for both demo and a public key with a channel-only membership fixture. Demo completed five stages; the public-key run completed three. Injected `--fail-stage purge_db`, waited for the retry deadline, and resumed the same failed stage successfully. Membership data remained intact. Repeated the CLI workflow on October 26 for demo and an npub backed by a fresh production inventory. Production access used read-only credentials and the real `inventory --pubkey "$SUBJECT"` handler. Copied only the relevant discovery fields into an isolated local PostgreSQL database; local channel parent rows supplied placeholder fields for foreign keys. The inventory saved by local submission exactly matched the production result. All workflow writes stayed local. For each input, ran `submit --pubkey "$SUBJECT" --requested-by local-e2e-submitter`, inspected the saved inventory, and confirmed execution before approval was rejected. Approved its digest, then ran each handler with `run "$REQUEST_ID" --next-step --stage-delay-ms 2500` in a new process. Captured all three workflow tables during and after every stage. Demo completed five stages across two communities; the npub completed three stages for its discovered community. Each handler saved `dummy: true`. Both requests finished with no current stage, no worker lease, and no retries. Copied memberships remained intact. This additional run exercised orderly resume; it did not inject a failure. The reproducible command sequence and fixture setup are in [the inventory walkthrough](docs/operator-user-deletion-inventory.md). Production was not changed. The earlier full-workflow runs used synthetic local fixtures; the additional npub run used captured production discovery fields. Repository-wide `just ci` was not run; validation focused on the affected packages and PostgreSQL workflow. The pre-push desktop build failed with “Too many open files in system.” Stopped the broader hook run and bypassed it to publish the earlier commits; the affected-package Clippy and formatting checks had passed. Generated with Codex --------- Signed-off-by: Ravneet Arora <rarora@squareup.com> |
||
|
Wes
|
8ceca8ac46 |
perf(mobile): batch initial channel read-state seeds (#8607)
## Summary Opening an uncached community seeded each previously unseen channel's read marker separately. Every seed copied and serialized the full read-state snapshot, started three preference writes, and notified listeners. Home now collects those seeds and saves/emits once, using the existing monotonic marker admission rules. For a 400-channel fixture with one existing marker, initial seeding falls from 399 full snapshot saves to one. Existing unread frontiers, local-only seeds and explicit read publishing keep their behavior. No transport, cache, relay or unread eligibility changes. This is independent of #8580 and #8490. ## Validation - `just mobile-install mobile-check mobile-test`: clean analysis; 3,779 tests passed, 19 skipped, plus four checks for builds without push configuration. - Real Home/channel-provider/read-state journey covers 400 channels, preservation of an existing unread highlight, persisted marks, no seed publication, and no repeated seed save on refresh. It replaces the earlier fake-notifier seed test. - Restoring the old per-channel loop makes the new journey fail at the single-snapshot assertion. - Diff and file-size checks pass. - Merged current main (`209fcd217`) at `d34d29627`; retained both the cold read-seeding and cached-community landing test registrations. The production diff is byte-for-byte unchanged from the version reviewed by the Court at `6a232ed37`. The complete mobile gates above passed again on the merged tree. Device latency is unmeasured. The Court source review found no blockers at `6a232ed37`; app/device testing and human acceptance remain unconfirmed. To try: open a community without a saved channel list, confirm its channels load and existing unread highlights remain correct, then refresh and reopen it. The performance claim here is reduced save work, not a measured end-to-end speedup. Signed-off-by: Wes <wesbillman@users.noreply.github.com> Co-authored-by: Star Lord <b89298dbe87c6b3fd8a425b535d9d161f23c88d555a47925ae0b04dfe02b201e@buzz.block.builderlab.xyz> |
||
|
morgmart
|
209fcd2172 |
fix(mobile): ignore unrelated mutes when building push subscriptions (#8608)
## Summary Muting a starter channel (#welcome-everyone or #general) could leave the iOS app on a blank screen after launch. Starter channels use UUIDv5 IDs. Push subscription setup checked every synced mute against the lease's UUIDv4 channel grammar and threw `FormatException: Push subscription channel ID is invalid.` from `pushSubscriptionSyncProvider` during build, so the first frame never rendered. Whether you saw a blank screen depended on whether channels and mutes finished loading before the first frame. Mutes only feed the lease as ignore filters for subscribed (DM) channels, and mutes sync across every community and channel type. So the builder now matches mutes against the subscribed channels before using them, instead of validating every mute. Subscribed channel IDs are still validated as before. ### Related issue None found. ### Testing - Captured the failure on a dogfood iPhone build from the device system log, with a stack trace through `buildDesiredBuzzPushSubscriptions` and the muted ID `fee88e92-da9f-5958-a910-48db3264ad38` (#welcome-everyone). Unmuting that channel from desktop fixed the device immediately, which confirms the cause. - New regression test `ignores mutes outside the push channel grammar` fails on `main` with the same exception and passes with this change. - The pre-push `mobile-checks` hook (`just mobile-check && just mobile-test`) passed on this head. - Screenshots: not applicable. This changes no UI. It only stops a startup failure. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: morgmart <98432065+morgmart@users.noreply.github.com> Co-authored-by: Carl <c217fe6b9d958f41c3a5e030dccc7f626775a923089cb6491305eade75ea1f1b@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Luke Tornquist
|
0e36dbc823 |
fix(db): keep the reader pool warm and drop its pre-acquire ping (#8495)
## Summary
The reader pool's 150ms `acquire_timeout` covers the whole acquire: the
pre-acquire idle-connection ping **and** any new-connection dial (TCP +
TLS + SCRAM). When readers slow down for a moment, the timeout:
- abandons in-flight dials mid-handshake (Postgres logs `could not
accept SSL connection: EOF detected`),
- closes healthy idle connections whose ping overran instead of
returning them to the pool, and
- falls every miss back to the writer, which then exhausts its own pool.
Across a fleet of relay pods this shows up as a burst of abandoned
handshakes on the readers that starts on nearly every pod at once,
followed seconds later by writer pool-acquire timeouts. The database
itself reports no statement, lock, or connection-limit errors before the
burst: the client is tearing down its own reader connections.
This PR removes that amplifier. It does not change the request budget,
writer fallback, or the writer pool.
### Change (`crates/buzz-db/src/runtime/mod.rs`)
- **Warm floor:** reader `min_connections` is now `max / 5`, at least 1,
derived from the existing reader pool size (no new config knob). sqlx
re-dials the floor in background maintenance, outside the request
budget, from its reaper and whenever a connection is closed instead of
released. The floor is per relay process, so standing reader connections
are the floor times the number of relay processes; size the reader's
`max_connections` for it.
- **Floor fills at boot:** with `max_lifetime`/`idle_timeout` set, a
lazy sqlx 0.9 pool does not dial `min_connections` at construction. The
existing boot ping now closes its probe connection instead of returning
it, which triggers sqlx's floor maintenance. Probe acquires that overrun
150ms retry with bounded backoff (250ms, 500ms, 1s, 2s). The doc comment
that said sqlx dials the minimum eagerly was wrong for this config and
is corrected.
- **`test_before_acquire(false)` on the reader:** the routed read's
`BEGIN … READ ONLY` already proves the connection and fails closed to
the writer (`reader_validation_error`). This removes one round trip per
checkout and the warm-connection churn.
- **Bounded fence proof:** with no pre-acquire ping, the proof (Aurora
capability probe, `BEGIN`, heartbeat observation) is the first round
trip on a checkout. It now runs under `READER_PROOF_TIMEOUT` (1s), so a
reader that stops answering without closing its sockets (partition, host
loss, failover) fails closed to the writer as the new route reason
`reader_proof_timeout` instead of hanging the request until TCP gives
up. 1s is well above a normal round trip, so a brief reader stall waits
on the reader rather than flooding the writer.
- **Bounded release:** every connection returned to the reader pool
first passes a release check bounded at 5s (`Db::read_pool_options`).
sqlx's own release ping has no timeout, so without this a socket whose
reader went silent would hold its slot until the kernel gives up; with
every slot held that way, the pool cannot dial the reader that replaced
it, and all routed reads fall back to the writer. A check that times out
or errors closes the socket and frees the slot, and floor maintenance
dials a replacement. A connection still inside a transaction is closed
instead of returned: a checkout dropped while its `BEGIN` was in flight
(a timed-out proof, a cancelled caller) has no `ROLLBACK` queued,
because sqlx counts the transaction only once `BEGIN` is acknowledged.
Closing it ends the session, and the server rolls it back. An expired
proof is therefore simply dropped. Maintenance checkouts (fleet usage
collection) also keep their close-on-drop: when one is abandoned at its
deadline, its socket is closed under sqlx's 5s close bound rather than
going through the release path.
- **Pinned isolation default:** reader sessions set
`default_transaction_isolation=read committed` in the startup packet,
which takes precedence over database and role defaults. The release
check depends on this, because it detects an open routed-read
transaction by its `REPEATABLE READ` isolation.
- **Metric:** each close is counted in
`buzz_db_reader_release_closed_total{reason}` (`timeout`, `error`,
`open_transaction`). Operator notes for the floor,
`reader_proof_timeout`, and this counter are in
`deploy/charts/buzz/README.md`.
### What got simpler
- The reader checkout path does one less round trip and no longer closes
healthy connections on a slow ping.
- Boot warm-up reuses sqlx's own `min_connections` maintenance rather
than re-implementing it.
- One release rule now covers every reader checkout, so the
open-transaction leak on cancelled reads no longer needs special
handling per call site.
- No new config knob.
- The reader test proxy now lives once in `test_support` instead of
inside `store/usage.rs`.
- Tests build their reader pools from the production options rather than
hand-copied ones.
### Tradeoffs
- The reader is now dialed at boot (previously "no reader dial at
boot"). If the reader is down at boot, this only produces log noise, and
reads still fail closed to the writer.
- The release check is one trivial statement (`SELECT
current_setting(...)`) per released reader connection. It runs in sqlx's
release task, after the response, so request latency is unchanged.
- The check detects an open transaction by comparing the current
isolation level with the session default. That is only sound because the
reader session default is pinned to `READ COMMITTED` and routed reads
begin `REPEATABLE READ`.
- **Residual window:** after the release check passes, sqlx sends its
own release ping (one more round trip), and that ping has no timeout. A
serving-read connection whose endpoint goes silent between those two
round trips still holds its slot until TCP gives up. Only releases
already mid-ping at that instant are exposed: later releases fail the
bounded check first and are closed. sqlx behaves the same way for every
pool on `main`. Maintenance checkouts avoid it through close-on-drop.
Bounding it for serving reads would mean replacing sqlx's release path
or putting a pooler in front of the reader, so it is left as a
follow-up.
- Boot fill is best effort. Each boot-probe retry still uses the 150ms
serving acquire budget, so if cold handshakes consistently take longer
than that, all five attempts fail and the floor is not filled at boot.
It fills the first time any reader connection is closed, which the
reaper and the release check both do, and until then reads dial as
before.
- This also closes pre-existing debt: a caller that dropped a routed
read mid-`BEGIN` (client disconnect) could leave a connection `idle in
transaction`. The release check covers that path too.
### Related issue
N/A (production incident follow-up; none found).
### Testing
- Postgres tests (`runtime/tests/reader_pool_postgres_tests.rs`):
- the boot ping leaves the reader pool at exactly its floor, all idle
(base leaves one connection);
- a warm reader whose heartbeat read blocks (exclusive lock) falls back
to the writer as `reader_proof_timeout` within one proof budget;
- a warm reader connection terminated server-side falls back as
`reader_validation_error` with no client error;
- a proof that times out mid-`BEGIN` on a paused reader (a test proxy
that holds bytes, like `docker pause`) falls back as
`reader_proof_timeout`. The reader database default is set to
`REPEATABLE READ` for this test. Once the reader resumes, it leaves no
`idle in transaction` reader backend, and the close is counted as
`open_transaction`. The test fails if the release check is made blind to
open transactions, or if the isolation pin is removed.
- Failover: every established reader session is stalled while new
sessions get through. Each stalled warm connection costs one
`reader_proof_timeout`, then routed reads reach the reader again through
new connections within 8s, and the closes are counted as `timeout`. The
test fails with the release bound removed.
- `store/usage.rs`: a fleet collection abandoned at its deadline on a
paused reader frees its slot within 3s while the reader is still paused,
which is faster than the 5s release check. The test fails with
maintenance close-on-drop removed.
- Unit tests: the min-connections mapping, the reader pool options, and
that construction still dials nothing.
- Blox, at `f3e3bf158`: `just fmt-check` and `just clippy` are clean,
and `buzz-db` unit tests are 191/191.
- Blox, at `01745b21e`: fmt is clean, the targeted reader tests passed 3
of 3 runs, and the release-bound and transaction-blind mutations both
turn the targeted tests red.
- Blox, at `2f8e482fb` (before the rebase onto `main`): fmt and clippy
are clean, the targeted reader tests passed 3 of 3 runs, `buzz-db` unit
tests pass, and each of four mutations turns its targeted test red:
release bound, transaction-blind check, isolation pin, and maintenance
close-on-drop.
- Blox, at the rebased head `e80bc2bed`: the same four mutations each
turn their targeted test red, and fmt, clippy, the targeted reader tests
(3 of 3 runs) and `buzz-db` unit tests are clean.
- CI Postgres lane at `e80bc2bed`: 1130/1130 passed, including every new
reader-pool test.
- Live check at `e80bc2bed`: a relay with a two-slot reader pool behind
a proxy, which was then frozen for 18s. All 246 history reads succeeded,
and the slowest took 1.0s. During the freeze, 2 proof timeouts and 2
release-timeout closes were counted. Reads returned to the reader within
36ms of the resume, and no reader backend was left `idle in
transaction`.
- Rebase note: `main` added a reader `after_connect` hook that tightens
`statement_timeout`. It now lives in `Db::read_pool_options`, so the
tests' reader pools get it too.
---------
Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Bradley Axen
|
a3a47e6294 |
feat(relay): add an unlisted channel setting (#8579)
🤖 ## Summary Sharing a Solo chat with a large channel today means adding every person as a member, one at a time, and the app blocks the share past 100 people. The plan is to share it the way Google Docs does with "anyone at the company with the link". Buzz's `open` visibility already gives that kind of access: anyone in the community can read and join an open channel. The problem is that an open channel also shows up in everyone's channel list and search, and a private chat shouldn't appear there just because its link was posted somewhere. This PR adds a channel setting, `unlisted`. An **unlisted open channel** can be read and joined by anyone who names it by id (from a shared link). For non-members, it is left out of every read that does not name it. Members see it as before. This is the relay side only; the buzz-app change comes later. #8575 (batch adds) remains the path for small audiences. Unlisted is not an access control. Anyone who has the id can read the channel, and that is intended. ## Solo flows this supports Each user action is one event. The buzz-app side comes later; the relay supports all of these now. | Flow | Event | Result | | --- | --- | --- | | New Solo chat | kind:9007 `visibility private` + `["unlisted", "true"]` | The chat starts unlisted, so it stays unlisted when any client later opens it. | | Share | One kind:9002: `visibility open` + `unlisted true` + `join_role guest\|member`. Then post the link. | Anyone with the link can read the chat by id, both history and live. Nobody is added as a member, so the chat isn't in anyone's channel list. A non-member can't post; `join_role` decides whether someone who joins can. | | View | A REQ that names the chat by `#h`, or by `#d` for its metadata | A non-member can read it. It is never listed for them. | | Reply | kind:9021 join (the app sends it before a viewer's first reply) | Joining makes them a member with the role from `join_role`, so with `member` they can post. A kind:9022 leave makes them a viewer again; the chat stays readable by id. | | Unshare | kind:9002 `visibility private` | The existing open→private eviction closes non-members' live subscriptions. New reads by non-members are refused. | | Convert to a channel | One kind:9002: `["t", ""]` + `unlisted false`, plus `visibility` | Clears the Solo label. Non-members can now find it in discovery. | **Sidebar pins are not relay state.** Pin and unpin live in the app's existing kind:30078 sidebar preferences, which sync across devices and don't touch membership. Don't map them to join/leave: someone a chat was shared with by member add is already a member, so a leave would remove their access. Your own Solo chats keep showing up, because you're their owner member, and members see unlisted channels everywhere. Existing Solo chats aren't backfilled. They get the flag when they're shared, because the share event sets it. **Open question, not built here:** someone who only views a shared chat isn't a member. A mention of them doesn't reach their inbox (`#p:[me]` without `#h`) and sends them no push. ## Behavior | Reader | Reads that don't name the channel (kind:39000 without `#d`, `#P` creator queries, messages, COUNT, NIP-50 search, artifact queries, channel list) | Reads that name it (`#h`, or `#d` on 39000–39003) | kind:9021 join | | --- | --- | --- | --- | | Member | Listed | Readable | n/a | | Non-member, unlisted open channel | **Not listed** | Readable | Works; `join_role` still decides the role | | Non-member, private channel (unlisted or not) | Not listed | Not readable | Refused | - **Setting it:** kind:9002 `["unlisted", "true"]` / `["unlisted", "false"]`. Owner or admin only, like `visibility`. A missing tag means no change. It can be sent in the same event as `["visibility", "open"]`, so a private chat becomes open and unlisted in one step. Any other value is refused before the event is stored. The relay posts an `unlisted_changed` system message and clears the accessible-channels cache across nodes. Nobody loses access when a channel is unlisted, so no subscriptions are evicted. - **At creation:** kind:9007 also takes `["unlisted", "true"|"false"]`. A missing tag means listed. Any other value is refused before the event is stored, as on kind:9002. It is stored with the channel row in the create's transaction, on both the client-UUID path and the relay-chosen-UUID path. - **kind:39000** carries a bare `["unlisted"]` flag while the setting is on. It does not add NIP-29 `hidden`, so the metadata stays readable by id. - **NIP-11** advertises `buzz-unlisted-channels` in `supported_extensions`. A relay without this PR would apply `["visibility", "open"]` and ignore `unlisted`, which would list the chat publicly. So buzz-app must check for the extension before it sends that pair. - **CLI:** `buzz channels update --channel <id> --unlisted true|false` and `buzz channels create ... --unlisted`. `buzz_sdk::build_update_channel` takes a new `unlisted: Option<bool>` argument, and `buzz_sdk::build_create_channel` takes `unlisted: bool`, which emits the tag only when true. ### Open question: does `unlisted` only mean something when the channel is open? **Decision: store it on its own; it only has an effect while the channel is open.** Private channels are already invisible to non-members, so on a private channel the flag does nothing. It is still stored and published, for three reasons: - a client can mark a Solo chat unlisted before opening it, or in the same event that opens it, so the chat is never listed even for a moment; - switching private → open → private → open keeps the setting; - there is no cross-field rule in kind:9002 to validate or keep in sync. kind:39000 publishes the flag as stored, whatever the visibility, so clients can show the setting. ## How The relay already has a "readable only when named" path for system channels (NIP-CL). Unlisted open channels reuse it: - **`get_accessible_channel_ids`**: the open half (the implicit scope of a REQ that doesn't name a channel) adds `AND NOT unlisted`. The member half is unchanged, so members still have the channel. - **`get_readable_named_only_channels`**: also returns unlisted open channels of the legacy types. Every reader of that set (REQ, COUNT, NIP-50 search, HTTP `/query` and `/count`, thread windows, workflow run reads) adds a named-only channel only to a filter that names it, through `filter_channel_scope`. An unlisted channel's type is `stream`/`forum`, so the `#t:["system"]`/`#P` system opt-in never adds it. - **`get_accessible_channels`** (the channel list): open channels are listed to non-members only when not unlisted. - **`query_artifacts`**: an artifact in an unlisted open channel reaches a non-member only when the query's `#h` names that channel. - **Live fan-out:** no change was needed. A global subscription never receives channel-scoped events, kind:39000 included (`fan_out_scoped`). A channel-scoped subscription has to name the channel, and that goes through the same named-only authorization. **Left alone on purpose:** access checks that read a named channel or event keep treating an unlisted channel as open. These are `handle_join_request` (kind:9021), the ingest write gate, operator-listener mention delivery (`load_deliverable_event`), personal read-state writes, and push matching. Push matching still requires membership, so non-members get no pushes from an unlisted channel. **Named reads checked:** these callers use the implicit channel set (`get_accessible_channel_ids`, its cache, `get_accessible_channels`). Every one that reads a channel the request names also adds the named-only set, so a non-member can still read an unlisted open chat by id. That covers WS REQ (history, live registration, NIP-50 search), WS COUNT, HTTP `/query` and `/count`, thread windows and workflow run reads. Other named reads don't use the implicit set: they check visibility or membership, so an unlisted open chat counts as open. Those are live fan-out, the ingest write gate, typing, kind:9021 join, huddle audio join, operator-listener delivery, the workflow sink and read-state writes. Feed queries (mentions, needs-action, activity) and the `/buzz/v1/me/sidebar` accessory API are lists, so leaving out chats you only view is intended. Media reads don't check channels. `buzz-auth`'s `ChannelAccessChecker` has no production implementation. ## Database (docs/database.md checklist) - **Phase:** expand only. The migration adds a column with a constant default, and the same release ships the code that reads and writes it. No backfill or contract step follows. - No trigger, function, procedure, rule or foreign key is added. - No migration changes existing rows. `ADD COLUMN ... BOOLEAN NOT NULL DEFAULT FALSE` stores the constant default in the catalog, so the table is not rewritten. - Every location in Rule 4 is updated: `migrations/0069_channel_unlisted.sql`, `schema/tables/public/channels.sql`, and the migration count. No seeds, types or reconcile changes are needed. `admin_schema_parity_between_desired_state_and_migrations` now compares `channels` columns between the pgschema desired state and the migrated database. I failed it once by removing the column from `channels.sql`. Only columns are compared for `channels`: on current `main`, pgschema already re-emits `idx_channels_id_live` without its `INCLUDE (community_id)`, an index drift that predates this PR and is noted in the test. - **Currently deployed relay against the new schema:** it never selects or writes `unlisted`, and every insert gets the default. All channel-record queries name their columns. Nobody can set the flag until this relay ships, because the old relay refuses a kind:9002 whose only tag is `unlisted`. - The migration starts with `SET LOCAL lock_timeout = '5s'`. No index is added: the new predicates ride the existing `(community_id, visibility)` index. ## Rollout 1. Deploy. Every channel stays listed, so nothing changes. 2. **Don't unlist any channel while an old relay is still running.** An old relay ignores the column and lists the channel to everyone. The NIP-11 extension lets clients gate on this. Unlisted isn't an access boundary, so a mixed fleet only exposes the channel in listings for a while. 3. **Rollback:** an older relay ignores the column, so unlisted channels show up as ordinary open channels again. Nothing breaks. ## Tests Postgres (`buzz-db`, discovered by the Postgres lane): - `unlisted_open_channel_is_named_only_for_non_members`: sets the flag through `update_channel`. Then checks that a non-member doesn't get it from `get_accessible_channel_ids` or `get_accessible_channels`, and does get it from `get_readable_named_only_channels`; that a member still lists it; that `put_member` (the kind:9021 path) lets the non-member join, after which they list it; and that listing it again shows it to a new reader. - `unlisted_private_channel_stays_private_and_opens_unlisted`: a private unlisted channel is not readable by name, and opening it keeps the flag. - `group_metadata_publishes_the_stored_unlisted_flag`, plus the unit test `publishes_the_unlisted_flag_as_stored`. - `unlisted_channel_artifacts_are_read_by_name_for_non_members`. - `channel_created_unlisted_is_never_listed`: creating through `create_channel_with_event` (the kind:9007 path) and `create_labeled_channel` stores the flag, and the channel is never listed to a non-member. - Mutation-checked: each of `NOT unlisted` in the implicit scope, `NOT c.unlisted` in the channel list, the unlisted branch of the named-only query, its `visibility = 'open'` guard, and the artifact condition was removed one at a time, and each removal failed at least one of these tests. Relay E2E (`test_nip29_unlisted_channel_wire`, added to the CI relay E2E step), against a local relay built from this branch: - A non-member or an unknown value is refused. - kind:39000 by `#d` carries `unlisted` and `public`, and not `hidden`. - kind:39000 discovery without `#d`: non-members don't see the channel, members do. - History and NIP-50 search: unnamed, a non-member doesn't see the channel's messages; with `#h`, they do; a member sees them unnamed. - A kind:9021 join works, and the joiner then lists the channel. - Listing it again shows it to everyone. - A reader whose channel set was cached before each change sees the change at once. I ran this against a relay with the cache invalidation removed, and it failed. Relay E2E (`test_nip29_solo_share_and_convert_wire`, added to the CI relay E2E step), each flow as a single event: - kind:9007 with an unknown `unlisted` value is refused. A chat created with `unlisted true` publishes the flag, and stays unlisted after a plain `visibility open`. - Share (open + unlisted + `join_role guest` in one kind:9002): a non-member doesn't list the chat. They read it by `#h`, both history and a live message, and can't post. A kind:9021 join lists it for them as a read-only guest. A kind:9022 leave unlists it again, and it stays readable by `#h`. - Unshare (`visibility private`): the viewer's live subscription gets CLOSED, and a new `#h` read is refused. - Convert (`["t", ""]` + `unlisted false`): the Solo label is cleared, and a non-member finds the chat in discovery. - I ran this against a relay that dropped the kind:9007 flag, and it failed. Also run: `cargo test` for buzz-db, buzz-sdk, buzz-cli, buzz-core, buzz-relay and buzz-test-client; the Postgres lane for buzz-db and buzz-relay (1045 tests); and the existing unlisted, announce, departure, guest, role-change and NIP-43 snapshot wire tests. In the lane, two tests failed on pool timeouts (`resend_validated_after_the_first_commit_is_a_duplicate`, `edit_replaces_and_clears_labels`) and passed when rerun alone. An earlier run had the same kind of flake: `routed_fallback_spends_one_acquire_budget_when_aurora_cache_is_cold` (a timing budget) and `resent_edit_is_a_duplicate` (a pool timeout). In a full local `e2e_relay` run, three tests outside CI's list also failed (`test_invite_*` twice, `test_unarchive_emits_member_added_notification`). They fail the same way on a relay built from `main`. ## Not in this PR - The buzz-app follow-up: support open Solo chats, the join-from-card view, the large-audience prompt, and gating on `buzz-unlisted-channels`. - A NIP-CL update. The behavior is documented in NOSTR.md ("Unlisted channels"). buzz-review-completed --------- Signed-off-by: Bradley Axen <baxen@squareup.com> Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: ART <ea39c1c8acf57180161ef4ba4ed68abf8589f5291956b909a5aa991fc66664ca@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: mbot <3754f8729004d95654c46dbab3129e4ab9ef05cc2534e2a3fbfc155983bd637b@buzz.block.builderlab.xyz> Co-authored-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> |
