mirror of
https://github.com/block/buzz.git
synced 2026-10-29 21:41:33 +00:00
974495fd4a
## Summary A relay rolling deploy closes every socket with 1012. On reconnect, mobile replayed each live subscription's **original filter** (startup `since`, full `limit`) unless that subscription had received an event, so quiet subscriptions re-read history back to app start every time. Reconnects used a fixed 1s delay with no jitter. The delay also reset to 1s on every successful connect, so a client bounced from pod to pod during a deploy kept coming back at the same moment as every other client. **This PR makes two changes:** 1. **Resume cursor.** If a subscription reached EOSE on a connection, it records `liveUntil`: the time of the last frame received from the relay on that connection, capped at the clock at close. The replay then uses `since = max(original since, lastSeen − 5s, liveUntil − 960s)`, and never goes past the filter's `until`. - **Why 960s:** the relay accepts `created_at` up to ±900s from its clock (`MAX_TIMESTAMP_DRIFT_SECS` in `crates/buzz-relay/src/handlers/ingest.rs`). An event stored after the close can therefore be backdated by that much, and the extra 60s covers client clock skew. - **Only relay frames count as liveness.** Close and error paths never extend `liveUntil`, so a close noticed after the device sleeps doesn't overstate how long the subscription was live. - **Ambiguous EOSE is ignored.** If a subscription has a second REQ on the same socket, that subscription doesn't advance `liveUntil` for the rest of the socket. A CLOSED ends liveness for the rest of that socket, because the retry is a second REQ. Subscription IDs are never reused, so an old EOSE can't mark a re-subscribe as live. - **No cursor:** a subscription that never reached EOSE keeps its previous cursor, or replays as before. 2. **Reconnect spreading.** - After a 1012 or 1001 close, the first attempt waits a uniform random delay in [0, 10s]. A client already backing off waits the longer of that delay and its backoff. - Ordinary backoff gets symmetric ±25% jitter, capped at 30s. - The backoff resets only after a connection stays up for 60s, not on connect. This matches desktop. - Foreground resume and a manual `reconnect()` still reset to the base delay, as before. **Simpler?** The pure reconnect math (cursor, jitter, spread, stability) lives in a new `relay_reconnect_policy.dart` and is table-tested on its own. `relay_session.dart` was near its 1200-line cap, so its five small private bookkeeping classes moved verbatim to `relay_session_subscriptions.dart`. Net concept count is unchanged. **Identified debt (not changed here):** - Dart's WebSocket ping/pong isn't visible to the session, so a long-idle but healthy connection that drops replays back to its last application frame − 960s. This is still bounded, and much cheaper than replaying from app start. - The unread catch-up in `channels_provider.dart` runs one REQ per active channel (`since = readAt + 1`, `limit: 1000`) on reconnect and every 60s. - A retryable CLOSED is retried with a 30s cap, no jitter and no attempt limit. - `lastSeenCreatedAt` advances on stored events received before EOSE. If a relay returns stored events newest-first and the socket dies before EOSE, the next replay can skip older stored events. - The `lastSeen − 5s` term keeps its existing gap for events backdated after a close. ### Related issue N/A. This is part of a set of reconnect-load reductions. Companion PRs: relay read-pool warm (#8651), desktop (#8652), agent harness (#8653). ### Testing - Mobile gate on a Linux workstation, at the head SHA: - `dart format --set-exit-if-changed`: pass. - `flutter analyze`: no issues. - Gateway recipe tests: pass. - `flutter test`: 3881 pass, 1 fail. `push_gateway_origin_validator_test` fails the same way on `main` on that host, due to a host git-config error. - New `relay_reconnect_resume_test.dart`: - pure table tests for the cursor and the delay - a real-socket test that reads the 1012 close code - journeys through `RelaySessionNotifier`. They check that a close or error after an hour asleep doesn't extend liveness, that a second REQ or a delayed EOSE for an unsubscribed ID can't advance the cursor, and that a backdated event inside the 960s window is still replayed. - 22 production mutations, each failing the new tests. They cover every cursor term, the 960s window, the `until` clamp, the liveness cap, the ambiguous-EOSE guard, CLOSED handling, jitter, the 1012 spread and floor, the backoff reset, and close-code capture. - `fake_relay.dart` now draws a fixed jitter midpoint, so existing journey tests stay deterministic. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>