9b1c4f98e2
Long threads (over ~100 replies) time out in desktop. The aux hop that
loads reactions and other e-tag references builds its filter as one
`tags @> …` term per id, joined with `OR`. Past about 100 ids the
planner switches to walking each partition's primary key backward,
expecting to stop early. When few rows actually match, it scans the
whole community. With a dirty GIN pending list, a 188-reply thread never
finishes.
### What changes
- **Set-based e-tag reads (`buzz-db`).** The aux e-tag query now puts
the id set behind a `MATERIALIZED` CTE and matches with `ANY`. Each
e-tag read also runs under a transaction-local `statement_timeout` of 20
s.
- **Terminal timeout contract (`buzz-relay`).** When a read hits the
deadline, the client gets a terminal timeout (`CLOSED` for REQ, the
timeout error for COUNT and search), not a retryable failure.
- **Subscription lifecycle (`buzz-relay`).** The per-connection
subscription map lock is now the lifecycle lock. Claims, replacement,
client `CLOSE`, owner-checked timeout and search cleanup, revoke, and
connection teardown all change the map, the registry, topic refcounts
and terminal frames while holding it. As a result, a timed-out
subscription's `CLOSED` goes out before any replacement with the same id
can claim it, and claims made after teardown are refused.
- **Dropped terminal frame disconnects (`buzz-relay`).** If the `CLOSED`
for a retired subscription (owner-checked timeout or channel eviction)
can't be queued, the relay cancels the connection instead of leaving the
client holding a subscription that will never receive fan-out. A
client-initiated `CLOSE` ack is exempt.
- **COUNT under the same deadline (`buzz-db`).** The fast COUNT path
runs e-tag counts under the same transaction-local 20 s cap as SELECT,
so WS COUNT and HTTP `/count` produce the terminal timeout too.
- **Replica reads surface the deadline (`buzz-db`).** When a
replica-routed read is cancelled by `statement_timeout` (SQLSTATE
`57014`), the error is returned instead of re-running the query on the
writer, which would run the same slow statement. Other replica errors
still fall back to the writer.
`PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md` records the exception.
- **Deadline handling (desktop).** Only the relay's `query timed out`
counts as a query deadline. A client-side `request timed out` stays
retryable. For one-shot reads a deadline is terminal: the global query
retry (budget 1), thread-replies retry and reaction hydration don't
re-send the same slow query. Reaction ids held back by a deadline are
released when the channel is opened again, so reopening retries them
while same-channel re-renders don't. Live subscriptions, in both the
renderer and the native archive-sync socket, keep treating a deadline
`CLOSED` as transient: they back off and resubscribe with the same
filter instead of being dropped.
- **Deadline handling (mobile).** Mobile recognizes the deadline on both
transports: HTTP `503 {"error":"query timed out"}` and the WS `CLOSED`
reason.
- The root `ProviderScope` retry policy returns no retry for deadline
errors and keeps Riverpod's default backoff for everything else.
- Background thread-summary recounts don't retry a deadline.
- Forum posts polling pauses while the last load settled on a deadline;
reopening the forum retries.
- A channel's newest-window load surfaces a deadline instead of falling
back to legacy WS history, and the channel list's batched queries return
"unavailable" on a deadline instead of fanning out to per-filter
fallbacks. An unavailable batch is not treated as empty. Known
last-message timestamps and unread state are kept, and unread catch-up
moves `lastMessageAt` forward (never backward) the same way live events
do, so a cold start with no known timestamp still gets one.
- A live subscription that gets a deadline `CLOSED` backs off and
resubscribes with the same filter. History requests still fail on it.
- Settled load failures show a Retry control, matching desktop's
thread-replies error card: a shared `LoadErrorView` on the channel
timeline, forum thread and forum posts list, and an icon button beside
the thread replies summary. While the thread query has no value, the
thread page shows the route snapshot plus live, cached and optimistic
replies, with the loading or error status still visible.
- Other re-sends (reconnect, foreground resume, live events, the unread
backstop, scroll) are not suppressed; each is bounded by the relay's 20
s statement deadline.
- **Long-thread seeder.** `just seed-long-thread [replies]` posts one
thread (default 187 replies) with some reactions to the local dev relay,
for reproducing long threads in the apps. It uses a throwaway key,
refuses non-loopback relays and ignores ambient `BUZZ_*` credentials.
Writes are paced to fit the stock relay rate limit.
**Lifecycle concurrency.** The lock serializes one connection's
subscription changes. Other connections can still be delayed by pressure
on the shared pubsub command queue, and by serial eviction loops waiting
behind a blocked connection. This PR does not claim strict isolation
between connections.
**Recovery vs. load.** Clients stop automatic retries of the same
one-shot read, but they don't remember deadlines across other triggers:
reconnect, resume, live events and the unread backstop can each send the
read again later, bounded by the 20 s statement deadline. A live
subscription that hits the deadline resubscribes with its original
filter under the existing 1–30 s backoff, and against a filter that
keeps timing out it retries at the 30 s cap indefinitely, so the worst
case is one bounded query per subscription every 30 s. `since` is
deliberately not advanced to now on resubscribe, because that would skip
events missed while the subscription was closed. Unchanged from `main`:
when mobile's channel history sync fails with cached messages available,
the cached messages are shown without the error or Retry control.
### Benchmark
Local reproduction on an isolated bench stack (not committed):
- PG16, migrations applied, one 188-reply thread plus 5M background
events in the same community.
- Each request replays the desktop thread-replies filter through the
relay, with `include_aux` on. Times are total ms for 3 runs, with a 60 s
client cap.
GIN pending-list state is controlled with `VACUUM` (clean) or by
refilling the pending list (dirty), and recorded per partition with
`pgstatginindex`. Dirty runs held 402 pending pages on every partition's
tags index at the start and end of each matrix. Clean runs held 0–1
pages (relay startup writes).
| build | GIN pending state | limit 10 | 50 | 100 | 200 (188 ids) |
|---|---|---|---|---|---|
| `main` | clean | 26 / 16 / 13 | 56 / 51 / 52 | 92 / 93 / 95 | 662 /
639 / 641 |
| `main` | dirty | 315 / 298 / 327 | 22141 / 15888 / 16769 | 28992 /
27819 / 28080 | timeout ×3 (>60 s) |
| this PR | clean | 35 / 21 / 14 | 57 / 49 / 48 | 82 / 80 / 75 | 153 /
150 / 135 |
| this PR | dirty | 290 / 254 / 251 | 1257 / 1204 / 1206 | 2432 / 2378 /
2382 | 4333 / 4706 / 5104 |
Baseline `origin/main` `b37e47721`; this PR at `5f2650f5f`, whose e-tag
read builders are identical to the current head. At head `1ad81d868`
against `origin/main` `15d44dcce`, limit 200 measured 145–179 ms clean
(`main` 683–715) and 4.41–4.51 s dirty (`main` timeout ×3).
Every completed cell returns the same signed event set: 209 events, 187
replies at limit 200. With `include_aux` off, every cell is ≤66 ms. The
thread-window loader (`select_aux`) is unchanged from `main`, and its
timings and captured plans match `main`'s.
Actual aux-hop statements captured from each relay binary with
`auto_explain` (analyze + buffers), execution ms:
| build | clean, 100 ids | clean, 188 ids | dirty, 100 ids | dirty, 188
ids |
|---|---|---|---|---|
| `main` (N-way `OR`) | 40 | 298 | 28785 | 62892 |
| this PR (`MATERIALIZED` fence + `ANY`) | 23 | 45 | 1992 | 3539 |
**What the fence changes**
- On `main`, the planner combines `ORDER BY created_at … LIMIT` with an
overestimated match count and walks each partition's primary key
backward, expecting to stop early. With few real matches it scans the
whole community: the dirty 188-id statement touches ~4.9M shared buffers
on `main` vs ~0.6M with the fence.
- The fence prevents that early-stop plan. Across the measured cases
(1–188 ids, both pending states), the planner used the GIN tags index.
The fence doesn't force GIN in general; a broad enough filter could
still get a different plan.
**Tradeoff:** the fence materializes every match before sorting. In this
bench small limits stayed within run-to-run noise of `main`. A single
hot id (5,000 reactions, limit 100) measured 30–34 ms unfenced vs 35 ms
fenced.
**Deadline:** each e-tag read (SELECT and COUNT) runs under a
transaction-local `statement_timeout` of 20 s. It only tightens: a
shorter operator timeout (`BUZZ_DB_STATEMENT_TIMEOUT_MS`) is kept. It's
a per-statement limit, not a per-request one, so a request that runs
several pages or hops can take longer than 20 s in total. The benchmark
does not force the deadline.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
Buzz Mobile
Flutter mobile client for Buzz.
See VISION_MOBILE.md for intended behavior and architecture.
Setup
Use the Flutter SDK pinned by the repository. Activate Hermit from the repo root before resolving packages or running any Flutter command:
cd /path/to/buzz
. ./bin/activate-hermit
./bin/just mobile-install
mobile-build-android intentionally builds with --no-pub. If an IDE or an
external Flutter SDK has touched mobile/.dart_tool, rerun mobile-install
with the pinned SDK before building so flutter_test, sky_engine, and the
engine all come from the same Flutter version.
Run
# From repo root (applies a worktree-isolated debug identity and starts/reuses Simulator):
just mobile-dev
# Direct (uses the app's configured community; apply worktree overrides first):
cd mobile && flutter run --dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
Worktree-aware debug identity
Debug builds produced from a git worktree get a unique app identifier keyed
to the worktree directory name
(xyz.block.buzz.dogfood.mobile.<slug> on iOS,
xyz.block.buzz.mobile.<slug> on Android) plus a display-only branch label
in the app name (Buzz (my-branch), or a short SHA when the worktree is
detached). Because the identifier follows the directory rather than the
branch, one worktree keeps exactly one installed app — and its login state —
across branch switches, and builds from multiple worktrees install side by
side, mirroring the desktop dev experience. Release and profile builds
always keep the production identity and name.
just mobile-dev and just mobile-build-android apply this automatically by
running scripts/mobile-worktree-overrides.sh, which writes two gitignored
files:
mobile/ios/Flutter/WorktreeOverrides.xcconfig(included by Debug builds only; a developer'sAppOverrides.xcconfigis included after it, so app-specific overrides like a personalBUNDLE_IDENTIFIERfor device signing always win)mobile/android/worktree.properties(read by the debug build type only)
Android developers can keep a stable local test identity that takes precedence
over the generated worktree values by creating the gitignored
mobile/android/AppOverrides.properties:
appName=Buzz Pairing
applicationIdSuffix=.device_pairing_e2e1
These values are consumed by the debug build type only. The standard
just mobile-build-android command can still be used; regenerating
worktree.properties does not overwrite AppOverrides.properties. Release
and profile builds keep the production Buzz name and application ID.
For direct Xcode / Android Studio / flutter run development, run
./scripts/mobile-worktree-overrides.sh from the repo root once per branch
switch to refresh the display label (the install identity never changes);
the persisted files are then picked up by any subsequent build. In the main
checkout the script is a no-op that removes stale override files, restoring
the plain Buzz identity. To enable push in direct Xcode builds and Runner tests, supply a
BUZZ_PUSH_GATEWAY_URL build setting in the gitignored
mobile/ios/Flutter/AppOverrides.xcconfig; the build phase validates and
passes it through as a Flutter Dart define. Since // begins an xcconfig
comment, spell the origin as BUZZ_PUSH_GATEWAY_URL = https:/$()/push.example.
For an Android debug build that must remain installed alongside other Buzz worktree builds, set an explicit launcher name and package suffix when invoking the generator or a recipe that invokes it:
BUZZ_PUSH_GATEWAY_URL="https://push.example" \
BUZZ_ANDROID_DEBUG_APP_NAME="Buzz Huddles" \
BUZZ_ANDROID_DEBUG_ID_SUFFIX=".huddles_829c" \
./bin/just mobile-build-android
This example produces the debug-only package
xyz.block.buzz.mobile.huddles_829c with the launcher label Buzz Huddles.
The suffix must start with a dot followed by a lowercase letter and may contain
only lowercase letters, digits, and underscores. Release and profile builds
ignore these overrides and retain the production package and name.
To remove leftover worktree-suffixed installs from booted iOS simulators and
connected Android emulators, run just mobile-clean (add --dry-run via
./scripts/mobile-worktree-clean.sh --dry-run to preview). Production
installs are never touched.
iOS push capability
Every iOS artifact builds and embeds the Notification Service Extension and
native push bridge. Runtime activation is fail-closed and scoped to the current
relay. After authenticated connectivity and a fully valid NIP-11 nip-pl push
descriptor, Buzz independently requests display permission and registers with
APNs. Display denial or request failure does not gate the device token, gateway
enrollment, or lease publication, so a later user opt-in can display pushes
without rebuilding transport authority. An absent, malformed, or unreachable
descriptor leaves push inactive without partial enrollment.
Mobile builds without a gateway origin succeed with push unavailable. To enable push, supply the gateway origin explicitly:
flutter build ios --dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
flutter build apk --dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
The iOS and Android build gates validate any supplied define, rejecting empty or malformed values. Release/profile builds require an HTTPS origin without an explicit port. An absent define disables permission requests, APNs registration, gateway enrollment, and lease publication; Settings shows push as unavailable. No production gateway is selected implicitly. Enrollment grants and crash-recovery journals are scoped to this origin. Push has not shipped to existing users, so there is no legacy-state or cross-gateway migration. Changing gateways requires fresh enrollment; old installations expire under their original gateway's lease policy. Current-gateway response loss is still retried from the exact journaled request.
Relay rollout remains an explicit deployment opt-in. Only deployments with
BUZZ_PUSH_ENABLED=true advertise the descriptor and process push. See
docs/push-gateway-deployment.md for the canonical gateway profile contract,
manual physical-device proof, measurements, and rollback procedure.
For local physical-device development, override the identity and sandbox
environments in the gitignored mobile/ios/Flutter/AppOverrides.xcconfig:
BUNDLE_IDENTIFIER = com.example.buzz.mobile
BUZZ_DEVELOPMENT_TEAM = YOUR_TEAM_ID
BUZZ_IOS_PUSH_ENVIRONMENT = development
BUZZ_APP_ATTEST_ENVIRONMENT = development
BUZZ_PUSH_GATEWAY_URL = https:/$()/push.example
Use your personal bundle ID and team above. Provision both the parent and its
.NotificationService extension. Configure an isolated gateway with the matching
App Attest application ID, APNs topic and sandbox certificate. Development
attestation requires the explicit personal-dev-app-attest gateway build feature
and BUZZ_PUSH_APP_ATTEST_ENVIRONMENT=development; ordinary gateway builds accept
production attestation only. See docs/push-gateway-deployment.md.
This validates the personal client/relay/gateway integration, not the internally
distributed dogfood artifact. Validate dogfood separately using the signed
internal release and its production gateway configuration.
Parent app identifiers require Apple's Communication Notifications capability and a regenerated app provisioning profile. The Notification Service Extension profile does not require that capability. Enable it on the personal development App ID for local rich-presentation validation. Enabling it on the Block dogfood and eventual App Store App IDs is a release follow-up and is not performed by this repository change. Without a matching parent profile, source and unit validation still work, but the app cannot be signed for a physical device.
APNs and the gateway continue to carry only the constant opaque wake-up. The extension fetches the message from the scoped relay, verifies message, sender profile, and channel-metadata signatures, and uses a bounded App Group cache for names and app-rendered avatar thumbnails. It never fetches an avatar URL; missing, stale, or invalid enrichment falls back to the verified message with a short sender pubkey, community subtitle, and no image.
Checks
dart format --output=none --set-exit-if-changed .
flutter analyze
flutter test --dart-define=BUZZ_PUSH_GATEWAY_URL=https://push.example
Or from the repo root: just mobile-check and just mobile-test.
Android release signing
Android release builds fail unless all upload-key inputs are supplied through the environment:
BUZZ_ANDROID_UPLOAD_KEYSTORE_PATH: path to a CI-vended keystore fileBUZZ_ANDROID_UPLOAD_KEYSTORE_PASSWORDBUZZ_ANDROID_UPLOAD_KEY_ALIASBUZZ_ANDROID_UPLOAD_KEY_PASSWORD
The keystore path must be absolute, and the keystore must remain outside the repository. Development and debug builds do not require these variables.
Release pipelines that sign through the central APK Signer service instead of
a local upload keystore must set BUZZ_ANDROID_RELEASE_SIGNING=external. That
mode produces an unsigned release bundle and refuses to run if any
BUZZ_ANDROID_UPLOAD_* value is also set.
Architecture
lib/
├── main.dart # Entry point, Riverpod bootstrap
├── app.dart # MaterialApp with theme
├── shared/
│ └── theme/ # Catppuccin light/dark, spacing tokens, extensions
└── features/
└── home/ # Placeholder home surface
- State management: Riverpod + Hooks (
HookConsumerWidget) - Theme: Catppuccin Latte (light) / Macchiato (dark) — matches desktop
- Spacing:
Gridtokens for consistent spacing - Linting:
flutter_lints+riverpod_lintviacustom_lint - Feature isolation: No cross-feature imports except
shared/