Files
buzz/PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md
Jemiah Westerman cec47c20a3 perf(db): hydrate search hits with exact event timestamps (#9126)
## Summary

Keep each search hit's existing signed `created_at` alongside its event
ID during canonical hydration. WebSocket search, HTTP search, and
profile-name search now use a bounded exact-pair database lookup so
PostgreSQL can prune unrelated event months. Also constrain
channel-create replay detection by the timestamp of the
already-available signed event.

## Scope / compatibility

- Combined-tree verification also passed: repository-wide `just ci`, 220
unit/source-policy tests, and the full 722-test buzz-db/search
PostgreSQL suite. All three temporary timestamp-constraint mutations
were killed by functional/plan assertions and restored.
- Independent of companion post-insert push and deletion PRs; based
directly on `main`.
- Deploy/code-only change. No schema/migrations/backfills, persistent or
serialized formats, indexes, triggers, functions, procedures, rules, or
foreign keys added.
- Existing ID-only event APIs remain available for callers without
timestamps.
- Maximum 500 input pairs, validated before checkout; duplicate inputs
do not multiply results; missing/deleted/wrong-community/wrong-timestamp
hits are omitted.
- Caller ordering, authorization, observability, bounded-only replica
routing, proved connection use, non-cancel fallback, and cancellation
propagation remain intact. A timestamp is not treated as a completeness
proof.
- Existing/new relays coexist against unchanged schema; no
rollout/prebuild dependency.

## Verification

- Compiled `buzz-db`, `buzz-search`, and `buzz-relay` library/tests.
- 220 infrastructure-free unit/source-policy tests passed.
- 35 focused PostgreSQL tests passed, including cross-month exact pairs,
tenant/deleted/missing scope, same-second IDs,
duplicate/empty/invalid/oversized/full batches,
routing/fallback/cancellation, channel replay, and existing search
regressions.
- `EXPLAIN ANALYZE BUFFERS` of production hydration SQL verifies only
requested event months execute under forced custom **and generic**
plans, including the full **500-distinct-pair** shape. A wrong-timestamp
decoy detects cross-product matching.
- Targeted clippy with `-D warnings`, formatting, diff checks,
PostgreSQL discovery, and differential file-size checks passed.
- Live isolated relay smoke: exact channel-create replay, HTTP NIP-50
search hydration, profile-name search hydration, and authenticated
WebSocket NIP-50 results/EOSE. All three hydration route labels observed
in local metrics.
- Independent review completed; optional maximum-batch plan coverage
added and independently re-reviewed. Compile fixes for SQLx 0.9 API
compatibility verified.

No production latency improvement or workload-percentage claim is made.

---------

Signed-off-by: Jemiah Westerman <jemiah@squareup.com>
Co-authored-by: Son of Anton <8e0ef2ab02d69a7ac089ef5eca90d1a3fbbdc546f6322dc34d0c4f3deecea7f4@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 15:43:28 -08:00

4.5 KiB

title, tags, status, created
title tags status created
Replica Full-Read Routing — Caller Classification
relay
replica-routing
consistency
active 2026-08-28

Replica Full-Read Routing — Caller Classification

Db::query_events always reads the writer pool. Db::query_events_routed (and its _bounded / count / feed siblings) opt a read into replica routing: the read may be served from a read replica when BUZZ_REPLICA_READ_MAX_AGE_MS is set, subject to the soundness predicate RoutePredicate::for_query derives from the query shape. The seam fails closed to the writer on replica errors, with one deliberate exception: a statement cancelled by statement_timeout (SQLSTATE 57014) is surfaced rather than re-run, because the writer would run the same slow statement. A busy reader is the other exception: a subscription-history read that cannot get a reader connection within its bounded wait is rejected as retryable instead of moving to the writer, and history reads are capped per pod below the reader pool size (Db::route_read). It is a genuine no-op until the budget is configured (crates/buzz-db/src/lib.rs).

Routing rule

If a read's result influences a write or a permission decision, it reads from the writer. A replica can lag behind the caller's own just-committed write; a read that gates the next write against that lag would make a wrong decision (e.g. validate a save against a stale head, or miss the caller's own accepted event during post-write verification and report a false conflict). Every such read stays on query_events / writer.

Display reads — a page the user scrolls, a count on a badge, a history list — tolerate bounded staleness and take the routed path.

Adding, removing, or reclassifying a caller requires updating the table below; the query_events_routed doc-comment points here.

Client-carried intent: the consistency extension field

Some write-influencing reads are issued by clients (Desktop, CLI) through the HTTP /query bridge, and are indistinguishable by query shape from display reads — a kind-40100 limit:1 read serves both get_canvas (display) and a canvas save's head precondition. The client therefore signals intent with the consistency extension field on the raw filter:

  • "consistency": "strong" → the bridge serves that filter from the writer (query_events), never a replica.
  • absent → the default routed path.
  • any other value → 400 Bad Request (fails loud, never silently degrades).

The field only ever forces the writer, which is always the sound direction; there is deliberately no inverse "force replica" value, so it cannot be sprayed to bypass the replica-staleness guard on reads that should not. Parsed in crates/buzz-relay/src/api/bridge.rs (extract_consistency).

Caller classification table

Rows below are every query_events_routed / query_events_routed_bounded call site at head, plus the writer-pinned canvas row this change adds. (The count and feed routed families — count_events_routed, get_events_by_ids_at_routed, query_feed_*_routed, and the get_channel_window cursor/head reads — are all display or bounded-count surfaces on the routed path; they carry their own soundness notes at their definitions in crates/buzz-db/src/lib.rs and are out of scope for this table.)

Caller / path label Pool Justification
bridge_query (default /query filter) routed Display reads over the HTTP bridge; bounded staleness acceptable.
bridge_query + consistency: strong writer Client-declared write-influencing read (canvas save precondition, restore precondition, post-write ancestry verification).
req_historical (WS REQ historical page) routed Display backfill of a subscription; per-row re-filter absorbs a briefly-stale row.
bridge_thread_aux (AuxReader::Routed, thread aux page) routed Thread reply hydration; display, post-verified against the fence wall.
bridge_count_fallback (query_events_routed_bounded) routed (bounded arm) COUNT fallback that materializes rows; bounded arm only, never covered.
count_req_fallback (query_events_routed_bounded) routed (bounded arm) WS COUNT fallback that materializes rows; bounded arm only.

The writer row is the only write-influencing entry; every other caller is a display or count surface that tolerates bounded staleness. Client canvas reads that gate a write set consistency: strong (Desktop desktop/src-tauri/src/commands/canvas.rs, CLI crates/buzz-cli/src/commands/channels.rs) so they land on the writer row.