a9aa1f73b0
## Why Compatible clients need incremental artifact reads for local collections on every Buzz relay. ## What Expose incremental artifact reads and capability discovery on every Buzz relay. ## How Serve incremental queries through the existing authentication and authorization checks. Keep the existing work budgets and the separate WebSocket sync capability. The HTTP test verifies capability discovery and a successful incremental query. ## Risk The relay starts no additional background work. Clients that use incremental reads consume database capacity. The net database load is not yet measured. Serving writers must maintain the modification timestamps. ## Testing Read-only SQL checks on November 3, 2026 verify the prerequisites below. Each database records successful migrations 76–78. All five required columns have the expected types, defaults, and nullability. All five indexes have the expected definitions and are valid, ready, and live. | Deployment | Schema and indexes | Running timestamp writers | | --- | --- | --- | | Production `bb-block` | Pass | `f6f6d9a` includes PR 8523 | | Production `bb-public` | Pass | `f3a7bab` includes PR 8523 | | Staging `bb-block` | Pass | `b4509de` includes PR 8523 | The checked columns are `artifact_heads.modified_at`, `artifact_revisions.modified_at`, `artifact_revisions.source_channel_id`, `channel_members.sync_modified_at`, and `communities.artifact_policy_modified_at`. The checked indexes are `idx_artifact_revisions_artifact`, `artifact_heads_modified`, `artifact_heads_channel`, `channel_members_sync`, and `artifact_revisions_moves`. At commit `62b26ad`, all 21 NIP-11 tests and the HTTP capability test pass on Linux with PostgreSQL 17 and Redis. The HTTP test checks capability discovery and a successful incremental query. All required push hooks also pass. No manual testing. Human confirmation remains pending. ## Bigger picture [Client PR 1507](https://github.com/block/buzz-app/pull/1507) uses the advertised capability for local artifact sync. Existing deployments receive this behavior when they deploy the relay version that includes this PR. Generated with Codex --------- Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
2.7 KiB
Incremental artifact reads deployment
Incremental artifact reads use a 120-second overlap and database modification timestamps. The schema, indexes, and every serving writer must support incremental reads.
Schema and indexes
Migration 0077 adds columns with constant defaults, then changes the defaults
for future inserts. It does not rewrite existing rows or run a backfill.
Old rows remain at the epoch timestamp. An initial read includes them.
New application writers stamp every head change, grant, rejoin, removal,
redaction, and move source. Changes to channel listing also require recovery. Old application updates do not maintain all these
stamps, so mixed-version serving cannot advertise incremental correctness.
For a populated database, first build the revision lookup index outside a
transaction, before migration 0076:
CREATE INDEX CONCURRENTLY idx_artifact_revisions_artifact
ON public.artifact_revisions (community_id, artifact_id);
Apply 0076_artifact_revision_lookup.sql in a transaction after that prebuild.
Then apply 0077_artifact_sync_timestamps.sql in a
transaction with its lock and statement budgets. The migration is idempotent
for this prebuild step. Then build these indexes outside a transaction:
CREATE INDEX CONCURRENTLY artifact_heads_modified
ON public.artifact_heads (community_id, modified_at, artifact_id);
CREATE INDEX CONCURRENTLY artifact_heads_channel
ON public.artifact_heads (community_id, channel_id, artifact_id);
CREATE INDEX CONCURRENTLY channel_members_sync
ON public.channel_members (community_id, pubkey, sync_modified_at, channel_id);
CREATE INDEX CONCURRENTLY artifact_revisions_moves
ON public.artifact_revisions (community_id, modified_at, event_id)
WHERE source_channel_id IS NOT NULL;
Migration 0078 verifies each index definition and its valid, ready, and live
flags. It refuses a populated table whose index is missing. A fresh empty
database can build the indexes under the migration's short budgets.
Migration 0076 also verifies the exact definition and all three flags.
Remove an invalid index with DROP INDEX CONCURRENTLY before a new build.
After all index builds succeed, run buzz-admin migrate with the target
DATABASE_URL. It verifies and records 0076, reapplies the idempotent 0077,
then verifies and records 0078. Direct SQL application does not record the
migration ledger.
Serving fleet
All instances must use the same configured relay signing key, because that key
authenticates the progress tokens. NIP-11 advertises artifacts.incremental.
Every serving writer must maintain the modification timestamps.
The 120-second stamp-to-visibility bound remains an operating assumption. Runtime measurements are tracked separately.