mirror of
https://github.com/block/buzz.git
synced 2026-11-04 16:31:38 +00:00
a9aa1f73b0
## Why Compatible clients need incremental artifact reads for local collections on every Buzz relay. ## What Expose incremental artifact reads and capability discovery on every Buzz relay. ## How Serve incremental queries through the existing authentication and authorization checks. Keep the existing work budgets and the separate WebSocket sync capability. The HTTP test verifies capability discovery and a successful incremental query. ## Risk The relay starts no additional background work. Clients that use incremental reads consume database capacity. The net database load is not yet measured. Serving writers must maintain the modification timestamps. ## Testing Read-only SQL checks on November 3, 2026 verify the prerequisites below. Each database records successful migrations 76–78. All five required columns have the expected types, defaults, and nullability. All five indexes have the expected definitions and are valid, ready, and live. | Deployment | Schema and indexes | Running timestamp writers | | --- | --- | --- | | Production `bb-block` | Pass | `f6f6d9a` includes PR 8523 | | Production `bb-public` | Pass | `f3a7bab` includes PR 8523 | | Staging `bb-block` | Pass | `b4509de` includes PR 8523 | The checked columns are `artifact_heads.modified_at`, `artifact_revisions.modified_at`, `artifact_revisions.source_channel_id`, `channel_members.sync_modified_at`, and `communities.artifact_policy_modified_at`. The checked indexes are `idx_artifact_revisions_artifact`, `artifact_heads_modified`, `artifact_heads_channel`, `channel_members_sync`, and `artifact_revisions_moves`. At commit `62b26ad`, all 21 NIP-11 tests and the HTTP capability test pass on Linux with PostgreSQL 17 and Redis. The HTTP test checks capability discovery and a successful incremental query. All required push hooks also pass. No manual testing. Human confirmation remains pending. ## Bigger picture [Client PR 1507](https://github.com/block/buzz-app/pull/1507) uses the advertised capability for local artifact sync. Existing deployments receive this behavior when they deploy the relay version that includes this PR. Generated with Codex --------- Signed-off-by: Daniel Rouleau <drouleau@squareup.com>
60 lines
2.7 KiB
Markdown
60 lines
2.7 KiB
Markdown
# Incremental artifact reads deployment
|
|
|
|
Incremental artifact reads use a 120-second overlap and database modification
|
|
timestamps. The schema, indexes, and every serving writer must support
|
|
incremental reads.
|
|
|
|
## Schema and indexes
|
|
|
|
Migration `0077` adds columns with constant defaults, then changes the defaults
|
|
for future inserts. It does not rewrite existing rows or run a backfill.
|
|
Old rows remain at the epoch timestamp. An initial read includes them.
|
|
New application writers stamp every head change, grant, rejoin, removal,
|
|
redaction, and move source. Changes to channel listing also require recovery. Old application updates do not maintain all these
|
|
stamps, so mixed-version serving cannot advertise incremental correctness.
|
|
|
|
For a populated database, first build the revision lookup index outside a
|
|
transaction, before migration `0076`:
|
|
|
|
```sql
|
|
CREATE INDEX CONCURRENTLY idx_artifact_revisions_artifact
|
|
ON public.artifact_revisions (community_id, artifact_id);
|
|
```
|
|
|
|
Apply `0076_artifact_revision_lookup.sql` in a transaction after that prebuild.
|
|
Then apply `0077_artifact_sync_timestamps.sql` in a
|
|
transaction with its lock and statement budgets. The migration is idempotent
|
|
for this prebuild step. Then build these indexes outside a transaction:
|
|
|
|
```sql
|
|
CREATE INDEX CONCURRENTLY artifact_heads_modified
|
|
ON public.artifact_heads (community_id, modified_at, artifact_id);
|
|
CREATE INDEX CONCURRENTLY artifact_heads_channel
|
|
ON public.artifact_heads (community_id, channel_id, artifact_id);
|
|
CREATE INDEX CONCURRENTLY channel_members_sync
|
|
ON public.channel_members (community_id, pubkey, sync_modified_at, channel_id);
|
|
CREATE INDEX CONCURRENTLY artifact_revisions_moves
|
|
ON public.artifact_revisions (community_id, modified_at, event_id)
|
|
WHERE source_channel_id IS NOT NULL;
|
|
```
|
|
|
|
Migration `0078` verifies each index definition and its valid, ready, and live
|
|
flags. It refuses a populated table whose index is missing. A fresh empty
|
|
database can build the indexes under the migration's short budgets.
|
|
Migration `0076` also verifies the exact definition and all three flags.
|
|
Remove an invalid index with `DROP INDEX CONCURRENTLY` before a new build.
|
|
|
|
After all index builds succeed, run `buzz-admin migrate` with the target
|
|
`DATABASE_URL`. It verifies and records `0076`, reapplies the idempotent `0077`,
|
|
then verifies and records `0078`. Direct SQL application does not record the
|
|
migration ledger.
|
|
|
|
## Serving fleet
|
|
|
|
All instances must use the same configured relay signing key, because that key
|
|
authenticates the progress tokens. NIP-11 advertises `artifacts.incremental`.
|
|
Every serving writer must maintain the modification timestamps.
|
|
|
|
The 120-second stamp-to-visibility bound remains an operating assumption.
|
|
Runtime measurements are tracked separately.
|