jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(relay): add atomic complete read-state snapshots (#7572)

Browse Source 
Opened by Brain on behalf of Wes (GitHub: wesbillman).

## Summary

Add an opt-in, host-scoped completeness primitive for NIP-RS read-state
clients. Ordinary query arrays and WebSocket behavior remain unchanged.

- Advertise a version-1 `read_state_snapshot` capability in NIP-11 only
when the request resolves a community.
- Accept exactly one self-author kind-30078 snapshot filter on
authenticated `POST /query`, after the existing admission, replay and
membership gates.
- Read all retained, nondeleted self-author kind-30078 coordinates from
the writer at one SQL-statement MVCC cut—including unrelated application
coordinates and events without a read-state tag.
- Return a complete versioned envelope bound to community, signer and a
content-identity hash. Reject malformed contracts,
reconstruction/signature/storage failures and whole-request overflow;
never return a truncated complete response.
- Bound snapshots at 4,096 events and 8 MiB of compact encoded
event-array bytes, advertised as `max_event_array_bytes`; envelope
metadata is additional. Retain the separate stored-payload preflight
budget.

## Safety and scope

This is point-in-time enumeration, **not** a live freshness guarantee,
cursor, CAS token, cross-relay transaction, or unread total. NIP-RS now
explicitly requires fresh complete enumeration from every write relay
before each override action, canonical publication, carry-forward or
deletion. It does not enable synchronized manual-unread overrides in any
client. No schema migration or deployment is included.

## Validation

Executed against the source committed as
`1fd5a5e0c0d893c1259386885517c0c44ae7630f`:

- Four real-router/PostgreSQL snapshot cases: beyond-page-cap
enumeration and overflow; corruption/storage failure; production
payload-bound NIP-98, Redis replay guard and membership enforcement;
host discovery/scope/strict request shape.
- Two database cases: replacement during preflight preserves a single
statement cut; encoded-byte overflow and writer-not-replica routing. The
cluster-wide probe uses the required `cluster_global_` serialization
prefix.
- Production-call-site mutations killed for dispatch, strict filter
validation, discovery tenant, snapshot tenant, encoded-byte guard,
membership enforcement and replay enforcement; isolated mutation tree
restored.
- Both affected crates pass all-target Clippy; formatting, differential
file-size and repository PostgreSQL test-discovery checks pass.
- Normal pre-push gates pass without bypass: branch skew, push-head
scope, file size, Rust unit lanes and desktop Tauri checks.
- Independent read-only review by Pinky found no remaining relay blocker
at this commit. The execution/mutation results above were run by Brain,
not duplicated by the reviewer.

Hosted CI and maintainer review remain separate gates. No production
traffic or deployment was used for these fixture tests.

Originating Buzz channel: `a394ecdb-9c67-4c4e-a3c6-4a95f69e8cc8`
(unread-but-good), thread
`2413d0ba7fa5b2ac5c777df6b3754e2099b042ec8fb96ca64302a0919a0550b3`.

## Review repair (2026-09-21)

Carl updated this existing branch to
`2f9c9b11266c24947412cf28cefae4e96e7167b7`, merging main
`5079c770fe30bb3d8204822ce6c2431eacac6d4b` without rewriting published
history.

- Resolved the DB export conflict, preserving both snapshot and
storage-accounting exports.
- Addressed the blocking review using its accepted alternative: rename
`max_bytes` to `max_event_array_bytes` and explicitly document that
success-envelope metadata is outside the array budget. No auth, query,
runtime-limit, or client-behavior redesign.
- Added a PostgreSQL/router regression proving discovery, exact
8,388,608-byte array success with a larger envelope, and one-byte-over
rejection (413, no partial complete response).

Exact-head validation: all seven snapshot PostgreSQL/router tests passed
in an isolated local database; both affected crates passed all-target
Clippy; mandatory pre-commit/pre-push gates passed without bypass (Rust
unit lanes, Tauri default/mesh Clippy and tests, differential file size,
branch-skew/ref checks). Reverting only the descriptor rename made the
new regression fail before the final commit. Independent bounded
source-delta review by Mordecai found no blockers; execution was Carl's.

A broader `cargo test -p buzz-db -p buzz-relay --all-targets` run on the
merged candidate passed 131 DB tests and 1,057 relay tests, then
encountered the same unrelated
`mesh_demo::demo_join_forwarded_arm_round_trips_echo` 504 recorded in
the prior review. Full repository `just ci` / `just test` were not rerun
locally; hosted CI remains a separate gate. No deployment or production
traffic was exercised.

---------

Signed-off-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Signed-off-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz>
Co-authored-by: Brain <1a02c72794dcd0f07058a353bc3a81f4028b8c77c92c87fce6d5c8b85970a20b@buzz.block.builderlab.xyz>
Co-authored-by: Carl <32a2e2c9d428ee08902cab75d956da2c1d235a22d4766b0dd4138bf6e2e5db1d@buzz.block.builderlab.xyz>
This commit is contained in:
Wes
2026-09-21 11:30:03 -07:00
committed by GitHub
parent a3117a4762
commit 6a86531ac9
8 changed files with 838 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

95
docs/nips/NIP-RS.md
View File

@@ -378,6 +378,97 @@ Until a complete load succeeds, the client MUST evaluate unread state from its o
The number of coordinates a full-state load must retrieve is bounded by the number of installations that have ever used the override layer, plus their not-yet-deleted rotation predecessors. It grows with the user's device history, not with elapsed time, and — because a coordinate carrying `ov_*` entries may not be deleted until it has been carried forward (see Client-ID Rotation) — it does not shrink on its own. Clients SHOULD carry forward and delete rotation predecessors promptly so the count stays near one coordinate per live installation.
#### Buzz Atomic Snapshot Extension (version 1)
A relay MAY offer a complete **point-in-time** alternative to the paginated
full-state load above through the existing authenticated HTTP Nostr query bridge.
This extension does not change kind 30078, encryption, register algebra, or the
ordinary `POST /query` array response. It does not assert the five WebSocket
conformance properties above.
Discovery: obtain NIP-11 from the configured, trusted relay origin. Its optional
`read_state_snapshot` descriptor contains `version: 1`, `community_id` (canonical
UUID string), `max_events: 4096`, and `max_event_array_bytes: 8388608`. The byte
budget applies only to the compact JSON `events` array, not the complete HTTP
response; clients MUST allow additional space for the success envelope metadata.
The community identifier is resolved from that request's Host through the active
community map; it is not supplied by the client or derived from its key. An
unresolved host omits the descriptor. This is TLS/origin-bound discovery, not an
independently signed global community identity. Clients MUST bind discovery and
queries to the same trusted relay origin and community session, and fence
asynchronous results on identity or community changes.
Request: NIP-98 authenticated `POST /query` with exactly one raw filter:
```json
[{"kinds":[30078],"authors":["<authenticated-self-hex>"],"read_state_snapshot":1}]
```
No extra keys, tags, time bounds, pagination, other kinds/authors, or mixed filters
are accepted. Admission, replay protection, and relay membership gates apply as
for ordinary bridge reads. The relay never substitutes an agent owner's identity
for the authenticated signing identity.
Success is a versioned envelope, **not an event array**:
```json
{
"read_state_snapshot": 1,
"complete": true,
"community_id": "<host-resolved-uuid>",
"pubkey": "<authenticated-self-hex>",
"snapshot_id": "<64-lowercase-hex>",
"events": []
}
```
`events` contains every non-deleted own-author kind-30078 event retained at one
writer-database MVCC statement cut, including unrelated application coordinates
and events without a `t` tag. There is no ordinary query cap, age window, replica
routing, or multi-request pagination. A coordinate replaced during the statement
is consistently represented by the version visible at its cut; a later snapshot
sees the replacement. This guarantee covers retained current state, not purged
history, uncommitted writes, or future commits.
`snapshot_id` is SHA-256 of the concatenation of UTF-8
`buzz-read-state-snapshot-v1` plus a zero byte, the 16 raw community UUID bytes,
the 32 author bytes, and each 32-byte event id in `created_at DESC, id ASC` order.
It is a content identifier, **not** a monotone write revision, CAS precondition,
or live-delivery cursor. An unchanged view may have the same identifier.
Both event count and bytes are hard bounded. More than 4096 events, more than
8 MiB of stored content plus PostgreSQL JSON tag text, or more than 8 MiB of
compact encoded event-array bytes fails the whole request (HTTP 413). No partial
envelope is returned. Row reconstruction, signature verification, serialization,
or database failure produces an explicit non-success response (HTTP 503 after
authentication). Invalid extension requests produce HTTP 400; normal auth,
unknown-host, membership, replay and admission errors retain their bridge status.
Clients MUST treat **all** such errors as `cannot prove complete`. There is no
permission to delete old coordinates to get under the snapshot cap.
Clients MUST reject absent discovery, version mismatch, ordinary arrays (including
an old relay silently ignoring the extension), missing/false completeness,
community/pubkey mismatch, invalid signatures, duplicate/conflicting coordinates,
or invalid event envelopes. Read-state selection/decryption is client-side. An
unreadable or unsupported recognized read-state coordinate MUST block override
actions/publication; it MUST NOT be interpreted as absent state.
A validated snapshot substitutes for the paginated enumeration and mutation fence
**only for its point-in-time load**. Before **each** override action, canonical
publication, or carry-forward/deletion decision, obtain a fresh snapshot from
**every** write relay and merge those snapshots with durable local state. Failure
on any required relay blocks that operation. Do not publish own coordinates while
loading. Ordinary read-before-write, primary co-location, permanent clear floors,
uint32/budget exhaustion, and per-relay acceptance-before-deletion rules remain.
A failed/ambiguous publish does not report synchronized action success.
Actions after different snapshot cuts are concurrent CRDT actions, not serialized
transactions. A snapshot does not reserve a coordinate, prevent another writer,
or make subsequent live delivery lossless. Clients MUST NOT cache `complete:true`
as a session-wide authority or use it to label live freshness. On reconnect,
identity changes, or later canonical publication, acquire a new snapshot; merge
arriving live state as evidence but do not infer a complete load from it.
### Merge Rule
After decrypting all fetched blobs, the effective read timestamp for each context is:
@@ -783,9 +874,9 @@ that expose read activity to other users MUST require explicit user consent.
## Backwards Compatibility
This NIP introduces no changes to existing event kinds and adds no new kind, wire message, or relay-stored read-state logic. It uses only standard NIP-01 event storage, NIP-33 addressable event semantics, NIP-44 encryption, and NIP-78 application data conventions. Clients that do not implement this NIP are unaffected, as are clients that implement everything but the manual-unread override layer.
This NIP introduces no changes to existing event kinds or relay-stored read-state logic. The baseline uses only standard NIP-01 event storage, NIP-33 addressable event semantics, NIP-44 encryption, and NIP-78 application data conventions. The optional Buzz Atomic Snapshot Extension adds explicitly discovered, versioned query/discovery envelopes; ordinary query arrays and WebSocket messages remain unchanged. Clients that do not implement this NIP are unaffected, as are clients that implement everything but the manual-unread override layer.
The override layer is the exception, and it is a relay-compatibility one rather than a client one. Its full-state load carries the completeness guarantee only against a relay that satisfies the ordering, capacity, floor, push, and barrier requirements enumerated in Full-State Load. Against a relay known or evidenced not to conform, every load resolves to *cannot prove complete* and the actions that depend on a complete load report as failed; against an undetectably nonconforming relay, a load may still return *complete*, and the completeness guarantee does not apply to that verdict. In either case the layer still runs and still merges, and frontier sync is unaffected.
The override layer is the exception, and it is a relay-compatibility one rather than a client one. Its paginated full-state load carries the completeness guarantee only against a relay that satisfies the ordering, capacity, floor, push, and barrier requirements enumerated in Full-State Load. A validated atomic snapshot supplies the alternative point-in-time guarantee described in Buzz Atomic Snapshot Extension, without asserting those WebSocket properties. Without either proof, every load resolves to *cannot prove complete* and the actions that depend on a complete load report as failed; against an undetectably nonconforming relay, a load may still return *complete*, and the completeness guarantee does not apply to that verdict. In either case the layer still runs and still merges, and frontier sync is unaffected.
## References