feat(relay): push per-community storage usage to kgoose (#8714)

### Why?
To bill hosted communities for storage in Orb, kgoose needs
per-community storage usage. The relay already computes it in a daily
snapshot, but nothing delivered it anywhere.

## Summary
- Adds opt-in `BUZZ_STORAGE_USAGE_URL`. When it is set, one
leader-elected relay pod checks every 5 minutes for a new storage
snapshot and POSTs it to kgoose's `POST /v1/buzz/storage-usage/record`
(squareup/cash-server#144285) in consecutive batches of up to 50
communities until every community is sent.
- Each batch is `{measured_at, communities: [{community_host,
storage_bytes, storage_objects}]}`. Active communities with no stored
bytes are reported as zero, and re-sends are safe because kgoose bills
each community once per UTC day.
- Retries on 408, 429, 5xx, and transport errors. No migration;
documented in `docs/storage-accounting.md`. kgoose authenticates the
relay as the `buzz` mesh service, so the relay needs mesh egress to
kgoose-builderbot.

### Related issue
None found. Complements #7899 (snapshot history in the DB). Receiving
endpoint: squareup/cash-server#144285.

### Testing
Unit tests cover batching, retry and reject handling, zero-filling, and
a real HTTP delivery that checks the exact kgoose request body. A
Postgres-gated DB test covers the new reads. End to end, a local relay
delivered to a fake endpoint once per new snapshot.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Shu Hui <shu@squareup.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Shu Hui
2026-11-04 09:40:06 +11:00
committed by GitHub
parent 8312419652
commit fa7a3bfbda
9 changed files with 1180 additions and 0 deletions

View File

@@ -54,3 +54,55 @@ The reader preserves leader-only emission, community scope filtering, and
cleanup of series for renamed or removed communities. It does not need S3
listing permission. The worker still needs that permission and its database
configuration.
## Usage delivery
Set `BUZZ_STORAGE_USAGE_URL` to push each new snapshot's per-community totals
to an operator endpoint. kgoose's `POST {prefix}/v1/buzz/storage-usage/record`
receives them and bills sponsored communities in Orb. Unset, nothing is sent.
Every five minutes, one relay pod (elected by a PostgreSQL session advisory
lock) compares the stored snapshot's `completed_at` with the last one it
handled and, when it is new, POSTs every community in consecutive batches of
at most 50:
```json
{
"measured_at": "2026-10-30T03:04:12Z",
"communities": [
{
"community_host": "acme.communities.buzz.xyz",
"storage_bytes": 123456789,
"storage_objects": 412
}
]
}
```
- `measured_at` is the snapshot's completion time in RFC 3339, the same for
every batch of a snapshot. Each batch is an independent request.
- Each request carries a NIP-98 `Authorization` header signed by the relay
key, binding the URL and body, as operator-listener mention notifications
do. kgoose authenticates the relay by its `buzz` mesh service identity
instead, so the relay needs mesh egress to kgoose-builderbot.
- `storage_bytes` and `storage_objects` are the logical per-community values
described above: a blob bound to several communities counts in full for
each.
- Every snapshot community with a community row is sent, including archived
or deleting ones that still hold bytes. An active community absent from the
snapshot is sent with zero, so a receiver keeping the latest value never
retains a stale one. A snapshot community with no community row has no host
and is skipped with a warning.
- A 2xx accepts a batch. A 408, 429, 5xx, or transport error stops the run and
re-sends the whole snapshot on the next check. Any other status abandons
that snapshot; the next worker run is delivered normally.
- Which snapshot was handled is kept in memory, so a restart or leader change
re-sends the latest snapshot. Receivers MUST treat a repeated community and
`measured_at` as already recorded; kgoose bills only a community's first
report for each UTC day.
- A snapshot replaced before it was delivered is never sent. Alert on
`time() - buzz_storage_usage_delivered_snapshot_timestamp_seconds` alongside
snapshot age; `buzz_storage_usage_deliveries_total{outcome}` counts
`accepted`, `retry`, and `rejected` checks. The leader re-publishes the
delivered timestamp on every check, so it survives the gauge idle timeout
between daily snapshots. After a restart or leader change it has no sample
until the next delivery succeeds, so treat a missing series as failing too.