mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
14a752fb24
## Why
The partition manager checks partition existence by name only and treats
Postgres 42P17 "would overlap partition" errors as success. On databases
where a catch-all partition (`events_p_future` /
`delivery_log_p_future`) covers future months, the monthly creation path
silently does nothing while real coverage erodes toward a write outage.
There is currently no visibility into partition catalog health and no
signal before writes start failing.
## What
- **Read-only catalog audit** in `buzz-db` (`partition.rs`): classifies
the live partition catalog for `events` and `delivery_log` via
`pg_inherits` and bound parsing — coverage bounds, catch-all detection,
anomalous children, and trigger parity checked recursively down to
routable leaf partitions.
- **Creation gated on the audit verdict**: the 42P17-to-success swallow
is removed; monthly CREATE runs only for months the audit proves
uncovered, a canonical-name mismatch is an error, and per-month outcome
capture replaces first-error abort. New
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED` kill switch (default `true`).
- **Prometheus metrics** per table: `buzz_partition_serving_safe`,
`buzz_partition_uncovered_months` (audit window is now through +3
months), `buzz_partition_catch_all_covered_months`,
`buzz_partition_anomalous_children`,
`buzz_partition_trigger_parity_missing`/`_extra`,
`buzz_partition_audit_duration_seconds`,
`buzz_partition_audit_runs_total{outcome}`,
`buzz_partition_audit_failures_total`,
`buzz_partition_create_attempts_total`, and
`buzz_partition_audit_last_success_timestamp_seconds` for staleness
alerting.
- **Cached diagnostics**: `/_status.partition_catalog` reports partition
serving-safety, audit age, and freshness. `/_readiness` stays
process-local, consistent with #7341; missing, unsafe, or stale audits
do not remove pods from service.
- **Periodic audit task** in the relay, plus config surface for the
audit interval and the monthly-create kill switch. Catch-alls are
detected from catalog bounds rather than configured names.
- Doc correction: the mesh heartbeat comment no longer claims readiness
gating (it never was gated; behavior unchanged).
## Behavior changes & risk
- No DETACH/DROP/ATTACH and no schema migrations. The only DDL is the
pre-existing monthly CREATE path, now strictly narrower
(audit-proven-uncovered months only) and behind a kill switch.
- Partition audit health does not gate readiness. Until the first
completed audit, read-only retries back off from 5 seconds to a
60-second cap. Any completed audit, including unsafe coverage, restores
the normal interval.
- Partition-overlap errors now surface as errors instead of being
swallowed as success.
- Rollback: revert the commit, or set
`BUZZ_PARTITION_MANAGER_CREATE_ENABLED=false` to disable the creation
path while keeping the audit and metrics.
## Validation
- New Postgres-backed regression tests for catalog classification and
trigger parity (disabled child trigger, `ENABLE ALWAYS` parent, disabled
nested leaf clone in a multi-level layout, healthy-catalog zero-parity
checks). Each new regression was verified to fail against the previous
logic before the fix.
- Local affected-target validation passed against a disposable
current-schema database, with the known mesh forwarded-echo and tracing
callsite failures excluded. This is not a claim that the full suite
passes.
Follow-up work (separate PRs): bounded catch-all advance (empty-only
golden path; non-empty reattach only behind an explicit operator flag,
default off), fresh-install convergence, and proactive coverage-runway
alerting.
---
## Update — 2026-08-21
- Fixed nested-leaf trigger parity auditing so child-only row triggers
are checked on every routable descendant leaf and reported as
`{leaf}:{trigger}` under the owning root child.
- Added a Postgres regression test for a child-only trigger on a nested
leaf; mutation-reverting to immediate-child-only extra-trigger scanning
makes it fail.
Generated with Codex
Signed-off-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Signed-off-by: Ian Oberst <ioberst@squareup.com>
Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: coder 0 <d97ebdbb198c7237c94f84ea8bb8a73583ea067407eebd0062abbb3962527fb1@buzz.block.builderlab.xyz>
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
44 lines
1.4 KiB
TOML
44 lines
1.4 KiB
TOML
[package]
|
|
name = "buzz-admin"
|
|
version.workspace = true
|
|
edition.workspace = true
|
|
rust-version.workspace = true
|
|
license.workspace = true
|
|
repository.workspace = true
|
|
description = "Operator CLI for Buzz relay administration"
|
|
|
|
[[bin]]
|
|
name = "buzz-admin"
|
|
path = "src/main.rs"
|
|
|
|
[dependencies]
|
|
buzz-db = { workspace = true }
|
|
buzz-deletion = { workspace = true }
|
|
buzz-core = { workspace = true }
|
|
buzz-auth = { workspace = true }
|
|
buzz-pubsub = { workspace = true }
|
|
buzz-search = { workspace = true }
|
|
buzz-audit = { workspace = true }
|
|
buzz-workflow = { workspace = true }
|
|
buzz-media = { workspace = true }
|
|
nostr = { workspace = true }
|
|
tokio = { workspace = true }
|
|
serde_json = { workspace = true }
|
|
serde = { workspace = true }
|
|
anyhow = { workspace = true }
|
|
hex = { workspace = true }
|
|
deadpool-redis = { workspace = true }
|
|
# Redis TLS (rediss://, e.g. ElastiCache) uses rustls, which needs a process
|
|
# CryptoProvider installed at startup. The workspace redis TLS feature compiles
|
|
# both aws-lc-rs and ring in transitively, so rustls can't auto-select one — we
|
|
# install ring explicitly in main(). Mirrors buzz-relay's setup.
|
|
rustls = { version = "0.23", default-features = false, features = ["ring", "std"] }
|
|
tracing = { workspace = true }
|
|
sqlx = { workspace = true }
|
|
url = { workspace = true }
|
|
uuid = { workspace = true }
|
|
clap = { version = "4", features = ["derive"] }
|
|
|
|
[dev-dependencies]
|
|
tokio = { workspace = true, features = ["test-util"] }
|