jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-11-06 17:31:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,303 Commits 1,344 Branches 263 Tags
4aae6c22aa734df508eee5f46a8e72a11ff89dd3
Commit Graph

3 Commits

Author SHA1 Message Date
Luke Tornquist
ff15c347c2 fix(db): auto-migrate prebuild-guarded indexes on small databases (#8802) 
## Problem

Migrations `0076`, `0078` and `0081` raise an exception on any table
with at least one row unless its index was prebuilt with `CREATE INDEX
CONCURRENTLY`. A relay that runs migrations at startup
(`BUZZ_AUTO_MIGRATE`, on by default in the Helm chart) and has data in
those tables therefore crash-loops on upgrade:

```
migration error: while executing migration 81: error returned from database:
prebuild idx_nip_fi_profile_heads_names concurrently; see docs/profile-search.md
```

`0081` (#9019) hits every auto-migrating relay that has published NIP-FI
verified profiles. `0076`/`0078` (#8523) hit any relay with artifact
data or channel members. Each of these tables is small enough on such
relays that a plain index build takes milliseconds.

The root cause is `docs/database.md` Rule 6. It told authors that an
index on a populated table MUST be built concurrently by an operator
ahead of time, so migrations were written to refuse populated tables.
With auto-migration that refusal is a crash loop, not a safe stop.

## Fix

The migrations are already applied on some databases, so they can't be
edited: sqlx would reject the changed checksums. Instead,
`run_migrations_locked` builds their missing indexes first. For each
entry in `INLINE_INDEX_BUILDS` (`0076`, `0078`, `0081`), it runs
migrations through the version before it, then builds any missing index.

Each build is a plain `CREATE INDEX` in its own transaction, under the
same 1s `lock_timeout` and 5s `statement_timeout` the migrations use.
That is the bounded inline build `0049` already does:

- **Small populated databases** upgrade the same way empty ones do.
- **Large databases** still fail closed. Only a lock or statement
timeout (`55P03` / `57014`) gets the "prebuild concurrently" hint,
pointing at that migration's own doc. Other errors pass through
unchanged. The blocked migration is not recorded.
- **Prebuilt indexes** are found with `to_regclass` and skipped before
`CREATE INDEX`, so deployments that prebuilt concurrently never request
the writer-conflicting lock.
- **Invalid prebuilt indexes** are left for the migration to reject, as
before.

`CONCURRENTLY` is not used in this path. The runner holds the migration
advisory lock, and another replica blocked on that lock holds a snapshot
that a concurrent build would wait on.

### Rules

- `docs/database.md` Rule 6: with `BUZZ_AUTO_MIGRATE` on, every
migration must apply its own change on a populated database and must
never refuse one because it has rows. A new index is built inline under
the migration's budgets. Only deployments that apply migrations out of
band (auto-migration off) prebuild concurrently.
- Root `AGENTS.md` (step 4) and `migrations/AGENTS.md` repeat the
never-refuse rule.
- `docs/artifact-sync-deployment.md` and `docs/profile-search.md`
describe the startup behavior first and the concurrent prebuild as the
out-of-band path.

## Tests

- `populated_database_auto_migrates_through_every_migration` (Postgres)
migrates to `0075`, seeds artifact rows, and checks four phases:
1. **Lock held:** while a writer lock is held, the `0076` build fails
closed.
2. **Index prebuilt:** with the `0076` index prebuilt, the build is
skipped.
3. **Partial `0078`:** a blocked `0078` build keeps the builds before it
committed.
4. **Verified profile seeded:** with a verified profile head inserted
before `0081`, a final startup succeeds. It asserts that the applied
versions equal every embedded migration and that all six indexes are
valid, ready and live.
- `frozen_prebuild_guards_have_inline_builds` (no database) scans
`migrations/`. It fails if any migration refuses a populated table
(`RAISE EXCEPTION 'prebuild …`) without an inline build, and if a new
guard is added.
- `migration_execution_cannot_bypass_schema_destruction_lock` allows
exactly one production `run_to`, inside `run_migrations_locked`.

## What got simpler

- Auto-migrating relays no longer need a manual, operator-only prebuild
step for small deployments. That restores the chart README's contract
that `helm upgrade` is the whole upgrade procedure.
- Rule 6 no longer steers authors toward refusing populated tables, and
a test enforces that.
- The change adds one table-driven step for three frozen migrations and
no new configuration.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-11-05 13:52:06 -05:00
ngoodger-block
f33ebaafa1 refactor(schema): stop auto-fencing new tables in desired-state bootstrap (#8620) 
### Why?

This is a cleanup that brings the schema bootstrap in line with how the
codebase works today, as asked for by
[`docs/database.md`](dc25198d53/docs/database.md (L103-L124))
(#8273). Writes are now blocked during community deletion by application
admission (`begin_community_write_transaction`), and that doc's Rule 1
says new tables must not add triggers. But
`schema/community_write_fence.sql` still looped over every table with a
`community_id` column and attached the old fence trigger to it. That
meant a new table got a trigger on a fresh database and none on a
migrated one, so the two schema paths disagreed and new tenant tables
were blocked. The doc says this bootstrap change must land "before the
feature, or as the feature's first PR."

### What?

Deletes the blanket fence loop and its `\i` from `schema/schema.sql`.
Existing legacy tables keep their fences through their explicit
`schema/triggers/` files, so no current table loses its fence and
runtime behavior is unchanged. Adds a test that a new `community_id`
table stays unfenced while `events` and `users` stay fenced, and updates
`docs/database.md` and `ARCHITECTURE.md` to match. It stands on its own;
the media-sharing PRs (#8451, #8444) are the first to rely on it.

buzz-review-completed

---------

Signed-off-by: Nikolaj Goodger <ngoodger@squareup.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-28 09:26:41 +11:00
Luke Tornquist
fcdabcf774 docs(db): database target architecture and schema-change rules (#8273) 
## Summary

Adds `docs/database.md`, which covers two things:

- **The target architecture.** `schema/tables/public/` is the
declarative source of truth, written within the constraints that
out-of-band, online schema tools can apply. Deployments at scale apply
schema changes out of band; auto-migration stays supported for smaller
and self-hosted deployments, and `migrations/` stays. The database has
no triggers, functions, or foreign keys; that behavior lives in the
owning application transaction or in durable application work. Data
repair runs through the backfill framework in `docs/data-backfills.md`,
not through migrations.
- **Rules for schema changes made now:**
1. Add no triggers, functions, procedures, rules, or foreign keys, with
no exceptions. New tenant-owned tables are fenced by community admission
in the application. Startup and deletion validation already tolerate
missing fences, but desired-state bootstraps still attach a fence
trigger to every `community_id` table, so a new tenant-owned table is
blocked until that loop stops (part of the fence-retirement work). The
doc includes checklists for adding a tenant-owned table and a
provenance-only table.
2. Migrations change schema only. They never mutate existing rows. Until
the backfill framework ships, a repair may be a reviewed, idempotent,
batched one-off script in `scripts/maintenance/` that takes the
community deletion lock and touches only active communities. A repair
that writes historical channel events must keep replica routing closed
until its WAL has replayed.
3. Update every schema location: `migrations/`, `schema/tables/public/`,
the manifest and sibling directories, the pgschema reconcile script
where it applies, and both homes of the existing `push_gateway_*` tables
(the target is for the gateway to own its tables alone). Never edit a
merged migration. Add catalog-level parity tests for what you touch.
4. Every migration must be safe for the relay that is currently
deployed, under both supported modes: auto-migration at startup and
out-of-band application. Either way the old relay runs against the new
schema for a while. Because auto-migration can apply expand and contract
in one boot, a contract migration must fail closed when its backfill has
not run, and a contract that depends on a writer release (such as
dropping a trigger whose work moved into the app) must name that release
as a required stop in the release notes, because no row check can prove
it. The order is expand, deploy, backfill, then contract. The doc
includes a table of changes that break the old relay and the safe form
of each.
5. Bound every lock, build `migrations/` indexes concurrently (the
declarative schema uses plain `CREATE INDEX`), and keep changes within
what out-of-band apply can make online.

AGENTS.md (Product Contract), ARCHITECTURE.md (buzz-db), and new
path-local `migrations/AGENTS.md`, `schema/AGENTS.md` (a pointer), and
`crates/buzz-push-gateway/migrations/AGENTS.md` point to the doc. The
path-local files mean agents editing those directories get the rules
automatically. AGENTS.md also had a stale claim that migrations are
"auto-applied on relay startup". This PR corrects it:
`BUZZ_AUTO_MIGRATE` defaults off in the binary and on in the Helm chart,
and both modes are supported.

## What got simpler

The rules were scattered across `schema/schema.sql` comments, AGENTS.md
Common Gotcha #7, the backfill spec, `docs/thread-window-deployment.md`,
and test names. They now live in one document, and the rest link to it.
No code changes.

## Testing

This PR changes documentation only. I checked links and diffs, and every
claim in the doc points to code at the base commit: the layout and
parity tests in `crates/buzz-db/src/runtime/migration.rs`,
`EXPECTED_SCOPED_TABLES` and the serving-catalog check in
`crates/buzz-db/src/store/deletion.rs`, and `BUZZ_AUTO_MIGRATE` in
`crates/buzz-relay/src/main.rs`. The new `AGENTS.md` files aren't
`.sql`, so neither the sqlx migrator nor the schema layout test picks
them up; CI's layout, migration-count, and deletion-parity tests pass.

At an earlier head, a real relay against isolated PostgreSQL 17 showed
that a new unfenced `community_id` table serves normally and makes
community deletion report `unknown=<table>`. Since then, the fence
requirement at startup was removed on main; the doc now cites
`migration_schema_fence_objects_are_optional_for_serving_and_deletion`
for that.

---------

Signed-off-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz>
 2026-10-20 15:47:05 -04:00