mirror of
https://github.com/block/buzz.git
synced 2026-11-03 07:51:22 +00:00
migrations: move 1321 cutover script out of the embedded sqlx dir
0002_backfill_default_community.sql is a psql operator script (\set, \ir, :'var' interpolation) but lived in migrations/, which sqlx::migrate! embeds wholesale. sqlx cannot run psql meta-commands, so every fresh deploy broke: BUZZ_AUTO_MIGRATE=true exited the relay with 'syntax error at or near "\"', buzz-admin migrate (the just setup path) failed identically with exit 5, and both left a wedged half-migrated DB. This also reddened the buzz-db unit test embedded_migrator_contains_consolidated_initial_schema (asserts exactly the consolidated 0001), which CI never ran. The module doc and that test already declare the intended invariant: the embedded migrator is the consolidated 0001 only; cutover/backfill is a separate operator script, not startup migration state. Restore it by relocating the script to scripts/cutover/1321_backfill_default_community.sql (dropping the 0002_ prefix so it never reads as sqlx state again), switching its schema include from \i to \ir so it resolves relative to the script rather than the operator's cwd, and adding scripts/cutover/README.md as the operator runbook. Wire the pure (infra-free) buzz-db tests into the unit path so this class of regression can't hide again. CI installs nextest, so just test-unit takes the nextest branch and bypasses run-tests.sh entirely — patch BOTH paths. Verified locally on a pristine Postgres: buzz-admin migrate -> exit 0, one migration row (v1); relay with BUZZ_AUTO_MIGRATE=true -> migrations complete, readiness 200, 36 tables, no wedge. buzz-db --lib goes from RED (left:2 right:1) to green. The relocated script's \ir resolves 0001 from a non-repo cwd. Co-authored-by: Tyler Longwell <tlongwell@block.xyz> Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
This commit is contained in:
parent
08669bca21
commit
666cfe4157
7
Justfile
7
Justfile
@@ -231,6 +231,13 @@ test-unit:
|
||||
#!/usr/bin/env bash
|
||||
if command -v cargo-nextest &>/dev/null; then
|
||||
cargo nextest run -p buzz-core -p buzz-auth --lib
|
||||
# buzz-db migrator/lint tests: pure SQL-parsing unit tests (no infra).
|
||||
# They guard the embedded-migrator invariant (exactly the consolidated
|
||||
# 0001; cutover/backfill stays an operator script, not startup state)
|
||||
# and the tenant-scoping lints. The Postgres-backed buzz-db tests are
|
||||
# #[ignore]d, so --lib runs only the infra-free set. Without this gate a
|
||||
# stray file in migrations/ or a broken lint ships green.
|
||||
cargo nextest run -p buzz-db --lib
|
||||
# Multi-tenant conformance gate (buzz-conformance): the independent
|
||||
# replay checker + golden fixtures. No infra — pure in-process trace
|
||||
# replay — so it belongs in the unit job. Run all targets (lib + the
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
-- 0002_backfill_default_community.sql
|
||||
-- scripts/cutover/1321_backfill_default_community.sql
|
||||
--
|
||||
-- One-off cutover: single-community (pre-1321) Postgres -> multi-tenant (1321)
|
||||
-- schema. Assigns EVERY existing row to one default community derived from the
|
||||
@@ -46,15 +46,18 @@
|
||||
-- PRECONDITIONS
|
||||
-- * DB is the pre-1321 single-community schema (no community_id columns, no
|
||||
-- `communities` table). Guard below refuses to run otherwise.
|
||||
-- * Run psql from the buzz repo root so the \i path resolves.
|
||||
-- * Run psql with this script via -f (not piped on stdin): the Phase B
|
||||
-- include below uses \ir, which resolves relative to THIS file, so the
|
||||
-- repo's migrations/0001 is found regardless of the operator's cwd.
|
||||
-- * TAKE A pg_dump / PVC SNAPSHOT FIRST. No down-path; the snapshot IS the
|
||||
-- rollback.
|
||||
--
|
||||
-- USAGE (from the buzz repo root)
|
||||
-- USAGE (run from anywhere; \ir below resolves the schema include relative to
|
||||
-- this file, so cwd does not matter)
|
||||
-- psql "$DATABASE_URL" \
|
||||
-- -v host="'sprout-oss.stage.blox.sqprod.co'" \
|
||||
-- -v ON_ERROR_STOP=1 \
|
||||
-- -f migrations/0002_backfill_default_community.sql
|
||||
-- -f scripts/cutover/1321_backfill_default_community.sql
|
||||
--
|
||||
-- After commit: boot the 1321 relay with BUZZ_AUTO_MIGRATE=false. The schema is
|
||||
-- already correct; the relay's idempotent boot-time
|
||||
@@ -128,7 +131,9 @@ END
|
||||
$move_types$;
|
||||
|
||||
-- ── Phase B: the authoritative 1321 schema, verbatim, into clean public ──────
|
||||
\i migrations/0001_initial_schema.sql
|
||||
-- \ir = include relative to THIS script's directory (scripts/cutover/), so the
|
||||
-- repo's migrations/0001 resolves regardless of the operator's cwd.
|
||||
\ir ../../migrations/0001_initial_schema.sql
|
||||
|
||||
-- ── Phase C: the one default community, normalized like the relay does ───────
|
||||
INSERT INTO public.communities (host)
|
||||
87
scripts/cutover/README.md
Normal file
87
scripts/cutover/README.md
Normal file
@@ -0,0 +1,87 @@
|
||||
# 1321 cutover: single-community → multi-tenant
|
||||
|
||||
`1321_backfill_default_community.sql` is a **one-off operator script**, not a
|
||||
startup migration. It takes a pre-1321 (single-community) Buzz Postgres to the
|
||||
1321 multi-tenant schema, assigning every existing row to one default community
|
||||
derived from the deployment host.
|
||||
|
||||
> It is **not** embedded in the relay binary. The relay's `sqlx::migrate!`
|
||||
> embeds only `migrations/0001_initial_schema.sql` — the consolidated 1321
|
||||
> schema. Fresh deployments need only that; this script exists solely to carry
|
||||
> existing pre-1321 data across the rewrite. (It uses psql client features —
|
||||
> `\set`, `\ir`, `:'var'` interpolation, `\gset` — that the embedded sqlx
|
||||
> migrator cannot run, which is why it must live here and not under
|
||||
> `migrations/`.)
|
||||
|
||||
## When you need it
|
||||
|
||||
Only when upgrading a Postgres that already holds **pre-1321 single-community
|
||||
data** to 1321. A brand-new deployment does **not** run this — it provisions
|
||||
from `migrations/0001_initial_schema.sql` (or `schema/schema.sql`) directly.
|
||||
|
||||
## Preconditions
|
||||
|
||||
- The DB is pristine pre-1321: no `communities` table, no `community_id`
|
||||
columns, no leftover `legacy` schema. The script's guard **refuses to run**
|
||||
otherwise (already-migrated, partially-migrated, or failed-prior-run states
|
||||
all raise and abort — see the guard block at the top of the script).
|
||||
- **Take a `pg_dump` / PVC snapshot first.** There is no down-path. The
|
||||
snapshot **is** the rollback.
|
||||
|
||||
## Run it
|
||||
|
||||
The whole script runs in **one transaction** (all-or-nothing). Pass the
|
||||
deployment host as `:host`; it is normalized exactly as the relay normalizes a
|
||||
community host (`rtrim(lower(host), '.')`).
|
||||
|
||||
```sh
|
||||
psql "$DATABASE_URL" \
|
||||
-v host="'your-deployment-host.example.com'" \
|
||||
-v ON_ERROR_STOP=1 \
|
||||
-f scripts/cutover/1321_backfill_default_community.sql
|
||||
```
|
||||
|
||||
`-f` (not stdin) matters: the script `\ir`-includes
|
||||
`../../migrations/0001_initial_schema.sql` **relative to its own location**, so
|
||||
it resolves the canonical 1321 schema regardless of your cwd.
|
||||
|
||||
### What it does
|
||||
|
||||
1. Renames all pre-1321 tables and enum types aside into a `legacy` schema.
|
||||
2. `\ir`-includes the repo's `0001` verbatim → the correct 1321 schema in a
|
||||
clean `public` (structurally identical to a fresh 1321 DB **by
|
||||
construction** — no hand-transcription).
|
||||
3. Creates the one default community from `:host`.
|
||||
4. Copies every row forward, stamped with that `community_id` (`search_tsv`
|
||||
regenerates as GENERATED; new-in-1321 columns default NULL).
|
||||
5. `DROP SCHEMA legacy CASCADE`.
|
||||
|
||||
## After it commits
|
||||
|
||||
Boot the 1321 relay with **`BUZZ_AUTO_MIGRATE=false`**. The schema is already
|
||||
correct; the relay's idempotent boot-time `ensure_configured_community` /
|
||||
allowlist→relay_members backfill finds this community and no-ops.
|
||||
|
||||
If you want the relay's sqlx migrator to consider `0001` already applied (so a
|
||||
later `BUZZ_AUTO_MIGRATE=true` boot doesn't try to re-run it), seed its
|
||||
`_sqlx_migrations` row to match a fresh 1321 install:
|
||||
|
||||
```sql
|
||||
-- version 1, description 'initial schema'; checksum must match the embedded 0001.
|
||||
-- Easiest: diff _sqlx_migrations against a freshly-migrated 1321 DB and copy the row.
|
||||
```
|
||||
|
||||
This is **not** required for correct operation — a fresh `0001` run against the
|
||||
already-correct schema is a no-op only if the migrator's checksum matches, so
|
||||
prefer leaving `BUZZ_AUTO_MIGRATE=false` unless you have a reason to seed.
|
||||
|
||||
## Verify (post-commit, outside the txn)
|
||||
|
||||
```sql
|
||||
SELECT count(*) FROM communities; -- expect 1
|
||||
SELECT count(*) FROM events WHERE community_id IS NULL; -- expect 0
|
||||
SELECT to_regnamespace('legacy'); -- expect NULL (gone)
|
||||
```
|
||||
|
||||
For a stronger guarantee, take a constraint/trigger/index/column diff of this
|
||||
DB against a freshly-migrated 1321 DB — it should be empty.
|
||||
@@ -84,6 +84,15 @@ run_unit_tests() {
|
||||
run_test_step "buzz-auth unit tests" \
|
||||
cargo test -p buzz-auth --lib -- --nocapture
|
||||
|
||||
# buzz-db migrator/lint unit tests (no infra): guard the embedded-migrator
|
||||
# invariant (exactly the consolidated 0001; cutover/backfill stays an operator
|
||||
# script, not startup state) and the tenant-scoping lints. The Postgres-backed
|
||||
# buzz-db tests are #[ignore]d; nothing here (or in integration mode below,
|
||||
# which runs `cargo test -p buzz-db` without --ignored) runs them — they need a
|
||||
# separate isolated-DB gate, so --lib keeps this step infra-free.
|
||||
run_test_step "buzz-db unit tests" \
|
||||
cargo test -p buzz-db --lib -- --nocapture
|
||||
|
||||
# Multi-tenant conformance gate: independent replay checker + golden
|
||||
# fixtures (buzz-conformance). Pure in-process trace replay, no infra.
|
||||
run_test_step "buzz-conformance tests" \
|
||||
|
||||
Reference in New Issue
Block a user