mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
## Summary Re-applying `schema/schema.sql` to a database that `pgschema apply` and `scripts/reconcile-schema-after-pgschema.sql` already bootstrapped fails with `cannot drop inherited constraint "events_p2026_01_pkey" of relation "events_p2026_01" (SQLSTATE 42P16)`. Because of that, `scripts/start-relay-for-tests.sh` breaks against a reused database. **Cause:** pgschema 1.7.4 moves a partitioned table's partition key to the front of its primary key when it creates the table. A pgschema-bootstrapped `events` therefore gets `PRIMARY KEY (created_at, community_id, id)`. `schema/tables/public/events.sql` and `migrations/0001` both declare `(community_id, created_at, id)`. Every later plan drops and re-adds `events_pkey` and each declared child's key. This happens even on a database that was never reconciled. The plan only becomes unexecutable after the reconcile script attaches the children, because their keys are then inherited. As a side effect, pgschema-bootstrapped databases also had a differently ordered PK index than migration-managed ones. **Fix:** after the partition ATTACH block, the reconcile script now compares `events_pkey` with the declared `PRIMARY KEY (community_id, created_at, id)`. If they differ, it rebuilds the key on the parent. It then asserts that `events` and every partition carry the declared key, which is the "convergence plus live assertion" shape `docs/database.md` requires. PostgreSQL recreates each partition's key with the same `<child>_pkey` names, so a reconciled re-apply now plans nothing. The block does nothing when the key is already correct. `migrations/0001` is untouched. Only pgschema-bootstrapped dev and test databases run this script, so it does not touch the out-of-band partitioned-table procedure. `delivery_log` is not affected. Its key `(delivered_at, id)` already starts with the partition key. **Existing local databases:** a database reconciled before this change would still fail its next `pgschema apply`, because the reconcile script only runs after apply. The repair therefore also lives in `scripts/repair-events-pkey.sql`. That file does nothing when `events` doesn't exist yet, and `scripts/start-relay-for-tests.sh` runs it before apply; it's the only launcher that reuses its database. The reconcile script keeps a verbatim copy for fresh bootstraps, and a unit test keeps the two identical. The rebuild drops the key with `IF EXISTS`, so a database that has lost its key still converges. It also takes its locks with a 10s `lock_timeout`. If a relay still holds `events`, it fails with "Stop any relay using this database, then rerun" instead of hanging. `start-relay-for-tests.sh` now builds the relay's `DATABASE_URL` from the `PG*` variables it already exports. The secret scanner flags the literal URI once the file is touched. The value is unchanged. pgschema 1.13.3 keeps the declared order and creates children as native `PARTITION OF`. Upgrading would make this block and the ATTACH blocks unnecessary, but it changes behavior for every caller, so it is left for a separate change. ### Related issue Follow-up to block/buzz#8487, which added the probe test and `.pgschemaignore`. Now rebased onto `main` after that PR merged. ### Testing - `pgschema_reapply_keeps_runtime_partitions` now asserts the plan after the maintenance pass is **empty**, which replaces the `DROP TABLE` and per-monthly plan checks. It then applies and reconciles a second time. Without the new block it fails with `ALTER TABLE events DROP CONSTRAINT events_pkey; ...`. With the block it passes. - It also includes the optional notes from the #8487 review: - After maintenance, it checks that both catch-alls are still present and attached, not only that the zipped bounds differ. - The first runtime monthly comes from the declared catch-all's lower bound instead of a hard-coded `p2026_07`. - The doc comment now states only what the test pins: bounds are not compared, and the plan is empty. - I broke the rebuild on purpose by re-adding the key in the wrong order. The new assertion fails the reconcile script with `events and its partitions must have PRIMARY KEY (community_id, created_at, id)`. - CLI runs against local Postgres 17: - Fresh DB: apply → reconcile → apply → reconcile → plan. The second apply reports no changes and the plan is empty. - DB reconciled with the old script: the next apply fails with SQLSTATE 42P16. After running the new reconcile script once, apply reports no changes and the plan is empty. - `events_pkey_repair_runs_before_reused_apply_and_matches_reconcile` checks two things: the reconcile script carries the repair block verbatim, and `start-relay-for-tests.sh` runs the repair before `pgschema apply`. It fails if the two copies drift. - I ran the repair script and then the bootstrap against four local probe databases: one reconciled with main's script, an empty one, one applied but never reconciled, and one with no `events` key. All four end with an empty plan. With another session holding a lock on `events`, the repair fails after 10s with the hint and leaves the key untouched. It succeeds once the lock is released. - `admin_schema_parity_between_desired_state_and_migrations` and the `runtime::migration` unit tests pass. fmt and clippy are clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
284 lines
12 KiB
SQL
284 lines
12 KiB
SQL
-- Reconcile schema details that pgschema does not preserve.
|
|
--
|
|
-- pgschema reconciles DDL, but it does not execute seed DML or preserve every
|
|
-- table storage parameter from schema/schema.sql. It also currently emits
|
|
-- partition children as standalone CREATE TABLE statements. Every pgschema
|
|
-- apply caller must run this idempotent script so fresh bootstraps converge on
|
|
-- the same live database contract as migration-managed databases.
|
|
|
|
DO $$
|
|
BEGIN
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p_past'::regclass
|
|
) THEN
|
|
-- pgschema may copy parent triggers onto standalone children. Drop
|
|
-- those copies before ATTACH; PostgreSQL recreates inherited parent
|
|
-- triggers while attaching and rejects same-named child triggers.
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p_past;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p_past;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p_past;
|
|
ALTER TABLE events ATTACH PARTITION events_p_past
|
|
FOR VALUES FROM (MINVALUE) TO ('2026-01-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_01'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_01;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_01;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_01;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_01
|
|
FOR VALUES FROM ('2026-01-01') TO ('2026-02-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_02'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_02;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_02;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_02;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_02
|
|
FOR VALUES FROM ('2026-02-01') TO ('2026-03-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_03'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_03;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_03;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_03;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_03
|
|
FOR VALUES FROM ('2026-03-01') TO ('2026-04-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_04'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_04;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_04;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_04;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_04
|
|
FOR VALUES FROM ('2026-04-01') TO ('2026-05-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_05'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_05;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_05;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_05;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_05
|
|
FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p2026_06'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_06;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_06;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_06;
|
|
ALTER TABLE events ATTACH PARTITION events_p2026_06
|
|
FOR VALUES FROM ('2026-06-01') TO ('2026-07-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'events'::regclass
|
|
AND inhrelid = 'events_p_future'::regclass
|
|
) THEN
|
|
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p_future;
|
|
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p_future;
|
|
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p_future;
|
|
ALTER TABLE events ATTACH PARTITION events_p_future
|
|
FOR VALUES FROM ('2026-07-01') TO (MAXVALUE);
|
|
END IF;
|
|
|
|
-- When pgschema creates partition children as standalone tables, it also
|
|
-- preserves the parent's identity column on delivery_log children. PostgreSQL
|
|
-- rejects attaching a child table that has its own identity column, so each
|
|
-- delivery_log attach path drops that standalone identity first. Raw
|
|
-- schema-created partitions are already attached, so these branches do not
|
|
-- run against inherited partition columns.
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p_past'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p_past ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p_past;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p_past
|
|
FOR VALUES FROM (MINVALUE) TO ('2026-03-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p2026_03'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p2026_03 ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_03;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_03
|
|
FOR VALUES FROM ('2026-03-01') TO ('2026-04-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p2026_04'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p2026_04 ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_04;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_04
|
|
FOR VALUES FROM ('2026-04-01') TO ('2026-05-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p2026_05'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p2026_05 ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_05;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_05
|
|
FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p2026_06'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p2026_06 ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_06;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_06
|
|
FOR VALUES FROM ('2026-06-01') TO ('2026-07-01');
|
|
END IF;
|
|
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_inherits
|
|
WHERE inhparent = 'delivery_log'::regclass
|
|
AND inhrelid = 'delivery_log_p_future'::regclass
|
|
) THEN
|
|
ALTER TABLE delivery_log_p_future ALTER COLUMN id DROP IDENTITY IF EXISTS;
|
|
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p_future;
|
|
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p_future
|
|
FOR VALUES FROM ('2026-07-01') TO (MAXVALUE);
|
|
END IF;
|
|
END $$;
|
|
|
|
-- pgschema moves the partition key to the front of a partitioned table's
|
|
-- primary key when it creates the table, so events gets
|
|
-- PRIMARY KEY (created_at, community_id, id) instead of the declared
|
|
-- (community_id, created_at, id). Every later plan then drops and re-adds
|
|
-- events_pkey and each declared child's key, which fails once the children are
|
|
-- attached because their keys are inherited. Rebuild the key on the parent in
|
|
-- the declared order; PostgreSQL recreates each partition's key with it.
|
|
-- delivery_log already declares its partition key first. This block is a
|
|
-- verbatim copy of scripts/repair-events-pkey.sql; keep the two identical.
|
|
DO $$
|
|
DECLARE
|
|
declared CONSTANT text := 'PRIMARY KEY (community_id, created_at, id)';
|
|
BEGIN
|
|
-- Also runs before pgschema apply, when events may not exist yet.
|
|
IF to_regclass('events') IS NULL THEN
|
|
RETURN;
|
|
END IF;
|
|
|
|
IF (
|
|
SELECT pg_get_constraintdef(oid) FROM pg_constraint
|
|
WHERE conrelid = 'events'::regclass AND contype = 'p'
|
|
) IS DISTINCT FROM declared THEN
|
|
-- The rebuild locks events and every partition ACCESS EXCLUSIVE. Fail
|
|
-- fast instead of queueing behind a relay that still uses them.
|
|
BEGIN
|
|
PERFORM set_config('lock_timeout', '10s', true);
|
|
ALTER TABLE events DROP CONSTRAINT IF EXISTS events_pkey;
|
|
ALTER TABLE events
|
|
ADD CONSTRAINT events_pkey PRIMARY KEY (community_id, created_at, id);
|
|
EXCEPTION WHEN lock_not_available THEN
|
|
RAISE EXCEPTION 'timed out locking events to rebuild events_pkey'
|
|
USING HINT = 'Stop any relay using this database, then rerun.';
|
|
END;
|
|
END IF;
|
|
|
|
IF EXISTS (
|
|
SELECT 1 FROM pg_constraint
|
|
WHERE contype = 'p'
|
|
AND conrelid IN (
|
|
SELECT 'events'::regclass
|
|
UNION ALL
|
|
SELECT inhrelid FROM pg_inherits WHERE inhparent = 'events'::regclass
|
|
)
|
|
AND pg_get_constraintdef(oid) <> declared
|
|
) THEN
|
|
RAISE EXCEPTION 'events and its partitions must have %', declared;
|
|
END IF;
|
|
END $$;
|
|
|
|
-- pgschema drops multi-column CHECK constraints. Restore the direct-action
|
|
-- shape CHECK from schema/schema.sql (and migration 0055) verbatim.
|
|
DO $$
|
|
BEGIN
|
|
IF NOT EXISTS (
|
|
SELECT 1 FROM pg_constraint
|
|
WHERE conrelid = 'relay_admin_actions'::regclass
|
|
AND conname = 'relay_admin_actions_direct_shape'
|
|
) THEN
|
|
ALTER TABLE relay_admin_actions
|
|
ADD CONSTRAINT relay_admin_actions_direct_shape CHECK (
|
|
report_id IS NOT NULL
|
|
OR (action = 'ban' AND enforcement_target_pubkey IS NOT NULL
|
|
AND timeout_secs IS NULL AND timeout_until IS NULL)
|
|
OR (action = 'timeout' AND enforcement_target_pubkey IS NOT NULL
|
|
AND timeout_secs IS NOT NULL AND timeout_until IS NOT NULL)
|
|
OR (action = 'delete' AND enforcement_target_event_id IS NOT NULL
|
|
AND enforcement_target_pubkey IS NOT NULL
|
|
AND timeout_secs IS NULL AND timeout_until IS NULL)
|
|
);
|
|
END IF;
|
|
END $$;
|
|
|
|
-- pgschema reconciles DDL but does not apply seed DML or table storage
|
|
-- parameters from schema/schema.sql. Restore those parts of the desired-state
|
|
-- contract explicitly and fail the bootstrap if the live catalog disagrees.
|
|
ALTER TABLE replica_heartbeat SET (vacuum_truncate = false);
|
|
|
|
INSERT INTO replica_heartbeat (id) VALUES (1)
|
|
ON CONFLICT (id) DO NOTHING;
|
|
|
|
DO $$
|
|
BEGIN
|
|
IF NOT EXISTS (
|
|
SELECT 1
|
|
FROM pg_class AS relation
|
|
JOIN pg_namespace AS namespace ON namespace.oid = relation.relnamespace
|
|
WHERE namespace.nspname = current_schema()
|
|
AND relation.relname = 'replica_heartbeat'
|
|
AND COALESCE(
|
|
relation.reloptions @> ARRAY['vacuum_truncate=false']::text[],
|
|
false
|
|
)
|
|
) THEN
|
|
RAISE EXCEPTION 'replica_heartbeat must disable vacuum truncation after pgschema apply';
|
|
END IF;
|
|
|
|
IF (SELECT count(*) FROM replica_heartbeat WHERE id = 1) <> 1 THEN
|
|
RAISE EXCEPTION 'replica_heartbeat must contain its singleton row after pgschema apply';
|
|
END IF;
|
|
END $$;
|