Files
buzz/scripts/reconcile-schema-after-pgschema.sql
Ian Oberst 20ba95ed89 fix(db): rebuild events_pkey in declared order after pgschema apply (#8525)
## Summary

Re-applying `schema/schema.sql` to a database that `pgschema apply` and
`scripts/reconcile-schema-after-pgschema.sql` already bootstrapped fails
with `cannot drop inherited constraint "events_p2026_01_pkey" of
relation "events_p2026_01" (SQLSTATE 42P16)`. Because of that,
`scripts/start-relay-for-tests.sh` breaks against a reused database.

**Cause:** pgschema 1.7.4 moves a partitioned table's partition key to
the front of its primary key when it creates the table. A
pgschema-bootstrapped `events` therefore gets `PRIMARY KEY (created_at,
community_id, id)`. `schema/tables/public/events.sql` and
`migrations/0001` both declare `(community_id, created_at, id)`. Every
later plan drops and re-adds `events_pkey` and each declared child's
key. This happens even on a database that was never reconciled. The plan
only becomes unexecutable after the reconcile script attaches the
children, because their keys are then inherited. As a side effect,
pgschema-bootstrapped databases also had a differently ordered PK index
than migration-managed ones.

**Fix:** after the partition ATTACH block, the reconcile script now
compares `events_pkey` with the declared `PRIMARY KEY (community_id,
created_at, id)`. If they differ, it rebuilds the key on the parent. It
then asserts that `events` and every partition carry the declared key,
which is the "convergence plus live assertion" shape `docs/database.md`
requires. PostgreSQL recreates each partition's key with the same
`<child>_pkey` names, so a reconciled re-apply now plans nothing. The
block does nothing when the key is already correct. `migrations/0001` is
untouched. Only pgschema-bootstrapped dev and test databases run this
script, so it does not touch the out-of-band partitioned-table
procedure.

`delivery_log` is not affected. Its key `(delivered_at, id)` already
starts with the partition key.

**Existing local databases:** a database reconciled before this change
would still fail its next `pgschema apply`, because the reconcile script
only runs after apply. The repair therefore also lives in
`scripts/repair-events-pkey.sql`. That file does nothing when `events`
doesn't exist yet, and `scripts/start-relay-for-tests.sh` runs it before
apply; it's the only launcher that reuses its database. The reconcile
script keeps a verbatim copy for fresh bootstraps, and a unit test keeps
the two identical.

The rebuild drops the key with `IF EXISTS`, so a database that has lost
its key still converges. It also takes its locks with a 10s
`lock_timeout`. If a relay still holds `events`, it fails with "Stop any
relay using this database, then rerun" instead of hanging.

`start-relay-for-tests.sh` now builds the relay's `DATABASE_URL` from
the `PG*` variables it already exports. The secret scanner flags the
literal URI once the file is touched. The value is unchanged.

pgschema 1.13.3 keeps the declared order and creates children as native
`PARTITION OF`. Upgrading would make this block and the ATTACH blocks
unnecessary, but it changes behavior for every caller, so it is left for
a separate change.

### Related issue

Follow-up to block/buzz#8487, which added the probe test and
`.pgschemaignore`. Now rebased onto `main` after that PR merged.

### Testing

- `pgschema_reapply_keeps_runtime_partitions` now asserts the plan after
the maintenance pass is **empty**, which replaces the `DROP TABLE` and
per-monthly plan checks. It then applies and reconciles a second time.
Without the new block it fails with `ALTER TABLE events DROP CONSTRAINT
events_pkey; ...`. With the block it passes.
- It also includes the optional notes from the #8487 review:
- After maintenance, it checks that both catch-alls are still present
and attached, not only that the zipped bounds differ.
- The first runtime monthly comes from the declared catch-all's lower
bound instead of a hard-coded `p2026_07`.
- The doc comment now states only what the test pins: bounds are not
compared, and the plan is empty.
- I broke the rebuild on purpose by re-adding the key in the wrong
order. The new assertion fails the reconcile script with `events and its
partitions must have PRIMARY KEY (community_id, created_at, id)`.
- CLI runs against local Postgres 17:
- Fresh DB: apply → reconcile → apply → reconcile → plan. The second
apply reports no changes and the plan is empty.
- DB reconciled with the old script: the next apply fails with SQLSTATE
42P16. After running the new reconcile script once, apply reports no
changes and the plan is empty.
- `events_pkey_repair_runs_before_reused_apply_and_matches_reconcile`
checks two things: the reconcile script carries the repair block
verbatim, and `start-relay-for-tests.sh` runs the repair before
`pgschema apply`. It fails if the two copies drift.
- I ran the repair script and then the bootstrap against four local
probe databases: one reconciled with main's script, an empty one, one
applied but never reconciled, and one with no `events` key. All four end
with an empty plan. With another session holding a lock on `events`, the
repair fails after 10s with the hint and leaves the key untouched. It
succeeds once the lock is released.
- `admin_schema_parity_between_desired_state_and_migrations` and the
`runtime::migration` unit tests pass. fmt and clippy are clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Ian Oberst <ioberst@block.xyz>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 10:57:49 -07:00

284 lines
12 KiB
SQL

-- Reconcile schema details that pgschema does not preserve.
--
-- pgschema reconciles DDL, but it does not execute seed DML or preserve every
-- table storage parameter from schema/schema.sql. It also currently emits
-- partition children as standalone CREATE TABLE statements. Every pgschema
-- apply caller must run this idempotent script so fresh bootstraps converge on
-- the same live database contract as migration-managed databases.
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p_past'::regclass
) THEN
-- pgschema may copy parent triggers onto standalone children. Drop
-- those copies before ATTACH; PostgreSQL recreates inherited parent
-- triggers while attaching and rejects same-named child triggers.
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p_past;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p_past;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p_past;
ALTER TABLE events ATTACH PARTITION events_p_past
FOR VALUES FROM (MINVALUE) TO ('2026-01-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_01'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_01;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_01;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_01;
ALTER TABLE events ATTACH PARTITION events_p2026_01
FOR VALUES FROM ('2026-01-01') TO ('2026-02-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_02'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_02;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_02;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_02;
ALTER TABLE events ATTACH PARTITION events_p2026_02
FOR VALUES FROM ('2026-02-01') TO ('2026-03-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_03'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_03;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_03;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_03;
ALTER TABLE events ATTACH PARTITION events_p2026_03
FOR VALUES FROM ('2026-03-01') TO ('2026-04-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_04'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_04;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_04;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_04;
ALTER TABLE events ATTACH PARTITION events_p2026_04
FOR VALUES FROM ('2026-04-01') TO ('2026-05-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_05'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_05;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_05;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_05;
ALTER TABLE events ATTACH PARTITION events_p2026_05
FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p2026_06'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p2026_06;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p2026_06;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p2026_06;
ALTER TABLE events ATTACH PARTITION events_p2026_06
FOR VALUES FROM ('2026-06-01') TO ('2026-07-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'events'::regclass
AND inhrelid = 'events_p_future'::regclass
) THEN
DROP TRIGGER IF EXISTS events_created_at_floor ON events_p_future;
DROP TRIGGER IF EXISTS community_write_fence_events ON events_p_future;
DROP TRIGGER IF EXISTS trg_events_guard_channel_roster_snapshot ON events_p_future;
ALTER TABLE events ATTACH PARTITION events_p_future
FOR VALUES FROM ('2026-07-01') TO (MAXVALUE);
END IF;
-- When pgschema creates partition children as standalone tables, it also
-- preserves the parent's identity column on delivery_log children. PostgreSQL
-- rejects attaching a child table that has its own identity column, so each
-- delivery_log attach path drops that standalone identity first. Raw
-- schema-created partitions are already attached, so these branches do not
-- run against inherited partition columns.
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p_past'::regclass
) THEN
ALTER TABLE delivery_log_p_past ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p_past;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p_past
FOR VALUES FROM (MINVALUE) TO ('2026-03-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p2026_03'::regclass
) THEN
ALTER TABLE delivery_log_p2026_03 ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_03;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_03
FOR VALUES FROM ('2026-03-01') TO ('2026-04-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p2026_04'::regclass
) THEN
ALTER TABLE delivery_log_p2026_04 ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_04;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_04
FOR VALUES FROM ('2026-04-01') TO ('2026-05-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p2026_05'::regclass
) THEN
ALTER TABLE delivery_log_p2026_05 ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_05;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_05
FOR VALUES FROM ('2026-05-01') TO ('2026-06-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p2026_06'::regclass
) THEN
ALTER TABLE delivery_log_p2026_06 ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p2026_06;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p2026_06
FOR VALUES FROM ('2026-06-01') TO ('2026-07-01');
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_inherits
WHERE inhparent = 'delivery_log'::regclass
AND inhrelid = 'delivery_log_p_future'::regclass
) THEN
ALTER TABLE delivery_log_p_future ALTER COLUMN id DROP IDENTITY IF EXISTS;
DROP TRIGGER IF EXISTS community_write_fence_delivery_log ON delivery_log_p_future;
ALTER TABLE delivery_log ATTACH PARTITION delivery_log_p_future
FOR VALUES FROM ('2026-07-01') TO (MAXVALUE);
END IF;
END $$;
-- pgschema moves the partition key to the front of a partitioned table's
-- primary key when it creates the table, so events gets
-- PRIMARY KEY (created_at, community_id, id) instead of the declared
-- (community_id, created_at, id). Every later plan then drops and re-adds
-- events_pkey and each declared child's key, which fails once the children are
-- attached because their keys are inherited. Rebuild the key on the parent in
-- the declared order; PostgreSQL recreates each partition's key with it.
-- delivery_log already declares its partition key first. This block is a
-- verbatim copy of scripts/repair-events-pkey.sql; keep the two identical.
DO $$
DECLARE
declared CONSTANT text := 'PRIMARY KEY (community_id, created_at, id)';
BEGIN
-- Also runs before pgschema apply, when events may not exist yet.
IF to_regclass('events') IS NULL THEN
RETURN;
END IF;
IF (
SELECT pg_get_constraintdef(oid) FROM pg_constraint
WHERE conrelid = 'events'::regclass AND contype = 'p'
) IS DISTINCT FROM declared THEN
-- The rebuild locks events and every partition ACCESS EXCLUSIVE. Fail
-- fast instead of queueing behind a relay that still uses them.
BEGIN
PERFORM set_config('lock_timeout', '10s', true);
ALTER TABLE events DROP CONSTRAINT IF EXISTS events_pkey;
ALTER TABLE events
ADD CONSTRAINT events_pkey PRIMARY KEY (community_id, created_at, id);
EXCEPTION WHEN lock_not_available THEN
RAISE EXCEPTION 'timed out locking events to rebuild events_pkey'
USING HINT = 'Stop any relay using this database, then rerun.';
END;
END IF;
IF EXISTS (
SELECT 1 FROM pg_constraint
WHERE contype = 'p'
AND conrelid IN (
SELECT 'events'::regclass
UNION ALL
SELECT inhrelid FROM pg_inherits WHERE inhparent = 'events'::regclass
)
AND pg_get_constraintdef(oid) <> declared
) THEN
RAISE EXCEPTION 'events and its partitions must have %', declared;
END IF;
END $$;
-- pgschema drops multi-column CHECK constraints. Restore the direct-action
-- shape CHECK from schema/schema.sql (and migration 0055) verbatim.
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint
WHERE conrelid = 'relay_admin_actions'::regclass
AND conname = 'relay_admin_actions_direct_shape'
) THEN
ALTER TABLE relay_admin_actions
ADD CONSTRAINT relay_admin_actions_direct_shape CHECK (
report_id IS NOT NULL
OR (action = 'ban' AND enforcement_target_pubkey IS NOT NULL
AND timeout_secs IS NULL AND timeout_until IS NULL)
OR (action = 'timeout' AND enforcement_target_pubkey IS NOT NULL
AND timeout_secs IS NOT NULL AND timeout_until IS NOT NULL)
OR (action = 'delete' AND enforcement_target_event_id IS NOT NULL
AND enforcement_target_pubkey IS NOT NULL
AND timeout_secs IS NULL AND timeout_until IS NULL)
);
END IF;
END $$;
-- pgschema reconciles DDL but does not apply seed DML or table storage
-- parameters from schema/schema.sql. Restore those parts of the desired-state
-- contract explicitly and fail the bootstrap if the live catalog disagrees.
ALTER TABLE replica_heartbeat SET (vacuum_truncate = false);
INSERT INTO replica_heartbeat (id) VALUES (1)
ON CONFLICT (id) DO NOTHING;
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1
FROM pg_class AS relation
JOIN pg_namespace AS namespace ON namespace.oid = relation.relnamespace
WHERE namespace.nspname = current_schema()
AND relation.relname = 'replica_heartbeat'
AND COALESCE(
relation.reloptions @> ARRAY['vacuum_truncate=false']::text[],
false
)
) THEN
RAISE EXCEPTION 'replica_heartbeat must disable vacuum truncation after pgschema apply';
END IF;
IF (SELECT count(*) FROM replica_heartbeat WHERE id = 1) <> 1 THEN
RAISE EXCEPTION 'replica_heartbeat must contain its singleton row after pgschema apply';
END IF;
END $$;