fix(db): disable heartbeat vacuum truncation (#6898)

## Why
`replica_heartbeat` is updated continuously, but autovacuum heap
truncation can take an `ACCESS EXCLUSIVE` lock whose replay cancels
concurrent hot-standby reads. Truncating a fixed, single-row table
provides no meaningful space benefit.

## What
- Add migration 0034 to set `vacuum_truncate = false` on
`replica_heartbeat`
- Keep the desired-state schema aligned for fresh installations
- Extend migration coverage so both paths retain the reloption

## Risk Assessment
Low — this disables only autovacuum's heap-truncation phase for a
single-row table; normal vacuum cleanup remains enabled. The tradeoff is
retaining at most the table's otherwise-truncatable tail pages.

## References
- [Investigation
thread](https://sq-block.slack.com/archives/C0B2WT43BDH/p1787760831375989)
- [PostgreSQL table storage
parameters](https://www.postgresql.org/docs/current/sql-createtable.html#SQL-CREATETABLE-STORAGE-PARAMETERS)

Generated with Codex

---

**Update Aug 27, 2026:** Hardened fresh `pgschema` bootstraps after
review.

- Renamed the shared post-apply script to
`reconcile-schema-after-pgschema.sql` and made it restore the heartbeat
reloption and singleton row idempotently.
- Added live catalog and row assertions that stop bootstrap when the
database does not match the required state.
- Updated every repository `pgschema apply` caller, including the
desktop release smoke path.
- Added a regression test that requires future `pgschema apply` callers
to run reconciliation.
- Documented the `pgschema` DML and storage-parameter limitation in
`AGENTS.md` and at the reconciliation entry point.
- Merged current `main` and renumbered this migration to 0034 after
`main` added migration 0033.

---------

Signed-off-by: Luke Tornquist <tornquist@squareup.com>
This commit is contained in:
Luke Tornquist
2026-08-27 13:01:13 -04:00
committed by GitHub
parent c363ce13c0
commit 0ccf934b88
9 changed files with 135 additions and 14 deletions

View File

@@ -1,11 +1,10 @@
-- Attach partition child tables after pgschema apply.
-- Reconcile schema details that pgschema does not preserve.
--
-- pgschema currently emits existing partition children as standalone CREATE TABLE
-- statements when applying schema/schema.sql in CI. The tables exist, but they
-- are not attached to their partitioned parents, so inserts into events or
-- delivery_log fail with "no partition of relation ... found for row". Keep this
-- idempotent: raw psql/schema.sql already attaches these partitions, while
-- pgschema-created schemas need this repair step.
-- pgschema reconciles DDL, but it does not execute seed DML or preserve every
-- table storage parameter from schema/schema.sql. It also currently emits
-- partition children as standalone CREATE TABLE statements. Every pgschema
-- apply caller must run this idempotent script so fresh bootstraps converge on
-- the same live database contract as migration-managed databases.
DO $$
BEGIN
@@ -197,3 +196,32 @@ BEGIN
FOR VALUES FROM ('2026-07-01') TO (MAXVALUE);
END IF;
END $$;
-- pgschema reconciles DDL but does not apply seed DML or table storage
-- parameters from schema/schema.sql. Restore those parts of the desired-state
-- contract explicitly and fail the bootstrap if the live catalog disagrees.
ALTER TABLE replica_heartbeat SET (vacuum_truncate = false);
INSERT INTO replica_heartbeat (id) VALUES (1)
ON CONFLICT (id) DO NOTHING;
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1
FROM pg_class AS relation
JOIN pg_namespace AS namespace ON namespace.oid = relation.relnamespace
WHERE namespace.nspname = current_schema()
AND relation.relname = 'replica_heartbeat'
AND COALESCE(
relation.reloptions @> ARRAY['vacuum_truncate=false']::text[],
false
)
) THEN
RAISE EXCEPTION 'replica_heartbeat must disable vacuum truncation after pgschema apply';
END IF;
IF (SELECT count(*) FROM replica_heartbeat WHERE id = 1) <> 1 THEN
RAISE EXCEPTION 'replica_heartbeat must contain its singleton row after pgschema apply';
END IF;
END $$;

View File

@@ -89,7 +89,7 @@ export PGSCHEMA_PLAN_HOST=localhost PGSCHEMA_PLAN_PORT=5432 PGSCHEMA_PLAN_DB="${
export PGSCHEMA_PLAN_USER=buzz PGSCHEMA_PLAN_PASSWORD=buzz_dev
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d "${DB_NAME}" -v ON_ERROR_STOP=1 < scripts/attach-schema-partitions.sql
psql -U buzz -d "${DB_NAME}" -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
BUZZ_DB_NAME="${DB_NAME}" BUZZ_COMMUNITY_HOST="${COMMUNITY_HOST}" ./scripts/setup-desktop-test-data.sh
docker exec buzz-redis redis-cli -n "${REDIS_DB}" FLUSHDB >/dev/null
phase database "${phase_start}"

View File

@@ -98,7 +98,7 @@ export PGSCHEMA_PLAN_HOST=localhost PGSCHEMA_PLAN_PORT=${PG_PORT}
export PGSCHEMA_PLAN_DB=buzz PGSCHEMA_PLAN_USER=buzz PGSCHEMA_PLAN_PASSWORD=buzz_dev
export PGHOST=localhost PGPORT=${PG_PORT} PGUSER=buzz PGDATABASE=buzz
./bin/pgschema apply --file schema/schema.sql --auto-approve
psql_h < scripts/attach-schema-partitions.sql
psql_h < scripts/reconcile-schema-after-pgschema.sql
ok "Schema applied"
# ── Deployment community + channels + members ────────────────────────────────

View File

@@ -105,7 +105,7 @@ export PGSCHEMA_PLAN_PASSWORD=buzz_dev
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD="${PGPASSWORD}" buzz-postgres \
psql -U "${PGUSER}" -d "${PGDATABASE}" -v ON_ERROR_STOP=1 < scripts/attach-schema-partitions.sql
psql -U "${PGUSER}" -d "${PGDATABASE}" -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
ok "Schema applied"
# ── Seed the deployment community ────────────────────────────────────────────