mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
## Summary Re-applying `schema/schema.sql` to a database that `pgschema apply` and `scripts/reconcile-schema-after-pgschema.sql` already bootstrapped fails with `cannot drop inherited constraint "events_p2026_01_pkey" of relation "events_p2026_01" (SQLSTATE 42P16)`. Because of that, `scripts/start-relay-for-tests.sh` breaks against a reused database. **Cause:** pgschema 1.7.4 moves a partitioned table's partition key to the front of its primary key when it creates the table. A pgschema-bootstrapped `events` therefore gets `PRIMARY KEY (created_at, community_id, id)`. `schema/tables/public/events.sql` and `migrations/0001` both declare `(community_id, created_at, id)`. Every later plan drops and re-adds `events_pkey` and each declared child's key. This happens even on a database that was never reconciled. The plan only becomes unexecutable after the reconcile script attaches the children, because their keys are then inherited. As a side effect, pgschema-bootstrapped databases also had a differently ordered PK index than migration-managed ones. **Fix:** after the partition ATTACH block, the reconcile script now compares `events_pkey` with the declared `PRIMARY KEY (community_id, created_at, id)`. If they differ, it rebuilds the key on the parent. It then asserts that `events` and every partition carry the declared key, which is the "convergence plus live assertion" shape `docs/database.md` requires. PostgreSQL recreates each partition's key with the same `<child>_pkey` names, so a reconciled re-apply now plans nothing. The block does nothing when the key is already correct. `migrations/0001` is untouched. Only pgschema-bootstrapped dev and test databases run this script, so it does not touch the out-of-band partitioned-table procedure. `delivery_log` is not affected. Its key `(delivered_at, id)` already starts with the partition key. **Existing local databases:** a database reconciled before this change would still fail its next `pgschema apply`, because the reconcile script only runs after apply. The repair therefore also lives in `scripts/repair-events-pkey.sql`. That file does nothing when `events` doesn't exist yet, and `scripts/start-relay-for-tests.sh` runs it before apply; it's the only launcher that reuses its database. The reconcile script keeps a verbatim copy for fresh bootstraps, and a unit test keeps the two identical. The rebuild drops the key with `IF EXISTS`, so a database that has lost its key still converges. It also takes its locks with a 10s `lock_timeout`. If a relay still holds `events`, it fails with "Stop any relay using this database, then rerun" instead of hanging. `start-relay-for-tests.sh` now builds the relay's `DATABASE_URL` from the `PG*` variables it already exports. The secret scanner flags the literal URI once the file is touched. The value is unchanged. pgschema 1.13.3 keeps the declared order and creates children as native `PARTITION OF`. Upgrading would make this block and the ATTACH blocks unnecessary, but it changes behavior for every caller, so it is left for a separate change. ### Related issue Follow-up to block/buzz#8487, which added the probe test and `.pgschemaignore`. Now rebased onto `main` after that PR merged. ### Testing - `pgschema_reapply_keeps_runtime_partitions` now asserts the plan after the maintenance pass is **empty**, which replaces the `DROP TABLE` and per-monthly plan checks. It then applies and reconciles a second time. Without the new block it fails with `ALTER TABLE events DROP CONSTRAINT events_pkey; ...`. With the block it passes. - It also includes the optional notes from the #8487 review: - After maintenance, it checks that both catch-alls are still present and attached, not only that the zipped bounds differ. - The first runtime monthly comes from the declared catch-all's lower bound instead of a hard-coded `p2026_07`. - The doc comment now states only what the test pins: bounds are not compared, and the plan is empty. - I broke the rebuild on purpose by re-adding the key in the wrong order. The new assertion fails the reconcile script with `events and its partitions must have PRIMARY KEY (community_id, created_at, id)`. - CLI runs against local Postgres 17: - Fresh DB: apply → reconcile → apply → reconcile → plan. The second apply reports no changes and the plan is empty. - DB reconciled with the old script: the next apply fails with SQLSTATE 42P16. After running the new reconcile script once, apply reports no changes and the plan is empty. - `events_pkey_repair_runs_before_reused_apply_and_matches_reconcile` checks two things: the reconcile script carries the repair block verbatim, and `start-relay-for-tests.sh` runs the repair before `pgschema apply`. It fails if the two copies drift. - I ran the repair script and then the bootstrap against four local probe databases: one reconciled with main's script, an empty one, one applied but never reconciled, and one with no `events` key. All four end with an empty plan. With another session holding a lock on `events`, the repair fails after 10s with the hint and leaves the key untouched. It succeeds once the lock is released. - `admin_schema_parity_between_desired_state_and_migrations` and the `runtime::migration` unit tests pass. fmt and clippy are clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Ian Oberst <ioberst@block.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
208 lines
8.4 KiB
Bash
Executable File
208 lines
8.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# =============================================================================
|
|
# start-relay-for-tests.sh — Start the Buzz relay and its backing services
|
|
# =============================================================================
|
|
# Shared script for CI jobs that need a running relay. Starts docker compose
|
|
# services, waits for health, applies the schema, builds the relay, starts it,
|
|
# and polls readiness.
|
|
#
|
|
# Usage:
|
|
# ./scripts/start-relay-for-tests.sh [--profile <cargo-profile>] [--no-build]
|
|
#
|
|
# Options:
|
|
# --profile <profile> Cargo build profile (default: ci)
|
|
# --no-build Use existing target/<profile>/ binaries (CI artifact reuse)
|
|
#
|
|
# Exports:
|
|
# RELAY_URL=ws://localhost:3000
|
|
# =============================================================================
|
|
set -euo pipefail
|
|
|
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
|
|
# ── Defaults ──────────────────────────────────────────────────────────────────
|
|
|
|
CARGO_PROFILE="${CARGO_PROFILE:-ci}"
|
|
SKIP_BUILD=false
|
|
|
|
# ── Parse args ────────────────────────────────────────────────────────────────
|
|
|
|
while [[ $# -gt 0 ]]; do
|
|
case "$1" in
|
|
--profile)
|
|
CARGO_PROFILE="$2"
|
|
shift 2
|
|
;;
|
|
--no-build)
|
|
SKIP_BUILD=true
|
|
shift
|
|
;;
|
|
*)
|
|
echo "Unknown option: $1" >&2
|
|
exit 1
|
|
;;
|
|
esac
|
|
done
|
|
|
|
# ── Colors ────────────────────────────────────────────────────────────────────
|
|
|
|
BLUE='\033[0;34m'
|
|
GREEN='\033[0;32m'
|
|
RED='\033[0;31m'
|
|
NC='\033[0m'
|
|
|
|
log() { echo -e "${BLUE}[relay-test]${NC} $*"; }
|
|
ok() { echo -e "${GREEN}[relay-test]${NC} $*"; }
|
|
err() { echo -e "${RED}[relay-test]${NC} $*" >&2; }
|
|
|
|
# ── Start docker compose services ────────────────────────────────────────────
|
|
|
|
cd "${REPO_ROOT}"
|
|
|
|
log "Starting docker compose services..."
|
|
docker compose up -d postgres redis minio minio-init
|
|
|
|
# ── Wait for services to be healthy ──────────────────────────────────────────
|
|
|
|
wait_healthy() {
|
|
local service="$1"
|
|
local container="$2"
|
|
log "Waiting for ${service}..."
|
|
for attempt in $(seq 1 60); do
|
|
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
|
|
if [ "${status}" = "healthy" ]; then
|
|
ok "${service} is healthy"
|
|
return 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
err "${service} did not become healthy within 120s"
|
|
docker logs "${container}" || true
|
|
return 1
|
|
}
|
|
|
|
wait_healthy "Postgres" "buzz-postgres"
|
|
wait_healthy "Redis" "buzz-redis"
|
|
wait_healthy "MinIO" "buzz-minio"
|
|
|
|
# ── Apply database schema ────────────────────────────────────────────────────
|
|
|
|
log "Applying database schema..."
|
|
export PGHOST=localhost
|
|
export PGPORT=5432
|
|
export PGUSER=buzz
|
|
export PGPASSWORD=buzz_dev
|
|
export PGDATABASE=buzz
|
|
|
|
# Use the already-running docker postgres for desired-state planning instead of
|
|
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
|
|
export PGSCHEMA_PLAN_HOST=localhost
|
|
export PGSCHEMA_PLAN_PORT=5432
|
|
export PGSCHEMA_PLAN_DB=buzz
|
|
export PGSCHEMA_PLAN_USER=buzz
|
|
export PGSCHEMA_PLAN_PASSWORD=buzz_dev
|
|
|
|
# This database persists across runs. One reconciled before the events_pkey
|
|
# repair existed would fail apply, so repair it first (see the script).
|
|
docker exec -i -e PGPASSWORD="${PGPASSWORD}" buzz-postgres \
|
|
psql -U "${PGUSER}" -d "${PGDATABASE}" -v ON_ERROR_STOP=1 < scripts/repair-events-pkey.sql
|
|
./bin/pgschema apply --file schema/schema.sql --auto-approve
|
|
docker exec -i -e PGPASSWORD="${PGPASSWORD}" buzz-postgres \
|
|
psql -U "${PGUSER}" -d "${PGDATABASE}" -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
|
|
ok "Schema applied"
|
|
|
|
# ── Seed the deployment community ────────────────────────────────────────────
|
|
# Multi-tenant: the relay resolves every connection's tenant from the durable
|
|
# communities host map (WHERE host = normalize_host($1)). normalize_host keeps
|
|
# non-default ports, so the host must be 'localhost:3000' verbatim to match
|
|
# RELAY_URL=ws://localhost:3000. The relay never auto-seeds a community
|
|
# (ensure_configured_community has no callers) and fails closed on an unmapped
|
|
# host, so without this row every e2e connection would 404 at host-binding.
|
|
# The unique index is on lower(host), so ON CONFLICT must target that expression.
|
|
# psql is not on PATH in the hermit env; postgres runs as the buzz-postgres
|
|
# docker container, so exec into it (same fallback as setup-desktop-test-data.sh).
|
|
log "Seeding deployment community (host=localhost:3000)..."
|
|
if command -v psql >/dev/null 2>&1; then
|
|
seed_psql() { PGPASSWORD="${PGPASSWORD}" psql -h "${PGHOST}" -p "${PGPORT}" -U "${PGUSER}" -d "${PGDATABASE}" -qtA "$@"; }
|
|
else
|
|
seed_psql() { docker exec -e PGPASSWORD="${PGPASSWORD}" buzz-postgres psql -U "${PGUSER}" -d "${PGDATABASE}" -qtA "$@"; }
|
|
fi
|
|
seed_psql -c "
|
|
INSERT INTO communities (id, host)
|
|
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
|
|
ON CONFLICT (lower(host)) DO NOTHING
|
|
;
|
|
"
|
|
ok "Community seeded"
|
|
|
|
# ── Build relay ──────────────────────────────────────────────────────────────
|
|
|
|
if [[ "${SKIP_BUILD}" == "true" ]]; then
|
|
for bin in buzz-relay git-credential-nostr; do
|
|
if [[ ! -x "./target/${CARGO_PROFILE}/${bin}" ]]; then
|
|
err "--no-build: ./target/${CARGO_PROFILE}/${bin} missing or not executable"
|
|
exit 1
|
|
fi
|
|
done
|
|
log "Skipping relay build (--no-build); using existing target/${CARGO_PROFILE}/ binaries"
|
|
else
|
|
log "Building relay (profile: ${CARGO_PROFILE})..."
|
|
cargo build --profile "${CARGO_PROFILE}" -p buzz-relay -p git-credential-nostr
|
|
ok "Relay built"
|
|
fi
|
|
|
|
# ── Start relay ──────────────────────────────────────────────────────────────
|
|
|
|
log "Starting relay..."
|
|
|
|
TEST_RELAY_PRIVATE_KEY="${BUZZ_RELAY_PRIVATE_KEY:-$(openssl rand -hex 32)}"
|
|
|
|
# Optional NIP-43 membership gating: exported by callers that need a
|
|
# membership-gated relay (e.g. the mesh lifecycle smoke). All three must be
|
|
# set together — the relay fails fast otherwise.
|
|
MEMBERSHIP_ENV=()
|
|
if [[ "${BUZZ_REQUIRE_RELAY_MEMBERSHIP:-}" == "true" ]]; then
|
|
MEMBERSHIP_ENV+=(
|
|
BUZZ_REQUIRE_RELAY_MEMBERSHIP=true
|
|
RELAY_OWNER_PUBKEY="${RELAY_OWNER_PUBKEY:?RELAY_OWNER_PUBKEY required with BUZZ_REQUIRE_RELAY_MEMBERSHIP=true}"
|
|
)
|
|
: "${BUZZ_RELAY_PRIVATE_KEY:?BUZZ_RELAY_PRIVATE_KEY required with BUZZ_REQUIRE_RELAY_MEMBERSHIP=true}"
|
|
log "Membership gating enabled (NIP-43)"
|
|
fi
|
|
|
|
nohup env \
|
|
DATABASE_URL="postgres://${PGUSER}:${PGPASSWORD}@${PGHOST}:${PGPORT}/${PGDATABASE}" \
|
|
REDIS_URL=redis://localhost:6379 \
|
|
RELAY_URL=ws://localhost:3000 \
|
|
BUZZ_BIND_ADDR=0.0.0.0:3000 \
|
|
BUZZ_RELAY_PRIVATE_KEY="${TEST_RELAY_PRIVATE_KEY}" \
|
|
BUZZ_REQUIRE_AUTH_TOKEN=false \
|
|
BUZZ_RECONCILE_CHANNELS=true \
|
|
BUZZ_GIT_PROBE_WRITERS=8 \
|
|
${MEMBERSHIP_ENV[@]+"${MEMBERSHIP_ENV[@]}"} \
|
|
"./target/${CARGO_PROFILE}/buzz-relay" > /tmp/buzz-relay.log 2>&1 &
|
|
echo $! > /tmp/buzz-relay.pid
|
|
|
|
# ── Poll readiness ───────────────────────────────────────────────────────────
|
|
|
|
log "Waiting for relay readiness..."
|
|
for attempt in $(seq 1 60); do
|
|
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
|
|
err "Relay process died"
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
fi
|
|
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
|
|
if [ "${status_code}" = "200" ]; then
|
|
ok "Relay is ready at ws://localhost:3000"
|
|
export RELAY_URL=ws://localhost:3000
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
|
|
err "Relay did not become ready within 60s"
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|