Files
buzz/.github/workflows/_ci-relay.yml
Brad Seiler 96af488d96 fix(media): revoke private channel attachment reads (#9077)
## Summary

Removed private-channel members can currently download attachments if
they know the URL. With `BUZZ_MEDIA_READ_MODE=enforce`, fresh GET, HEAD
and Range requests check current membership and readable references on
the primary database. Removed members receive 404; remaining members
retain access, and rejoining restores access. Thumbnail aliases use the
parent attachment’s permissions.

Publication checks also prevent copied private URLs from granting new
access. They cover attachments, link previews, profile pictures,
banners, group pictures and emoji. Persona creation and changed-avatar
edits, plus team and individual snapshot imports, transfer animated
posters and animations under the agent’s signer before profile
publication. Failed persona transfers preserve existing local records;
import failures appear in the results.

Authorization uses bounded queries, a two-second statement deadline and
transaction-local JIT suppression. An index on edit targets avoids broad
scans. Publications have a 512-reference limit, and capacity waits up to
500 ms before refusal.

Protection defaults off. Existing downloaded bytes cannot be recalled.
All modes stop creating immutable HTTP cache entries.

### Related issue

Related approaches: #7565, #5185, #8451 and #8444. This change evaluates
live stored references directly.

### Testing

Published head: `3d1aa7431cc495a001a0f13e95e2e422c5a77ab1`, rebased on
main `2cd40badb85de059a6916206e133da7a0a88fb2e`. Push hooks were
**SKIPPED** with explicit author approval. Current-head HTTP, full
package, PostgreSQL migration/media and profile-filter mutation checks
are pending; CI is pending. The earlier complete HTTP journey passed at
`f20b449262` (identical source tree), with its evidence attributed
separately. Earlier failing preview/report-fixture runs are preserved;
they do not count as passes.

The HTTP journey covers upload/publication, fresh GET/HEAD/Range
revocation and rejoin, emoji replacement, edited previews, persona
visibility and non-reference decoys. The migration regression forces an
index timeout, verifies no 0086 history or media tables, then proves a
migrator through the base's 0085 can restart.

Human attachment removal/rejoin testing passed on the pinned
`1f08a8e0fd2edf19dc94321f7258518149fe1bf6` trial. Native animated
persona edits and team/individual import acceptance remain incomplete.
That old-snapshot database stays intact and cannot run the reconciled
binary; a new trial needs a separate fresh database.

### Rollout prerequisites

- Build and verify the reference index before shadow mode. Small
databases can auto-migrate through one atomic 0086 migration within the
configured timeouts. It builds the index before creating the media
tables, so failure records no new migration version. Large deployments
must disable auto-migration and prebuild indexes concurrently.
- **Enforce remains blocked unless the operator can rule out SDK/CLI
attachment edits and the unverified agent-harness edit producer.**
SDK/CLI edits still omit attachment metadata. That behavior is deferred,
not fixed. Removing this blocker requires latest-effective attachment
metadata across successive edits, no restoration of removed files, and
an upload → successive edits → GET/HEAD regression.
- Validate client signer compatibility and split emoji publications
above 512 references before enforce mode.
- Complete the human workflows above in an isolated community with
enforce enabled.
- Follow the [index deployment
guide](https://github.com/block/buzz/blob/honor/private-channel-media/docs/private-media-index-deployment.md)
and [operator
guidance](https://github.com/block/buzz/blob/honor/private-channel-media/docs/operator-private-media.md)
for rollout and rollback.

The [prebuild timeout
finding](https://github.com/block/buzz/pull/9077#discussion_r4486221478)
and [avatar-transfer
finding](https://github.com/block/buzz/pull/9077#discussion_r4486221458)
are implemented and independently reviewed.

### Schema change checklist

- [x] Additive expand and code deployment; no backfill or contract
phase. Prebuild/verify before shadow under either schema deployment
path.
- [x] No SQL function, trigger, procedure, rule or foreign key added.
Signed events are untouched.
- [x] No migration mutates existing rows.
- [x] Migration and desired-state schema match. Catalog parity, target
parsing, future partition inheritance and upgrade refusal are covered.
- [x] Old/new binaries coexist for serving. Suspend community deletion
during expansion and while rolled back to a binary with the old deletion
catalog.
- [x] Keep auto-migration disabled on old binaries after schema
expansion; sqlx rejects unknown versions.
- [x] Migration lock timeout 1 s; statement timeout 5 s. Small populated
databases build inline within those budgets; large deployments use
concurrent child-index prebuild/attachment with auto-migration off.
Missing, invalid or wrong indexes fail verification.

Generated with Codex

---------

Signed-off-by: Brad Seiler <seiler@squareup.com>
Co-authored-by: Honor <e0bf479eeebe47dde1e35705212a4df70616a7bc834d907ddfb18cf0405f8b1b@seiler-test.communities.public.builderlabstaging.xyz>
2026-11-06 22:35:00 +00:00

724 lines
34 KiB
YAML

name: CI / Relay and PostgreSQL
on:
workflow_call:
inputs:
rust:
required: true
type: boolean
desktop:
required: true
type: boolean
desktop_rust:
required: true
type: boolean
lane:
required: true
type: string
outputs:
desktop_e2e_relay_result:
value: ${{ jobs.results.outputs.desktop_e2e_relay_result }}
desktop_e2e_integration_result:
value: ${{ jobs.results.outputs.desktop_e2e_integration_result }}
backend_integration_result:
value: ${{ jobs.results.outputs.backend_integration_result }}
relay_e2e_result:
value: ${{ jobs.results.outputs.relay_e2e_result }}
postgres_tests_result:
value: ${{ jobs.results.outputs.postgres_tests_result }}
env:
CARGO_TERM_COLOR: always
BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
jobs:
# The relay binaries and the two nextest archives compile independently, so
# they build in parallel; consumers download every desktop-e2e-relay-*
# artifact into target/ci as before.
desktop-e2e-relay:
name: Desktop E2E Relay (${{ matrix.artifact }})
runs-on: ubuntu-latest
timeout-minutes: 45
if: inputs.lane == 'artifacts' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
strategy:
fail-fast: false
matrix:
include:
- artifact: relay-binaries
paths: |
target/ci/buzz-relay
target/ci/git-credential-nostr
- artifact: backend-integration-tests
paths: target/ci/backend-integration-tests.tar.zst
- artifact: postgres-tests
paths: target/ci/postgres-tests.tar.zst
permissions:
contents: read
env:
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ (github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.number == 5224)) && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# Reuse each artifact when none of its inputs changed (desktop-only PRs
# hit this every time). The key covers everything they embed, including
# migrations via sqlx migrate!.
- name: Restore relay artifacts cache
id: relay-artifacts-cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ matrix.paths }}
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
- uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
# All lanes restore the shared Rust cache; only the postgres-tests lane,
# which compiles the widest test graph, writes it so a faster lane
# cannot save a narrower target dir under the same key.
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
with:
workspaces: |
.
desktop/src-tauri
save-if: ${{ github.event_name != 'pull_request' && matrix.artifact == 'postgres-tests' }}
# Cache rustc outputs for unchanged workspace crates. Trusted pushes write;
# the bounded PR 5224 trial writes only to its isolated merge-ref scope.
- name: Set up sccache
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
uses: Mozilla-Actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11 # zizmor: ignore[cache-poisoning] Bounded trial: only PR 5224 writes to its isolated merge-ref scope; trusted pushes retain production writes.
with:
version: v0.16.0
# The sccache server checks its GHA cache storage at startup and refuses to
# start when that fails (egress over the account limit, a failed read),
# which used to fail cargo's first `sccache rustc -vV` and the build.
# SCCACHE_IGNORE_SERVER_IO_ERROR doesn't cover startup in v0.16.0, so start
# it here and build with plain rustc if it won't start.
- name: Start sccache
id: sccache
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
run: |
if sccache --start-server; then
echo "wrapper=sccache" >> "$GITHUB_OUTPUT"
else
echo "::warning::sccache did not start; building without it"
echo "wrapper=" >> "$GITHUB_OUTPUT"
fi
- name: Install cargo-nextest
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact != 'relay-binaries'
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Build relay binaries
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'relay-binaries'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: cargo build --profile ci -p buzz-relay -p git-credential-nostr
- name: Build backend integration test archive
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'backend-integration-tests'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: |
cargo nextest archive \
--cargo-profile ci \
-p buzz-db \
-p buzz-relay \
-p buzz-relay-mesh \
-p buzz-test-client \
--lib \
--test e2e_event_reminder \
--test e2e_author_only_deletion \
--test e2e_existence_oracle \
--archive-file target/ci/backend-integration-tests.tar.zst
- name: Build PostgreSQL test archive
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'postgres-tests'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: |
postgres_package_args=()
while IFS= read -r package; do
postgres_package_args+=(-p "$package")
done < <(scripts/postgres-test-packages.sh)
if [[ "${#postgres_package_args[@]}" -eq 0 ]]; then
echo "no PostgreSQL test packages were discovered" >&2
exit 1
fi
cargo nextest archive \
--cargo-profile ci \
"${postgres_package_args[@]}" \
--lib \
--tests \
--archive-file target/ci/postgres-tests.tar.zst
- name: Save relay artifacts cache
# PR-scoped exact-source entries cannot warm main or other PRs and churn
# the shared cache pool. sccache provides read-only PR reuse instead.
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && github.event_name == 'push'
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ matrix.paths }}
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
- name: Upload relay artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: desktop-e2e-relay-${{ matrix.artifact }}
path: ${{ matrix.paths }}
if-no-files-found: error
retention-days: 1
postgres-tests:
name: PostgreSQL Tests
runs-on: ubuntu-latest
timeout-minutes: 10
if: inputs.lane == 'postgres' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: buzz
POSTGRES_PASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
POSTGRES_DB: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U buzz -d postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 12
redis:
image: redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 12
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PG_BIN_DIR: /usr/bin
REDIS_URL: redis://localhost:6379
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Tune disposable PostgreSQL
env:
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
run: |
# Database-per-test cloning forces checkpoints. Durability is redundant
# for this disposable service and makes runtime depend on runner disk I/O.
psql --dbname postgres --set ON_ERROR_STOP=1 <<'SQL'
ALTER SYSTEM SET fsync = off;
ALTER SYSTEM SET full_page_writes = off;
ALTER SYSTEM SET synchronous_commit = off;
SELECT pg_reload_conf();
SQL
psql --dbname postgres --tuples-only --no-align --command \
"SELECT bool_and(setting = 'off') FROM pg_settings WHERE name IN ('fsync', 'full_page_writes', 'synchronous_commit')" \
| grep --fixed-strings --line-regexp t
- name: Install cargo-nextest
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Download backend test archive
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Restore relay executable permission
run: chmod +x ./target/ci/buzz-relay
- name: PostgreSQL-backed tests
env:
BUZZ_POSTGRES_ADMIN_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/postgres
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
run: |
scripts/postgres-test-run.sh \
--archive-file target/ci/postgres-tests.tar.zst
desktop-e2e-integration-shard:
name: Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})
runs-on: ubuntu-latest
# A shard needs ~10m; Playwright install-deps is bounded by
# ci-apt-retry.sh (~25m worst case), so a degraded mirror fails with its
# own error instead of the job timeout.
timeout-minutes: 40
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Start integration services
run: |
for attempt in 1 2 3; do
if docker compose up -d postgres redis minio minio-init; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "docker compose up failed after 3 attempts" >&2
exit 1
fi
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
sleep $((attempt * 5))
done
- name: Get pnpm store directory
id: pnpm-cache
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: pnpm-${{ runner.os }}-
- name: Install desktop dependencies
run: just desktop-install-ci
- name: Get Playwright version
id: pw-version
run: echo "version=$(cd desktop && node -e "console.log(require('@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"
- name: Restore Playwright browser cache
id: playwright-cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Install Playwright Chromium
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: cd desktop && pnpm exec playwright install chromium
- name: Install Playwright system dependencies
run: cd desktop && ../scripts/ci-apt-retry.sh pnpm exec playwright install-deps chromium
- name: Save Playwright browser cache
if: steps.playwright-cache.outputs.cache-hit != 'true' && github.event_name == 'push' && matrix.shard == 1
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Desktop E2E build
run: pnpm -C desktop build:e2e
- name: Wait for integration services
run: |
wait_healthy() {
local service="$1"
local container="$2"
for attempt in $(seq 1 60); do
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
if [ "${status}" = "healthy" ]; then
echo "${service} is healthy"
return 0
fi
sleep 2
done
docker logs "${container}" || true
return 1
}
wait_healthy "Postgres" "buzz-postgres"
wait_healthy "Redis" "buzz-redis"
wait_healthy "MinIO" "buzz-minio"
- name: Download relay binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Apply schema and seed deployment community
# MT: the relay resolves each request's tenant from the communities host
# map and fails closed on an unmapped host. The channel reconciler binds
# the deployment community ONCE at boot (outside its retry loop) and
# exits permanently on an unmapped host, so the 'localhost:3000'
# community MUST exist before the relay starts — the retry loop only
# handles late-seeded channels, not a late-seeded community. The relay
# migrates at boot via BUZZ_AUTO_MIGRATE, but that's too late for the
# pre-boot seed, so apply the schema here first (then drop AUTO_MIGRATE
# below). lower(host) is the unique index → ON CONFLICT target. psql
# isn't on PATH in hermit → exec into the buzz-postgres container.
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PGPASSWORD: buzz_dev
PGDATABASE: buzz
# Use the already-running docker postgres for desired-state planning instead of
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_DB: buzz
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
run: |
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -qtA -c "
INSERT INTO communities (id, host)
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
ON CONFLICT (lower(host)) DO NOTHING
;"
- name: Start relay
run: |
chmod +x ./target/ci/buzz-relay
nohup env \
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
REDIS_URL=redis://localhost:6379 \
RELAY_URL=ws://localhost:3000 \
BUZZ_BIND_ADDR=0.0.0.0:3000 \
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
BUZZ_REQUIRE_AUTH_TOKEN=false \
BUZZ_RECONCILE_CHANNELS=true \
BUZZ_RATE_LIMIT_HUMAN_MESSAGES_PER_MIN=100000 \
BUZZ_RATE_LIMIT_HUMAN_API_CALLS_PER_MIN=100000 \
BUZZ_RATE_LIMIT_HUMAN_WS_EVENTS_PER_SEC=10000 \
BUZZ_GIT_PROBE_WRITERS=8 \
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
echo $! > /tmp/buzz-relay.pid
for attempt in $(seq 1 60); do
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
cat /tmp/buzz-relay.log
exit 1
fi
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
if [ "${status_code}" = "200" ]; then
exit 0
fi
sleep 1
done
cat /tmp/buzz-relay.log
exit 1
- name: Seed desktop e2e data
run: bash scripts/setup-desktop-test-data.sh
- name: Desktop relay-backed e2e
run: cd desktop && pnpm exec playwright test --project=integration --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Summarize flaky tests
if: ${{ !cancelled() }}
run: node scripts/summarize-flaky-tests.mjs playwright-report.json "Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})"
working-directory: desktop
- name: Upload desktop integration artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: desktop-e2e-integration-artifacts-${{ matrix.shard }}
path: |
desktop/playwright-report
desktop/playwright-report.json
desktop/test-results
/tmp/buzz-relay.log
if-no-files-found: ignore
retention-days: 7
- name: Save pnpm store cache
if: github.event_name == 'push'
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
desktop-e2e-integration:
name: Desktop E2E Integration
runs-on: ubuntu-latest
timeout-minutes: 5
needs: [desktop-e2e-integration-shard]
if: always() && inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
permissions:
contents: read
steps:
- name: Check integration shards
run: |
if [ "${{ needs.desktop-e2e-integration-shard.result }}" != "success" ]; then
echo "Desktop E2E Integration shards finished with: ${{ needs.desktop-e2e-integration-shard.result }}"
exit 1
fi
echo "Desktop E2E Integration shards passed"
backend-integration:
name: Backend Integration (relay e2e)
runs-on: ubuntu-latest
timeout-minutes: 20
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Install cargo-nextest
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Start integration services
run: |
for attempt in 1 2 3; do
if docker compose up -d postgres redis minio minio-init; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "docker compose up failed after 3 attempts" >&2
exit 1
fi
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
sleep $((attempt * 5))
done
- name: Wait for integration services
run: |
wait_healthy() {
local service="$1"
local container="$2"
for attempt in $(seq 1 60); do
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
if [ "${status}" = "healthy" ]; then
echo "${service} is healthy"
return 0
fi
sleep 2
done
docker logs "${container}" || true
return 1
}
wait_healthy "Postgres" "buzz-postgres"
wait_healthy "Redis" "buzz-redis"
wait_healthy "MinIO" "buzz-minio"
- name: Download relay artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Apply schema and seed deployment community
# MT: the relay resolves each request's tenant from the communities host
# map and fails closed on an unmapped host. The reminder scheduler binds
# the deployment community ONCE at boot and exits permanently on an
# unmapped host (no retry, unlike the channel reconciler), so the
# 'localhost:3000' community MUST exist before the relay starts — seeding
# after boot leaves the scheduler dead. The relay migrates at boot via
# BUZZ_AUTO_MIGRATE, but that's too late for the pre-boot seed, so apply
# the schema here first (then drop AUTO_MIGRATE below). lower(host) is the
# unique index → ON CONFLICT target. psql isn't on PATH in hermit → exec
# into the buzz-postgres container.
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PGPASSWORD: buzz_dev
PGDATABASE: buzz
# Use the already-running docker postgres for desired-state planning instead of
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_DB: buzz
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
run: |
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -qtA -c "
INSERT INTO communities (id, host)
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
ON CONFLICT (lower(host)) DO NOTHING
;"
- name: Git default-branch route and clone regressions
env:
# The preceding step already applied and reconciled schema/schema.sql.
BUZZ_TEST_SCHEMA_MODE: desired
BUZZ_TEST_DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
BUZZ_TEST_REDIS_URL: redis://localhost:6379
BUZZ_TEST_S3_ENDPOINT: http://localhost:9000
BUZZ_TEST_S3_BUCKET: buzz-media
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/api::git::settings::tests::external_infra::/)' \
--run-ignored ignored-only
- name: Admin attachment access-log regression
env:
# This test builds its state from Config::for_test(), which reads the
# relay's own DATABASE_URL and BUZZ_S3_* variables.
DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
BUZZ_S3_ENDPOINT: http://localhost:9000
BUZZ_S3_BUCKET: buzz-media
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/api::admin::postgres_tests::external_infra_minio_tests::/)' \
--run-ignored ignored-only
- name: Relay quota accounting and rejection regressions
env:
BUZZ_TEST_REDIS_URL: redis://localhost:6379
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and (test(/^rejection::/) or test(/^admission::/))' \
--run-ignored all
- name: NIP-FI cross-pod command replay regressions
env:
BUZZ_TEST_REDIS_URL: redis://localhost:6379
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/^api::nip_fi::route_integration_tests::external_infra_redis::/)' \
--run-ignored ignored-only
- name: Mesh ready-registry index regressions
env:
BUZZ_TEST_REDIS_URL: redis://localhost:6379
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay-mesh) and test(/^registry::tests::external_infra_redis::/)' \
--run-ignored ignored-only
- name: Workflow message provenance unit tests
# The relay's workflow_sink suite is not selected by the infra-free
# unit job. Its ignored database cases run in the isolated PostgreSQL
# lane; keep the pure provenance cases covered here without duplication.
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/workflow_sink/)'
- name: Start relay
run: |
chmod +x ./target/ci/buzz-relay
nohup env \
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
REDIS_URL=redis://localhost:6379 \
RELAY_URL=ws://localhost:3000 \
BUZZ_BIND_ADDR=0.0.0.0:3000 \
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
BUZZ_REQUIRE_AUTH_TOKEN=false \
BUZZ_RECONCILE_CHANNELS=true \
BUZZ_GIT_PROBE_WRITERS=8 \
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
echo $! > /tmp/buzz-relay.pid
for attempt in $(seq 1 60); do
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
cat /tmp/buzz-relay.log
exit 1
fi
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
if [ "${status_code}" = "200" ]; then
exit 0
fi
sleep 1
done
cat /tmp/buzz-relay.log
exit 1
- name: NIP-ER reminder e2e
# Feature e2e for NIP-ER (Event Reminders, kind:30300): write-path
# validation, author-only read filtering, and scheduler delivery against
# a live relay. The schema-drift / migration-version guarantee is owned
# by the buzz-db migration.rs unit tests, not this suite.
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'binary(e2e_event_reminder) | binary(e2e_author_only_deletion) | binary(e2e_existence_oracle)' \
--run-ignored ignored-only
env:
RELAY_URL: ws://localhost:3000
# schema/schema.sql (applied above) indexes kind:5 content.
BUZZ_TEST_SEARCH_INDEXES_DELETIONS: "1"
- name: Upload relay log
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: backend-integration-relay-log
path: /tmp/buzz-relay.log
if-no-files-found: ignore
relay-e2e:
name: Relay E2E
runs-on: ubuntu-latest
timeout-minutes: 20
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
save-if: ${{ github.event_name != 'pull_request' }}
# Reuse the relay + git-credential-nostr built by Desktop E2E Relay
# instead of compiling them a second time.
- name: Download relay binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Start relay
env:
BUZZ_MEDIA_READ_MODE: enforce
run: |
chmod +x ./target/ci/buzz-relay ./target/ci/git-credential-nostr
./scripts/start-relay-for-tests.sh --no-build
- name: Relay E2E tests
run: |
cargo test -p buzz-test-client --test e2e_persona --test e2e_team_catalog --test e2e_nostr_interop --test e2e_thread_roots --test e2e_project -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay invite -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip43_membership_snapshots_are_rejected -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_departure_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_batch_put_user_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_announce_channel_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_unlisted_channel_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_solo_share_and_convert_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_member_raises_guest_agent -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_guest_is_read_only_in_normal_channel -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_relay_rejects_role_change_by_unprivileged_actor -- --ignored --nocapture
env:
RELAY_URL: ws://localhost:3000
GIT_CREDENTIAL_NOSTR_BIN: ${{ github.workspace }}/target/ci/git-credential-nostr
- name: Media read-auth e2e
# Reads require kind:24242 `t=get` auth, so these binaries are the only
# coverage that a real relay rejects bare reads and honours host- and
# hash-scoped tokens. They were #[ignore]d and selected by no CI job, so
# the lane never ran; select it here, where MinIO and the seeded
# 'localhost:3000' community already exist.
# --no-fail-fast: without it cargo stops after the first failing binary,
# so one broken case hides every later binary's result.
run: |
cargo test -p buzz-test-client --no-fail-fast --test e2e_media --test e2e_media_extended --test e2e_media_video -- --ignored --nocapture
env:
RELAY_URL: ws://localhost:3000
RELAY_HTTP_URL: http://localhost:3000
- name: Upload relay logs
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: relay-e2e-artifacts
path: /tmp/buzz-relay.log
if-no-files-found: ignore
results:
name: Results
if: ${{ always() }}
needs: [desktop-e2e-relay, postgres-tests, desktop-e2e-integration, backend-integration, relay-e2e]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions: {}
outputs:
desktop_e2e_relay_result: ${{ needs.desktop-e2e-relay.result }}
desktop_e2e_integration_result: ${{ needs.desktop-e2e-integration.result }}
backend_integration_result: ${{ needs.backend-integration.result }}
relay_e2e_result: ${{ needs.relay-e2e.result }}
postgres_tests_result: ${{ needs.postgres-tests.result }}
steps:
- run: echo "Captured Relay and PostgreSQL job results"