mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
🤖 A sender who cannot use or read a referenced event, artifact or workflow now gets exactly the response it would get for an ID that does not exist. #8197 (merged) set this rule for deletions; this applies it to the other writes that look up a target, plus the workflow read and webhook endpoints. Artifact `create` and workflow definition saves are the exceptions, described below. Access is checked before any message that depends on the target, readability goes through `event_visible_to_reader` (including #8197's private-deletion rule), and soft-deleted targets keep matching missing ones. | Path | Checks, in order | Response for any unusable target | |---|---|---| | Reaction (kind 7) | target lookup → `event_visible_to_reader` → every channel gate for the derived channel, including archived | `invalid: reaction target event not found` | | Edit (kind 40003) | target in the edit's `h` channel and readable → author / agent-owner check, including an author since removed from a private channel | `invalid: edit target not found or not editable by you` | | Forum vote (kind 45002) | target in the vote's `h` channel and readable → kind check | `invalid: vote target event not found` | | Report by event (kind 1984 `e`) | `event_visible_to_reader` → token channel scope → channel membership or open | `invalid: report target event not found` | | Reply (NIP-10 and workflow `send_message` replies) | parent in the reply's channel and readable | `reply parent not found` | | Artifact update / delete / restore / move | head exists and is homed in `h` (move: in the source the sender can write) → revision, type, deleted and root checks | `conflict: artifact head unavailable` | | Workflow trigger (kind 46020) | workflow exists and sender is its owner | `invalid: workflow not found or not triggerable by you` | | `GET /workflows/{id}/runs` and `/runs/{run_id}/approvals` | workflow exists, has a channel, and the caller can read it | `404 {"error":"workflow not found"}` | | `POST /hooks/{id}` | workflow exists and the provided secret matches its stored secret → definition and trigger-type checks | `404 {"error":"workflow not found"}` | Two of these reject writes that used to be accepted: a reaction to an event the sender cannot read (such as someone else's author-only event), and a report on an event the reporter cannot read. Reports by pubkey and by blob are unchanged. In an archived channel, writes whose channel comes from their target (reactions and deletions) get that target's not-found response, and an artifact move out of an archived source channel gets `conflict: artifact head unavailable` like any other source the sender cannot write; every other write keeps `invalid: channel is archived`. A webhook call with an absent or wrong secret, or to a workflow without a webhook secret, now gets `404` instead of `401`/`400`. Artifact `create` and workflow definition saves still report a taken ID, as NIP-AR allows for artifacts. Removed messages, none of which any client depends on from these endpoints: `edit target event not found`, `must be event author to edit`, `target event belongs to a different channel`, `target event has no channel`, `parent event belongs to a different channel`, `parent event has no channel association`, `artifact home changed`, `workflow is not accessible`, `workflow is not channel-scoped`, the webhook's `authentication failed` and `webhook secret required but not configured`, `forbidden: not authorized to trigger this workflow` for non-owners (it remains for an owner whose channel authority lapsed), and `restricted: not a channel member` for an edit by an author removed from a private channel. `e2e_existence_oracle` checks each event write over HTTP and WS, and the workflow read and webhook endpoints over HTTP, against missing, soft-deleted, other-channel, private-channel, archived, author-only and wrong-kind targets as applicable, with a positive control per path. It runs in the Backend Integration job alongside `e2e_author_only_deletion`. `removed_author_edit_does_not_reveal_target` checks that a removed author's edits of their remaining message, their deleted message and a missing ID get identical answers. In the PostgreSQL suite, `head_outside_authorized_channel_matches_missing_head` covers the artifact transaction, `workflow_reply_to_unusable_parent_matches_missing` covers workflow replies through `RelayActionSink::send_message`, `scoped_token_report_does_not_reveal_target` and `scoped_token_reaction_does_not_reveal_target` cover channel-scoped tokens, and `channelless_workflow_runs_read_as_missing` covers the runs read for a workflow with no channel. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <211b96e6a2b7f45fd4047988976c7bbbeeda0c15f3ae7b32eec20834b5a55118@buzz.block.builderlab.xyz>
670 lines
30 KiB
YAML
670 lines
30 KiB
YAML
name: CI / Relay and PostgreSQL
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
rust:
|
|
required: true
|
|
type: boolean
|
|
desktop:
|
|
required: true
|
|
type: boolean
|
|
desktop_rust:
|
|
required: true
|
|
type: boolean
|
|
lane:
|
|
required: true
|
|
type: string
|
|
outputs:
|
|
desktop_e2e_relay_result:
|
|
value: ${{ jobs.results.outputs.desktop_e2e_relay_result }}
|
|
desktop_e2e_integration_result:
|
|
value: ${{ jobs.results.outputs.desktop_e2e_integration_result }}
|
|
backend_integration_result:
|
|
value: ${{ jobs.results.outputs.backend_integration_result }}
|
|
relay_e2e_result:
|
|
value: ${{ jobs.results.outputs.relay_e2e_result }}
|
|
postgres_tests_result:
|
|
value: ${{ jobs.results.outputs.postgres_tests_result }}
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
|
|
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
|
|
|
|
jobs:
|
|
# The relay binaries and the two nextest archives compile independently, so
|
|
# they build in parallel; consumers download every desktop-e2e-relay-*
|
|
# artifact into target/ci as before.
|
|
desktop-e2e-relay:
|
|
name: Desktop E2E Relay (${{ matrix.artifact }})
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
if: inputs.lane == 'artifacts' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- artifact: relay-binaries
|
|
paths: |
|
|
target/ci/buzz-relay
|
|
target/ci/git-credential-nostr
|
|
- artifact: backend-integration-tests
|
|
paths: target/ci/backend-integration-tests.tar.zst
|
|
- artifact: postgres-tests
|
|
paths: target/ci/postgres-tests.tar.zst
|
|
permissions:
|
|
contents: read
|
|
env:
|
|
SCCACHE_GHA_ENABLED: "true"
|
|
SCCACHE_GHA_RW_MODE: ${{ (github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.number == 5224)) && 'READ_WRITE' || 'READ_ONLY' }}
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
# Reuse each artifact when none of its inputs changed (desktop-only PRs
|
|
# hit this every time). The key covers everything they embed, including
|
|
# migrations via sqlx migrate!.
|
|
- name: Restore relay artifacts cache
|
|
id: relay-artifacts-cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ matrix.paths }}
|
|
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
|
|
- uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
# All lanes restore the shared Rust cache; only the postgres-tests lane,
|
|
# which compiles the widest test graph, writes it so a faster lane
|
|
# cannot save a narrower target dir under the same key.
|
|
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
with:
|
|
workspaces: |
|
|
.
|
|
desktop/src-tauri
|
|
save-if: ${{ github.event_name != 'pull_request' && matrix.artifact == 'postgres-tests' }}
|
|
# Cache rustc outputs for unchanged workspace crates. Trusted pushes write;
|
|
# the bounded PR 5224 trial writes only to its isolated merge-ref scope.
|
|
- name: Set up sccache
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
uses: Mozilla-Actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11 # zizmor: ignore[cache-poisoning] Bounded trial: only PR 5224 writes to its isolated merge-ref scope; trusted pushes retain production writes.
|
|
with:
|
|
version: v0.16.0
|
|
- name: Install cargo-nextest
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact != 'relay-binaries'
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Build relay binaries
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'relay-binaries'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: cargo build --profile ci -p buzz-relay -p git-credential-nostr
|
|
- name: Build backend integration test archive
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'backend-integration-tests'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: |
|
|
cargo nextest archive \
|
|
--cargo-profile ci \
|
|
-p buzz-db \
|
|
-p buzz-relay \
|
|
-p buzz-test-client \
|
|
--lib \
|
|
--test e2e_event_reminder \
|
|
--test e2e_author_only_deletion \
|
|
--test e2e_existence_oracle \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst
|
|
- name: Build PostgreSQL test archive
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'postgres-tests'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: |
|
|
postgres_package_args=()
|
|
while IFS= read -r package; do
|
|
postgres_package_args+=(-p "$package")
|
|
done < <(scripts/postgres-test-packages.sh)
|
|
if [[ "${#postgres_package_args[@]}" -eq 0 ]]; then
|
|
echo "no PostgreSQL test packages were discovered" >&2
|
|
exit 1
|
|
fi
|
|
cargo nextest archive \
|
|
--cargo-profile ci \
|
|
"${postgres_package_args[@]}" \
|
|
--lib \
|
|
--tests \
|
|
--archive-file target/ci/postgres-tests.tar.zst
|
|
- name: Save relay artifacts cache
|
|
# PR-scoped exact-source entries cannot warm main or other PRs and churn
|
|
# the shared cache pool. sccache provides read-only PR reuse instead.
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && github.event_name == 'push'
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ matrix.paths }}
|
|
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
|
|
- name: Upload relay artifacts
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: desktop-e2e-relay-${{ matrix.artifact }}
|
|
path: ${{ matrix.paths }}
|
|
if-no-files-found: error
|
|
retention-days: 1
|
|
|
|
postgres-tests:
|
|
name: PostgreSQL Tests
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
if: inputs.lane == 'postgres' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
services:
|
|
postgres:
|
|
image: postgres:16
|
|
env:
|
|
POSTGRES_USER: buzz
|
|
POSTGRES_PASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
POSTGRES_DB: postgres
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U buzz -d postgres"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 12
|
|
redis:
|
|
image: redis:7
|
|
ports:
|
|
- 6379:6379
|
|
options: >-
|
|
--health-cmd "redis-cli ping"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 12
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PG_BIN_DIR: /usr/bin
|
|
REDIS_URL: redis://localhost:6379
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Tune disposable PostgreSQL
|
|
env:
|
|
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
run: |
|
|
# Database-per-test cloning forces checkpoints. Durability is redundant
|
|
# for this disposable service and makes runtime depend on runner disk I/O.
|
|
psql --dbname postgres --set ON_ERROR_STOP=1 <<'SQL'
|
|
ALTER SYSTEM SET fsync = off;
|
|
ALTER SYSTEM SET full_page_writes = off;
|
|
ALTER SYSTEM SET synchronous_commit = off;
|
|
SELECT pg_reload_conf();
|
|
SQL
|
|
psql --dbname postgres --tuples-only --no-align --command \
|
|
"SELECT bool_and(setting = 'off') FROM pg_settings WHERE name IN ('fsync', 'full_page_writes', 'synchronous_commit')" \
|
|
| grep --fixed-strings --line-regexp t
|
|
- name: Install cargo-nextest
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Download backend test archive
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Restore relay executable permission
|
|
run: chmod +x ./target/ci/buzz-relay
|
|
- name: PostgreSQL-backed tests
|
|
env:
|
|
BUZZ_POSTGRES_ADMIN_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/postgres
|
|
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
run: |
|
|
scripts/postgres-test-run.sh \
|
|
--archive-file target/ci/postgres-tests.tar.zst
|
|
|
|
desktop-e2e-integration-shard:
|
|
name: Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})
|
|
runs-on: ubuntu-latest
|
|
# A shard needs ~10m; Playwright install-deps is bounded by
|
|
# ci-apt-retry.sh (~25m worst case), so a degraded mirror fails with its
|
|
# own error instead of the job timeout.
|
|
timeout-minutes: 40
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3]
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Start integration services
|
|
run: |
|
|
for attempt in 1 2 3; do
|
|
if docker compose up -d postgres redis minio minio-init; then
|
|
break
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "docker compose up failed after 3 attempts" >&2
|
|
exit 1
|
|
fi
|
|
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
|
|
sleep $((attempt * 5))
|
|
done
|
|
- name: Get pnpm store directory
|
|
id: pnpm-cache
|
|
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
|
|
- name: Restore pnpm store cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
|
|
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
|
|
restore-keys: pnpm-${{ runner.os }}-
|
|
- name: Install desktop dependencies
|
|
run: just desktop-install-ci
|
|
- name: Get Playwright version
|
|
id: pw-version
|
|
run: echo "version=$(cd desktop && node -e "console.log(require('@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"
|
|
- name: Restore Playwright browser cache
|
|
id: playwright-cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
|
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
|
- name: Install Playwright Chromium
|
|
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
|
run: cd desktop && pnpm exec playwright install chromium
|
|
- name: Install Playwright system dependencies
|
|
run: cd desktop && ../scripts/ci-apt-retry.sh pnpm exec playwright install-deps chromium
|
|
- name: Save Playwright browser cache
|
|
if: steps.playwright-cache.outputs.cache-hit != 'true' && github.event_name == 'push' && matrix.shard == 1
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
|
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
|
- name: Desktop E2E build
|
|
run: pnpm -C desktop build:e2e
|
|
- name: Wait for integration services
|
|
run: |
|
|
wait_healthy() {
|
|
local service="$1"
|
|
local container="$2"
|
|
for attempt in $(seq 1 60); do
|
|
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
|
|
if [ "${status}" = "healthy" ]; then
|
|
echo "${service} is healthy"
|
|
return 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
docker logs "${container}" || true
|
|
return 1
|
|
}
|
|
wait_healthy "Postgres" "buzz-postgres"
|
|
wait_healthy "Redis" "buzz-redis"
|
|
wait_healthy "MinIO" "buzz-minio"
|
|
- name: Download relay binary
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Apply schema and seed deployment community
|
|
# MT: the relay resolves each request's tenant from the communities host
|
|
# map and fails closed on an unmapped host. The channel reconciler binds
|
|
# the deployment community ONCE at boot (outside its retry loop) and
|
|
# exits permanently on an unmapped host, so the 'localhost:3000'
|
|
# community MUST exist before the relay starts — the retry loop only
|
|
# handles late-seeded channels, not a late-seeded community. The relay
|
|
# migrates at boot via BUZZ_AUTO_MIGRATE, but that's too late for the
|
|
# pre-boot seed, so apply the schema here first (then drop AUTO_MIGRATE
|
|
# below). lower(host) is the unique index → ON CONFLICT target. psql
|
|
# isn't on PATH in hermit → exec into the buzz-postgres container.
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PGPASSWORD: buzz_dev
|
|
PGDATABASE: buzz
|
|
# Use the already-running docker postgres for desired-state planning instead of
|
|
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_DB: buzz
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
run: |
|
|
./bin/pgschema apply --file schema/schema.sql --auto-approve
|
|
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
|
|
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -qtA -c "
|
|
INSERT INTO communities (id, host)
|
|
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
|
|
ON CONFLICT (lower(host)) DO NOTHING
|
|
;"
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay
|
|
nohup env \
|
|
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
|
|
REDIS_URL=redis://localhost:6379 \
|
|
RELAY_URL=ws://localhost:3000 \
|
|
BUZZ_BIND_ADDR=0.0.0.0:3000 \
|
|
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
|
|
BUZZ_REQUIRE_AUTH_TOKEN=false \
|
|
BUZZ_RECONCILE_CHANNELS=true \
|
|
BUZZ_RATE_LIMIT_HUMAN_MESSAGES_PER_MIN=100000 \
|
|
BUZZ_RATE_LIMIT_HUMAN_API_CALLS_PER_MIN=100000 \
|
|
BUZZ_RATE_LIMIT_HUMAN_WS_EVENTS_PER_SEC=10000 \
|
|
BUZZ_GIT_PROBE_WRITERS=8 \
|
|
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
|
|
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
|
|
echo $! > /tmp/buzz-relay.pid
|
|
for attempt in $(seq 1 60); do
|
|
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
fi
|
|
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
|
|
if [ "${status_code}" = "200" ]; then
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
- name: Seed desktop e2e data
|
|
run: bash scripts/setup-desktop-test-data.sh
|
|
- name: Desktop relay-backed e2e
|
|
run: cd desktop && pnpm exec playwright test --project=integration --shard=${{ matrix.shard }}/${{ strategy.job-total }}
|
|
- name: Summarize flaky tests
|
|
if: ${{ !cancelled() }}
|
|
run: node scripts/summarize-flaky-tests.mjs playwright-report.json "Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})"
|
|
working-directory: desktop
|
|
- name: Upload desktop integration artifacts
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: desktop-e2e-integration-artifacts-${{ matrix.shard }}
|
|
path: |
|
|
desktop/playwright-report
|
|
desktop/playwright-report.json
|
|
desktop/test-results
|
|
/tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
- name: Save pnpm store cache
|
|
if: github.event_name == 'push'
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
|
|
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
|
|
|
|
desktop-e2e-integration:
|
|
name: Desktop E2E Integration
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
needs: [desktop-e2e-integration-shard]
|
|
if: always() && inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- name: Check integration shards
|
|
run: |
|
|
if [ "${{ needs.desktop-e2e-integration-shard.result }}" != "success" ]; then
|
|
echo "Desktop E2E Integration shards finished with: ${{ needs.desktop-e2e-integration-shard.result }}"
|
|
exit 1
|
|
fi
|
|
echo "Desktop E2E Integration shards passed"
|
|
|
|
backend-integration:
|
|
name: Backend Integration (relay e2e)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Install cargo-nextest
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Start integration services
|
|
run: |
|
|
for attempt in 1 2 3; do
|
|
if docker compose up -d postgres redis minio minio-init; then
|
|
break
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "docker compose up failed after 3 attempts" >&2
|
|
exit 1
|
|
fi
|
|
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
|
|
sleep $((attempt * 5))
|
|
done
|
|
- name: Wait for integration services
|
|
run: |
|
|
wait_healthy() {
|
|
local service="$1"
|
|
local container="$2"
|
|
for attempt in $(seq 1 60); do
|
|
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
|
|
if [ "${status}" = "healthy" ]; then
|
|
echo "${service} is healthy"
|
|
return 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
docker logs "${container}" || true
|
|
return 1
|
|
}
|
|
wait_healthy "Postgres" "buzz-postgres"
|
|
wait_healthy "Redis" "buzz-redis"
|
|
wait_healthy "MinIO" "buzz-minio"
|
|
- name: Download relay artifacts
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Apply schema and seed deployment community
|
|
# MT: the relay resolves each request's tenant from the communities host
|
|
# map and fails closed on an unmapped host. The reminder scheduler binds
|
|
# the deployment community ONCE at boot and exits permanently on an
|
|
# unmapped host (no retry, unlike the channel reconciler), so the
|
|
# 'localhost:3000' community MUST exist before the relay starts — seeding
|
|
# after boot leaves the scheduler dead. The relay migrates at boot via
|
|
# BUZZ_AUTO_MIGRATE, but that's too late for the pre-boot seed, so apply
|
|
# the schema here first (then drop AUTO_MIGRATE below). lower(host) is the
|
|
# unique index → ON CONFLICT target. psql isn't on PATH in hermit → exec
|
|
# into the buzz-postgres container.
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PGPASSWORD: buzz_dev
|
|
PGDATABASE: buzz
|
|
# Use the already-running docker postgres for desired-state planning instead of
|
|
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_DB: buzz
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
run: |
|
|
./bin/pgschema apply --file schema/schema.sql --auto-approve
|
|
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
|
|
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -qtA -c "
|
|
INSERT INTO communities (id, host)
|
|
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
|
|
ON CONFLICT (lower(host)) DO NOTHING
|
|
;"
|
|
- name: Git default-branch route and clone regressions
|
|
env:
|
|
# The preceding step already applied and reconciled schema/schema.sql.
|
|
BUZZ_TEST_SCHEMA_MODE: desired
|
|
BUZZ_TEST_DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
BUZZ_TEST_S3_ENDPOINT: http://localhost:9000
|
|
BUZZ_TEST_S3_BUCKET: buzz-media
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/api::git::settings::tests::external_infra::/)' \
|
|
--run-ignored ignored-only
|
|
- name: Relay quota accounting and rejection regressions
|
|
env:
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and (test(/^rejection::/) or test(/^admission::/))' \
|
|
--run-ignored all
|
|
- name: NIP-FI cross-pod command replay regressions
|
|
env:
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/^api::nip_fi::route_integration_tests::external_infra_redis::/)' \
|
|
--run-ignored ignored-only
|
|
- name: Workflow message provenance unit tests
|
|
# The relay's workflow_sink suite is not selected by the infra-free
|
|
# unit job. Its ignored database cases run in the isolated PostgreSQL
|
|
# lane; keep the pure provenance cases covered here without duplication.
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/workflow_sink/)'
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay
|
|
nohup env \
|
|
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
|
|
REDIS_URL=redis://localhost:6379 \
|
|
RELAY_URL=ws://localhost:3000 \
|
|
BUZZ_BIND_ADDR=0.0.0.0:3000 \
|
|
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
|
|
BUZZ_REQUIRE_AUTH_TOKEN=false \
|
|
BUZZ_RECONCILE_CHANNELS=true \
|
|
BUZZ_GIT_PROBE_WRITERS=8 \
|
|
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
|
|
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
|
|
echo $! > /tmp/buzz-relay.pid
|
|
for attempt in $(seq 1 60); do
|
|
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
fi
|
|
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
|
|
if [ "${status_code}" = "200" ]; then
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
- name: NIP-ER reminder e2e
|
|
# Feature e2e for NIP-ER (Event Reminders, kind:30300): write-path
|
|
# validation, author-only read filtering, and scheduler delivery against
|
|
# a live relay. The schema-drift / migration-version guarantee is owned
|
|
# by the buzz-db migration.rs unit tests, not this suite.
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'binary(e2e_event_reminder) | binary(e2e_author_only_deletion) | binary(e2e_existence_oracle)' \
|
|
--run-ignored ignored-only
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
# schema/schema.sql (applied above) indexes kind:5 content.
|
|
BUZZ_TEST_SEARCH_INDEXES_DELETIONS: "1"
|
|
- name: Upload relay log
|
|
if: failure()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: backend-integration-relay-log
|
|
path: /tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
|
|
relay-e2e:
|
|
name: Relay E2E
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
|
|
with:
|
|
save-if: ${{ github.event_name != 'pull_request' }}
|
|
# Reuse the relay + git-credential-nostr built by Desktop E2E Relay
|
|
# instead of compiling them a second time.
|
|
- name: Download relay binary
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay ./target/ci/git-credential-nostr
|
|
./scripts/start-relay-for-tests.sh --no-build
|
|
- name: Relay E2E tests
|
|
run: |
|
|
cargo test -p buzz-test-client --test e2e_persona --test e2e_team_catalog --test e2e_nostr_interop --test e2e_thread_roots --test e2e_project -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay invite -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip43_membership_snapshots_are_rejected -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_departure_wire -- --ignored --nocapture
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
GIT_CREDENTIAL_NOSTR_BIN: ${{ github.workspace }}/target/ci/git-credential-nostr
|
|
- name: Media read-auth e2e
|
|
# Reads require kind:24242 `t=get` auth, so these binaries are the only
|
|
# coverage that a real relay rejects bare reads and honours host- and
|
|
# hash-scoped tokens. They were #[ignore]d and selected by no CI job, so
|
|
# the lane never ran; select it here, where MinIO and the seeded
|
|
# 'localhost:3000' community already exist.
|
|
# --no-fail-fast: without it cargo stops after the first failing binary,
|
|
# so one broken case hides every later binary's result.
|
|
run: |
|
|
cargo test -p buzz-test-client --no-fail-fast --test e2e_media --test e2e_media_extended --test e2e_media_video -- --ignored --nocapture
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
RELAY_HTTP_URL: http://localhost:3000
|
|
- name: Upload relay logs
|
|
if: failure()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: relay-e2e-artifacts
|
|
path: /tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
|
|
|
|
results:
|
|
name: Results
|
|
if: ${{ always() }}
|
|
needs: [desktop-e2e-relay, postgres-tests, desktop-e2e-integration, backend-integration, relay-e2e]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
permissions: {}
|
|
outputs:
|
|
desktop_e2e_relay_result: ${{ needs.desktop-e2e-relay.result }}
|
|
desktop_e2e_integration_result: ${{ needs.desktop-e2e-integration.result }}
|
|
backend_integration_result: ${{ needs.backend-integration.result }}
|
|
relay_e2e_result: ${{ needs.relay-e2e.result }}
|
|
postgres_tests_result: ${{ needs.postgres-tests.result }}
|
|
steps:
|
|
- run: echo "Captured Relay and PostgreSQL job results"
|