Files
buzz/.github/workflows/_ci-relay.yml
Will Pfleger 0a06c0b3cb feat(relay): log reads, logins and connections on access target (#8376)
🤖 Adds an access log for relay reads, logins, connections and HTTP
requests on the dedicated `buzz_relay::access` target. It writes to
stdout, not `audit_log`, so reads never take the audit advisory lock.
The log is shaped for volume: one line per HTTP request, one line per
WebSocket read, and one line per WebSocket connection.

## Lines per operation

| Operation | Lines |
|---|---|
| HTTP request on a matched route | 1 (`http`), except the skip list
below |
| HTTP request matching no route | 1 (`http`, `route="<unmatched>"`) at
status 400 and above, otherwise none |
| WebSocket connection | 1 (`close`, carrying the login) |
| WebSocket `REQ` / `COUNT` / NIP-AR sync | 1 each (`req` / `count` /
`sync`), plus the `buzz_ws_reads_total` counter |
| WebSocket NIP-42 login | 1 (`auth`) per decided or cut-off attempt,
capped after the decision as below |
| Audio login | 1 (`auth`, at its final admission check) |
| WebSocket `EVENT` | 1 (`Event ingested`, not on the access target) |

## HTTP requests
One middleware, `http_access_log`, writes every HTTP line. It sits
between `track_metrics` and `http_trace_layer`, so rebuilt errors,
body-limit refusals and localhost refusals all reach it. It writes the
line when the response is ready, which is not proof the body was
delivered. A request dropped before then logs `request=aborted` at WARN
with no `status`.
- Every line carries `route` (the matched route template, never the raw
path), `peer_addr` when the listener has one, `status`,
`time_to_response_ms` and `http_method` (the HTTP verb). The verb stays
on the line itself because the `http.request` span is absent under an
access-only filter and when a pending request is dropped.
- Handlers record what they learn on the request's `AccessSlot`: the
login (`method`, `pubkey`, `pubkey_verified`, `community`), the
decision, the filter summary, and the route's own fields. The decision
is `outcome` (`success` / `denied` / `failed` / `challenged`) with a
fixed `reason`, or `undecided` when the route recorded none. A refusal
is final: a later admission doesn't replace it, so a replayed NIP-98
proof that verifies and is then refused logs `failed` / `nip98_replay`.
- `outcome` describes the login. What the route did is in `status` and
the route's fields: a rejected `/events` write logs `outcome=success`
with `status=400`, `accepted=false` and its `reject_reason`.
- Route fields:
- `/query` and `/count`: the filter summary and `result_count`. An
incremental `/query` response (`{changes:[...]}`) counts its changes,
removals included. Artifact reads record their summary before the query
runs, so a read dropped mid-query keeps it.
- `/events`: `accepted`, `kind`, `event_id`, and on a refused write the
parse `category` / `line` / `column` or a fixed `reject_reason` (such as
`invalid` or `restricted`), never the ingest message.
- Invite mint: `invite_id`, `expires_at` and `max_uses`. Invite claim:
`joined`.
- Admin feedback attachment reads: `feedback_id` and
`attachment_sha256`. With `BUZZ_ADMIN_AUTH=disabled` they log
`method=network_trusted` with no key, and Host or Origin refusals log
their fixed reason.
  - Every other route logs only the login fields.
- No field can hold an invite code, a token, or a Blossom read hash.
Media routes log the `/media/{sha256_ext}` template.
- Skip list: NIP-11 and other non-upgrade requests on `/`, `/info`,
`/.well-known/nostr.json`, `/health`, `/_liveness`, `/_readiness`, and
the SPA and static files the relay serves when enabled (web assets,
invite landing pages, git web pages, the admin bundle) at any status.
Successful WebSocket upgrades (101) log no `http` line. Refusals before
the upgrade on `/` and the audio path log one, with the status actually
sent.
- Routes that aren't Nostr-key logins (the command-token disconnect
endpoint, the workflow webhook, the git server's internal callback) log
`outcome=undecided` with no key.
- Dev-mode `X-Pubkey` requests log `method=x_pubkey`,
`pubkey_verified=false`.
- A successful `/query` or `/count` that returned nothing
(`result_count=0`) logs at DEBUG: on staging, 93% of `/query` lines were
empty polls. Denied, failed, aborted and error reads, reads with no
`result_count`, writes and logins keep their levels.
- Other successful lines and git's credential-less first request
(`challenged`, with its 401) are INFO. Refusals, statuses of 400 and
above, and aborted requests are WARN. In Enforce mode the line records
the remapped status (for example 403). Denial bodies and headers are
unchanged.

## WebSocket reads
- `REQ` / `COUNT` / `sync` lines carry `pubkey`, `conn_id`, `sub_id`,
the filter summary, the outcome (`served` / `denied` / `aborted` /
`error` plus a fixed reason), and `events_queued` (REQ and sync) or
`count` (a COUNT that counted). A `REQ` is `served` only once its EOSE
is queued to the client. A read logs WARN only when it ends in `error`.
`served`, `denied` and `aborted` are INFO, since `aborted` means the
read ended before completing: the client disconnected or sent CLOSE, a
same-ID `REQ` replaced it, or the session was cancelled.
- A NIP-AR sync subscription writes its `sync` line when its initial set
completes, not when live delivery ends. Its summary is the registered
live filter (kinds and channels, never artifact predicates), and
`events_queued` counts the initial set.
- `REQ` lines stay at INFO even when empty at EOSE, because the
subscription can still deliver live events later.
- Filter summary, shared with HTTP `/query` and `/count`: `kinds`,
channel ids, authors and `p` values each list the first 10 distinct
values. The exact `kind_count` / `channel_count` / `author_count` /
`p_count` (summed per filter) appears only when the list is `*` or shows
fewer values than the filters held. A dimension that any filter leaves
open logs `*`. `since` / `until` / `limit` take the widest bound and are
absent when any filter leaves them open, so no field understates what
the read covered. `ids` is a count only. NIP-50 search logs `search:
true`, never the text. Only well-formed channel ids and pubkeys are
listed. An artifact read's summary uses only the standard dimensions of
each raw filter, so artifact predicates and client-defined tags (such as
`#1`) are never logged. The summary is skipped when the access target is
off. Under `=warn`, every WARN line, HTTP or WebSocket, still carries
it.
- Per connection, only the first 16 reads that are refused before they
run (`auth_required`, quota, concurrency, overload) or denied are
logged. The rest are counted as `reads_suppressed` on the close line. A
refusal that arrives after the close line has been written, from a read
still running when the client left, logs its own line, so every refusal
is either logged or counted, never both. Served reads are never capped.

## WebSocket read counter
`buzz_ws_reads_total{type="req"|"count"|"sync"}` counts every parsed
`REQ`, `COUNT` and NIP-AR sync frame before admission, exported at zero
from startup. It is a metric, so it counts at any `RUST_LOG`, including
production's current `error`.

## WebSocket logins
A NIP-42 login writes one `auth` line when it is decided. A successful
login logs INFO `outcome=success` at its commit point, decided under the
connection's close-reason lock, so a disconnect racing the login
(session expiry, admin disconnect, revocation, ban or roster removal)
either wins, and the login logs no success, or loses and closes the
connection right after the success line. A refused or failed login
writes one WARN `auth` line with `outcome`, a fixed `reason` from the
shared `AuthReason` code set, `pubkey` and `pubkey_verified`. Root
WebSocket `auth` lines carry `peer_addr`. AUTH frames after the login is
decided are capped at 16 per connection, and the rest are counted as
`auth_suppressed`. An attempt cut off by close or session expiry logs
`aborted`. Audio logins log one `auth` line at their final admission
check.

## Connections
The close line is the connection's only line. It is owned by the
upgraded socket, so it is written exactly once however the connection
ends: a normal close, a refusal while checking the community is active
(`refused=not_admitted`), the connection limit
(`refused=connection_limit`), or the handler being dropped. It carries
`conn_id`, `community`, `peer_addr`, `duration_ms`, the login,
`events_queued` / `bytes_queued` (frames the writer accepted and
flushed, not client delivery), and the two suppression counters.

## Error text
- Database errors log as `DbError::kind()` (SQLSTATE or variant),
including the command-executor event insert. Search errors, on both read
paths and in the search-cancel task, log as `SearchError::kind()` / the
same SQLx classifier. Workflow database and webhook errors log as their
codes, since webhook URLs can be secrets.
- Git NIP-98 failures log a fixed code. Git subprocess stderr moved to
DEBUG because hook output can contain message text.

## Operator notes
`.env.example` documents the target, its `event` values (`http`, `auth`,
`req`, `count`, `sync`, `close`), the skip list, the levels, and how to
turn it down: `buzz_relay::access=warn` keeps refusals, errors and
aborted HTTP requests, and of WebSocket reads only those that end in
`error`, and `=off` drops everything. It also lists the removed lines:
`NIP-42 auth successful/failed`, `WebSocket connection
established/closed`, `event=connect`, `HTTP bridge request`, `Event
ingested via pipeline`, `relay invite minted`, `relay member added via
invite`, `admin feedback attachment read`, and `addr` (now `peer_addr`).
JSON lines no longer repeat the current span as `span`, since `spans`
already ends with it. A line logged under an explicit `parent:` span
loses that parent's fields. The default OTel export filter sets
`buzz_relay::access=off`, so access lines aren't exported a second time
as span events.

`peer_addr` is the TCP peer. `BUZZ_CLIENT_IP_HEADER` optionally names a
request header (for example `cf-connecting-ip` behind Cloudflare or
`x-real-ip` behind nginx, matched case-insensitively). When set, `http`,
`auth` (root and audio), `close` and connection-limit lines also log
`client_ip`: the rightmost comma-separated entry of the header's last
field, only if it parses as an IP address. It never replaces
`peer_addr`, and it is only trustworthy when every request arrives
through a proxy that overwrites the header.

---------

Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <wpfleger@block.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
2026-11-03 17:48:59 -05:00

713 lines
33 KiB
YAML

name: CI / Relay and PostgreSQL
on:
workflow_call:
inputs:
rust:
required: true
type: boolean
desktop:
required: true
type: boolean
desktop_rust:
required: true
type: boolean
lane:
required: true
type: string
outputs:
desktop_e2e_relay_result:
value: ${{ jobs.results.outputs.desktop_e2e_relay_result }}
desktop_e2e_integration_result:
value: ${{ jobs.results.outputs.desktop_e2e_integration_result }}
backend_integration_result:
value: ${{ jobs.results.outputs.backend_integration_result }}
relay_e2e_result:
value: ${{ jobs.results.outputs.relay_e2e_result }}
postgres_tests_result:
value: ${{ jobs.results.outputs.postgres_tests_result }}
env:
CARGO_TERM_COLOR: always
BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
jobs:
# The relay binaries and the two nextest archives compile independently, so
# they build in parallel; consumers download every desktop-e2e-relay-*
# artifact into target/ci as before.
desktop-e2e-relay:
name: Desktop E2E Relay (${{ matrix.artifact }})
runs-on: ubuntu-latest
timeout-minutes: 45
if: inputs.lane == 'artifacts' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
strategy:
fail-fast: false
matrix:
include:
- artifact: relay-binaries
paths: |
target/ci/buzz-relay
target/ci/git-credential-nostr
- artifact: backend-integration-tests
paths: target/ci/backend-integration-tests.tar.zst
- artifact: postgres-tests
paths: target/ci/postgres-tests.tar.zst
permissions:
contents: read
env:
SCCACHE_GHA_ENABLED: "true"
SCCACHE_GHA_RW_MODE: ${{ (github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.number == 5224)) && 'READ_WRITE' || 'READ_ONLY' }}
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# Reuse each artifact when none of its inputs changed (desktop-only PRs
# hit this every time). The key covers everything they embed, including
# migrations via sqlx migrate!.
- name: Restore relay artifacts cache
id: relay-artifacts-cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ matrix.paths }}
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
- uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
# All lanes restore the shared Rust cache; only the postgres-tests lane,
# which compiles the widest test graph, writes it so a faster lane
# cannot save a narrower target dir under the same key.
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
with:
workspaces: |
.
desktop/src-tauri
save-if: ${{ github.event_name != 'pull_request' && matrix.artifact == 'postgres-tests' }}
# Cache rustc outputs for unchanged workspace crates. Trusted pushes write;
# the bounded PR 5224 trial writes only to its isolated merge-ref scope.
- name: Set up sccache
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
uses: Mozilla-Actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11 # zizmor: ignore[cache-poisoning] Bounded trial: only PR 5224 writes to its isolated merge-ref scope; trusted pushes retain production writes.
with:
version: v0.16.0
# The sccache server checks its GHA cache storage at startup and refuses to
# start when that fails (egress over the account limit, a failed read),
# which used to fail cargo's first `sccache rustc -vV` and the build.
# SCCACHE_IGNORE_SERVER_IO_ERROR doesn't cover startup in v0.16.0, so start
# it here and build with plain rustc if it won't start.
- name: Start sccache
id: sccache
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
run: |
if sccache --start-server; then
echo "wrapper=sccache" >> "$GITHUB_OUTPUT"
else
echo "::warning::sccache did not start; building without it"
echo "wrapper=" >> "$GITHUB_OUTPUT"
fi
- name: Install cargo-nextest
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact != 'relay-binaries'
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Build relay binaries
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'relay-binaries'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: cargo build --profile ci -p buzz-relay -p git-credential-nostr
- name: Build backend integration test archive
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'backend-integration-tests'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: |
cargo nextest archive \
--cargo-profile ci \
-p buzz-db \
-p buzz-relay \
-p buzz-test-client \
--lib \
--test e2e_event_reminder \
--test e2e_author_only_deletion \
--test e2e_existence_oracle \
--archive-file target/ci/backend-integration-tests.tar.zst
- name: Build PostgreSQL test archive
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'postgres-tests'
env:
RUSTC_WRAPPER: ${{ steps.sccache.outputs.wrapper }}
run: |
postgres_package_args=()
while IFS= read -r package; do
postgres_package_args+=(-p "$package")
done < <(scripts/postgres-test-packages.sh)
if [[ "${#postgres_package_args[@]}" -eq 0 ]]; then
echo "no PostgreSQL test packages were discovered" >&2
exit 1
fi
cargo nextest archive \
--cargo-profile ci \
"${postgres_package_args[@]}" \
--lib \
--tests \
--archive-file target/ci/postgres-tests.tar.zst
- name: Save relay artifacts cache
# PR-scoped exact-source entries cannot warm main or other PRs and churn
# the shared cache pool. sccache provides read-only PR reuse instead.
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && github.event_name == 'push'
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ matrix.paths }}
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
- name: Upload relay artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: desktop-e2e-relay-${{ matrix.artifact }}
path: ${{ matrix.paths }}
if-no-files-found: error
retention-days: 1
postgres-tests:
name: PostgreSQL Tests
runs-on: ubuntu-latest
timeout-minutes: 10
if: inputs.lane == 'postgres' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: buzz
POSTGRES_PASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
POSTGRES_DB: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U buzz -d postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 12
redis:
image: redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 12
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PG_BIN_DIR: /usr/bin
REDIS_URL: redis://localhost:6379
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Tune disposable PostgreSQL
env:
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
run: |
# Database-per-test cloning forces checkpoints. Durability is redundant
# for this disposable service and makes runtime depend on runner disk I/O.
psql --dbname postgres --set ON_ERROR_STOP=1 <<'SQL'
ALTER SYSTEM SET fsync = off;
ALTER SYSTEM SET full_page_writes = off;
ALTER SYSTEM SET synchronous_commit = off;
SELECT pg_reload_conf();
SQL
psql --dbname postgres --tuples-only --no-align --command \
"SELECT bool_and(setting = 'off') FROM pg_settings WHERE name IN ('fsync', 'full_page_writes', 'synchronous_commit')" \
| grep --fixed-strings --line-regexp t
- name: Install cargo-nextest
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Download backend test archive
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Restore relay executable permission
run: chmod +x ./target/ci/buzz-relay
- name: PostgreSQL-backed tests
env:
BUZZ_POSTGRES_ADMIN_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/postgres
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
run: |
scripts/postgres-test-run.sh \
--archive-file target/ci/postgres-tests.tar.zst
desktop-e2e-integration-shard:
name: Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})
runs-on: ubuntu-latest
# A shard needs ~10m; Playwright install-deps is bounded by
# ci-apt-retry.sh (~25m worst case), so a degraded mirror fails with its
# own error instead of the job timeout.
timeout-minutes: 40
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Start integration services
run: |
for attempt in 1 2 3; do
if docker compose up -d postgres redis minio minio-init; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "docker compose up failed after 3 attempts" >&2
exit 1
fi
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
sleep $((attempt * 5))
done
- name: Get pnpm store directory
id: pnpm-cache
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
- name: Restore pnpm store cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
restore-keys: pnpm-${{ runner.os }}-
- name: Install desktop dependencies
run: just desktop-install-ci
- name: Get Playwright version
id: pw-version
run: echo "version=$(cd desktop && node -e "console.log(require('@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"
- name: Restore Playwright browser cache
id: playwright-cache
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Install Playwright Chromium
if: steps.playwright-cache.outputs.cache-hit != 'true'
run: cd desktop && pnpm exec playwright install chromium
- name: Install Playwright system dependencies
run: cd desktop && ../scripts/ci-apt-retry.sh pnpm exec playwright install-deps chromium
- name: Save Playwright browser cache
if: steps.playwright-cache.outputs.cache-hit != 'true' && github.event_name == 'push' && matrix.shard == 1
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
- name: Desktop E2E build
run: pnpm -C desktop build:e2e
- name: Wait for integration services
run: |
wait_healthy() {
local service="$1"
local container="$2"
for attempt in $(seq 1 60); do
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
if [ "${status}" = "healthy" ]; then
echo "${service} is healthy"
return 0
fi
sleep 2
done
docker logs "${container}" || true
return 1
}
wait_healthy "Postgres" "buzz-postgres"
wait_healthy "Redis" "buzz-redis"
wait_healthy "MinIO" "buzz-minio"
- name: Download relay binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Apply schema and seed deployment community
# MT: the relay resolves each request's tenant from the communities host
# map and fails closed on an unmapped host. The channel reconciler binds
# the deployment community ONCE at boot (outside its retry loop) and
# exits permanently on an unmapped host, so the 'localhost:3000'
# community MUST exist before the relay starts — the retry loop only
# handles late-seeded channels, not a late-seeded community. The relay
# migrates at boot via BUZZ_AUTO_MIGRATE, but that's too late for the
# pre-boot seed, so apply the schema here first (then drop AUTO_MIGRATE
# below). lower(host) is the unique index → ON CONFLICT target. psql
# isn't on PATH in hermit → exec into the buzz-postgres container.
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PGPASSWORD: buzz_dev
PGDATABASE: buzz
# Use the already-running docker postgres for desired-state planning instead of
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_DB: buzz
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
run: |
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -qtA -c "
INSERT INTO communities (id, host)
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
ON CONFLICT (lower(host)) DO NOTHING
;"
- name: Start relay
run: |
chmod +x ./target/ci/buzz-relay
nohup env \
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
REDIS_URL=redis://localhost:6379 \
RELAY_URL=ws://localhost:3000 \
BUZZ_BIND_ADDR=0.0.0.0:3000 \
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
BUZZ_REQUIRE_AUTH_TOKEN=false \
BUZZ_RECONCILE_CHANNELS=true \
BUZZ_RATE_LIMIT_HUMAN_MESSAGES_PER_MIN=100000 \
BUZZ_RATE_LIMIT_HUMAN_API_CALLS_PER_MIN=100000 \
BUZZ_RATE_LIMIT_HUMAN_WS_EVENTS_PER_SEC=10000 \
BUZZ_GIT_PROBE_WRITERS=8 \
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
echo $! > /tmp/buzz-relay.pid
for attempt in $(seq 1 60); do
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
cat /tmp/buzz-relay.log
exit 1
fi
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
if [ "${status_code}" = "200" ]; then
exit 0
fi
sleep 1
done
cat /tmp/buzz-relay.log
exit 1
- name: Seed desktop e2e data
run: bash scripts/setup-desktop-test-data.sh
- name: Desktop relay-backed e2e
run: cd desktop && pnpm exec playwright test --project=integration --shard=${{ matrix.shard }}/${{ strategy.job-total }}
- name: Summarize flaky tests
if: ${{ !cancelled() }}
run: node scripts/summarize-flaky-tests.mjs playwright-report.json "Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})"
working-directory: desktop
- name: Upload desktop integration artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: desktop-e2e-integration-artifacts-${{ matrix.shard }}
path: |
desktop/playwright-report
desktop/playwright-report.json
desktop/test-results
/tmp/buzz-relay.log
if-no-files-found: ignore
retention-days: 7
- name: Save pnpm store cache
if: github.event_name == 'push'
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
desktop-e2e-integration:
name: Desktop E2E Integration
runs-on: ubuntu-latest
timeout-minutes: 5
needs: [desktop-e2e-integration-shard]
if: always() && inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
permissions:
contents: read
steps:
- name: Check integration shards
run: |
if [ "${{ needs.desktop-e2e-integration-shard.result }}" != "success" ]; then
echo "Desktop E2E Integration shards finished with: ${{ needs.desktop-e2e-integration-shard.result }}"
exit 1
fi
echo "Desktop E2E Integration shards passed"
backend-integration:
name: Backend Integration (relay e2e)
runs-on: ubuntu-latest
timeout-minutes: 20
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
# pgschema has no --version; --help exits 0 once Hermit has fetched it.
- name: Fetch Hermit tools
run: scripts/ci-hermit-retry.sh ./bin/pgschema --help
- name: Install cargo-nextest
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Start integration services
run: |
for attempt in 1 2 3; do
if docker compose up -d postgres redis minio minio-init; then
break
fi
if [ "$attempt" -eq 3 ]; then
echo "docker compose up failed after 3 attempts" >&2
exit 1
fi
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
sleep $((attempt * 5))
done
- name: Wait for integration services
run: |
wait_healthy() {
local service="$1"
local container="$2"
for attempt in $(seq 1 60); do
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
if [ "${status}" = "healthy" ]; then
echo "${service} is healthy"
return 0
fi
sleep 2
done
docker logs "${container}" || true
return 1
}
wait_healthy "Postgres" "buzz-postgres"
wait_healthy "Redis" "buzz-redis"
wait_healthy "MinIO" "buzz-minio"
- name: Download relay artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Apply schema and seed deployment community
# MT: the relay resolves each request's tenant from the communities host
# map and fails closed on an unmapped host. The reminder scheduler binds
# the deployment community ONCE at boot and exits permanently on an
# unmapped host (no retry, unlike the channel reconciler), so the
# 'localhost:3000' community MUST exist before the relay starts — seeding
# after boot leaves the scheduler dead. The relay migrates at boot via
# BUZZ_AUTO_MIGRATE, but that's too late for the pre-boot seed, so apply
# the schema here first (then drop AUTO_MIGRATE below). lower(host) is the
# unique index → ON CONFLICT target. psql isn't on PATH in hermit → exec
# into the buzz-postgres container.
env:
PGHOST: localhost
PGPORT: "5432"
PGUSER: buzz
PGPASSWORD: buzz_dev
PGDATABASE: buzz
# Use the already-running docker postgres for desired-state planning instead of
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
PGSCHEMA_PLAN_HOST: localhost
PGSCHEMA_PLAN_PORT: "5432"
PGSCHEMA_PLAN_DB: buzz
PGSCHEMA_PLAN_USER: buzz
PGSCHEMA_PLAN_PASSWORD: buzz_dev
run: |
./bin/pgschema apply --file schema/schema.sql --auto-approve
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
psql -U buzz -d buzz -qtA -c "
INSERT INTO communities (id, host)
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
ON CONFLICT (lower(host)) DO NOTHING
;"
- name: Git default-branch route and clone regressions
env:
# The preceding step already applied and reconciled schema/schema.sql.
BUZZ_TEST_SCHEMA_MODE: desired
BUZZ_TEST_DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
BUZZ_TEST_REDIS_URL: redis://localhost:6379
BUZZ_TEST_S3_ENDPOINT: http://localhost:9000
BUZZ_TEST_S3_BUCKET: buzz-media
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/api::git::settings::tests::external_infra::/)' \
--run-ignored ignored-only
- name: Admin attachment access-log regression
env:
# This test builds its state from Config::for_test(), which reads the
# relay's own DATABASE_URL and BUZZ_S3_* variables.
DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
BUZZ_S3_ENDPOINT: http://localhost:9000
BUZZ_S3_BUCKET: buzz-media
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/api::admin::postgres_tests::external_infra_minio_tests::/)' \
--run-ignored ignored-only
- name: Relay quota accounting and rejection regressions
env:
BUZZ_TEST_REDIS_URL: redis://localhost:6379
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and (test(/^rejection::/) or test(/^admission::/))' \
--run-ignored all
- name: NIP-FI cross-pod command replay regressions
env:
BUZZ_TEST_REDIS_URL: redis://localhost:6379
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/^api::nip_fi::route_integration_tests::external_infra_redis::/)' \
--run-ignored ignored-only
- name: Workflow message provenance unit tests
# The relay's workflow_sink suite is not selected by the infra-free
# unit job. Its ignored database cases run in the isolated PostgreSQL
# lane; keep the pure provenance cases covered here without duplication.
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'package(buzz-relay) and test(/workflow_sink/)'
- name: Start relay
run: |
chmod +x ./target/ci/buzz-relay
nohup env \
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
REDIS_URL=redis://localhost:6379 \
RELAY_URL=ws://localhost:3000 \
BUZZ_BIND_ADDR=0.0.0.0:3000 \
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
BUZZ_REQUIRE_AUTH_TOKEN=false \
BUZZ_RECONCILE_CHANNELS=true \
BUZZ_GIT_PROBE_WRITERS=8 \
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
echo $! > /tmp/buzz-relay.pid
for attempt in $(seq 1 60); do
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
cat /tmp/buzz-relay.log
exit 1
fi
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
if [ "${status_code}" = "200" ]; then
exit 0
fi
sleep 1
done
cat /tmp/buzz-relay.log
exit 1
- name: NIP-ER reminder e2e
# Feature e2e for NIP-ER (Event Reminders, kind:30300): write-path
# validation, author-only read filtering, and scheduler delivery against
# a live relay. The schema-drift / migration-version guarantee is owned
# by the buzz-db migration.rs unit tests, not this suite.
run: |
cargo nextest run \
--archive-file target/ci/backend-integration-tests.tar.zst \
-E 'binary(e2e_event_reminder) | binary(e2e_author_only_deletion) | binary(e2e_existence_oracle)' \
--run-ignored ignored-only
env:
RELAY_URL: ws://localhost:3000
# schema/schema.sql (applied above) indexes kind:5 content.
BUZZ_TEST_SEARCH_INDEXES_DELETIONS: "1"
- name: Upload relay log
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: backend-integration-relay-log
path: /tmp/buzz-relay.log
if-no-files-found: ignore
relay-e2e:
name: Relay E2E
runs-on: ubuntu-latest
timeout-minutes: 20
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
save-if: ${{ github.event_name != 'pull_request' }}
# Reuse the relay + git-credential-nostr built by Desktop E2E Relay
# instead of compiling them a second time.
- name: Download relay binary
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: desktop-e2e-relay-*
merge-multiple: true
path: target/ci
- name: Start relay
run: |
chmod +x ./target/ci/buzz-relay ./target/ci/git-credential-nostr
./scripts/start-relay-for-tests.sh --no-build
- name: Relay E2E tests
run: |
cargo test -p buzz-test-client --test e2e_persona --test e2e_team_catalog --test e2e_nostr_interop --test e2e_thread_roots --test e2e_project -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay invite -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip43_membership_snapshots_are_rejected -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_departure_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_batch_put_user_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_announce_channel_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_unlisted_channel_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_solo_share_and_convert_wire -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_member_raises_guest_agent -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_guest_is_read_only_in_normal_channel -- --ignored --nocapture
cargo test -p buzz-test-client --test e2e_relay nip29_relay_rejects_role_change_by_unprivileged_actor -- --ignored --nocapture
env:
RELAY_URL: ws://localhost:3000
GIT_CREDENTIAL_NOSTR_BIN: ${{ github.workspace }}/target/ci/git-credential-nostr
- name: Media read-auth e2e
# Reads require kind:24242 `t=get` auth, so these binaries are the only
# coverage that a real relay rejects bare reads and honours host- and
# hash-scoped tokens. They were #[ignore]d and selected by no CI job, so
# the lane never ran; select it here, where MinIO and the seeded
# 'localhost:3000' community already exist.
# --no-fail-fast: without it cargo stops after the first failing binary,
# so one broken case hides every later binary's result.
run: |
cargo test -p buzz-test-client --no-fail-fast --test e2e_media --test e2e_media_extended --test e2e_media_video -- --ignored --nocapture
env:
RELAY_URL: ws://localhost:3000
RELAY_HTTP_URL: http://localhost:3000
- name: Upload relay logs
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: relay-e2e-artifacts
path: /tmp/buzz-relay.log
if-no-files-found: ignore
results:
name: Results
if: ${{ always() }}
needs: [desktop-e2e-relay, postgres-tests, desktop-e2e-integration, backend-integration, relay-e2e]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions: {}
outputs:
desktop_e2e_relay_result: ${{ needs.desktop-e2e-relay.result }}
desktop_e2e_integration_result: ${{ needs.desktop-e2e-integration.result }}
backend_integration_result: ${{ needs.backend-integration.result }}
relay_e2e_result: ${{ needs.relay-e2e.result }}
postgres_tests_result: ${{ needs.postgres-tests.result }}
steps:
- run: echo "Captured Relay and PostgreSQL job results"