mirror of
https://github.com/block/buzz.git
synced 2026-10-27 04:30:52 +00:00
🤖 ## Summary Channels are Buzz's main way to control access, but today they cannot separate reading from writing. In an open channel, anyone who can read can also write. So there is no way to say "many people read, a few people write". Announcements, release feeds, plugin release records and shared configuration all need that. This PR adds **announce channels**: anyone can follow and read them, but only members can post. It does this with one new channel setting and without changing what any role means. - **Roles mean the same thing in every channel.** A *guest* can read and leave, but cannot write. A *member* (or *bot*) can write. *Admins* and *owners* also manage the channel. The relay already had the guest role, but no client used it until now. - **A new setting, `join_role` (`member` | `guest`), picks the role a new person gets** when nobody chooses one: when they join by themselves, or when someone adds them without a role. The default, `member`, keeps today's behavior for every existing channel. - **An open channel with `join_role = guest` is an announce channel.** Anyone can find, read and join it. People who join are guests, so they cannot write until someone makes them a member. The relay enforces these rules, so a client cannot get around them. What each role can do (the same in every channel): | Action | Guest | Member or bot | Admin / owner | | --- | --- | --- | --- | | Read, leave | ✅ | ✅ | ✅ | | Post, reply, edit, delete own, react, artifacts, canvas, forum, pins, topic, typing, workflow messages, save or run workflows, huddles | ❌ | ✅ | ✅ | | Add people | ❌ | ✅ as member, guest or bot | ✅ any role | | Make a guest a member (person or agent) | ❌ | ✅ | ✅ | | Any other role change | ❌ | ❌ | ✅ | | Change name, about, visibility, TTL, `join_role` | ❌ | ❌ | ✅ | Other rules: - **Joining by yourself.** A person who joins (kind:9021), or adds themselves (kind:9000 with their own key), gets the channel's `join_role`. They may ask for `join_role` or a lower role, never a higher one. So in an announce channel nobody can join as a member by themselves. - **Non-members.** A non-member may write only where joining would let them write: an open channel with `join_role = member`. That is today's behavior for open channels. - **Agents.** Agents follow the same rules as people. Any member who can write may make a guest agent a member. Limits that only an agent's owner may lift belong to the later IFC work (its "restricted" agent level), not to this PR. That work only refuses more actions, so it can be added later without breaking anything here. - **Leaving always works.** A member may leave an archived channel too (kind:9022, or kind:9001 naming themselves). The last owner still cannot leave. - **Agent owners keep edit and delete.** Today the human owner of an agent can edit (kind:40003) and delete (kind:9005) that agent's messages without being a member. This PR keeps that existing behavior in announce channels too, and documents and tests it. It does not let the owner post. - **Changing the setting changes no roles.** Existing members can still write after a change to `guest`. Existing guests stay read-only after a change back to `member`. A refused write gets `restricted: only members can post in this channel`. ## Details - **One write rule.** The relay's channel write check decides from the actor's role and the channel's `join_role`. Some event types skip that check and run their own: message edit, deleting your own message (kind:9005), topic and purpose (kind:9002), add-member (kind:9000), workflow messages and joining a huddle. Each of these now uses the same rule. Membership changes (add, remove, leave) do not need write access, so a guest can always leave. If the relay finds a stored `join_role` value it does not know, it treats it as `guest`, so an unknown value can never give more access. - **Admission is decided in the database transaction.** The membership write (`put_member`) makes the whole admission decision inside its transaction, after it takes the channel's membership lock: `join_role` for a missing role tag, the self-join ceiling, guests cannot add people, who may grant admin or owner, and role changes (`MemberRole::may_change_role`). The relay's request check uses the same rules, but the database is the authority, so a write that skips the request check cannot get around them. A change to `visibility` or `join_role` takes the same lock, so an add and a setting change cannot interleave. - **No creator exception.** `put_member` used to give owner to a channel's creator who added themselves to a private channel without an active membership. A removed creator could use that: their join (kind:9021) passes the open-channel check, the channel becomes private before the join takes the membership lock, and the join restored them as owner. Channel creation already makes the creator owner in the same transaction, so the exception is removed. A removed creator is now admitted like anyone else. - **Huddles.** Joining a huddle needs write access in the huddle channel **and** in its parent channel. The relay checks both before the join. When it commits the join, it applies the same rule again while it holds both channels' membership locks: a member needs a role that can post, and a non-member needs an open channel whose `join_role` is `member`. A guest in either channel cannot join. A join is also refused when the joiner loses membership or the channel's `join_role` changes before the commit. - **One lock order.** Each writer that takes more than one of a channel's locks takes them in this order: membership lock, TTL lock, channel row. Settings changes, roster publication, TTL refresh and huddle joins all use this order. Before this change, a `join_role` change on an ephemeral channel at the same time as a roster publication could deadlock. - **Workflows.** Saving a workflow (kind:30620) skips the generic write check, so it now checks the sender's role itself and needs a writing role before anything is stored. Every path that starts a workflow run, including the manual trigger (kind:46020), needs the workflow's owner to hold a writing role now (and an elevated role for an elevated workflow). So a workflow saved while its owner was a member stops running after the owner is made a guest. - **Channel names are checked before storage.** kind:9002 and kind:9007 refuse a name longer than the `channels.name` column (255 characters) before the event is stored. Before this, the relay accepted the event, then the name update failed, and the rest of the update (for example `join_role`) was silently skipped. This PR only adds this check. Storing the event and applying the update in one transaction is a larger change, and it is not in this PR. - **Changing the setting.** Owners and admins send a channel-metadata update (kind:9002) with `["join_role", "guest"]` or `["join_role", "member"]`. Buzz applies these updates as partial updates; a missing tag means "no change". The relay posts a `join_role_changed` system message in the channel. - **What clients see.** The relay signs a channel description event (kind:39000) that clients read. One shared builder now writes it for both the relay and the `buzz-admin` backfill, which had a copy that had drifted from the relay's. It always publishes an explicit `["join_role", …]` tag, plus the standard NIP-29 group access flags: | `visibility` | `join_role` | `public` / `private` | `restricted` | `closed` | | --- | --- | --- | --- | --- | | open | `member` | public | | | | open | `guest` | public | ✓ | | | private | either | private | ✓ | ✓ | This also fixes an old bug: `closed` ("join requests are ignored") was published for every channel, even open channels that accept joins. - **CLI and SDK.** `buzz channels update --join-role guest|member`. `buzz_sdk::build_update_channel` takes a new `join_role` argument and always writes it as an explicit tag. - **Docs.** `NOSTR.md` has an "Announce channels" section with the role table and the flag table. - **Tests.** Relay end-to-end tests, run in the CI relay E2E step: - `test_nip29_announce_channel_wire`: who can set `join_role`, the kind:39000 flags, joining as a guest, refused guest and non-member posts, reactions and adds, member posts, a member making a guest a member who then posts, changing the setting back without changing roles, and leaving. - `test_nip29_member_raises_guest_agent`: a member who is not the agent's owner can make a guest agent a member. - `test_nip29_announce_channel_wire_workflow_send_needs_write_role`: a guest's workflow message is refused; a member's is accepted. - `test_nip29_departure_wire_archived_channel_allows_self_leave`: guests and members can leave an archived channel. - `test_private_channel_any_member_can_invite` now expects a private-channel guest to be refused when it adds someone (it used to expect success). - `test_nip29_announce_channel_wire_workflow_commands_need_write_role`: saving and triggering a workflow, by a member and by a guest, in announce and normal channels; a guest cannot run a workflow saved while it was a member. - `test_nip29_announce_channel_wire_overlong_name_is_refused_before_storage`: a kind:9002 with an overlong name and `join_role` is refused, and kind:39000 still shows the old settings. - `test_nip29_announce_channel_wire_agent_owner_exception`: an agent's owner, as a guest and as a non-member, can edit and delete the agent's messages, while ordinary guests are still refused. - `test_nip29_guest_is_read_only_in_normal_channel`: a guest in an ordinary channel cannot write, and joining an ordinary channel still gives `member`. Unit tables cover the self-join rule for each `join_role`, and every actor, current role and requested role for the role-change rule. Database tests check the admission rules in `put_member` (`put_member_enforces_announce_admission`), that an add which waits on an uncommitted change to `guest` is decided on the new setting (`put_member_waits_for_a_concurrent_join_role_change`), and that a member can make a guest a member but not an admin. Unit tests cover the name limit and the workflow owner rule (`owner_authority_denies_roles_that_cannot_write`). A relay test covers huddle joins with different roles in the huddle and its parent, and a commit after a demotion, a `join_role` change or a lost membership (`huddle_join_needs_a_write_role_in_the_huddle_and_its_parent`). Lock-order tests: `join_role_change_waits_for_a_roster_publication_on_an_ephemeral_channel` reproduced a real deadlock (`40P01`) against the old order. `f2d_join_lock_order_serializes_membership_and_settings_writers` checks that membership and settings writers wait for a join and then succeed. `removed_creator_cannot_rejoin_a_private_channel` checks that `put_member` refuses a removed creator on a private channel, and `removed_creator_join_racing_a_private_switch_is_refused` holds the membership lock so a removed creator's kind:9021 passes the open-channel check, then makes the channel private before the join continues. Each new test was run against the old code and failed. ## Rollout 1. **Before the deploy, count guest rows** (`SELECT count(*) FROM channel_members WHERE role = 'guest' AND removed_at IS NULL`). We expect none. Any guest that exists loses the ability to post, change the topic and add people when this ships. If some exist, decide first whether to make them members. 2. Deploy. Every channel stays `join_role = member`, so nothing else changes. 3. **Do not set any channel to `join_role = guest` until no old relay is running.** Old relays ignore the column, so they would let guests and non-members write in that channel. 4. Rerun the `buzz-admin` channel reconcile so open channels drop the wrong `closed` flag. ## Migration and rollback - Migration `0067` adds `channels.join_role TEXT NOT NULL DEFAULT 'member'` with a CHECK constraint. Postgres stores a constant default in table metadata, so existing rows are not rewritten and there is no backfill. The relay runs the migration at startup. - Existing channels keep the old `closed` flag until their next change, or until the reconcile in Rollout step 4. - **Before rolling back the relay**, set every announce channel back to `join_role = member`. An older relay ignores the column and does not stop guests from writing, so those channels would accept posts from their guests and give `member` to new joiners again. ## Not in this PR Client UI: a Follow button, hiding the composer for guests, and an admin toggle. Clients that ignore the new tag keep working. A guest who tries to post sees the `restricted` error. ## Stack 1. #8143 (this PR): announce channels 2. #8494: store the bot role only for keys with a recorded agent owner --------- Signed-off-by: Logan Johnson <loganj@squareup.com> Co-authored-by: Larry <627498bd4bd1f281a16431e3c6cce3b5c25b6692798c78672298aefbf2f8f8b5@buzz.block.builderlab.xyz>
674 lines
31 KiB
YAML
674 lines
31 KiB
YAML
name: CI / Relay and PostgreSQL
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
rust:
|
|
required: true
|
|
type: boolean
|
|
desktop:
|
|
required: true
|
|
type: boolean
|
|
desktop_rust:
|
|
required: true
|
|
type: boolean
|
|
lane:
|
|
required: true
|
|
type: string
|
|
outputs:
|
|
desktop_e2e_relay_result:
|
|
value: ${{ jobs.results.outputs.desktop_e2e_relay_result }}
|
|
desktop_e2e_integration_result:
|
|
value: ${{ jobs.results.outputs.desktop_e2e_integration_result }}
|
|
backend_integration_result:
|
|
value: ${{ jobs.results.outputs.backend_integration_result }}
|
|
relay_e2e_result:
|
|
value: ${{ jobs.results.outputs.relay_e2e_result }}
|
|
postgres_tests_result:
|
|
value: ${{ jobs.results.outputs.postgres_tests_result }}
|
|
|
|
env:
|
|
CARGO_TERM_COLOR: always
|
|
BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
|
|
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
|
|
|
|
jobs:
|
|
# The relay binaries and the two nextest archives compile independently, so
|
|
# they build in parallel; consumers download every desktop-e2e-relay-*
|
|
# artifact into target/ci as before.
|
|
desktop-e2e-relay:
|
|
name: Desktop E2E Relay (${{ matrix.artifact }})
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
if: inputs.lane == 'artifacts' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- artifact: relay-binaries
|
|
paths: |
|
|
target/ci/buzz-relay
|
|
target/ci/git-credential-nostr
|
|
- artifact: backend-integration-tests
|
|
paths: target/ci/backend-integration-tests.tar.zst
|
|
- artifact: postgres-tests
|
|
paths: target/ci/postgres-tests.tar.zst
|
|
permissions:
|
|
contents: read
|
|
env:
|
|
SCCACHE_GHA_ENABLED: "true"
|
|
SCCACHE_GHA_RW_MODE: ${{ (github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.pull_request.number == 5224)) && 'READ_WRITE' || 'READ_ONLY' }}
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
# Reuse each artifact when none of its inputs changed (desktop-only PRs
|
|
# hit this every time). The key covers everything they embed, including
|
|
# migrations via sqlx migrate!.
|
|
- name: Restore relay artifacts cache
|
|
id: relay-artifacts-cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ matrix.paths }}
|
|
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
|
|
- uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
# All lanes restore the shared Rust cache; only the postgres-tests lane,
|
|
# which compiles the widest test graph, writes it so a faster lane
|
|
# cannot save a narrower target dir under the same key.
|
|
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
with:
|
|
workspaces: |
|
|
.
|
|
desktop/src-tauri
|
|
save-if: ${{ github.event_name != 'pull_request' && matrix.artifact == 'postgres-tests' }}
|
|
# Cache rustc outputs for unchanged workspace crates. Trusted pushes write;
|
|
# the bounded PR 5224 trial writes only to its isolated merge-ref scope.
|
|
- name: Set up sccache
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true'
|
|
uses: Mozilla-Actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11 # zizmor: ignore[cache-poisoning] Bounded trial: only PR 5224 writes to its isolated merge-ref scope; trusted pushes retain production writes.
|
|
with:
|
|
version: v0.16.0
|
|
- name: Install cargo-nextest
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact != 'relay-binaries'
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Build relay binaries
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'relay-binaries'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: cargo build --profile ci -p buzz-relay -p git-credential-nostr
|
|
- name: Build backend integration test archive
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'backend-integration-tests'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: |
|
|
cargo nextest archive \
|
|
--cargo-profile ci \
|
|
-p buzz-db \
|
|
-p buzz-relay \
|
|
-p buzz-test-client \
|
|
--lib \
|
|
--test e2e_event_reminder \
|
|
--test e2e_author_only_deletion \
|
|
--test e2e_existence_oracle \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst
|
|
- name: Build PostgreSQL test archive
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && matrix.artifact == 'postgres-tests'
|
|
env:
|
|
RUSTC_WRAPPER: sccache
|
|
run: |
|
|
postgres_package_args=()
|
|
while IFS= read -r package; do
|
|
postgres_package_args+=(-p "$package")
|
|
done < <(scripts/postgres-test-packages.sh)
|
|
if [[ "${#postgres_package_args[@]}" -eq 0 ]]; then
|
|
echo "no PostgreSQL test packages were discovered" >&2
|
|
exit 1
|
|
fi
|
|
cargo nextest archive \
|
|
--cargo-profile ci \
|
|
"${postgres_package_args[@]}" \
|
|
--lib \
|
|
--tests \
|
|
--archive-file target/ci/postgres-tests.tar.zst
|
|
- name: Save relay artifacts cache
|
|
# PR-scoped exact-source entries cannot warm main or other PRs and churn
|
|
# the shared cache pool. sccache provides read-only PR reuse instead.
|
|
if: steps.relay-artifacts-cache.outputs.cache-hit != 'true' && github.event_name == 'push'
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ matrix.paths }}
|
|
key: relay-artifacts-${{ matrix.artifact }}-${{ runner.os }}-${{ hashFiles('crates/**', 'migrations/**', 'Dockerfile', 'Cargo.toml', 'Cargo.lock', 'rust-toolchain.toml', '.cargo/config.toml', '.config/nextest.toml', 'scripts/postgres-test-*.sh', 'scripts/check-postgres-test-discovery.py', '.github/workflows/ci.yml', '.github/workflows/_ci-*.yml') }}
|
|
- name: Upload relay artifacts
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: desktop-e2e-relay-${{ matrix.artifact }}
|
|
path: ${{ matrix.paths }}
|
|
if-no-files-found: error
|
|
retention-days: 1
|
|
|
|
postgres-tests:
|
|
name: PostgreSQL Tests
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
if: inputs.lane == 'postgres' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
services:
|
|
postgres:
|
|
image: postgres:16
|
|
env:
|
|
POSTGRES_USER: buzz
|
|
POSTGRES_PASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
POSTGRES_DB: postgres
|
|
ports:
|
|
- 5432:5432
|
|
options: >-
|
|
--health-cmd "pg_isready -U buzz -d postgres"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 12
|
|
redis:
|
|
image: redis:7
|
|
ports:
|
|
- 6379:6379
|
|
options: >-
|
|
--health-cmd "redis-cli ping"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 12
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PG_BIN_DIR: /usr/bin
|
|
REDIS_URL: redis://localhost:6379
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Tune disposable PostgreSQL
|
|
env:
|
|
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
run: |
|
|
# Database-per-test cloning forces checkpoints. Durability is redundant
|
|
# for this disposable service and makes runtime depend on runner disk I/O.
|
|
psql --dbname postgres --set ON_ERROR_STOP=1 <<'SQL'
|
|
ALTER SYSTEM SET fsync = off;
|
|
ALTER SYSTEM SET full_page_writes = off;
|
|
ALTER SYSTEM SET synchronous_commit = off;
|
|
SELECT pg_reload_conf();
|
|
SQL
|
|
psql --dbname postgres --tuples-only --no-align --command \
|
|
"SELECT bool_and(setting = 'off') FROM pg_settings WHERE name IN ('fsync', 'full_page_writes', 'synchronous_commit')" \
|
|
| grep --fixed-strings --line-regexp t
|
|
- name: Install cargo-nextest
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Download backend test archive
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Restore relay executable permission
|
|
run: chmod +x ./target/ci/buzz-relay
|
|
- name: PostgreSQL-backed tests
|
|
env:
|
|
BUZZ_POSTGRES_ADMIN_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/postgres
|
|
PGPASSWORD: ${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}
|
|
run: |
|
|
scripts/postgres-test-run.sh \
|
|
--archive-file target/ci/postgres-tests.tar.zst
|
|
|
|
desktop-e2e-integration-shard:
|
|
name: Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})
|
|
runs-on: ubuntu-latest
|
|
# A shard needs ~10m; Playwright install-deps is bounded by
|
|
# ci-apt-retry.sh (~25m worst case), so a degraded mirror fails with its
|
|
# own error instead of the job timeout.
|
|
timeout-minutes: 40
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
shard: [1, 2, 3]
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Start integration services
|
|
run: |
|
|
for attempt in 1 2 3; do
|
|
if docker compose up -d postgres redis minio minio-init; then
|
|
break
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "docker compose up failed after 3 attempts" >&2
|
|
exit 1
|
|
fi
|
|
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
|
|
sleep $((attempt * 5))
|
|
done
|
|
- name: Get pnpm store directory
|
|
id: pnpm-cache
|
|
run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
|
|
- name: Restore pnpm store cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
|
|
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
|
|
restore-keys: pnpm-${{ runner.os }}-
|
|
- name: Install desktop dependencies
|
|
run: just desktop-install-ci
|
|
- name: Get Playwright version
|
|
id: pw-version
|
|
run: echo "version=$(cd desktop && node -e "console.log(require('@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"
|
|
- name: Restore Playwright browser cache
|
|
id: playwright-cache
|
|
uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
|
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
|
- name: Install Playwright Chromium
|
|
if: steps.playwright-cache.outputs.cache-hit != 'true'
|
|
run: cd desktop && pnpm exec playwright install chromium
|
|
- name: Install Playwright system dependencies
|
|
run: cd desktop && ../scripts/ci-apt-retry.sh pnpm exec playwright install-deps chromium
|
|
- name: Save Playwright browser cache
|
|
if: steps.playwright-cache.outputs.cache-hit != 'true' && github.event_name == 'push' && matrix.shard == 1
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
|
|
key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
|
|
- name: Desktop E2E build
|
|
run: pnpm -C desktop build:e2e
|
|
- name: Wait for integration services
|
|
run: |
|
|
wait_healthy() {
|
|
local service="$1"
|
|
local container="$2"
|
|
for attempt in $(seq 1 60); do
|
|
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
|
|
if [ "${status}" = "healthy" ]; then
|
|
echo "${service} is healthy"
|
|
return 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
docker logs "${container}" || true
|
|
return 1
|
|
}
|
|
wait_healthy "Postgres" "buzz-postgres"
|
|
wait_healthy "Redis" "buzz-redis"
|
|
wait_healthy "MinIO" "buzz-minio"
|
|
- name: Download relay binary
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Apply schema and seed deployment community
|
|
# MT: the relay resolves each request's tenant from the communities host
|
|
# map and fails closed on an unmapped host. The channel reconciler binds
|
|
# the deployment community ONCE at boot (outside its retry loop) and
|
|
# exits permanently on an unmapped host, so the 'localhost:3000'
|
|
# community MUST exist before the relay starts — the retry loop only
|
|
# handles late-seeded channels, not a late-seeded community. The relay
|
|
# migrates at boot via BUZZ_AUTO_MIGRATE, but that's too late for the
|
|
# pre-boot seed, so apply the schema here first (then drop AUTO_MIGRATE
|
|
# below). lower(host) is the unique index → ON CONFLICT target. psql
|
|
# isn't on PATH in hermit → exec into the buzz-postgres container.
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PGPASSWORD: buzz_dev
|
|
PGDATABASE: buzz
|
|
# Use the already-running docker postgres for desired-state planning instead of
|
|
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_DB: buzz
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
run: |
|
|
./bin/pgschema apply --file schema/schema.sql --auto-approve
|
|
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
|
|
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -qtA -c "
|
|
INSERT INTO communities (id, host)
|
|
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
|
|
ON CONFLICT (lower(host)) DO NOTHING
|
|
;"
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay
|
|
nohup env \
|
|
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
|
|
REDIS_URL=redis://localhost:6379 \
|
|
RELAY_URL=ws://localhost:3000 \
|
|
BUZZ_BIND_ADDR=0.0.0.0:3000 \
|
|
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
|
|
BUZZ_REQUIRE_AUTH_TOKEN=false \
|
|
BUZZ_RECONCILE_CHANNELS=true \
|
|
BUZZ_RATE_LIMIT_HUMAN_MESSAGES_PER_MIN=100000 \
|
|
BUZZ_RATE_LIMIT_HUMAN_API_CALLS_PER_MIN=100000 \
|
|
BUZZ_RATE_LIMIT_HUMAN_WS_EVENTS_PER_SEC=10000 \
|
|
BUZZ_GIT_PROBE_WRITERS=8 \
|
|
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
|
|
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
|
|
echo $! > /tmp/buzz-relay.pid
|
|
for attempt in $(seq 1 60); do
|
|
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
fi
|
|
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
|
|
if [ "${status_code}" = "200" ]; then
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
- name: Seed desktop e2e data
|
|
run: bash scripts/setup-desktop-test-data.sh
|
|
- name: Desktop relay-backed e2e
|
|
run: cd desktop && pnpm exec playwright test --project=integration --shard=${{ matrix.shard }}/${{ strategy.job-total }}
|
|
- name: Summarize flaky tests
|
|
if: ${{ !cancelled() }}
|
|
run: node scripts/summarize-flaky-tests.mjs playwright-report.json "Desktop E2E Integration (${{ matrix.shard }}/${{ strategy.job-total }})"
|
|
working-directory: desktop
|
|
- name: Upload desktop integration artifacts
|
|
if: ${{ !cancelled() }}
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: desktop-e2e-integration-artifacts-${{ matrix.shard }}
|
|
path: |
|
|
desktop/playwright-report
|
|
desktop/playwright-report.json
|
|
desktop/test-results
|
|
/tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
retention-days: 7
|
|
- name: Save pnpm store cache
|
|
if: github.event_name == 'push'
|
|
uses: actions/cache/save@caa296126883cff596d87d8935842f9db880ef25 # v5
|
|
with:
|
|
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
|
|
key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
|
|
|
|
desktop-e2e-integration:
|
|
name: Desktop E2E Integration
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
needs: [desktop-e2e-integration-shard]
|
|
if: always() && inputs.lane == 'required' && (github.event_name == 'push' || inputs.desktop || inputs.desktop_rust || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- name: Check integration shards
|
|
run: |
|
|
if [ "${{ needs.desktop-e2e-integration-shard.result }}" != "success" ]; then
|
|
echo "Desktop E2E Integration shards finished with: ${{ needs.desktop-e2e-integration-shard.result }}"
|
|
exit 1
|
|
fi
|
|
echo "Desktop E2E Integration shards passed"
|
|
|
|
backend-integration:
|
|
name: Backend Integration (relay e2e)
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- name: Install cargo-nextest
|
|
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
|
|
with:
|
|
tool: cargo-nextest@0.9.136
|
|
- name: Start integration services
|
|
run: |
|
|
for attempt in 1 2 3; do
|
|
if docker compose up -d postgres redis minio minio-init; then
|
|
break
|
|
fi
|
|
if [ "$attempt" -eq 3 ]; then
|
|
echo "docker compose up failed after 3 attempts" >&2
|
|
exit 1
|
|
fi
|
|
echo "docker compose up failed (attempt $attempt), retrying in $((attempt * 5))s..." >&2
|
|
sleep $((attempt * 5))
|
|
done
|
|
- name: Wait for integration services
|
|
run: |
|
|
wait_healthy() {
|
|
local service="$1"
|
|
local container="$2"
|
|
for attempt in $(seq 1 60); do
|
|
status=$(docker inspect --format='{{.State.Health.Status}}' "${container}" 2>/dev/null || echo "not_found")
|
|
if [ "${status}" = "healthy" ]; then
|
|
echo "${service} is healthy"
|
|
return 0
|
|
fi
|
|
sleep 2
|
|
done
|
|
docker logs "${container}" || true
|
|
return 1
|
|
}
|
|
wait_healthy "Postgres" "buzz-postgres"
|
|
wait_healthy "Redis" "buzz-redis"
|
|
wait_healthy "MinIO" "buzz-minio"
|
|
- name: Download relay artifacts
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Apply schema and seed deployment community
|
|
# MT: the relay resolves each request's tenant from the communities host
|
|
# map and fails closed on an unmapped host. The reminder scheduler binds
|
|
# the deployment community ONCE at boot and exits permanently on an
|
|
# unmapped host (no retry, unlike the channel reconciler), so the
|
|
# 'localhost:3000' community MUST exist before the relay starts — seeding
|
|
# after boot leaves the scheduler dead. The relay migrates at boot via
|
|
# BUZZ_AUTO_MIGRATE, but that's too late for the pre-boot seed, so apply
|
|
# the schema here first (then drop AUTO_MIGRATE below). lower(host) is the
|
|
# unique index → ON CONFLICT target. psql isn't on PATH in hermit → exec
|
|
# into the buzz-postgres container.
|
|
env:
|
|
PGHOST: localhost
|
|
PGPORT: "5432"
|
|
PGUSER: buzz
|
|
PGPASSWORD: buzz_dev
|
|
PGDATABASE: buzz
|
|
# Use the already-running docker postgres for desired-state planning instead of
|
|
# downloading an embedded Postgres from Maven Central (transient-fetch flake source).
|
|
PGSCHEMA_PLAN_HOST: localhost
|
|
PGSCHEMA_PLAN_PORT: "5432"
|
|
PGSCHEMA_PLAN_DB: buzz
|
|
PGSCHEMA_PLAN_USER: buzz
|
|
PGSCHEMA_PLAN_PASSWORD: buzz_dev
|
|
run: |
|
|
./bin/pgschema apply --file schema/schema.sql --auto-approve
|
|
docker exec -i -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -v ON_ERROR_STOP=1 < scripts/reconcile-schema-after-pgschema.sql
|
|
docker exec -e PGPASSWORD=buzz_dev buzz-postgres \
|
|
psql -U buzz -d buzz -qtA -c "
|
|
INSERT INTO communities (id, host)
|
|
VALUES ('00000000-0000-4000-8000-00000000c0de', 'localhost:3000')
|
|
ON CONFLICT (lower(host)) DO NOTHING
|
|
;"
|
|
- name: Git default-branch route and clone regressions
|
|
env:
|
|
# The preceding step already applied and reconciled schema/schema.sql.
|
|
BUZZ_TEST_SCHEMA_MODE: desired
|
|
BUZZ_TEST_DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
BUZZ_TEST_S3_ENDPOINT: http://localhost:9000
|
|
BUZZ_TEST_S3_BUCKET: buzz-media
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/api::git::settings::tests::external_infra::/)' \
|
|
--run-ignored ignored-only
|
|
- name: Relay quota accounting and rejection regressions
|
|
env:
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and (test(/^rejection::/) or test(/^admission::/))' \
|
|
--run-ignored all
|
|
- name: NIP-FI cross-pod command replay regressions
|
|
env:
|
|
BUZZ_TEST_REDIS_URL: redis://localhost:6379
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/^api::nip_fi::route_integration_tests::external_infra_redis::/)' \
|
|
--run-ignored ignored-only
|
|
- name: Workflow message provenance unit tests
|
|
# The relay's workflow_sink suite is not selected by the infra-free
|
|
# unit job. Its ignored database cases run in the isolated PostgreSQL
|
|
# lane; keep the pure provenance cases covered here without duplication.
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'package(buzz-relay) and test(/workflow_sink/)'
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay
|
|
nohup env \
|
|
DATABASE_URL="postgres://buzz:${BUZZ_TEST_POSTGRES_PASSWORD}@localhost:5432/buzz" \
|
|
REDIS_URL=redis://localhost:6379 \
|
|
RELAY_URL=ws://localhost:3000 \
|
|
BUZZ_BIND_ADDR=0.0.0.0:3000 \
|
|
BUZZ_RELAY_PRIVATE_KEY="$(openssl rand -hex 32)" \
|
|
BUZZ_REQUIRE_AUTH_TOKEN=false \
|
|
BUZZ_RECONCILE_CHANNELS=true \
|
|
BUZZ_GIT_PROBE_WRITERS=8 \
|
|
SPROUT_REMINDER_SCHEDULER_INTERVAL_SECS=1 \
|
|
./target/ci/buzz-relay > /tmp/buzz-relay.log 2>&1 &
|
|
echo $! > /tmp/buzz-relay.pid
|
|
for attempt in $(seq 1 60); do
|
|
if ! kill -0 "$(cat /tmp/buzz-relay.pid)" 2>/dev/null; then
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
fi
|
|
status_code=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:3000/_readiness || true)
|
|
if [ "${status_code}" = "200" ]; then
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
cat /tmp/buzz-relay.log
|
|
exit 1
|
|
- name: NIP-ER reminder e2e
|
|
# Feature e2e for NIP-ER (Event Reminders, kind:30300): write-path
|
|
# validation, author-only read filtering, and scheduler delivery against
|
|
# a live relay. The schema-drift / migration-version guarantee is owned
|
|
# by the buzz-db migration.rs unit tests, not this suite.
|
|
run: |
|
|
cargo nextest run \
|
|
--archive-file target/ci/backend-integration-tests.tar.zst \
|
|
-E 'binary(e2e_event_reminder) | binary(e2e_author_only_deletion) | binary(e2e_existence_oracle)' \
|
|
--run-ignored ignored-only
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
# schema/schema.sql (applied above) indexes kind:5 content.
|
|
BUZZ_TEST_SEARCH_INDEXES_DELETIONS: "1"
|
|
- name: Upload relay log
|
|
if: failure()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: backend-integration-relay-log
|
|
path: /tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
|
|
relay-e2e:
|
|
name: Relay E2E
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
|
|
permissions:
|
|
contents: read
|
|
steps:
|
|
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
|
|
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
|
|
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
|
|
with:
|
|
save-if: ${{ github.event_name != 'pull_request' }}
|
|
# Reuse the relay + git-credential-nostr built by Desktop E2E Relay
|
|
# instead of compiling them a second time.
|
|
- name: Download relay binary
|
|
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
pattern: desktop-e2e-relay-*
|
|
merge-multiple: true
|
|
path: target/ci
|
|
- name: Start relay
|
|
run: |
|
|
chmod +x ./target/ci/buzz-relay ./target/ci/git-credential-nostr
|
|
./scripts/start-relay-for-tests.sh --no-build
|
|
- name: Relay E2E tests
|
|
run: |
|
|
cargo test -p buzz-test-client --test e2e_persona --test e2e_team_catalog --test e2e_nostr_interop --test e2e_thread_roots --test e2e_project -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay invite -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip43_membership_snapshots_are_rejected -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_departure_wire -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_announce_channel_wire -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_member_raises_guest_agent -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_guest_is_read_only_in_normal_channel -- --ignored --nocapture
|
|
cargo test -p buzz-test-client --test e2e_relay nip29_relay_rejects_role_change_by_unprivileged_actor -- --ignored --nocapture
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
GIT_CREDENTIAL_NOSTR_BIN: ${{ github.workspace }}/target/ci/git-credential-nostr
|
|
- name: Media read-auth e2e
|
|
# Reads require kind:24242 `t=get` auth, so these binaries are the only
|
|
# coverage that a real relay rejects bare reads and honours host- and
|
|
# hash-scoped tokens. They were #[ignore]d and selected by no CI job, so
|
|
# the lane never ran; select it here, where MinIO and the seeded
|
|
# 'localhost:3000' community already exist.
|
|
# --no-fail-fast: without it cargo stops after the first failing binary,
|
|
# so one broken case hides every later binary's result.
|
|
run: |
|
|
cargo test -p buzz-test-client --no-fail-fast --test e2e_media --test e2e_media_extended --test e2e_media_video -- --ignored --nocapture
|
|
env:
|
|
RELAY_URL: ws://localhost:3000
|
|
RELAY_HTTP_URL: http://localhost:3000
|
|
- name: Upload relay logs
|
|
if: failure()
|
|
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
|
|
with:
|
|
name: relay-e2e-artifacts
|
|
path: /tmp/buzz-relay.log
|
|
if-no-files-found: ignore
|
|
|
|
|
|
results:
|
|
name: Results
|
|
if: ${{ always() }}
|
|
needs: [desktop-e2e-relay, postgres-tests, desktop-e2e-integration, backend-integration, relay-e2e]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
permissions: {}
|
|
outputs:
|
|
desktop_e2e_relay_result: ${{ needs.desktop-e2e-relay.result }}
|
|
desktop_e2e_integration_result: ${{ needs.desktop-e2e-integration.result }}
|
|
backend_integration_result: ${{ needs.backend-integration.result }}
|
|
relay_e2e_result: ${{ needs.relay-e2e.result }}
|
|
postgres_tests_result: ${{ needs.postgres-tests.result }}
|
|
steps:
|
|
- run: echo "Captured Relay and PostgreSQL job results"
|