Files
buzz/.github/workflows/_ci-rust.yml
Matt Toohey eda26b9423 ci: repeat changed tests ten times on pull requests (#8613)
This ports the advisory flake check from block/buzz-app
(https://github.com/block/buzz-app/pull/890) to this repository: pull
requests repeat the tests they change ten times, with retries off, so a
test that passes CI by luck is caught before it merges.

## What it does

`flake-select` diffs the PR merge commit against its first parent
(`HEAD^1..HEAD`, `fetch-depth: 2`) and maps changed hunks to tests per
runner: a hunk inside a test body selects that test; a change anywhere
else in a test file, or a new file, selects the whole file. One lane per
runner then repeats only that selection 10× with retries disabled and
writes a test × iteration table to its step summary. Lanes skip when
nothing was selected, run only on `pull_request`, and are not read by
any required gate in `ci.yml` — their own statuses carry the result.

## Per runner

| Runner | Selection granularity | Repeat mechanism |
|---|---|---|
| Rust (cargo-nextest) | `crates/<crate>/<path>.rs::<module::test>` from
`#[test]`-style attribute spans (comments and literals blanked first). A
hunk outside any test body selects the file's tests. `#[ignore]`d and
Postgres / `external_infra` tests are excluded. | One `cargo nextest run
--profile flake` per package per iteration, with target flags derived
from the path (`src/**` → `--lib --bins`, `tests/<stem>.rs` → `--test
<stem>`), a filterset of exact `binary_id & test` pairs, and JUnit
parsed per iteration. New `[profile.flake]`: `retries = 0`, `fail-fast =
false`. |
| node:test | Whole changed `desktop/src/**/*.test.mjs`,
`*.jsdom-test.mjs`, `desktop/scripts/*.test.mjs` | 10× loop, each file
alone, with the flag sets from `desktop/package.json` `test` |
| Playwright `smoke` | `file:line` journeys from `test()`/`it()` spans
(TypeScript AST) intersected with `playwright test --list
--project=smoke`, deduplicated | Native `--repeat-each=10 --retries=0`,
sharded `min(10, ceil(journeys / 5))` ways; `playwright-summary`
tabulates the JSON report |
| Flutter | Whole changed `mobile/test/**/*_test.dart` | 10× `flutter
test --file-reporter=json` loop with the mobile lane's `--dart-define` |

A runner whose selection exceeds 64 KiB of entries, or more than 250
Playwright journeys, is skipped with a note rather than overrunning its
timeout.

## Differences from the buzz-app version

- Lanes live in a reusable `_ci-flake.yml` called from `ci.yml`,
matching this repo's `_ci-*.yml` convention, instead of inline jobs.
- Playwright specs are parsed with the desktop app's own `typescript`
package rather than `rolldown/parseAst`.
- No Playwright `integration` project lane: it needs the relay artifact
chain. Integration-only specs list nothing for `smoke` and the selection
stays empty.
- Playwright shards are capped at 10, the same bound as the regular
smoke lane.
- Rust entries carry the file and in-file module path, and the filterset
is anchored to exact `(binary, test)` pairs, so a same-named test in
another module is not repeated.
- nextest target flags are derived from the changed file's path, so a
`buzz-relay` change does not compile every test target of the crate.
- A journey-count cap (250) skips the Playwright lane for selections ten
repetitions could not finish within its 45-minute timeout.

## Heads-up

- The Unit Tests `rust-cache` moves from `key: sherpa-cache-v1` to
`shared-key: unit-tests-sherpa-cache-v1` so the flake lane can restore
it read-only. This changes the cache key once: the first Unit Tests run
after merge builds cold.
- This PR itself changes no file the selector recognises
(`scripts/flake-check.test.mjs` is a root script, run by the Desktop
lane), so its own Flake Check lanes are expected to skip.

## Verification

- `node --test scripts/flake-check.test.mjs` — all unit tests pass (also
wired into the Desktop lane).
- `actionlint` on the changed workflows; `biome check` on the two
scripts.
- `scripts/test-rust-cache-contract.sh` and its regression harness pass
with the new shared key.
- Synthetic-diff selection runs for each runner, plus `select` against
this PR's own merge-base (empty, as expected).
- 2-iteration repeats of a real selection on all four runners locally.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <contact@matttoohey.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-29 14:34:20 +11:00

226 lines
9.8 KiB
YAML

name: CI / Rust
on:
workflow_call:
inputs:
rust:
required: true
type: boolean
desktop_rust:
required: true
type: boolean
lane:
required: true
type: string
outputs:
rust_lint_result:
value: ${{ jobs.results.outputs.rust_lint_result }}
unit_tests_result:
value: ${{ jobs.results.outputs.unit_tests_result }}
windows_rust_result:
value: ${{ jobs.results.outputs.windows_rust_result }}
env:
CARGO_TERM_COLOR: always
BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
jobs:
rust-lint:
name: Rust Lint
runs-on: ubuntu-latest
timeout-minutes: 30
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust || inputs.desktop_rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
save-if: ${{ github.event_name != 'pull_request' }}
- name: Format check
run: just fmt-check
- name: Desktop Tauri format check
run: just desktop-tauri-fmt-check
- name: Clippy
run: just clippy
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
# Named so the advisory flake lane (_ci-flake.yml) can restore it;
# with shared-key set, rust-cache ignores `key`, so the cache-bust
# suffix lives here. Bump it to discard every saved cache.
shared-key: unit-tests-sherpa-cache-v1
save-if: ${{ github.event_name != 'pull_request' }}
- name: Install cargo-nextest
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cargo-nextest@0.9.136
- name: Unit tests
run: just test-unit
server-cross-compile:
name: Server Cross-Compile
runs-on: ubuntu-latest
timeout-minutes: 30
if: inputs.lane == 'cross-compile' && (github.event_name == 'push' || inputs.rust)
permissions:
contents: read
strategy:
fail-fast: false
matrix:
target:
- x86_64-unknown-linux-musl
- aarch64-unknown-linux-musl
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
- uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
key: cross-${{ matrix.target }}
save-if: ${{ github.event_name != 'pull_request' }}
- name: Install cross
uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
with:
tool: cross@0.2.5
- name: Build server binaries
env:
TARGET: ${{ matrix.target }}
# PRs: compile + build-script gate only (no codegen/link). Main: full link gate.
CARGO_CMD: ${{ github.event_name == 'pull_request' && 'check' || 'build' }}
run: |
cross "$CARGO_CMD" --release --target "$TARGET" \
-p buzz-relay \
-p buzz-acp \
-p buzz-agent \
-p buzz-dev-mcp \
-p git-credential-nostr \
-p git-sign-nostr
windows-rust:
name: Windows Rust (x86_64-pc-windows-msvc)
runs-on: windows-latest
# Windows runners are slow and this compiles the workspace + Tauri crate
# cold across four steps; budget generously.
timeout-minutes: 45
if: inputs.lane == 'required' && (github.event_name == 'push' || inputs.rust || inputs.desktop_rust)
permissions:
contents: read
env:
TARGET: x86_64-pc-windows-msvc
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
# MSVC needs windows.h (aws-lc-sys et al.), so this runs on a real Windows
# runner — hermit, used by the Linux jobs, does not provide MSVC. The
# toolchain (1.98.1 + clippy via profile = default) comes from the
# repo-root rust-toolchain.toml, which the runner's preinstalled rustup
# honors on demand; the host triple already is x86_64-pc-windows-msvc.
- uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
with:
workspaces: |
.
desktop/src-tauri
key: windows-msvc
save-if: ${{ github.event_name != 'pull_request' }}
# Tauri validates externalBin at compile time, so the Tauri-crate steps
# below fail without these stubs. Mirrors scripts/bundle-sidecars.sh's
# Windows naming (binaries/<bin>-<triple>.exe); empty files suffice for a
# type-check since nothing executes them.
- name: Create sidecar placeholders
shell: bash
run: |
mkdir -p desktop/src-tauri/binaries
for bin in buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz; do
touch "desktop/src-tauri/binaries/${bin}-${TARGET}.exe"
done
- name: Clippy (workspace)
run: cargo clippy --workspace --all-targets --target $env:TARGET -- -D warnings
- name: Check (workspace)
run: cargo check --workspace --all-targets --target $env:TARGET
- name: Test (buzz-dev-mcp)
# The Windows-only bash resolver lives in buzz-dev-mcp; its unit tests
# only gate if this crate is tested ON Windows.
# Serial: windows_resolver_tests mutate process-global env
# (BUZZ_SHELL/GIT_BASH/SystemRoot) that SharedState::new reads.
run: cargo test -p buzz-dev-mcp --target $env:TARGET -- --test-threads=1
- name: Test (ACP launch-prefix platform contract)
# Execute the non-Unix refusal branch; a cross-compile cannot test it.
run: cargo test -p buzz-acp --target $env:TARGET --lib acp::launch::platform_tests
- name: Test (buzz-agent auth coordinator)
# The auth coordinator single-flights on an OS advisory lock, which is
# LockFileEx on Windows; this integration suite drives real second
# processes on the same lock file, so it only exercises the Windows
# lock runtime if it runs ON Windows. Every other job compiles it but
# never executes it. Tests exercised on Windows: lock serialization
# (two coordinators race for the same key), cooldown sidecar sharing
# across processes, attempt-sidecar adoption (UserInitiated waiter
# adopts a predecessor's denial), and the in-process single-flight for
# same-key coalescing. Tests that are UNIX-ONLY and NOT executed here:
# crash-release (flock drop on SIGKILL, guarded by #[cfg(unix)]) and
# cross-process cache success/race (on-disk token handoff, also
# #[cfg(unix)]).
run: cargo test -p buzz-agent --target $env:TARGET --test databricks_auth_coordinator
# Smoke-test the new host-prereq contract: Git for Windows (which provides
# bash) is available on the runner, a shell command round-trips, and bash
# does NOT resolve from System32 (so WSL's launcher is never picked up).
# windows-latest runners have Git for Windows pre-installed; the unit tests
# above exercise the MCP resolver itself. This step verifies the host env.
- name: Smoke-test host Git Bash prereq (host env check)
shell: bash
run: |
set -euo pipefail
# Git for Windows ships bash.exe under its bin/ directory; confirm it
# resolves from the standard location the runtime resolver probes first.
bash_path=$(command -v bash 2>/dev/null || true)
[[ -n "$bash_path" ]] || { echo "ERROR: bash not found on PATH — host Git for Windows missing" >&2; exit 1; }
echo "Resolved bash: $bash_path"
[[ "$bash_path" != *System32* ]] || { echo "ERROR: resolved bash is WSL's System32 launcher" >&2; exit 1; }
# Run a basic pipeline through the resolved bash (same invocation the
# agent uses: bash -c '...').
out=$(bash -c 'echo hello | tr a-z A-Z')
[[ "$out" == "HELLO" ]] || { echo "bash pipeline failed: got '$out'" >&2; exit 1; }
# Confirm git itself works — agents run git commands frequently.
git --version
repo=$(mktemp -d)
cd "$repo"
git init -q
git -c user.name=ci -c user.email=ci@example.com commit -q --allow-empty -m smoke
git log -1 --format=%s | grep -qx smoke
echo "Host bash resolved and functional; git commit round-trip passed"
- name: Check (Tauri crate)
run: cargo check --manifest-path desktop/src-tauri/Cargo.toml --workspace --all-targets --target $env:TARGET
env:
CMAKE_POLICY_VERSION_MINIMUM: "3.5"
- name: Test (Tauri crate)
run: cargo test --manifest-path desktop/src-tauri/Cargo.toml --target $env:TARGET
env:
CMAKE_POLICY_VERSION_MINIMUM: "3.5"
results:
name: Results
if: ${{ always() }}
needs: [rust-lint, unit-tests, server-cross-compile, windows-rust]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions: {}
outputs:
rust_lint_result: ${{ needs.rust-lint.result }}
unit_tests_result: ${{ needs.unit-tests.result }}
windows_rust_result: ${{ needs.windows-rust.result }}
steps:
- run: echo "Captured Rust job results"