ci: repeat changed tests ten times on pull requests (#8613)

This ports the advisory flake check from block/buzz-app
(https://github.com/block/buzz-app/pull/890) to this repository: pull
requests repeat the tests they change ten times, with retries off, so a
test that passes CI by luck is caught before it merges.

## What it does

`flake-select` diffs the PR merge commit against its first parent
(`HEAD^1..HEAD`, `fetch-depth: 2`) and maps changed hunks to tests per
runner: a hunk inside a test body selects that test; a change anywhere
else in a test file, or a new file, selects the whole file. One lane per
runner then repeats only that selection 10× with retries disabled and
writes a test × iteration table to its step summary. Lanes skip when
nothing was selected, run only on `pull_request`, and are not read by
any required gate in `ci.yml` — their own statuses carry the result.

## Per runner

| Runner | Selection granularity | Repeat mechanism |
|---|---|---|
| Rust (cargo-nextest) | `crates/<crate>/<path>.rs::<module::test>` from
`#[test]`-style attribute spans (comments and literals blanked first). A
hunk outside any test body selects the file's tests. `#[ignore]`d and
Postgres / `external_infra` tests are excluded. | One `cargo nextest run
--profile flake` per package per iteration, with target flags derived
from the path (`src/**` → `--lib --bins`, `tests/<stem>.rs` → `--test
<stem>`), a filterset of exact `binary_id & test` pairs, and JUnit
parsed per iteration. New `[profile.flake]`: `retries = 0`, `fail-fast =
false`. |
| node:test | Whole changed `desktop/src/**/*.test.mjs`,
`*.jsdom-test.mjs`, `desktop/scripts/*.test.mjs` | 10× loop, each file
alone, with the flag sets from `desktop/package.json` `test` |
| Playwright `smoke` | `file:line` journeys from `test()`/`it()` spans
(TypeScript AST) intersected with `playwright test --list
--project=smoke`, deduplicated | Native `--repeat-each=10 --retries=0`,
sharded `min(10, ceil(journeys / 5))` ways; `playwright-summary`
tabulates the JSON report |
| Flutter | Whole changed `mobile/test/**/*_test.dart` | 10× `flutter
test --file-reporter=json` loop with the mobile lane's `--dart-define` |

A runner whose selection exceeds 64 KiB of entries, or more than 250
Playwright journeys, is skipped with a note rather than overrunning its
timeout.

## Differences from the buzz-app version

- Lanes live in a reusable `_ci-flake.yml` called from `ci.yml`,
matching this repo's `_ci-*.yml` convention, instead of inline jobs.
- Playwright specs are parsed with the desktop app's own `typescript`
package rather than `rolldown/parseAst`.
- No Playwright `integration` project lane: it needs the relay artifact
chain. Integration-only specs list nothing for `smoke` and the selection
stays empty.
- Playwright shards are capped at 10, the same bound as the regular
smoke lane.
- Rust entries carry the file and in-file module path, and the filterset
is anchored to exact `(binary, test)` pairs, so a same-named test in
another module is not repeated.
- nextest target flags are derived from the changed file's path, so a
`buzz-relay` change does not compile every test target of the crate.
- A journey-count cap (250) skips the Playwright lane for selections ten
repetitions could not finish within its 45-minute timeout.

## Heads-up

- The Unit Tests `rust-cache` moves from `key: sherpa-cache-v1` to
`shared-key: unit-tests-sherpa-cache-v1` so the flake lane can restore
it read-only. This changes the cache key once: the first Unit Tests run
after merge builds cold.
- This PR itself changes no file the selector recognises
(`scripts/flake-check.test.mjs` is a root script, run by the Desktop
lane), so its own Flake Check lanes are expected to skip.

## Verification

- `node --test scripts/flake-check.test.mjs` — all unit tests pass (also
wired into the Desktop lane).
- `actionlint` on the changed workflows; `biome check` on the two
scripts.
- `scripts/test-rust-cache-contract.sh` and its regression harness pass
with the new shared key.
- Synthetic-diff selection runs for each runner, plus `select` against
this PR's own merge-base (empty, as expected).
- 2-iteration repeats of a real selection on all four runners locally.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Matt Toohey <contact@matttoohey.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

.config/nextest.toml

@@ -43,3 +43,13 @@ threads-required = "num-test-threads"
 filter = "(test(/postgres_tests::/) or binary(/^postgres_/)) and not test(/(^|::)external_infra[^:]*::/)"
 setup = "postgres-template"
 run-wrapper = "postgres-isolation"
+
+[profile.flake]
+# The advisory pull-request flake lane (scripts/flake-check.mjs rust) repeats
+# a selection and must see every failed repetition, so no retries; the JUnit
+# report gives it per-test statuses.
+retries = 0
+fail-fast = false
+
+[profile.flake.junit]
+path = "junit.xml"

.github/workflows/_ci-desktop.yml

@@ -52,6 +52,8 @@ jobs:
         run: just desktop-check
       - name: Desktop unit tests
         run: just desktop-test
+      - name: Flake check selection tests
+        run: node --test scripts/flake-check.test.mjs
       - name: Desktop build
         run: just desktop-build
       - name: Save pnpm store cache

.github/workflows/_ci-flake.yml

@@ -0,0 +1,330 @@
+name: CI / Flake Check
+on:
+  workflow_call:
+
+env:
+  CARGO_TERM_COLOR: always
+  BUZZ_TEST_POSTGRES_PASSWORD: buzz_dev
+  PLAYWRIGHT_BROWSERS_PATH: ${{ github.workspace }}/.cache/ms-playwright
+
+# Advisory flake check: repeat the tests a pull request changes, ten times
+# each with retries off, so a race inside a test surfaces before it lands on
+# main. The lanes skip when no test changed, so they stay out of the required
+# checks in ci.yml (whose gates reject skipped lanes); their own statuses and
+# step summaries carry the result. Selections reach the lanes through env
+# vars, never interpolated into run scripts; flake-check.mjs admits only plain
+# path and identifier characters.
+jobs:
+  flake-select:
+    name: Flake check selection
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    permissions:
+      contents: read
+    outputs:
+      rust: ${{ steps.select.outputs.rust }}
+      tauri: ${{ steps.select.outputs.tauri }}
+      node: ${{ steps.select.outputs.node }}
+      playwright: ${{ steps.select.outputs.playwright }}
+      playwright_shards: ${{ steps.select.outputs.playwright_shards }}
+      playwright_shard_total: ${{ steps.select.outputs.playwright_shard_total }}
+      dart: ${{ steps.select.outputs.dart }}
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+        with:
+          # The merge commit's first parent is the base branch tip, so this
+          # diff is exactly the pull request's change without a full clone.
+          fetch-depth: 2
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      # Only a changed spec needs desktop dependencies: Playwright lists its
+      # smoke journeys and the desktop app's TypeScript parses it. Pull
+      # requests without one skip the install.
+      - name: Check for changed Playwright specs
+        id: specs
+        run: |
+          if git diff --quiet --diff-filter=AM HEAD^1 HEAD -- 'desktop/tests/e2e/*.spec.ts'; then
+            echo "changed=false" >> "$GITHUB_OUTPUT"
+          else
+            echo "changed=true" >> "$GITHUB_OUTPUT"
+          fi
+      - name: Get pnpm store directory
+        if: steps.specs.outputs.changed == 'true'
+        id: pnpm-cache
+        run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
+      - name: Restore pnpm store cache
+        if: steps.specs.outputs.changed == 'true'
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
+          key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: pnpm-${{ runner.os }}-
+      - name: Install desktop dependencies
+        if: steps.specs.outputs.changed == 'true'
+        run: just desktop-install-ci
+      - name: Map changed hunks to tests
+        id: select
+        run: node scripts/flake-check.mjs select HEAD^1 HEAD
+
+  flake-rust:
+    name: Flake check (Rust)
+    needs: [flake-select]
+    if: needs.flake-select.outputs.rust != ''
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      - uses: rui314/setup-mold@339e96b4ff1f3bd865856ba4cbbe8818a7acb038 # v1
+      # Restores the Unit Tests lane's cache (_ci-rust.yml) read-only; this
+      # lane never saves one of its own. The restore only hits while the cache
+      # key matches: the same shared-key, and rust-cache's environment hash —
+      # the CARGO*/RUST*/CC* environment variables, the lockfiles and the rustc
+      # version — must stay identical to the Unit Tests job's, so any env this
+      # job sets must be set there too (scripts/test-rust-cache-contract.sh
+      # checks the key).
+      - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
+        with:
+          shared-key: unit-tests-sherpa-cache-v1
+          save-if: false
+      - name: Install cargo-nextest
+        uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
+        with:
+          tool: cargo-nextest@0.9.136
+      # nextest has no repeat flag; the script runs the selection once per
+      # iteration under the `flake` profile (.config/nextest.toml).
+      - name: Repeat changed Rust tests
+        env:
+          SELECTED: ${{ needs.flake-select.outputs.rust }}
+        run: node scripts/flake-check.mjs rust --iterations=10 $SELECTED
+      - name: Rust iteration reports
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: flake-rust
+          path: test-results/flake
+          if-no-files-found: ignore
+          retention-days: 7
+
+  flake-tauri:
+    name: Flake check (Desktop Tauri)
+    needs: [flake-select]
+    if: needs.flake-select.outputs.tauri != ''
+    runs-on: ubuntu-latest
+    # A Desktop Tauri lane's bound (_ci-desktop.yml: ci-apt-retry.sh bounds
+    # the apt step at ~34m worst case and a cold build needs ~15m), plus a
+    # second cold build: a lib selection also builds the mesh-llm graph.
+    timeout-minutes: 70
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      - uses: rui314/setup-mold@339e96b4ff1f3bd865856ba4cbbe8818a7acb038 # v1
+      # Restores the Desktop Tauri core lane's cache (_ci-desktop.yml)
+      # read-only, under the same constraints as flake-rust's restore above:
+      # the workspace, shared-key and environment hash must stay identical to
+      # that job's.
+      - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
+        with:
+          workspaces: desktop/src-tauri
+          shared-key: desktop-core
+          save-if: false
+      - name: Install Tauri dependencies (Linux)
+        env:
+          DEBIAN_FRONTEND: noninteractive
+        run: |
+          CI_APT_ATTEMPT_SECONDS=90 scripts/ci-apt-retry.sh sudo apt-get update
+          scripts/ci-apt-retry.sh sudo apt-get install -y --no-install-recommends \
+            build-essential \
+            curl \
+            file \
+            libasound2-dev \
+            libayatana-appindicator3-dev \
+            libgtk-3-dev \
+            librsvg2-dev \
+            libssl-dev \
+            libwebkit2gtk-4.1-dev \
+            libxdo-dev \
+            patchelf \
+            wget
+      - name: Install cargo-nextest
+        uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15
+        with:
+          tool: cargo-nextest@0.9.136
+      # tauri-build validates the sidecar binaries at compile time; the Desktop
+      # Tauri recipes create the same empty stubs.
+      - name: Create sidecar stubs
+        run: just _ensure-sidecar-stubs
+      # Same script as flake-rust, against the Tauri workspace's manifest and
+      # its own .config/nextest.toml `flake` profile. A selected lib test also
+      # repeats under mesh-llm, as desktop-tauri-test-mesh runs it; the
+      # restored desktop-core cache does not hold that dependency graph, so a
+      # lib selection builds it cold, and an integration-test one never does.
+      - name: Repeat changed desktop Tauri tests
+        env:
+          CMAKE_POLICY_VERSION_MINIMUM: "3.5"
+          SELECTED: ${{ needs.flake-select.outputs.tauri }}
+        run: node scripts/flake-check.mjs tauri --iterations=10 $SELECTED
+      - name: Desktop Tauri iteration reports
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: flake-tauri
+          path: test-results/flake
+          if-no-files-found: ignore
+          retention-days: 7
+
+  flake-node:
+    name: Flake check (node:test)
+    needs: [flake-select]
+    if: needs.flake-select.outputs.node != ''
+    runs-on: ubuntu-latest
+    timeout-minutes: 15
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      - name: Get pnpm store directory
+        id: pnpm-cache
+        run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
+      - name: Restore pnpm store cache
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
+          key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: pnpm-${{ runner.os }}-
+      - name: Install desktop dependencies
+        run: just desktop-install-ci
+      - name: Repeat changed node:test files
+        env:
+          SELECTED: ${{ needs.flake-select.outputs.node }}
+        run: node scripts/flake-check.mjs node --iterations=10 $SELECTED
+      - name: node:test iteration logs
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: flake-node
+          path: test-results/flake
+          if-no-files-found: ignore
+          retention-days: 7
+
+  flake-playwright:
+    name: Flake check (Playwright smoke, ${{ matrix.shard }}/${{ needs.flake-select.outputs.playwright_shard_total }})
+    needs: [flake-select]
+    if: needs.flake-select.outputs.playwright != ''
+    runs-on: ubuntu-latest
+    # Same bound as a smoke shard: Chromium dependency setup is bounded by
+    # ci-apt-retry.sh (~25m worst case), and flake-check.mjs sizes the shards
+    # so the repetitions fit the rest.
+    timeout-minutes: 45
+    strategy:
+      fail-fast: false
+      matrix:
+        shard: ${{ fromJSON(needs.flake-select.outputs.playwright_shards) }}
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      - name: Get pnpm store directory
+        id: pnpm-cache
+        run: echo "STORE_PATH=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
+      - name: Restore pnpm store cache
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
+          key: pnpm-${{ runner.os }}-${{ hashFiles('**/pnpm-lock.yaml') }}
+          restore-keys: pnpm-${{ runner.os }}-
+      - name: Install desktop dependencies
+        run: just desktop-install-ci
+      - name: Get Playwright version
+        id: pw-version
+        run: echo "version=$(cd desktop && node -e "console.log(require('@playwright/test/package.json').version)")" >> "$GITHUB_OUTPUT"
+      - name: Restore Playwright browser cache
+        id: playwright-cache
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ${{ env.PLAYWRIGHT_BROWSERS_PATH }}
+          key: playwright-${{ runner.os }}-${{ steps.pw-version.outputs.version }}
+      - name: Install Playwright Chromium
+        if: steps.playwright-cache.outputs.cache-hit != 'true'
+        run: cd desktop && pnpm exec playwright install chromium
+      - name: Install Playwright system dependencies
+        run: cd desktop && ../scripts/ci-apt-retry.sh pnpm exec playwright install-deps chromium
+      - name: Desktop E2E build
+        run: pnpm -C desktop build:e2e
+      # --retries=0 overrides the config's CI retries, so every failed
+      # repetition reports as unexpected and fails the step.
+      - name: Repeat changed smoke journeys
+        env:
+          PLAYWRIGHT_JSON_OUTPUT_FILE: playwright-report.json
+          SELECTED: ${{ needs.flake-select.outputs.playwright }}
+          SHARD: ${{ matrix.shard }}
+          SHARD_TOTAL: ${{ needs.flake-select.outputs.playwright_shard_total }}
+        run: cd desktop && pnpm exec playwright test --project=smoke --repeat-each=10 --retries=0 --shard="$SHARD/$SHARD_TOTAL" --reporter=list,json $SELECTED
+      - name: Summarize repetitions
+        if: ${{ !cancelled() }}
+        env:
+          SHARD: ${{ matrix.shard }}
+          SHARD_TOTAL: ${{ needs.flake-select.outputs.playwright_shard_total }}
+        run: node scripts/flake-check.mjs playwright-summary desktop/playwright-report.json "Flake check (Playwright smoke, $SHARD/$SHARD_TOTAL)"
+      - name: Playwright repetition artifacts
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: flake-playwright-${{ matrix.shard }}
+          path: |
+            desktop/playwright-report.json
+            desktop/test-results
+          if-no-files-found: ignore
+          retention-days: 7
+
+  flake-dart:
+    name: Flake check (Flutter)
+    needs: [flake-select]
+    if: needs.flake-select.outputs.dart != ''
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    permissions:
+      contents: read
+    steps:
+      - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
+      - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1
+      # Restores the Mobile lane's Hermit and pub caches read-only; that lane
+      # saves them.
+      - name: Compute Hermit cache key
+        id: hermit-bin-hash
+        run: |
+          hash="$(find ./bin ! -type d | sort | xargs openssl sha256 | openssl sha256 -r | cut -d' ' -f1)"
+          echo "hash=$hash" >> "$GITHUB_OUTPUT"
+      - name: Restore Hermit package cache
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ~/.cache/hermit/pkg
+          key: ${{ runner.os }}-hermit-cache-${{ steps.hermit-bin-hash.outputs.hash }}
+          restore-keys: ${{ runner.os }}-hermit-cache-
+      - name: Prime Flutter SDK
+        run: flutter --version
+      - name: Restore pub cache
+        uses: actions/cache/restore@caa296126883cff596d87d8935842f9db880ef25 # v5
+        with:
+          path: ~/.pub-cache
+          key: pub-${{ runner.os }}-${{ hashFiles('mobile/pubspec.lock') }}
+          restore-keys: pub-${{ runner.os }}-
+      - name: Install dependencies
+        run: cd mobile && flutter pub get
+      - name: Repeat changed Flutter tests
+        env:
+          SELECTED: ${{ needs.flake-select.outputs.dart }}
+        run: node scripts/flake-check.mjs dart --iterations=10 $SELECTED
+      - name: Flutter iteration reports
+        if: ${{ !cancelled() }}
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: flake-dart
+          path: test-results/flake
+          if-no-files-found: ignore
+          retention-days: 7

.github/workflows/_ci-rust.yml

@@ -58,7 +58,10 @@ jobs:
       - uses: rui314/setup-mold@7e4f20ad28a2e8ca6fd0892ccf72e2abb706b9c3 # v1
       - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32
         with:
-          key: sherpa-cache-v1
+          # Named so the advisory flake lane (_ci-flake.yml) can restore it;
+          # with shared-key set, rust-cache ignores `key`, so the cache-bust
+          # suffix lives here. Bump it to discard every saved cache.
+          shared-key: unit-tests-sherpa-cache-v1
           save-if: ${{ github.event_name != 'pull_request' }}
       - name: Install cargo-nextest
         uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15

.github/workflows/ci.yml

@@ -83,6 +83,9 @@ jobs:
               - 'scripts/normative-corpus.json'
               - 'desktop/**'
               - 'pnpm-lock.yaml'
+              - 'scripts/flake-check.mjs'
+              - 'scripts/flake-check.test.mjs'
+              - '.github/workflows/_ci-flake.yml'
             desktop-rust:
               - 'desktop/src-tauri/**'
             web:
@@ -259,6 +262,14 @@ jobs:
       mobile: ${{ needs.changes.outputs.mobile == 'true' }}
       lane: mobile-swift
 
+  # Advisory flake check: repeats the tests a pull request changes. Its lanes
+  # skip when no test changed and none of the required gates below reads them;
+  # their own statuses and step summaries carry the result.
+  flake-check:
+    name: Flake Check
+    if: github.event_name == 'pull_request'
+    uses: ./.github/workflows/_ci-flake.yml
+
   security-domain:
     name: Security Domain
     needs: [changes]

CONTRIBUTING.md

@@ -304,6 +304,34 @@ just ci
 This is the same check that runs in CI. PRs that fail `just ci` will not be
 merged. If `just ci` fails on formatting, `just fix-all` fixes it in one shot (`rustfmt` + Tauri fmt + desktop, web, and mobile formatters).
 
+Pull requests also get an advisory **flake check**. `scripts/flake-check.mjs
+select` maps the diff against the base branch to the Rust tests (in the root
+and desktop Tauri workspaces), desktop `node --test` files, Playwright smoke
+journeys and Flutter test files it changed: a hunk inside a test body selects
+that test; a change anywhere else in a test file (imports, helpers, hooks,
+the code under test beside Rust tests) or a new file selects the whole file.
+An integration test binary's tests include those of the file modules its
+root declares (`mod x;`, `#[path = "…"] mod x;`), and a change to a helper
+module under a crate's `tests/` selects every integration test binary that
+includes it. The `Flake check (…)` lanes then
+run that selection ten times with retries off; a repetition that fails is
+usually a race in the test or the code under it, worth fixing before it
+lands, though a test that needs infrastructure or the regular lane's
+cross-file concurrency can fail every repetition instead. Some changed tests
+are deliberately not repeated: `#[ignore]`d and Postgres-backed Rust tests,
+Playwright specs outside the `smoke` project, `*.perf.ts` measurements, and a
+selection too large for its lane (over 64 KiB of entries, or more Playwright
+journeys than ten repetitions can finish within the lane's timeout).
+A Rust test repeats in every feature graph the regular lanes test its
+package in that lists it (LaunchDarkly, the App Attest development feature,
+the desktop crate's lib under `mesh-llm`), since a `#[cfg(feature = …)]` arm
+compiles only in its own graph; the step summary marks a row from a feature
+graph with that feature. A selected test that no such graph lists on the
+Linux runner, such as a macOS-only one, fails its lane by name rather than
+passing unrepeated.
+The lanes skip when no test changed and are not required to merge; their
+step summaries show the selection and the test × iteration table.
+
 ---
 
 ## Code Style

desktop/src-tauri/.config/nextest.toml

@@ -0,0 +1,14 @@
+# nextest reads its configuration from the workspace root, so the desktop
+# Tauri workspace carries its own. Only the advisory pull-request flake lane
+# (scripts/flake-check.mjs tauri) runs nextest here; the Desktop Tauri lanes
+# use `cargo test`.
+
+[profile.flake]
+# Mirrors the root .config/nextest.toml profile: the lane repeats a selection
+# and must see every failed repetition, so no retries; the JUnit report gives
+# it per-test statuses.
+retries = 0
+fail-fast = false
+
+[profile.flake.junit]
+path = "junit.xml"

scripts/test-rust-cache-contract-regressions.sh

@@ -139,13 +139,28 @@ YAML
 run_contract >/dev/null
 rm -rf "$tmp/.github/actions" "$tmp/.github/workflows/new-cache-user.yaml"
 
+cp "$root/.github/workflows/_ci-flake.yml" "$tmp/flake.orig"
+python3 - "$tmp/.github/workflows/_ci-flake.yml" <<'PY'
+import pathlib
+import sys
+
+path = pathlib.Path(sys.argv[1])
+workflow = path.read_text()
+needle = "          shared-key: unit-tests-sherpa-cache-v1\n          save-if: false\n"
+if workflow.count(needle) != 1:
+    raise SystemExit("expected one flake-rust cache restore key")
+path.write_text(workflow.replace(needle, "          shared-key: unit-tests\n          save-if: false\n"))
+PY
+expect_failure 'flake-rust rust-cache action must restore with.shared-key unit-tests-sherpa-cache-v1'
+mv "$tmp/flake.orig" "$tmp/.github/workflows/_ci-flake.yml"
+
 python3 - "$tmp/.github/workflows/_ci-rust.yml" <<'PY'
 import pathlib
 import sys
 
 path = pathlib.Path(sys.argv[1])
 workflow = path.read_text()
-needle = "          key: sherpa-cache-v1\n          save-if:"
+needle = "          shared-key: unit-tests-sherpa-cache-v1\n          save-if:"
 replacement = "          save-if:"
 if workflow.count(needle) != 1:
     raise SystemExit("expected one Unit Tests cache generation key")
@@ -153,9 +168,9 @@ workflow = workflow.replace(needle, replacement)
 job = "  unit-tests:\n"
 if workflow.count(job) != 1:
     raise SystemExit("expected one Unit Tests job")
-workflow = workflow.replace(job, job + "    env:\n      key: sherpa-cache-v1\n", 1)
+workflow = workflow.replace(job, job + "    env:\n      shared-key: unit-tests-sherpa-cache-v1\n", 1)
 path.write_text(workflow)
 PY
-expect_failure 'rust-cache action must keep with.key set to sherpa-cache-v1'
+expect_failure 'rust-cache action must keep with.shared-key set to unit-tests-sherpa-cache-v1'
 
 echo "rust cache contract regressions passed"

scripts/test-rust-cache-contract.sh

@@ -104,9 +104,30 @@ cache_steps = steps.select do |step|
 end
 abort "Unit Tests must contain exactly one rust-cache action step" unless cache_steps.length == 1
 
+# With shared-key set, rust-cache ignores `key`, so the cache-bust suffix
+# lives in the shared key; the advisory flake lane restores the same cache
+# and must name it identically.
+shared_key = "unit-tests-sherpa-cache-v1"
 cache_with = cache_steps.first["with"]
-unless cache_with.is_a?(Hash) && cache_with["key"] == "sherpa-cache-v1"
-  abort "Unit Tests rust-cache action must keep with.key set to sherpa-cache-v1"
+unless cache_with.is_a?(Hash) && cache_with["shared-key"] == shared_key && !cache_with.key?("key")
+  abort "Unit Tests rust-cache action must keep with.shared-key set to #{shared_key} and no with.key"
+end
+
+flake_ci_path = root / ".github" / "workflows" / "_ci-flake.yml"
+flake_ci = workflows.fetch(flake_ci_path) { load_workflow.call(flake_ci_path) }
+flake_jobs = flake_ci["jobs"]
+flake_rust = flake_jobs.is_a?(Hash) ? flake_jobs["flake-rust"] : nil
+abort "flake-rust job missing" unless flake_rust.is_a?(Hash)
+flake_steps = flake_rust["steps"]
+abort "flake-rust steps missing" unless flake_steps.is_a?(Array)
+flake_cache_steps = flake_steps.select do |step|
+  step.is_a?(Hash) && step["uses"].is_a?(String) &&
+    step["uses"].match?(/\ASwatinem\/rust-cache@/i)
+end
+abort "flake-rust must contain exactly one rust-cache action step" unless flake_cache_steps.length == 1
+flake_with = flake_cache_steps.first["with"]
+unless flake_with.is_a?(Hash) && flake_with["shared-key"] == shared_key && flake_with["save-if"] == false
+  abort "flake-rust rust-cache action must restore with.shared-key #{shared_key} with save-if false"
 end
 
 renovate = JSON.parse(Pathname(renovate_name).binread.force_encoding("UTF-8"))