Address the review of the flake check port (99b21e4fb).
An integration-only Playwright selection no longer fails flake-select:
Playwright exits 1 with a lone "No tests found" error when the file
filters match nothing in the smoke project, which listPlaywright now
reads as an empty selection, as buzz-app's listBrowser does. Listed
journeys are deduplicated, since a looped test() declaration is listed
once per iteration, and a selection over 250 journeys skips the
Playwright lane with a note: the smoke lane runs ~150 journey-runs per
shard in ~11 minutes, so 250 journey-runs per shard across ten shards
is what fits the 45-minute timeout after setup and the E2E build.
Rust entries now carry the file and the test's in-file module path
(`crates/.../auth.rs::tests::name`). repeatRust runs one cargo
invocation per package with target flags derived from the path
(src/** builds --lib --bins, tests/<stem>.rs builds --test <stem>), so
a buzz-relay change no longer compiles every test target of the crate.
It lists candidates by in-file path, prefers the test whose full name
the file's path predicts, and falls back to every suffix match only
when nothing matches (a `#[path]` module), so a same-named test in
another module is no longer repeated for a change it never saw.
The rust-cache bust knob moves into the shared key
(`unit-tests-sherpa-cache-v1`) in both the Unit Tests and flake-rust
jobs, since rust-cache ignores `key` once shared-key is set; the
contract script checks both and the flake restore's save-if, and the
regression harness covers both. flake-select installs desktop
dependencies only when a spec changed. Changed .rs files without test
functions get a summary row, test.fail() declarations count as tests,
a captured nextest 0.9.136 list/JUnit pair pins the classname contract,
and CONTRIBUTING lists what the lanes deliberately do not repeat.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
Add an advisory flake check that selects the tests a pull request
changes and repeats each one ten times with retries off, so a test
that passes CI by luck is caught before it merges. This mirrors
block/buzz-app#890 (and its sharding follow-up), adapted to the
runners this repository actually has.
scripts/flake-check.mjs `select` diffs HEAD^1..HEAD and maps hunks to
tests per runner: Rust files to `<package>::<fn>` entries by parsing
`#[test]`-style attributes (shared code changes select the file's
tests; `#[ignore]`d and Postgres tests are excluded), desktop
Playwright specs to `file:line` smoke journeys via `playwright --list`
(non-smoke and unregistered specs are excluded), and desktop node:test
and Flutter test files whole. Deleted and renamed-away files drop out
of the selection, and any runner whose selection exceeds 64 KiB is
dropped with a note so GitHub outputs stay within limits.
The reusable _ci-flake.yml adds `flake-select` plus one repeat lane per
runner (`flake-rust`, `flake-node`, `flake-playwright` sharded by
journey count, `flake-dart`). Each lane skips when its selection is
empty, runs only on pull_request, and is not read by any required
gate; its status and step summary carry the result. A `flake` nextest
profile disables retries and emits JUnit so the Rust lane can report
per-test, per-iteration results. The Unit Tests rust-cache gains a
`shared-key` so the flake lane can restore it read-only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
