Address the Codex security review and the zizmor code-scanning alerts
(386-390) on #8613.
repeatRust merged each package's cargo target flags through a Set of
argv tokens, so two integration-test files in one package collapsed
`--test codec --test connection` into `--test codec connection`.
nextest reads the trailing `connection` as a test-name filter, lists
nothing, and the lane took the "nothing to repeat" branch and reported
success without running a test (verified against buzz-audio-client:
the collapsed argv lists no tests and exits 0). rustTarget now returns
one flag per entry and the new rustGroups keys each whole flag, so a
`--test <name>` is emitted once per name while `--lib`/`--bins` stay
single; the regression test asserts the exact argv for two integration
files and for a src/ plus tests/ file in the same package, through the
seam repeatRust uses. The `-p`, `-E` and filterset arguments were
re-read for the same class of bug and pass single values.
In _ci-flake.yml, actions/upload-artifact 043fb46d is v7.0.1 (the
`v7` tag has since moved to v7.0.2), so its four comments now say so,
as the other workflows do. rui314/setup-mold 7e4f20ad is a commit the
`v1` tag no longer points at (339e96b4, mold 3.0.0), so the flake
workflow pins the current `v1`; _ci-rust.yml, _ci-relay.yml and
_ci-desktop.yml carry the same stale pin and are left for a separate
change.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
Address the review of the flake check port (99b21e4fb).
An integration-only Playwright selection no longer fails flake-select:
Playwright exits 1 with a lone "No tests found" error when the file
filters match nothing in the smoke project, which listPlaywright now
reads as an empty selection, as buzz-app's listBrowser does. Listed
journeys are deduplicated, since a looped test() declaration is listed
once per iteration, and a selection over 250 journeys skips the
Playwright lane with a note: the smoke lane runs ~150 journey-runs per
shard in ~11 minutes, so 250 journey-runs per shard across ten shards
is what fits the 45-minute timeout after setup and the E2E build.
Rust entries now carry the file and the test's in-file module path
(`crates/.../auth.rs::tests::name`). repeatRust runs one cargo
invocation per package with target flags derived from the path
(src/** builds --lib --bins, tests/<stem>.rs builds --test <stem>), so
a buzz-relay change no longer compiles every test target of the crate.
It lists candidates by in-file path, prefers the test whose full name
the file's path predicts, and falls back to every suffix match only
when nothing matches (a `#[path]` module), so a same-named test in
another module is no longer repeated for a change it never saw.
The rust-cache bust knob moves into the shared key
(`unit-tests-sherpa-cache-v1`) in both the Unit Tests and flake-rust
jobs, since rust-cache ignores `key` once shared-key is set; the
contract script checks both and the flake restore's save-if, and the
regression harness covers both. flake-select installs desktop
dependencies only when a spec changed. Changed .rs files without test
functions get a summary row, test.fail() declarations count as tests,
a captured nextest 0.9.136 list/JUnit pair pins the classname contract,
and CONTRIBUTING lists what the lanes deliberately do not repeat.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
Add an advisory flake check that selects the tests a pull request
changes and repeats each one ten times with retries off, so a test
that passes CI by luck is caught before it merges. This mirrors
block/buzz-app#890 (and its sharding follow-up), adapted to the
runners this repository actually has.
scripts/flake-check.mjs `select` diffs HEAD^1..HEAD and maps hunks to
tests per runner: Rust files to `<package>::<fn>` entries by parsing
`#[test]`-style attributes (shared code changes select the file's
tests; `#[ignore]`d and Postgres tests are excluded), desktop
Playwright specs to `file:line` smoke journeys via `playwright --list`
(non-smoke and unregistered specs are excluded), and desktop node:test
and Flutter test files whole. Deleted and renamed-away files drop out
of the selection, and any runner whose selection exceeds 64 KiB is
dropped with a note so GitHub outputs stay within limits.
The reusable _ci-flake.yml adds `flake-select` plus one repeat lane per
runner (`flake-rust`, `flake-node`, `flake-playwright` sharded by
journey count, `flake-dart`). Each lane skips when its selection is
empty, runs only on pull_request, and is not read by any required
gate; its status and step summary carry the result. A `flake` nextest
profile disables retries and emits JUnit so the Rust lane can report
per-test, per-iteration results. The Unit Tests rust-cache gains a
`shared-key` so the flake lane can restore it read-only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Matt Toohey <contact@matttoohey.com>
