docs: add review-proven failure-path & async-state rules to AGENTS.md (#7061)

Mining the last 25 PRs' review threads (45 substantive findings, 11
reviewed PRs, avg **4.8 review rounds** each) shows **53% of findings
are repeats** of five clusters: swallowed failures, stale-async-state
races, tests that don't bind the production seam, unbounded
resources/retry loops, and non-atomic multi-step persistence. PR #6956
alone burned 4 rounds converging on one of these classes.

A second, independent mining pass over **71 agent-review rooms (303
findings, Aug 18–29)** confirmed the same clusters and added outcome
data — how often authors actually fix each finding class once flagged:
test-seam binding and unbounded-resource findings **100%**, swallowed
errors **90%**, stale-state races **70%**. It also surfaced two clusters
the GitHub-thread pass under-sampled: **assistive-semantics defects**
(44 findings, second-largest cluster) and **input-modality divergence**
(27 findings), now rules 7–8.

This PR distills those clusters into eight imperative rules in AGENTS.md
so agents apply them **before writing code**, adds one
client-consumption invariant to ARCHITECTURE.md §5, and places the
test-quality rule in TESTING.md (per the team decision that testing docs
are the canonical guide for review standards), cross-referenced from
AGENTS.md. Each rule cites the PRs where it was litigated. Raw mining
data: `reviews.jsonl` / `comments.jsonl` +
`backfill/buzz-review-findings.jsonl` (review-mining artifacts, not
committed).

No code changes. CLAUDE.md is a symlink to AGENTS.md and picks this up
automatically.

🤖 Drafted by Jude's agent from automated mining of this repo's last 25
PRs' review threads and 71 agent-review rooms; every rule cites the PRs
where it was litigated. Jude reviews and owns the result. Mining method
+ raw cluster data available on request.

---------

Signed-off-by: Jude Edwards <judeedwards@squareup.com>
This commit is contained in:
Jude Edwards
2026-08-31 15:13:58 -07:00
committed by GitHub
parent b47b5a5012
commit 4952f58102
3 changed files with 109 additions and 0 deletions

View File

@@ -16,6 +16,21 @@ just test # unit + integration (starts Docker if needed)
cargo test -p buzz-test-client -- --ignored
```
### Review-Proven Test Standards
Mined from the last 25 PRs' review threads (see Review-Proven Rules in
[AGENTS.md](AGENTS.md)); this is the test-quality rule reviewers litigated
most:
**Regression tests must bind the production seam and be falsifiable.**
A guard whose removal doesn't fail any test protects nothing — mutations
survived the full mobile suite twice (PRs #6996, #7013). Don't bind a
regression test to a test-only helper instead of the production code
path (PR #7013). Give pure predicates a table test over the full input
combination space (PR #6807). Scope Playwright locators — unscoped
`getByText` in a required smoke test is a strict-mode flake (PR #6980).
(PRs #6807, #6980, #6996, #7013)
---
## Live Local Relay