mirror of
https://github.com/block/buzz.git
synced 2026-10-27 04:30:52 +00:00
docs: add review-proven failure-path & async-state rules to AGENTS.md (#7061)
Mining the last 25 PRs' review threads (45 substantive findings, 11 reviewed PRs, avg **4.8 review rounds** each) shows **53% of findings are repeats** of five clusters: swallowed failures, stale-async-state races, tests that don't bind the production seam, unbounded resources/retry loops, and non-atomic multi-step persistence. PR #6956 alone burned 4 rounds converging on one of these classes. A second, independent mining pass over **71 agent-review rooms (303 findings, Aug 18–29)** confirmed the same clusters and added outcome data — how often authors actually fix each finding class once flagged: test-seam binding and unbounded-resource findings **100%**, swallowed errors **90%**, stale-state races **70%**. It also surfaced two clusters the GitHub-thread pass under-sampled: **assistive-semantics defects** (44 findings, second-largest cluster) and **input-modality divergence** (27 findings), now rules 7–8. This PR distills those clusters into eight imperative rules in AGENTS.md so agents apply them **before writing code**, adds one client-consumption invariant to ARCHITECTURE.md §5, and places the test-quality rule in TESTING.md (per the team decision that testing docs are the canonical guide for review standards), cross-referenced from AGENTS.md. Each rule cites the PRs where it was litigated. Raw mining data: `reviews.jsonl` / `comments.jsonl` + `backfill/buzz-review-findings.jsonl` (review-mining artifacts, not committed). No code changes. CLAUDE.md is a symlink to AGENTS.md and picks this up automatically. 🤖 Drafted by Jude's agent from automated mining of this repo's last 25 PRs' review threads and 71 agent-review rooms; every rule cites the PRs where it was litigated. Jude reviews and owns the result. Mining method + raw cluster data available on request. --------- Signed-off-by: Jude Edwards <judeedwards@squareup.com>
This commit is contained in:
15
TESTING.md
15
TESTING.md
@@ -16,6 +16,21 @@ just test # unit + integration (starts Docker if needed)
|
||||
cargo test -p buzz-test-client -- --ignored
|
||||
```
|
||||
|
||||
### Review-Proven Test Standards
|
||||
|
||||
Mined from the last 25 PRs' review threads (see Review-Proven Rules in
|
||||
[AGENTS.md](AGENTS.md)); this is the test-quality rule reviewers litigated
|
||||
most:
|
||||
|
||||
**Regression tests must bind the production seam and be falsifiable.**
|
||||
A guard whose removal doesn't fail any test protects nothing — mutations
|
||||
survived the full mobile suite twice (PRs #6996, #7013). Don't bind a
|
||||
regression test to a test-only helper instead of the production code
|
||||
path (PR #7013). Give pure predicates a table test over the full input
|
||||
combination space (PR #6807). Scope Playwright locators — unscoped
|
||||
`getByText` in a required smoke test is a strict-mode flake (PR #6980).
|
||||
(PRs #6807, #6980, #6996, #7013)
|
||||
|
||||
---
|
||||
|
||||
## Live Local Relay
|
||||
|
||||
Reference in New Issue
Block a user