jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
buzz/benchmarks/buzz-dataset/interleaved-agent-reports
History
Salman Mohammed 79df468255 Add Buzz benchmark evaluation layers (#6823) 
## Summary

- classify every Buzz-native benchmark task as `regression` or
`workflow`
- add metadata-backed `--layer` selection with k=1 regression and k=3
workflow defaults, while preserving explicit `--attempts`/`-k`
- split an unqualified mixed Buzz dataset run into two Harbor jobs so
each layer gets the correct default
- document the layer contract and cover metadata, selection, defaults,
overrides, invalid input, and command construction

---------

Signed-off-by: Salman Mohammed <smohammed@squareup.com>
2026-08-26 17:41:56 -04:00
..
environment
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
tests
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
instruction.md
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
README.md
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
task.toml
Add Buzz benchmark evaluation layers (#6823)
2026-08-26 17:41:56 -04:00

README.md

interleaved-agent-reports

Three signed bot identities publish independent reports immediately after the human request. The solo agent must retain every input, calculate 87, notify the human exactly once, and avoid waking the reporters again. The case exercises the batching/steering family reported in block/buzz#5839 and block/buzz#4942.