jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
buzz/benchmarks/buzz-dataset/ambiguous-user-mention
History
Salman Mohammed 79df468255 Add Buzz benchmark evaluation layers (#6823) 
## Summary

- classify every Buzz-native benchmark task as `regression` or
`workflow`
- add metadata-backed `--layer` selection with k=1 regression and k=3
workflow defaults, while preserving explicit `--attempts`/`-k`
- split an unqualified mixed Buzz dataset run into two Harbor jobs so
each layer gets the correct default
- document the layer contract and cover metadata, selection, defaults,
overrides, invalid input, and command construction

---------

Signed-off-by: Salman Mohammed <smohammed@squareup.com>
2026-08-26 17:41:56 -04:00
..
environment
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
tests
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
instruction.md
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
README.md
test(benchmarks): expand Buzz-native dataset (#6448)
2026-08-21 12:00:04 -04:00
task.toml
Add Buzz benchmark evaluation layers (#6823)
2026-08-26 17:41:56 -04:00

README.md

ambiguous-user-mention

The channel contains two real identities with the exact same three-word display name. Their profile about fields carry different routing codes. The agent must discover the intended pubkey, notify it exactly once, never notify the twin, and separately callback the requester. This guards the silent ambiguity family reported in block/buzz#4303 and block/buzz#6257.