jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
main
buzz/benchmarks/buzz-dataset/cross-thread-requests/README.md
Salman Mohammed b56a52ca11 test(benchmarks): expand Buzz-native dataset (#6448) 
## Why
Terminal task correctness alone does not guard Buzz-native collaboration
behavior such as exact reply routing, non-waking narrative names,
batched agent synthesis, thread isolation, and ambiguous identity
targeting.

## What
- Add five medium/hard Harbor tasks for multiline delivery, narrative
agent names, interleaved agent reports, cross-thread requests, and
ambiguous user mentions
- Add signed scripted-event fixtures, duplicate-display-name profiles,
and public-only evidence export support
- Add positive and adversarial verifier fixtures covering dropped
inputs, extra posts, incorrect routing, and unintended mentions

## Risk Assessment
Low — changes are limited to benchmark tooling and datasets; production
behavior is exercised but not modified.

## References
- Grounded in behavior reported in `buzz-community` and public issues
#5787, #5176, #5839, #4942, #4072, #4303, and #6257
- Live Claude Sonnet 4.6 smoke run: 2/5 passed; the three failures
exposed top-level instead of threaded delivery, empty narrative output,
and silent ambiguous-identity completion
- Live GPT-5.6 Luna run (`buzz-native-solo-luna.yaml`): 3/5 passed.
Passed `ambiguous-user-mention`, `cross-thread-requests`, and
`interleaved-agent-reports`. Failed `multiline-message` and
`narrative-agent-names` because both omitted the event-level user
mention despite otherwise-correct content and threading.



<img width="1176" height="805" alt="Screenshot 2026-08-21 at 11 11
17 AM"
src="https://github.com/user-attachments/assets/fc259ec9-892c-4436-b022-1bf85415617d"
/>

Signed-off-by: Salman Mohammed <smohammed@squareup.com>
2026-08-21 12:00:04 -04:00

492 B
Raw Permalink Blame History

cross-thread-requests

The harness posts ALPHA and BETA as separate top-level human mentions in the same channel before the queue flushes. Passing requires two different replies, each anchored to its own triggering event with only its own answer. This is a deliberately hard guard for the cross-thread contamination reported in block/buzz#5839 and the exact reply-target contract in block/buzz#4072.