## Why
Terminal task correctness alone does not guard Buzz-native collaboration
behavior such as exact reply routing, non-waking narrative names,
batched agent synthesis, thread isolation, and ambiguous identity
targeting.
## What
- Add five medium/hard Harbor tasks for multiline delivery, narrative
agent names, interleaved agent reports, cross-thread requests, and
ambiguous user mentions
- Add signed scripted-event fixtures, duplicate-display-name profiles,
and public-only evidence export support
- Add positive and adversarial verifier fixtures covering dropped
inputs, extra posts, incorrect routing, and unintended mentions
## Risk Assessment
Low — changes are limited to benchmark tooling and datasets; production
behavior is exercised but not modified.
## References
- Grounded in behavior reported in `buzz-community` and public issues
#5787, #5176, #5839, #4942, #4072, #4303, and #6257
- Live Claude Sonnet 4.6 smoke run: 2/5 passed; the three failures
exposed top-level instead of threaded delivery, empty narrative output,
and silent ambiguous-identity completion
- Live GPT-5.6 Luna run (`buzz-native-solo-luna.yaml`): 3/5 passed.
Passed `ambiguous-user-mention`, `cross-thread-requests`, and
`interleaved-agent-reports`. Failed `multiline-message` and
`narrative-agent-names` because both omitted the event-level user
mention despite otherwise-correct content and threading.
<img width="1176" height="805" alt="Screenshot 2026-08-21 at 11 11
17 AM"
src="https://github.com/user-attachments/assets/fc259ec9-892c-4436-b022-1bf85415617d"
/>
Signed-off-by: Salman Mohammed <smohammed@squareup.com>
## Why
Terminal-Bench can show task correctness while missing Buzz-specific
collaboration failures, such as posting a final answer outside the
user's thread. Add a small Buzz-native suite that scores relay behavior
directly.
## What
- Add Harbor tasks for direct thread replies, event-level callback
mentions to a three-word user identity, targeted reads of explicitly
named paths outside the workspace, and exact temporary-channel
membership against a seeded directory of 50 users and 10 bots
- Export sanitized post-agent relay evidence and extend solo
runtime/provisioning for deterministic verification
- Expose channel TTL in CLI search output and cover positive and
negative transcript/state fixtures
## Risk Assessment
Low to medium — this primarily adds benchmark and testbed machinery. The
CLI change is an additive `ttl_seconds` field in channel-search JSON;
relay behavior is unchanged.
## References
- Motivated by the Buzz-specific behavior observed while benchmarking
https://github.com/block/buzz/pull/6128
- The named-path regression case covers the issue being fixed by
https://github.com/block/buzz/pull/6261
- Local live Sonnet runs: `reply-to-thread`, `user-mention`, and
`create-channel-invite-users` each scored reward 1.0 with zero
exceptions
- The current branch prompt scored 3/5 on a concurrent
`read-named-path-outside-workspace` batch: one explicit
workspace-boundary refusal and one missing final response
Generated with Codex
---------
Signed-off-by: Salman Mohammed <smohammed@squareup.com>
