mirror of
https://github.com/block/buzz.git
synced 2026-10-28 04:51:40 +00:00
eed74bde2f
## What Two telemetry additions to make a known silent-death failure mode visible in harness logs. **1. `stop` field on `llm: call completed` INFO** (`crates/buzz-agent/src/llm.rs`) The `ProviderStop` value was already parsed and stored on `LlmResponse` but never emitted in the log line. Without it, "model chose `end_turn`" vs "gateway truncated/refused" is indistinguishable from telemetry alone. **2. WARN on silent-turn signature** (`crates/buzz-agent/src/agent.rs`) Emits a `WARN` when a turn produces no publish, no visible assistant text, and either near-zero or absent output tokens. The WARN logic is extracted into `warn_if_silent_turn` (pure synchronous function) so the seam is testable without the async run loop. Three independent gates before the WARN fires: 1. **`!buzz_reply_call_seen`** — no publish attempt in any round, tracked unconditionally via the existing `is_buzz_reply_call` matcher. Read-only tool calls do NOT suppress the WARN; a turn that ran tools but never published and died at 3 tokens is still a silent death. 2. **`text_is_empty`** — no visible assistant text in the final round. A terse reply like "OK" (≤12 tokens, non-empty) is not a silent death. 3. **Token check** — two distinct WARN messages: - `Some(t) where t <= 12`: near-zero token count, the observed failure signature (2–12 tokens) - `None` usage: provider omitted token counts entirely, separately diagnostic Tests use a scoped `tracing_subscriber` layer (same pattern as the existing stall-warn tests in `llm.rs`) to exercise the WARN seam directly: - Canonical signature (no publish, no text, 4 tokens) → 1 WARN - Non-empty assistant text → 0 WARNs - Publish seen → 0 WARNs - `None` usage (no publish, no text) → 1 WARN ## Why Recurring silent-death incident in a specific agent×channel combination: sessions die with 1 LLM call, 2–12 output tokens, no tool calls, no message, no error — recorded as a "successful" turn. The harness log shows the token count but not the `stop_reason`, leaving the root cause undiagnosable without request-level tracing. The observed shape also includes tool-step-then-3-token-death (one tool call, then silence) — the publish-aware gate catches both shapes. Context thread: buzz://message?channel=91fd9ca1-cf04-4ef7-b18f-aa2aee55692b&id=e3f1693f2e29f26a0c840f8054d592270c1504beacdc1d9c2063d8ab82960a06 ## Scope Logging and telemetry only. No behavior change, no retry-logic change, no stop-reason mapping change. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>