mirror of
https://github.com/block/buzz.git
synced 2026-11-02 07:22:55 +00:00
a0dd2d455463da985ce61ebbbb5d516ef81cdd83
91 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jimmy
|
55edc4aef5 |
fix(agent): preserve migration behavior
Co-authored-by: Jimmy <1fe240cd1a8cf775f6f3060f115e5a303181f3abf28ad4cb0c2515f4a02b36a8@meshllm.communities.buzz.xyz> Signed-off-by: Jimmy <1fe240cd1a8cf775f6f3060f115e5a303181f3abf28ad4cb0c2515f4a02b36a8@meshllm.communities.buzz.xyz> |
||
|
Michael Neale
|
678dc68384 |
chore(buzz-agent): drop the resurrected model_capabilities.rs copy
The merge brought #5597's `buzz-agent/src/model_capabilities.rs` back as an untracked-by-any-`mod` file. The live copy is in `buzz-model-catalog`, where I moved it so the desktop can read the manifest without linking goose; the two were byte-identical. Nothing declared `mod model_capabilities` in buzz-agent, so it was never compiled — which is exactly why it survived a green build, a full test run and clippy. Found by counting files, not by a failure. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
e5f73cd27c |
fix(buzz-agent): carry BUZZ_AGENT_LLM_TIMEOUT_SECS onto goose
PR #6115 seats a 660 s client budget for mesh agents in the desktop (`managed_agents/relay_mesh.rs`) because MeshLLM's own backend budget is 600 s and a cold prefill of a large agent prompt can legitimately take ~500 s. On this branch that seed reached nothing: buzz-agent no longer has a request loop, and `project_goose_env` had no projection for the variable, so goose's own default applied instead. goose's default is 600 s (`DEFAULT_PROVIDER_TIMEOUT_SECS`) — just *under* the mesh server's, which is the same wrong-side-of-the-server ordering #6115 diagnosed. So the bug #6115 fixes would have come back on merge, silently and only on shared compute. goose reads the timeout per provider rather than globally, so this projects onto the variable belonging to the configured provider. `relay-mesh` maps to goose's `openai` provider and so reads `OPENAI_TIMEOUT` (`goose/src/providers/openai_def.rs:118`). Providers goose gives no timeout knob — databricks — return `None` rather than getting a variable goose never reads. Two tests. The mesh one asserts the invariant (`> 600`) rather than the literal, and I verified it fails with the projection removed and the test kept, reconstructing that baseline from the file rather than `git stash`. README corrected: it claimed this knob was "no longer read", and its limits table still listed the pre-goose 240 s default. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
9ca88061fa |
Merge origin/main into micn/buzz-agent-goose-core
82 commits of main, plus the goose pin moved from bf332b9 to 7c4ba22 (60 commits) because main's changes and goose's API changes overlap. Conflict resolutions worth knowing about: * `llm.rs`, `handoff.rs`, `tests/regressions.rs` — main changed files this branch deletes. Reviewed each change rather than dropping it silently: #5475 raised `BUZZ_AGENT_MAX_OUTPUT_TOKENS` and the truncation-recovery allowance in buzz's own request loop, which no longer exists; goose owns retries and output limits now, so there is nothing to port. * `model_capabilities.rs` (#5597) moved to `buzz-model-catalog`, not kept in `buzz-agent`. The manifest is read by the desktop model picker, which cannot link goose (`libsqlite3-sys` collision), so it has to live in the crate the desktop already depends on. `ThinkingEffort` moved with it, reduced to the vocabulary the manifest is typed in — the request-path mapping went with buzz's HTTP transport. The 103-vector corpus guard passes in its new home; `just ci` and `regen-model-corpus` now point at it, and `run-tests.sh` runs the new crate's lib tests. * Curated model labels reach the picker. main added them to the Databricks discovery path; on this branch `session/new` enumerates through goose's provider API instead, so the label lookup had to be applied there or #5597 would have been silently reverted for every provider. goose API changes this bump required: * `StateMachine`/`Step`/`Operation` are generic over session and effect type, and the loop moved into a new `goose-agent` crate. * `StateEffect` split: buzz now names `ConversationEffect`, the narrow set, rather than goose's `GooseEffect`. `ReplaceConversation` lost its `usage` field — buzz always passed `None`, and the driving loop already resets the running total, so this is a rename not a behaviour change. 78 unit + 22 integration tests pass, clippy `-D warnings` and fmt clean on the pinned 1.95.0 toolchain. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Will Pfleger
|
1b7e5ac1be |
feat(model-capabilities): drive model capabilities and labels from one manifest (#5597)
## Summary Centralizes model capability knowledge — thinking mode, supported effort levels, wire routes, and human-readable labels — into a single manifest, `scripts/model-capabilities.json`. Rust and TypeScript each get a small interpreter that reads the same manifest, replacing hand-maintained tables scattered across both languages that had already drifted apart. A capability change is now a data edit, not parallel edits to two code paths. Supersedes the codegen approach explored in #3603. A cross-language contract keeps the two interpreters honest: `scripts/normative-corpus.json` is a golden snapshot generated from the Rust resolver (103 vectors covering all six capability axes) and replayed natively in TS. CI fails if either language disagrees with the corpus or the corpus drifts from the resolver. Regenerate with `just regen-model-corpus`. ## Behavior changes - **Effort dropdown for `openai-compat` providers** no longer offers `max`. The request path always clamped `max` to `xhigh` on the wire, so the UI stops offering a value that was silently rewritten. UI-only, wire-identical. - **Databricks v2 routing (wire-visible):** uncurated endpoint names carrying a bare Claude code-name segment (e.g. `goose-opus-5`) now route to the MLflow chat wire instead of Anthropic Messages — they lose Anthropic prompt caching but still succeed on a valid OpenAI-compatible wire. Curated `databricks-claude-*` records and any name starting with `claude` are unchanged. A handful of other uncurated/adversarial name shapes similarly fall back to MLflow chat instead of pattern-matched routes; every curated model resolves identically to before, all axes. - **Curated model labels on the real discovery path.** The Databricks API returns no display name, so discovery emits the raw endpoint id as the model `name` (`{id, name: id}`) on every path. `ModelEntry.name` is now curated at all four construction seams in `buzz-agent` — v2 discovery, v1 parse, the auth-empty default catalog, and the configured-model fallback — via a read-only `databricks_registry_label` lookup over the manifest's `databricks_v2` exact records; `id` stays the raw wire/config value. A known id renders its curated label (`databricks-gpt-5-5` → `GPT-5.5`), an unknown id passes through unchanged, and the default-catalog row reads `GPT-5.5 (default catalog)`. As a defense against older `buzz-agent` binaries and any harness that echoes ids, `resolveModelLabel` treats a discovered name equal to the trimmed id as absent and falls through to the registry tier; a genuinely distinct name (including the suffixed default-catalog label) still wins. ## Cleanup Deletes the duplicated capability tables and their tests: the `config.rs` gpt5 matchers, effort tables, and clamp logic; the legacy segment-based Databricks v2 route classifier in `llm.rs`; and the TS hand tables plus `effortTable.fixture.json`. All are replaced by manifest lookups through the shared resolver — no line of capability data exists in two places. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Atish Patel
|
72d56e7bd3 |
fix(agent): raise output limit and allow 3 recoveries (#5475)
## Summary Raise the built-in output and recovery defaults so long-running agents have more room to finish useful work instead of terminating after repeated 32,768-token reasoning-only responses. - Raise `BUZZ_AGENT_MAX_OUTPUT_TOKENS` from 32,768 to 65,536 - Raise the finite output-truncation recovery allowance from 2 to 3 via `BUZZ_AGENT_MAX_TOKEN_RECOVERIES`; `0` still disables recovery - Strengthen the recovery prompt so the model stops prolonged reasoning, uses tools immediately, and builds scripts or artifacts in small verifiable steps - Preserve the safety invariant that incomplete truncated tool calls are discarded and never executed - Keep proactive handoff independently at 90% of `BUZZ_AGENT_MAX_CONTEXT_TOKENS` (180,000 tokens with the 200,000 default), regardless of the output allowance - Add request-loop and configuration regressions for exact-N recovery, disabled recovery, successful tool-first recovery, discarded truncated calls, and finite round bounds `BUZZ_AGENT_MAX_OUTPUT_TOKENS` remains an explicit per-agent deployment setting. Operators should configure it at or below the served model's output limit; this PR does not perform live provider capability discovery or automatic clamping. **Risk:** Medium — this increases the default request size and permits one additional recovery attempt by default. Recovery remains finite and bounded by `BUZZ_AGENT_MAX_ROUNDS`. Deployments whose served model rejects 65,536 output tokens must set a lower per-agent value. Current output limits - model - output token max - DeepSeek V4 Flash - 384,000 tokens - Qwen 3.8 (Max) - 131,072 tokens - GLM 5.2 - 131,072 tokens - GPT 5.6 - 128,000 tokens - Claude Opus 5 - 128,000 tokens - Gemini 3.6 Flash - 65,536 tokens - Kimi K3 (Moonshot)- 131,072 tokens ### Related issue None found. Originating benchmark analysis: `buzz://message?channel=c3252dd2-0142-4e01-88c7-a2183c3960a5&id=91e991aab5fd49094583c3937477f6c12db57a41d86edf7fd4745d0d57d10017` ### Testing - `cargo fmt --all -- --check` - `cargo test -p buzz-agent` — 595 passed, 0 failed, 0 ignored at `bd6de557b367850f50325bafdd3c046131942bef` - `cargo clippy -p buzz-agent --all-targets -- -D warnings` - Previously failing `cancelled_turn_with_usage_emits_notification_before_response` passed alone and in the full rerun - Push hooks passed: organization guard, branch skew, Rust tests, and Desktop Tauri checks ### Update — 2026-08-11 Per review feedback, the recovery default is 3. The OpenRouter live `/models` output-cap discovery, cache, request clamp, and related tests/documentation were removed. Per-agent output configuration is now the sole output-cap mechanism. Proactive handoff and its pre-usage byte fallback now depend only on 90% of `BUZZ_AGENT_MAX_CONTEXT_TOKENS`; with the 200,000 default, the handoff threshold is 180,000 regardless of `BUZZ_AGENT_MAX_OUTPUT_TOKENS`. Generated with Brainy Bumble ### Targeted validation — 2026-08-11 Ran the exact PR binary once on each of the 11 benchmark tasks causally affected by the previous 32,768-token ceiling, using OpenRouter with `deepseek/deepseek-v4-flash-0731` pinned to Fireworks and maximum reasoning effort. Relay-429 collection failures were excluded and rerun at concurrency 2. - **6/11 passed:** `circuit-fibsqrt`, `feal-linear-cryptanalysis`, `model-extraction-relu-logits`, `path-tracing`, `schemelike-metacircular-eval`, and `sqlite-db-truncate` - **5/11 reached the benchmark deadline:** `adaptive-rejection-sampler`, `dna-assembly`, `path-tracing-reverse`, `regex-chess`, and `write-compressor` - `regex-chess` reached exactly 65,536 output tokens, triggered one output-limit recovery, and then reached the deadline. This directly confirms that the larger ceiling and recovery path were active, but not that recovery guarantees completion. For context, ten of these tasks were 0/5 in the historical baseline; `sqlite-db-truncate`, the clean control, was 4/5. This is targeted one-attempt-per-task validation rather than a statistically powered comparison. The result should not be attributed solely to the recovery default of 3: this PR also raises the output ceiling and strengthens recovery behavior, and OpenRouter routing conditions may differ from the historical direct-Fireworks runs. Generated with Brainy Bumble --------- Signed-off-by: Atish Patel <atish@squareup.com> Signed-off-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Signed-off-by: Hardworking Honey <c5c455215c2506cb8ba776518cec804af62d3a0526e32d496a22072e395042b9@buzz.block.builderlab.xyz> Co-authored-by: Brainy Bumble <0ed7657b57c0e8a9f5288390dd6c8d5d0a3a06abe9b01b9006814f52077d6cdf@buzz.block.builderlab.xyz> Co-authored-by: Hardworking Honey <c5c455215c2506cb8ba776518cec804af62d3a0526e32d496a22072e395042b9@buzz.block.builderlab.xyz> |
||
|
Michael Neale
|
e0d5f20475 |
test(buzz-agent): cover the steer and compaction operations
The two operations added in
|
||
|
Michael Neale
|
60e63a065d |
docs(buzz-agent): record why [Reflect] stays out of the state machine
Reflect was step 3 of the migration plan. It should not be an Operation, and the reason is worth writing down rather than leaving as an omission. StateEffect can append a message or patch a tool request's metadata, but nothing edits tool result *content*. PatchToolRequestMeta is applied via SessionManager -- the store this branch deliberately does not write to -- and patches metadata, not what the model reads. So as an operation the reflection would arrive as a separate message after the result rather than inside it. That is exactly the arrangement this port moved away from when it stopped delivering reflections as steers. Structure follows behaviour. Also corrected reflect.rs's module comment, which still described the steer-based delivery an intermediate commit used. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
099bacbe6a |
refactor(buzz-agent): express steer and compaction as goose Operations
Steps 4 and 5 of PLANS/BUZZ_OPERATIONS_MIGRATION.md. Behaviour is unchanged: steer.rs, cancel.rs and the real-MCP integration tests all pass without modification. BuzzSteerOperation drains the steer queue; BuzzCompactionOperation wraps goose's check_if_compaction_needed/compact_messages plus the _PostCompact re-injection that buzz's old context handoff existed for. These need a second StateMachine (round_start), not more steps on the existing one: round_gate also runs when a turn wants to END, and draining steers or compacting there would change when they happen. Since step() stops at the first operation that applies, the start machine runs to exhaustion -- a turn that both steers and compacts needs two passes. The gate now carries StateEffect rather than Message, because compaction replaces the conversation instead of appending to it. apply_effects reports whether state actually changed, which is what stops a no-op-applied operation from spinning the loop. Not goose's CompactionOperation: it yields to the client and emits a user-facing notification. In buzz a yield ends the turn, and the notification would post 'compacting' into the channel -- so a long conversation would stop mid-work and narrate its own housekeeping. Nearly dropped the [PostCompact] prefix on the re-injected state while moving it. Caught by re-reading the old function before deleting it; the model uses that prefix to tell re-injected state from a user turn. 74 unit + 21 integration green, handtest --all 18/18. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
ab5b289488 |
fix(buzz-agent): restore the env surface main had
The port dropped BUZZ_AGENT_* knobs whose implementation moved to goose. That conflated two different things: goose owning the mechanism, and the setting silently ceasing to work. Someone who set BUZZ_AGENT_TOOL_TIMEOUT_SECS=1200 got no error and a 300s timeout. Restored as mappings onto goose, defaults matching main: - TOOL_TIMEOUT_SECS -> GOOSE_DEFAULT_EXTENSION_TIMEOUT - MAX_TOOL_RESULT_TEXT_BYTES -> GOOSE_MAX_TOOL_RESPONSE_SIZE - NO_HINTS -> CONTEXT_FILE_NAMES=[] - STOP_MAX_REJECTIONS -> the veto cap, configurable again (0 disables) Both timeout and truncation project buzz's *default* as well as an explicit value, because goose's differ (300s vs 660s, 200KB vs 50KB) -- mapping only explicit values would still have changed behaviour for everyone who never set them. Two silent regressions fixed: - MAX_SESSIONS defaulted to 8; main was unlimited. A busy agent would have started refusing sessions it used to accept. - The reply guard was gone entirely. Desktop still enables it by default for shared-compute agents (relay_mesh.rs), so the flag was set and ignored. Restored as BuzzReplyGuardOperation with main's nag text, two-reminder budget, and publish-shaped-call latch. README: every removed knob is now listed as 'No longer read' with what replaced it, rather than documented as if still live. 74 unit + 21 integration green, handtest --all 18/18. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
15f6fd57f8 |
refactor(buzz-agent): express the _Stop veto as a goose Operation
Step 2 of PLANS/BUZZ_OPERATIONS_MIGRATION.md. Behaviour is unchanged: the existing stop_hook and real_dev_mcp integration tests pass without modification, which is the point of this step. BuzzStopVetoOperation replaces the inline stop_veto() branch. The loop now runs the gate a second time when a round wants to end, which is where the operation applies -- it declines unless the last message ends the turn, so the top-of-round call is a no-op for it. Two things move rather than change: - The block cap was a loop-local u32 in run_turn; objections now carry a metadata note and the cap counts them since kickoff. run_turn is per session/prompt, so both are turn-scoped -- same three-strikes behaviour, but now a pure function of the conversation, which is why goose tags its own denials the same way. - MAX_STOP_BLOCKS was declared in both loop_drive and hooks with the same value. Dropped the loop_drive copy. Deliberately not goose's StopHookOperation: it drives goose's private HookManager plugin system, while buzz's hook is an MCP tool that can see buzz-dev-mcp's in-process todo state. It also emits a user-facing notification per denial, which in buzz would post 'stop hook blocked ending this turn' into a channel every time an agent had an open todo. The objection reaches the model, not the room. 65 unit + 21 integration green, handtest --all 18/18. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
42fb755040 |
fix(buzz-agent): the agent's configured provider must beat an inherited one
An agent configured for OpenAI sent its OpenAI model to Anthropic and 404'd every turn (llm model not found: gpt-5.6-sol at api.anthropic.com). Found in live testing, not in a test. project_goose_env used set_if_absent, so an inherited GOOSE_PROVIDER won over the agent's own BUZZ_AGENT_PROVIDER. That looks defensive but inverts the intended precedence: BUZZ_AGENT_PROVIDER/MODEL are not ambient config, they are derived by the desktop from the agent record's structured provider/model fields at spawn time, and the desktop deliberately refuses to persist GOOSE_PROVIDER/GOOSE_MODEL in an agent's env so they cannot shadow those fields (env_vars.rs:DERIVED_PROVIDER_MODEL_ENV_KEYS). The subprocess still inherits the desktop's environment, though, and anyone with goose installed exports GOOSE_PROVIDER from their login shell. The failure is silent in the worst way: the agent's settings read correctly in the UI while its traffic goes to another provider. Set both unconditionally when the agent has them, and likewise map OPENAI_COMPAT_API_KEY over OPENAI_API_KEY: an inherited OPENAI_API_KEY would otherwise authenticate and bill the agent as whoever that key belongs to. With no BUZZ_AGENT_* value there is nothing to override with, so an ambient GOOSE_* is still honoured. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
69c6091216 |
feat(buzz-agent): back goose's Databricks provider with buzz's PKCE engine
Both sides ran OAuth PKCE against the same Databricks host with separate token caches: goose's providers::oauth for inference, buzz's PkceOAuthTokenSource for the desktop model picker. A user could be authenticated for one and not the other and face two browser flows for one host. Buzz's engine is the hardened one (#5534: O_NOFOLLOW, 0600 from creation, unique temp names) and the only one the desktop can link, so it wins. goose supports this without a fork: DatabricksV2Provider::new takes an oauth token provider closure, and databricks_v2_def::from_env is just the variant passing goose's own. Retries, the agent-session-id header, endpoint resolution and the wire format all stay goose's. Uses bearer_no_browser: inference runs in a headless subprocess where a browser step would hang unseen. Interactive login stays in buzz-agent auth databricks and the desktop picker. A static DATABRICKS_TOKEN still takes precedence, as in goose's own from_env, and no DATABRICKS_HOST falls through to goose's registry unchanged. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
8e1ba34207 |
refactor(buzz-agent): use goose's skills instead of buzz's duplicate
buzz-agent grew its own skill discovery, prompt index and load_skill tool before goose had one. goose now ships goose::skills plus a SkillsClient registered in PLATFORM_EXTENSIONS, so buzz's version was a strictly smaller duplicate: the same three project directories, but goose also reads the global directories, plugin-installed skills and its own builtins, and supports args templating buzz never had. Delete builtin.rs, builtin_client.rs and hints.rs (1,204 lines) and register goose's skills extension by name so its factory runs. goose declares it unprefixed_tools, so the model now sees load_skill rather than buzz__load_skill. skills.rs keeps only the prompt index, because the goose code that renders it lives on SkillOperation, which is pub(super). The hint-file guard from hints.rs moves there rather than being dropped, and now matches filename literals instead of the bare words so it can document itself. Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
3190102efb |
merge origin/main into micn/buzz-agent-goose-core-v2
Conflicts, and how each was resolved: * crates/buzz-agent/src/mesh.rs + tests/mesh_auto.rs — DELETED. main's #5289 removed buzz-agent's client-side mesh catalog probe on purpose: MeshLLM v0.75 decides per request, the desktop resolves the wire model before spawn (relay_mesh_wire_model), and BUZZ_AGENT_PREFER_MESH_FOR_AUTO is no longer set by anything. Keeping my MeshAutoProvider would have resurrected policy main deliberately retired. * crates/buzz-agent/src/llm.rs — stays deleted; goose owns the transport. main's changes to it were the Databricks OAuth hardening (#5534), which applies to auth.rs, now living in buzz-model-catalog. * crates/buzz-agent/src/config.rs — ours; main's only change was dropping prefer_mesh_for_auto, which this branch had already dropped. * buzz-model-catalog: picked up #5534's O_NOFOLLOW/0600 token-cache hardening, which needs the nix dep the extraction had left behind. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Mic Neale
|
240cdd3ea1 |
chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh (#5289)
Shared compute now has exactly two model choices: MeshLLM's virtual `mesh` model, or a model you name. Buzz picks between them in one place, and buzz-agent no longer knows meshes exist. ## What changed - **MeshLLM v0.74.0 → v0.75.1.** v0.75.0 added `degrade_to_single_model`, so a `model=mesh` request is answered by one served model when there is no committee to form, instead of failing. v0.75.1 adds Mesh-LLM#1196, which skips stale pre-0.75 runtime cache entries rather than aborting startup on them — without it, anyone who had run mesh on 0.73/0.74 could not start. - **Deleted the client-side mesh catalog probe.** buzz-agent used to poll `/v1/models` (5s TTL, 30s cooldown, two-observation debounce) to decide whether `mesh` was safe to send. MeshLLM now decides per request, so the polling, its hysteresis, and its 503 fallback are gone. - **One mapping point.** `relay_mesh_wire_model()` turns the stored value into a wire name: `auto` becomes `mesh`, a named model passes through. The spawn env, the ACP harness, and the readiness probe all use it, so they cannot disagree — previously `BUZZ_ACP_MODEL` and the probe both said `auto`, a name the mesh does not advertise. - **Removed the `nostr-relay-pool` advisory exception.** #5404 allowed RUSTSEC-2026-0243 "after mesh-llm migrates to nostr-sdk >= 0.45". v0.75.1 does, so the retired crate is gone from both lockfiles and the exception would only mask a future advisory for it. - **Deleted `scripts/ensure-mesh-native-runtime.sh`** and its six justfile call sites. It built llama.cpp from source into the runtime cache; the app already downloads the signed release runtime itself, and CI never called it. ## Why it is better **−639 lines of Rust.** Availability is decided by the node that knows the answer, per request, instead of by a client cache that could be stale for up to 30 seconds. A second worker joining now takes effect on the next request rather than after two confirming probes. ## Behaviour change A 503 on an explicit `mesh` request takes the ordinary transport retry under the same model instead of failing over to a second one — there is no second model to fail over to now. MoA repairs partial committee results internally before it reaches that point. ## Validation `crates/buzz-relay/examples/mesh_agent_e2e.rs` now sends `mesh` where it previously sent `auto` or the physical model id, so no leg was covering what Buzz actually puts on the wire. 4/4 on gemma-4-E4B, gemma-4-26B-A4B, and Qwen3-8B — including a real ACP tool call through `mesh` into buzz-dev-mcp, asserted by reading the written file back off disk. Hand-tested in the desktop app on both gemma-4 sizes: picked Auto, agent logged `model_id=mesh`, replied in channel. ## Not covered A committee that forms and then loses a worker returns 502, and that needs two workers to reproduce — not testable on one machine. --------- Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com> |
||
|
Will Pfleger
|
5e4d0fe925 |
fix(buzz-agent): harden Databricks OAuth token cache and callback (#5534)
Hardens the Databricks PKCE OAuth code in `crates/buzz-agent/src/auth.rs`. Two fixes. ## Token cache is owner-only across its whole lifecycle, and race-safe The PKCE cache holds both the access and refresh tokens, but `save()` wrote it with a bare `fs::write` + `fs::rename`. Under a `022` umask the file landed world-readable, and the fixed `*.json.tmp` temp name races across concurrent savers sharing `$HOME` — one writer's `rename` can fail on another's half-written temp. **On write**, `write_private_cache()` creates a temp file with owner-only permissions from the moment it exists — mode `0o600` on Unix via `OpenOptions::mode` — writes and fsyncs it, then renames over the destination. The rename swaps the inode wholesale, so a pre-existing cache file with loose permissions is *replaced* by the new private inode rather than inheriting its mode. `unique_suffix()` (getrandom, timestamp fallback) gives each write a distinct temp name, and a drop guard removes the temp on any failure path. **On load**, owner-only is enforced as a cache lifecycle invariant, not just a write-path property. A world-readable cache left by an older buzz-agent was previously read straight into memory and returned on the fresh cache-hit path without ever invoking `save()`, so a token file with no advertised expiry could stay exposed indefinitely. `read_cache()` now funnels every load — initial and cross-process re-reads — through `read_private_cache()`, which on Unix opens with `O_NOFOLLOW` (kernel-level symlink refusal, no stat/open TOCTOU), requires a regular file, and `fchmod`s the pinned handle to `0o600` when any group/other bit is set. A cache that cannot be secured is treated as absent, so callers fail closed to a fresh flow rather than trusting an exposed file. ## OAuth callback no longer reflects untrusted input The localhost callback embedded the untrusted `error` query param straight into the HTML response — an XSS sink on the redirect page — and routed that same raw value into the error string that reaches the logs. `callback_outcome()` is now a pure function returning `(result, static_page)`: the browser always sees a fixed literal page that embeds no request parameter, and failure detail travels only through the result channel. `sanitize_callback_detail()` strips control characters (CR/LF log-line injection) and caps length before that detail enters the error string bound for the logs. ## Deferred: Windows owner-only ACLs Windows owner-only protection is out of scope for this change. The goose-parity route (`CreateFileW` with an owner-only SDDL `D:P(A;;FA;;;OW)`) requires `unsafe` FFI, which this crate's `#![forbid(unsafe_code)]` prohibits; reconciling that conflict is a separate decision. Both platform seams — `create_private_temp_file` (write) and `read_private_cache` (load) — have a `#[cfg(not(unix))]` branch that relies on the default per-user ACLs and is the drop-in point if Windows protection is added later. No new dependency and no `unsafe` are introduced here. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: Hayt <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz> |
||
|
Michael Neale
|
45584a9c7a |
refactor(buzz-agent): hold the provider on buzz's side, not in goose's session row
I claimed the remaining session row was needed because update_provider persists the model config there and goose reads it back for the context limit. That was wrong, and the fix is straightforward: buzz builds the provider and model config itself, so it can hold them. New crate::model::SessionModel owns the pair. The turn reads the provider and config from it instead of Agent::provider and model_config_for_session, and session/set_model swaps it there rather than calling update_provider. That removes the last write buzz makes to goose's store. The row itself still has to exist, but for a different reason than I gave. Agent::add_extension_inner (goose agent.rs:1465) resolves each MCP extension's working directory by loading the session, and fails setup if the row is missing -- 'Failed to get session: Session not found'. I found that by removing create_session and watching every tool-using test fail. So: one row per session, created for extension setup, and zero writes after it. Removing it needs goose to take the working directory as an argument instead of reading it back. Verified: a session with an MCP server, a tool call and two turns leaves 1 sessions row and 0 messages rows, still recalls turn 1 in turn 2, and still advertises 9 catalog models. handtest.py --all 18/18; 87 unit + 21 integration green. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
ce844719f8 |
refactor(buzz-agent): keep the turn conversation in memory, not sqlite
The turn loop round-tripped every message through goose's SessionManager
-- a sqlite database -- once per round. That was never required. None of
the goose APIs buzz calls reads the store:
- Provider::stream takes &[Message].
- Agent::dispatch_tool_call takes &Session but reads only working_dir
and id.
- context_mgmt::{check_if_compaction_needed, compact_messages} read
usage.total_tokens and model_config, and carry session_id only into a
task-local that becomes an agent-session-id HTTP header for provider
request attribution (goose/src/session_context.rs).
So the turn now owns a Session value directly (crate::turn_state) and
buzz-agent keeps the conversation across turns itself. goose still gets
the &Session its signatures ask for; nothing backs it with a database.
This is the right shape for buzz regardless of dependencies: a Buzz
agent's durable record is the relay, not a local sqlite file. Agents are
chatty and long-lived, and every one of them was growing a database
nothing ever read back.
One row remains, created at session/new: update_provider persists the
provider/model config against the session id, and that config is what
goose reads back to resolve the context limit for compaction. Declining
it would make auto-compaction fire at the wrong point. Zero message rows
are written for a conversation of any length.
Verified: a two-turn conversation recalls the first turn's content with
0 rows in the messages table; handtest.py --all 18/18; 84 unit + 21
integration green.
Also fixes the Desktop Core CI failure this branch introduced: the log
filter added earlier pushed runtime.rs past the 1000-line ratchet, so it
moves to its own module with its tests.
Co-authored-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
|
||
|
Michael Neale
|
979eff99fd |
refactor(buzz-agent): express the round budget as a goose Operation
First step of moving buzz's loop decisions into goose's StateMachine, one
at a time (PLANS/BUZZ_OPERATIONS_MIGRATION.md). The remaining decisions
still live in loop_drive.rs; this adds the gate they will move into.
Two things the state-machine shape buys immediately:
- The budget now counts assistant turns since the kickoff message rather
than loop iterations, so it survives a process restart. The loop-local
counter did not.
- Precedence between the guards becomes list order rather than control
flow, which is what makes the remaining conversions safe to do
incrementally.
Deliberately not goose's MaxTurnsOperation: that appends an assistant
message ('Would you like me to continue?') before yielding. buzz-acp
publishes assistant messages into a channel, so a synthesised one would
be indistinguishable from the agent saying it. buzz signals the exhausted
budget through the max_turn_requests stop reason instead.
Two things found while building it, both now pinned by tests:
- goose's operation helpers (messages_since_kickoff, assistant_turn_count)
are pub in a *private* module, so they are not reachable from outside
the crate -- only the names re-exported from state_machine are.
Reimplemented here against goose's semantics.
- A mid-turn steer is user-visible, so it becomes the new kickoff and
restarts the budget. That is the behaviour we want, but it is emergent
rather than chosen, so a test pins it.
StepResult cannot carry a stop reason, so operations record one in a
shared Outcome cell; first writer wins, since the machine stops at the
first operation that applies.
Verified: max_rounds=2 against a real provider returns max_turn_requests
with the gate firing from buzz_agent::ops; handtest.py --all 18/18;
79 unit + 21 integration green.
Co-authored-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
|
||
|
Michael Neale
|
c312a70e53 |
fix(buzz-agent): mark steered messages with with_steer()
goose's own steer path sets metadata.steer (agent.rs:540) and its ACP surface republishes it as _meta.goose.steer. buzz-agent queued steers as plain user messages, so nothing downstream could tell a mid-turn steer apart from the user simply sending another message -- which is the one distinction the flag exists to preserve. buzz-acp does not read it today; this keeps the conversation record honest for anything that later does. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
c918877d60 |
fix(buzz-agent): stop rendering every AGENTS.md into the prompt twice
buzz-agent scanned for hint files itself and added the result as a system-prompt extra. But goose's PromptManager::build_system_prompt already calls .with_hints() (prompt_manager.rs:255), which runs the same walk. Every AGENTS.md therefore reached the model twice -- once under our '## Project Hints', again under goose's '### Project Hints'. Verified against a scratch repo with a canary string: 2 occurrences before, 1 after. The walk really is the same algorithm, which is why this went unnoticed: goose's find_git_root + get_local_directories collect cwd->root and reverse for root->cwd ordering, exactly as buzz's did. goose's loader is a superset -- .goosehints as a second filename, CONTEXT_FILE_NAMES to configure the list, gitignore-aware filtering, and @file expansion bounded at the git root. So hint-file loading is deleted here and left to goose. What stays is skill discovery, because load_skill is served by our builtin_client and needs the same SkillEntry list the tool dispatches against. hints.rs 732 -> 479 lines. A guard test fails if hint-file loading is reintroduced, since the duplicate rendered silently -- the prompt was merely longer, never wrong-looking. Verified end-to-end against Anthropic: handtest.py --all, 18/18, persona and hints still reach the model and load_skill still resolves. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
626b079fa3 |
fix(buzz-agent): keep agent conversations out of the user's goose session DB
goose's SessionManager::instance() resolves to Paths::data_dir(), so buzz-agent was persisting every turn into the same sessions.db as the user's own goose CLI history. On this machine that file is already 132 MB and 892 sessions, with desktop agent turns from minutes ago interleaved into it. Not a correctness bug -- SQLite handles concurrent writers -- but agent conversations do not belong in a human's personal session history, and buzz-agent should not grow a shared file it does not own and cannot prune. Resolves its own directory instead: BUZZ_AGENT_SESSION_DIR when absolute, else <data-dir>/buzz-agent, else goose's singleton. A relative env value is ignored rather than resolved against the agent's CWD, which is the user's workspace. Deliberately not GOOSE_PATH_ROOT: that would move goose's config and keyring lookups too, and provider configuration should keep resolving where goose puts it. Resolved once behind a OnceLock -- SessionManager::new builds its own SQLite pool, so a per-session call would open a pool per session against the same file. Verified: with BUZZ_AGENT_SESSION_DIR set, the turn's session lands in that directory and the shared DB is untouched. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
20a731b6e8 |
fix(desktop): admit buzz_agent logs to per-agent log files
buzz-acp inherits its stderr to the agent subprocess (acp.rs:463), so the desktop's RUST_LOG filter for the harness also decides what the agent can write to disk. It named only buzz_acp, so every buzz-agent diagnostic was filtered out before reaching the per-agent log. That is invisible until something goes wrong: an agent misbehaving in the desktop left no trace of compaction, the _Stop veto, max-rounds, or provider errors -- exactly the lines you need to tell whether a turn did what it should. Found while trying to confirm from the logs alone that a desktop agent was running the new loop; the logs could not answer it. Adds buzz_agent=info to the default, appends only the missing directive when an operator filter is present (so RUST_LOG=buzz_agent=debug is not downgraded to info), and takes a fully-specified filter verbatim. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
9869f24319 |
refactor(buzz-agent): buzz owns the agent loop; goose supplies the parts
Inverts the previous design. buzz-agent no longer calls
goose::agents::Agent::reply -- it drives its own round loop and calls
goose for the four heavy components:
model call Provider::stream (via Agent::provider)
tool surface Agent::list_tools
tool execution Agent::dispatch_tool_call
system prompt PromptManager (our own instance)
compaction context_mgmt::{check_if_compaction_needed, compact_messages}
Under Agent::reply every buzz-specific behaviour had to be smuggled in
around goose's turn policy: the _Stop veto needed an outer loop purely to
re-enter reply(), and [Reflect] had to be delivered as a *steer* because
the tool result itself was out of reach. Owning the loop removes the
smuggling -- the veto is a branch, and [Reflect] goes back on the tool
result where buzz-agent originally put it and where the model reads it in
context.
New modules:
loop_drive.rs the round loop: inference, tools, compaction, _Stop veto,
steer drain, cancellation, max-rounds bound
tools.rs parallel dispatch, announce->terminal wire invariant,
[Reflect] on failure
prompt.rs our PromptManager (goose's is pub(super) to Agent::reply)
steer.rs our steer queue (goose's drain is pub(crate) to reply)
agent.rs shrinks to what goose knows nothing about: ACP session/update
emission and the keepalive ticker. Tool lifecycle now emits from tools.rs,
which is the only place that knows when a call starts and ends -- emitting
from streamed content as well would double-announce every tool.
Behaviour preserved and covered by the existing suite: Fizz persona,
_Stop veto (cap 3), [Reflect] (cap 8), _PostCompact re-injection,
steering without cancellation, cancel leaving no unresolved tool call,
usage accounting, and the model catalog.
81 unit + 21 integration tests pass; clippy clean (the one hints.rs
warning predates this branch).
Co-authored-by: Michael Neale <michael.neale@gmail.com>
Signed-off-by: Michael Neale <michael.neale@gmail.com>
|
||
|
Michael Neale
|
80a9f9a360 |
chore(buzz-agent): bump goose pin to bf332b9 (unrolled loop) and fix rmcp 3.x breaks
Moves the goose git pin from 305849b to bf332b9, which is past ca52cce (#9574, the unrolled agent loop). Consequences of the bump: - goose now uses rmcp 3.x; buzz-agent's own rmcp dep moves 1 -> 3 so the two do not resolve to different crate versions of the same types. - rmcp 3 renames Content -> ContentBlock and adds fields to ListToolsResult; builtin_client.rs updated to match. - buzz-model-catalog needs an explicit dirs dep after the merge. - TurnTotalState / PricingIdentity, added on main while this branch was away, ported back into types.rs for wire.rs's usage_update_payload. cargo check -p buzz-agent passes. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
d35f7042a5 | Merge origin/main into buzz-agent-goose-core | ||
|
Will Pfleger
|
5e4c05f90b |
feat(desktop): NIP-AM agent-usage backend — P2 emission/transport/archive + P4a aggregation/D6 (#4000)
## What Implements Phases 2 and 4a of the Usage v2 plan (plan events `d0268cd0`/`0e95b035`), extending the archive backend to emit, transport, archive, and aggregate both cache categories and billing identity fail-closed. ### P2 — emission, transport, archive **Tri-state accumulators** (`Unseen`/`Exact`/`Unknown`) for cache-read and cache-write in `buzz-agent` turn and session state. Absent field = Unknown (never zero) through the full pipeline. No `unwrap_or(0)` on the cache path. Both cache folds are gated on usage-bearing responses (same gate as the total-state and identity folds) — a response with no usage at all must not poison either accumulator. **Overflow-aware input token parsing and accumulation** — closed end-to-end from parse through wire to ACP: - `sum_usage()` returns `SumUsageResult` (`Exact(u64)` | `Overflow`) — checked arithmetic, never clamps. `anthropic_input_tokens()` returns `Option<SumUsageResult>` since it sums three fields (`input_tokens + cache_read_input_tokens + cache_creation_input_tokens`) that can collectively overflow. Single-field callers (`prompt_tokens`, `completion_tokens`, etc.) convert via `.into_exact()` — their single-field sums cannot overflow. - `LlmResponse.input_tokens_overflowed: bool` propagates the parse-layer signal into the run loop. When set, `input_tokens` is `None` (clamped value discarded), the context-gate baseline (`last_request_input_tokens`) is frozen at its prior reading, and `turn_input_tokens` is poisoned to `TurnIOState::Poisoned` before any emission — including mid-turn `emit_usage_update` calls. A dedicated enum on `LlmResponse.input_tokens` would ripple into ~20 existing test assertions on `r.input_tokens == Some(...)`; the bool flag confines the change to the two call sites that check it. - `TurnIOState` (`Unseen`/`Exact`/`Poisoned`) for input and output: per-round fold uses `checked_add`; overflow poisons permanently at turn and session level, no healing. Absence does not poison (pass-2-cleared contract unchanged). Wire emission omits `accumulatedInputTokens`/`accumulatedOutputTokens` when poisoned — never null, never `u64::MAX`. ACP treats absent = publisher-poisoned: `delta_reliable: false`, null turn fields, null cumulative for that category; session cumulative stays unknown for all subsequent turns once poisoned. **Conditional wire emission** for `accumulatedCachedInputTokens` and new `accumulatedCacheWriteTokens`: fields are omitted when the cumulative is Unseen or Unknown. ACP `_goose/unstable/session/update` contract documented next to the payload with tests for all absence/zero variants. **`PricingIdentity` stamping (publisher-side)**: - `pricing_authority()`: canonical parsed-URL endpoint comparison against the official allowlist — HTTPS only, exact allowlisted host (lookalike-safe), default port (omitted or explicit :443), required API base path, rejects userinfo/query/fragment/path-prefix lookalikes. - Model: the actually-requested `request_model` after mesh/auto resolution (not `effective_model_str`). - Turn discipline: identity retained only while ALL usage in the current turn carries one identical proven identity; any mismatch, unproven-usage-bearing response, or unpaired cumulative snapshot poisons to absent; a later matching notification does not heal a mixed turn. **ACP `UsageTracker` identity fold**: per-in-flight-turn tri-state identity accumulator replacing last-update-wins. Any absent identity on a token-advancing notification or exact mismatch poisons to absent; poison survives later updates; reset in `begin_turn()`/`take()`; reset also when a request fails (baseline cleared so preflight gate cannot stay frozen sub-threshold on retries). **M3 migration**: adds `turn_cache_write_tokens`, `cumulative_cache_write_tokens`, `pricing_authority`, `pricing_model`, `pricing_cache_class` to `agent_metric_index`. Additive, idempotent, guarded per-column by marker. M2 migration also guarded per-column (turn and cumulative cache-read columns checked and added independently; marker commits only after both are present). Fresh-DB schema includes all columns. **First-turn baselines**: `seed_zero_baseline` seeds `last_input: Some(0)`, `last_output: Some(0)`, `last_cached_input: Some(0)`, `last_cache_write: Some(0)`, and `last_total: Some(0)` — all have the known-zero-at-spawn argument. Absent fields from incoming snapshots still produce unknown (tri-state unchanged). Sessions buzz-acp did not spawn (no seed) remain fail-closed on turn one. **`ReportedUsage` TS mirror**: `cacheReadTokens`, `cacheWriteTokens`, `freshInputTokens` added to `tauriArchive.ts` as `UsageField` members, field-for-field with the Rust struct. ### P4a — aggregation layer **Extended S-1 ladder** to cache-read and cache-write via the same `ladder_token` path as the existing token fields. **`freshInputTokens` derivation**: checked arithmetic, fail-closed — absent cache fields produce Unknown (not zero), overflow and `cacheRead+cacheWrite > input` both produce `incomplete: true`. Aggregated as a `UsageField`. **D6 comparator**: `sort_value()` = provider total when known, else `input+output` when both known, else `None` (unknown-last). Replaces the prior total-only comparator for both agent-level and model-level sort. Ships a pinned test vector that the TS render layer (P5) must match. ## Test coverage - `buzz-agent`: 440 lib + 15 integration (golden_transcripts) — includes 13 new `cache_total_state_tests`; 14 new `turn_io_state_tests`; 3 new `sum_usage_*` tests (exact single-field, exact two-field, overflow signals correctly); 3 new `parse_anthropic_*` tests (overflow flag set + value cleared, normal sum no flag, absent usage no flag); end-to-end golden transcript drives real subprocess with Anthropic-shaped `input_tokens: u64::MAX, cache_read: 1` response and asserts `accumulatedInputTokens` absent from the emitted `usage_update` — no logic duplication; 3 wire pin tests; 4 `fold_pricing_identity_*` tests; `pricing_authority()` explicit-:443 acceptance - `buzz-acp`: 700 tests (691 lib + 9 integration) — 4 new usage tests (absent input → unreliable+null; absent output → unreliable+null; goose-shaped both present unchanged; poison mid-session); 3 ACP behavior tests; 7 pool lifecycle tests - Desktop (Rust): 2259+ tests — 14 new P4a pinned tests; 2 M3 round-trip tests; 1 serde key-shape test; 2 M2 partial-schema migration tests; first-turn cache round-trip test ## Related PRs - P1 NIP-AM spec: [#4632](https://github.com/block/buzz/pull/4632) - P3 pricing table: [#4629](https://github.com/block/buzz/pull/4629) - UI (P5): [#4001](https://github.com/block/buzz/pull/4001) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Tyler
|
5bf78671f4 |
fix(agent): retry LLM completion on malformed 2xx JSON body (#5351)
## Problem
A provider can return HTTP 200 with a **truncated JSON body** — cleanly
closed connection, correct framing, content cut off mid-value. Both LLM
HTTP loops treated this as a terminal error on the first attempt:
`AgentError::Llm("json: EOF while parsing a value")`, surfaced as code
-32000 at the ACP boundary, killing the agent turn before it produced
anything.
Observed live in a tb2.1 bench trial (write-compressor, tb21-twins-1):
deepseek via OpenRouter returned a truncated body, the agent died
mid-prompt with 0 turns completed, and the trial scored 0 on a provider
hiccup.
Meanwhile the same loops already retry timeouts, 429s, 5xxs, 499s, and
mid-body stream stalls — a truncated-but-complete body was the one
transient upstream fault that fell through to terminal.
## Fix
In both `post()` and `openrouter_post()`
(`crates/buzz-agent/src/llm.rs`): when the fully-received success body
fails `serde_json::from_slice`, `continue` the **existing** retry loop
instead of returning terminal — same `MAX_RETRIES` (3) bound, same
`backoff_with_jitter`. On exhaustion, the error goes through
`terminal_llm_error` so it carries cumulative duration + attempt count
like every other retried failure (previously the `json:` error carried
neither).
`post_anthropic` routes through `post()`, so
Anthropic/OpenAI/Databricks/mesh and OpenRouter are all covered.
## Why this cannot re-run a tool call
Hard requirement: tool calls are not idempotent, and this change must
not introduce any possibility of replaying one.
1. **The retry lives inside the HTTP POST helper, below the parse
boundary.** Tool calls are only ever extracted from a *successfully
parsed* response value
(`parse_openai`/`parse_anthropic`/`parse_responses`, all downstream of
these helpers' `Ok` return). A malformed body never parses, therefore no
tool call was ever extracted from it, therefore nothing downstream of it
ever dispatched.
2. **What is re-sent is the completion request itself** — the identical
`body_bytes` captured once at function entry. Sending a completion
request executes no tools; it asks the model for the next message.
3. **Same safety class as existing behavior.** The loop already re-sends
this identical request on 429/5xx/timeout/stream-stall; this adds one
more transient-fault arm to the same loop with the same bytes.
## Tests
Three new tests mirroring the existing 499/dropped-connection fixtures
(raw `TcpListener` stubs):
- `post_retries_malformed_json_body_and_succeeds` — truncated 200 body
on attempt 1, valid JSON on attempt 2; asserts success and **exactly 2**
server-side requests
- `post_exhausts_retries_on_persistent_malformed_json` —
always-truncated body; asserts exactly `MAX_RETRIES` attempts and a
terminal error carrying `json:` + cumulative/attempt context
- `openrouter_post_retries_malformed_json_body_and_succeeds` — same
recovery through OpenRouter's separate loop
Full `cargo test -p buzz-agent` green at
|
||
|
Tyler
|
261c460761 |
fix(buzz-agent): recover from 400-shaped image rejections; unbound benchmark agent rounds (#5318)
## Problem
Two failure modes from the `tb21-glm52-crusoe-1` benchmark run (GLM-5.2
solo, TB2.1) wedged or killed 13 of 89 trials without the model being at
fault:
1. **Conversation poisoning on text-only endpoints.** Crusoe's
serverless `crusoeai/GLM-5.2-NVFP4` rejects any request whose history
contains an image with `400: ... is not a multimodal model`. The
recovery machinery for exactly this case already exists —
`AgentError::UnsupportedImageInput` → `replace_unsupported_images()`
strips the image blocks, marks the tool result as an error, substitutes
a text placeholder, and continues the turn. But classification only
matched OpenRouter's 404 body (`no endpoints found that support image
input`) and was only consulted on the 404 arms. The Crusoe 400 fell
through to terminal `AgentError::Llm`: the image stayed in history,
every subsequent call failed identically, buzz-acp rode its 10-retry
ladder (~40 min), and the trial idled to budget death. Measured blast
radius: **8 trials wedged, 12.7h aggregate idle-after-poison.**
2. **Bounded agent rounds in benchmark trials.** The harness default
`DEFAULT_MAX_AGENT_ROUNDS = 32` ended solo trials mid-work when turns
rotated (thinking-heavy models hit max_tokens rotation fast; 4 trials
died this way). Benchmark trials already have a wall-clock budget as the
real limit — the round cap only converts recoverable rotation into trial
death.
## Fix
- `is_unsupported_image_input_error()` also matches the verbatim `is not
a multimodal model` body. Matcher stays deliberately tight (same
doctrine as `is_context_length_error`): misclassifying a generic 400 as
recoverable would mutate history for an error that removing images
cannot fix.
- Both status ladders — shared `post()` and `openrouter_post()` —
consult it on their 400 arms and return the typed
`UnsupportedImageInput` (OpenAI-compatible providers report this as 400;
a BYOK/passthrough upstream can surface the provider's own 400 through
OpenRouter).
- Harness `DEFAULT_MAX_AGENT_ROUNDS` → `0` (unbounded —
`BUZZ_AGENT_MAX_ROUNDS=0` is the agent config's documented unbounded
value). Per-agent `budget.max_calls` in manifests still overrides.
## Acceptance
- A 400 with the image-rejection body reaches the existing image-strip
recovery path instead of wedging the session — asserted through
`complete()` (covers the return path into the convergence mapper) and at
the `openrouter_post` terminal, both proving single-attempt (a
deterministic capability rejection must never be retried).
- Ordinary 400s stay terminal `AgentError::Llm` (existing negative tests
unchanged).
- Benchmark trials run unbounded rounds by default; python tests updated
for 0-is-legal with a negative arm at -1.
## Verification
- `cargo test -p buzz-agent`: 427 + 18 + 20 + 15 + 8 + 1 + 48 passed, 0
failed (full package, 3 consecutive clean runs)
- `cargo clippy -p buzz-agent --all-targets`, `cargo fmt --check`: clean
- `uv run --extra dev pytest tests/` in harbor-buzz-orchestra: 35 passed
- Pre-push hooks (full workspace rust-tests + desktop-tauri-checks)
green on rustc 1.95.0 at head
|
||
|
Tyler
|
c7b663680a |
fix(buzz-agent): budget summarizer reasoning separately so it cannot starve the handoff summary (#5248)
## Problem
The handoff summarizer sends `max_tokens: 8192`
(`HANDOFF_MAX_OUTPUT_TOKENS`) with no reasoning budget separation. On
reasoning models, thinking tokens count against that cap: the model can
spend the entire budget reasoning, length-stop with empty `content`, and
`summarize()` — which only reads `content` — reports an empty summary.
The handoff then degrades to lossy history truncation.
Observed on deepseek-v4-flash during a terminal-bench 2.1 run
(tb21-solo-3, 89 tasks): **13 consecutive handoff attempts across 5
trials failed exactly this way** (`handoff returned empty summary;
truncating`), each burning ~3 minutes of full-cap reasoning, before a
stochastically-short reasoning run finally fit. circuit-fibsqrt alone: 5
failures, 5 truncations, then success on attempt 6. video-processing
failed its task by one frame after 3 context truncations.
## Fix
`openrouter_summary_body` now grants reasoning its own equal-sized
budget and excludes it from the response:
- `reasoning.max_tokens = max_output_tokens` — thinking gets a dedicated
budget instead of competing with the summary text
- `reasoning.exclude = true` — reasoning is never in the response body;
`summarize()` only reads `content`
- `max_tokens = max_output_tokens * 2` — the total cap covers both
budgets, so the text budget the caller asked for is actually available
for text
Non-reasoning endpoints ignore the `reasoning` object. Deliberately not
paired with `provider.require_parameters`, for the reasons documented at
`apply_openrouter_mutations` (it hard-404s valid model ids).
The prior test
`openrouter_summary_carries_neither_reasoning_nor_provider` asserted
`reasoning` absent from the summary body — that assertion guarded
against *effort-based* reasoning leaking in from config (the body is
built independently of `cfg`, which is still true and still tested:
`reasoning.effort` stays unset). Replaced with
`openrouter_summary_budgets_reasoning_separately_and_carries_no_provider`.
## Verification
- `cargo test -p buzz-agent`: 422 unit + 110 integration tests pass at
|
||
|
Tyler
|
2b873cf208 |
Recover from max-token response truncation (#5223)
## Summary - treat provider `max_tokens` as an interrupted assistant response and continue the same turn with actionable feedback - discard tool calls from truncated responses, including malformed partial arguments, so they are neither executed nor replayed with invalid tool-result pairing - bound recovery to two retries while preserving normal finite `max_rounds` accounting ## Verification - `cargo fmt --all -- --check` - `cargo test -p buzz-agent` (422 unit tests plus all package integration/doc suites passed) - `cargo clippy -p buzz-agent --all-targets -- -D warnings` ## Notes The pre-push repository-wide hook also ran. Its Rust tests passed (2,270 passed, 14 ignored), but its `buzz-db` unit-test build was blocked because local rustc 1.89 is below sqlx 0.9's rustc 1.94 requirement. The affected package suite above is green on the exact pushed commit. Originating Buzz channel: `c3252dd2-0142-4e01-88c7-a2183c3960a5` Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
742e8d1197 |
fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId (#5195)
Three pre-existing gaps in the buzz-agent observer feed fixed together
per Will's ruling ("all 3 in the current PR"):
1. **OpenAI/DBv2-GPT route** — `responses_body` never requested
`reasoning.summary`; GPT-family models billed thinking tokens but
returned `summary: []`.
2. **Anthropic/DBv2-Claude route** — `anthropic_thinking_config()` never
sent `thinking.display`; newest Claude models (Opus 5, Sonnet 5, Fable
5, Mythos 5, Opus 4.7/4.8, Mythos Preview) default to
`display:"omitted"`, returning thinking blocks with an empty `thinking`
field — observer rendered nothing.
3. **ACP v2 compliance** — buzz-agent negotiates ACP v2 but emitted
`agent_thought_chunk` and `agent_message_chunk` without `messageId`,
which ACP v2's `ContentChunk` requires (`messageId` + `content` both
required at schema head `d13d1baa`).
## Changes
**`crates/buzz-agent/src/config.rs`**
- New `ThinkingSummary` enum (`Auto`/`Concise`/`Detailed`) with
`BUZZ_AGENT_THINKING_SUMMARY` env var (default `Auto`); mirrors
`BUZZ_AGENT_THINKING_EFFORT` pattern
- `anthropic_thinking_config()` now emits `"display": "summarized"` in
both the adaptive shape and the manual-budget shape whenever thinking is
enabled
- Rewrote `is_adaptive_thinking_model` and `anthropic_thinking_config`
doc comments to match Anthropic's exact three-way per-model terminology
(doc:
https://platform.claude.com/docs/en/build-with-claude/thinking-troubleshooting#supported-models):
- Opus 4.6/4.7/4.8, Sonnet 4.6: **Off** — thinking OFF by default;
`type:"adaptive"` required to enable
- Opus 5, Sonnet 5: **On** — thinking on by default, can be disabled; we
still send `type:"adaptive"` to activate `output_config.effort`
- Fable 5, Mythos 5, Mythos Preview: **Always on** — thinking cannot be
disabled; we still send `type:"adaptive"` to activate
`output_config.effort`
**`crates/buzz-agent/src/llm.rs`**
- `responses_body` emits `reasoning.summary` alongside
`reasoning.effort` when effort is set (gated — no bare
`reasoning:{summary}` without effort)
- Covers both the pure-OpenAI Responses path and the DBv2 GPT-family
Responses path
**`crates/buzz-agent/src/agent.rs`**
- `agent_thought_chunk` carries `"messageId":
format!("{run_id}-thought-{round}")`
- `agent_message_chunk` carries `"messageId":
format!("{run_id}-message-{round}")`
- The two IDs are distinct (thought and assistant are two logical
messages per the ACP v2 Message ID RFD)
- `run_id` is a fresh random token per `session/prompt` invocation so
IDs are session-unique across multiple prompts
**`crates/buzz-agent/src/lib.rs`**
- `run_id` plumbed into `RunCtx` (was already generated in `run_prompt`,
just not threaded through)
**`crates/buzz-agent/tests/golden_transcripts.rs`**
- `test_acp_v2_chunks_carry_message_id` — negotiates v2, drives two
consecutive `session/prompt` calls, asserts: both chunk types carry
non-empty `messageId`; thought and message IDs are **distinct**; IDs do
**not** recur across the two prompts in the same ACP session
**`desktop/src-tauri/src/managed_agents/env_vars.rs`**
- `BUZZ_AGENT_THINKING_SUMMARY` added to `is_safe_to_reveal` allowlist
**`desktop/src-tauri/src/commands/agent_config_tests.rs`**
- Tests for `BUZZ_AGENT_THINKING_SUMMARY` allowlist entry
(case-insensitive)
## Tests added
- `parse_thinking_summary_round_trips_all_values`
- `parse_thinking_summary_unset_and_empty_yield_auto`
- `parse_thinking_summary_is_case_insensitive`
- `parse_thinking_summary_rejects_unknown_value`
- `thinking_summary_as_str_mapping`
- `responses_body_summary_present_iff_effort_set`
- `responses_body_emits_configured_summary_mode`
- `responses_body_concise_summary_mode`
- `anthropic_thinking_config_adaptive_emits_display_summarized`
- `anthropic_thinking_config_manual_budget_emits_display_summarized`
- `test_acp_v2_chunks_carry_message_id` (integration test — two-prompt
cross-session case)
## Notes
- **DBv2 gateway parity for `display`**: unverified — the DBv2 Claude
route proxies Anthropic Messages shape, but whether the gateway passes
`thinking.display` through is not confirmed. Flagged here rather than
blocking on it.
- buzz-acp and Desktop TS are unchanged — they already parse `messageId`
as optional and will pick it up from the wire automatically.
- Chat Completions and OpenRouter paths: untouched.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
346ae8cadc |
fix(buzz-agent): escalate LLM timeouts per retry and log per-call latency (#5130)
Non-streaming LLM calls (`"stream": false`) through slow model/provider combinations routinely take longer than the fixed `BUZZ_AGENT_LLM_TIMEOUT_SECS` window (default 240 s) to return their first response byte. The retry loop then re-ran the identical 240 s bet three times, failed the turn, and the ACP harness requeued the whole turn from scratch: agents spent 30+ minutes producing nothing while every attempt died at the same wall. And because the LLM path only logged WARN lines on failure, a healthy-but-slow call was indistinguishable from a wedged one. ### Timeout handling - **Per-attempt escalation**: the per-request budget doubles after each timeout failure (`base × 2^n`, capped at `max(1200 s, base)` — `escalated_timeout()` in `llm.rs`), shared by the main `post()` loop and `openrouter_post()`. Non-timeout retryables (429/5xx/connect) do not escalate. A call that needs six minutes now succeeds on a later attempt instead of never. - **Per-request total timeouts**: enforcement moved from the client-level `read_timeout` to `RequestBuilder::timeout()` on each LLM request, so escalated budgets aren't silently floored by the shared client and each attempt's bound covers connect through body completion. Timeout error messages were updated to match the new semantics and still point at `BUZZ_AGENT_LLM_TIMEOUT_SECS`. ### Observability - One INFO line per completed LLM call: model, provider, `duration_ms`, `input_tokens`, `cached_input_tokens`, `output_tokens`. Slowness and prompt-cache effectiveness are now visible in harness logs without waiting for a failure, and `None` vs `0` token reports stay distinguishable. Handoff summarization calls log the same line with duration only. - The agent main loop wraps the call in a `session_id` tracing span so each line is attributable to a session. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> |
||
|
Kalvin C
|
c293b3cd40 |
fix(agent): resolve oauth cache home cross-platform (#5151)
## Summary - resolve the OAuth cache home directory with the repository's platform-aware `dirs` convention - preserve the existing `.config/buzz-agent/oauth` cache layout on macOS and Linux - make the cache-path regression assertion portable across path separators ## Problem `buzz-agent` read only `$HOME` when constructing the OAuth token cache path. Packaged Windows processes do not guarantee that Unix variable, so OAuth source construction failed with `oauth cache: $HOME not set` even though Windows had a valid user profile. ## Validation Independent reviewers validated exact commit `836d820483b141b7291170cb33535ac7cb49b2eb` on Windows/MSVC with `HOME` unset and `USERPROFILE` present: - `cargo +1.94.1 clippy -p buzz-agent --all-targets --locked -- -D warnings` - `cargo +1.94.1 fmt --all -- --check` - `git diff --check f53bbd1..836d820` - `auth::` tests: 11/11 passed with `HOME` unset - full package lib target: 396 passed / 2 failed; identical-base controls classified both as pre-existing Windows failures The changed regression fails on the base with `$HOME not set` and passes on this branch. `dirs 6.0.0` was already locked by other workspace crates; the lockfile change adds only the `buzz-agent` dependency edge. Signed-off-by: Kalvin C <kalvinnchau@users.noreply.github.com> |
||
|
Will Pfleger
|
bd2fdf4a2f |
fix(buzz-agent): classify read timeouts distinctly in LLM error messages (#4959)
## Problem
When `buzz-agent` exhausts retries on a stalled LLM call, the error
message reads:
```
transport: error sending request for url (...) (cumulative 721s, 3 attempts)
```
That text is reqwest's generic pre-response failure string — identical
whether the cause is a TLS abort, a reset connection, or a
`read_timeout` fire. An operator reading the log cannot tell whether
something broke or whether the LLM generation legitimately took longer
than the configured timeout.
## Root cause (probe-confirmed)
A live probe against `goose-claude-fable-5` with a 900s client timeout
completed in **370s** — well past the default
`BUZZ_AGENT_LLM_TIMEOUT_SECS=240`. Extended-thinking models emit zero
bytes on non-streaming calls until generation is complete, so reqwest's
`read_timeout` fires on byte-silence regardless of whether the server is
healthy. The 46× exact-721s stall signatures in production logs (3 ×
240s + backoff) are deterministic self-inflicted timeouts, not network
faults.
## Fix
### Pure classifier over `{is_connect, llm_timeout, phase}`
A new `timeout_message(is_connect: bool, llm_timeout: Duration, phase:
TimeoutPhase)` pure function produces factual messages with the
configured duration value embedded verbatim. Two thin wrappers
(`classify_transport_error`, `classify_body_read_error`) extract the
reqwest flags and delegate. The duration reaches the classifiers through
a new `read_timeout: Duration` parameter on `post()` and
`openrouter_post()`; callers pass `cfg.llm_timeout`.
### Messages emitted
| Case | Message |
|---|---|
| Connect-phase timeout (`is_connect && is_timeout`) | `connect timeout:
no connection established within 10s` |
| Transport read-timeout | `read timeout: no response bytes received
within 240s (consider raising BUZZ_AGENT_LLM_TIMEOUT_SECS)` |
| Body-read timeout | `read timeout: no further response bytes received
within 240s (consider raising BUZZ_AGENT_LLM_TIMEOUT_SECS)` |
| Non-timeout | `transport: {reqwest text}` / `body read: {reqwest
text}` (unchanged) |
`LLM_CONNECT_TIMEOUT` is now a named `const` (was inline
`from_secs(10)`).
**Out of scope by explicit decision:** streaming support, changes to
`MAX_RETRIES` or backoff.
## Files changed
- `crates/buzz-agent/src/llm.rs` — `timeout_message` pure fn +
`TimeoutPhase` enum + `LLM_CONNECT_TIMEOUT` const; two classifier
wrappers updated; `post()` and `openrouter_post()` gain `read_timeout`
param; tests replaced.
## Tests
`cargo test -p buzz-agent`: **397 passed, 0 failed** at `294ce5897`.
**Pure-function tests (no network):**
- `timeout_message_connect_true_shows_connect_timeout` —
`is_connect=true` → connect-flavored text with 10s value; both phases
checked
- `timeout_message_transport_phase_shows_read_timeout_and_duration` —
transport phase includes 240s and config knob
- `timeout_message_body_read_phase_says_no_further_bytes_and_duration` —
body phase says "no further", shows 300s
- `timeout_message_duration_is_not_hardcoded` — 600s supplied → 600s in
output, not 240s
**Loopback reqwest integration tests:**
- `classify_transport_error_read_timeout_is_loopback_verified` — TCP
connect succeeds, server sends no bytes; verifies reqwest sets
`is_timeout && !is_connect` and message contains 50ms value
- `classify_transport_error_non_timeout_preserves_reqwest_text` —
controlled accept-then-close on an owned loopback listener → non-timeout
error; asserts exact `transport: {err}` output equality
- `classify_body_read_error_timeout_says_no_further_bytes` — loopback
server sends headers + 4 bytes of a declared-1024-byte body, then holds;
verifies `is_timeout`, "no further", 100ms value, config knob
No test performs egress beyond loopback (`127.0.0.1`). The TEST-NET-3
dial is deleted.
---------
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
ed4b3e7afa |
fix(buzz-agent): recover from context-window 400s instead of sticking (#4946)
Provider `context_length_exceeded` 400s permanently wedged agent sessions: the turn errored, the oversized history persisted in the in-memory session, and the usage baseline stayed frozen at the last successful sub-threshold reading (failed requests report no usage), so the preflight handoff gate never fired again — every later prompt failed identically until an agent restart. The byte-truncation fallback never intervened because it is a request-body limiter (`estimated_bytes`), not a context-window defence; at context-window scale it is a measured no-op. This adds the reactive recovery path: - **Typed classification.** `AgentError::LlmContextExceeded` is classified at both non-success provider terminals — the shared `post()` (Anthropic, OpenAI, Databricks) and `openrouter_post()` — on `status == 400` plus a context-window body match, so ordinary 400s stay terminal. - **Forced handoff.** A context-400 forces a summarize-handoff that bypasses `should_handoff()` and `BUZZ_AGENT_MAX_HANDOFFS`, bounded by its own per-turn budget (`MAX_CONTEXT_RECOVERIES_PER_RUN = 3`). - **Shrink ladder.** The summarize prompt budget halves from the observed rejected history size — not from `max_context_tokens`, the number the provider just contradicted — rung to rung, with a 4096-byte floor. A summarize call rejected for the same reason takes the next rung instead of re-sticking. At the floor (overflow dominated by unshrinkable frame: system prompt, tool schemas, live prompt) recovery is refused and the provider error surfaces clearly instead of self-healing. - **Baseline reset.** The stale usage baseline is cleared when a request fails, so the preflight gate cannot stay frozen sub-threshold on retries. Named behavior changes: 1. **Anthropic and OpenRouter errors now carry the `(model)` stamp.** Provider arms return their `Result` into the central error mapper instead of early-returning past it, making the code match its documented single-convergence contract at that mapper. 2. **`max_rounds` now counts completions the loop acts on.** A request rejected with a context-400 that is then successfully recovered refunds its round before the retry, paired 1:1 with a consumed recovery rung, so the round cap is neither weakened nor able to drop a recovered turn unanswered. Related: #4805 — the complementary proactive fix (per-session handoff-cap kill switch that let sessions grow to the provider wall). #4805 prevents reaching the wall; this PR recovers at it. --------- Co-authored-by: npub17jjz49l9jjmhhk7cac63j8yt9z555n9cw8vk7v5jz4vzw4ppld5qgj57cc <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Will Pfleger
|
6df7eba24d |
fix(buzz-agent): scope handoff cap per turn, not per session lifetime (#4805)
`BUZZ_AGENT_MAX_HANDOFFS` compared against the session-cumulative `handoff_count` (persisted across prompts). After N handoffs a long-lived session hit the cap permanently: `maybe_handoff()` returned `Skipped` on every subsequent prompt, the 16 MiB byte-truncation fallback never bound before a 1M-token provider wall, and the session wedged on the first 400 with no recovery path. Thufir's session log shows 8 days of cap-forced truncation before the first `context_length_exceeded` 400. The fix replaces the session-level cap comparison with a local `handoff_attempts` counter constructed at the start of `run()` and passed into `maybe_handoff()`. The counter resets on every `session/prompt` turn so `BUZZ_AGENT_MAX_HANDOFFS` caps compaction loops within a single turn while allowing unbounded compactions across a session's lifetime. The session-cumulative `handoff_count` is retained for log context only and is not reset. Steer-driven rounds share the per-turn budget automatically since steers inject into the running `run()` loop, not a new call. - Move `handoff_attempts` increment to before `summarize()` so failed, empty, and cancelled summarize calls each consume one budget slot — the cap cannot be bypassed by a repeatedly-failing summarizer - Upgrade cap-forced `Skipped` from `INFO` to `WARN`; add structured fields for `session_id`, attempt count, projected tokens, and threshold so the cap→wall pairing is attributable per session - Document `max_handoffs` in `config.rs` as a per-`session/prompt`-turn bound - Three new behavioral regression tests: per-turn reset proven across two separate turns; within-turn cap proven via multi-round tool-call turn; failed summarize proven to burn the attempt budget Note: this is the proactive half of the context-window fix. The reactive `context_length_exceeded` 400 recovery path is owned by Sami's branch (`buzz-ctxfix-sami`, Tyler's crew); this PR is intended to land after that one. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> |
||
|
Tyler
|
8a7eb8d3d7 |
fix(agent): recover from unsupported image input instead of poisoning the turn (#4896)
## Problem
`buzz-dev-mcp` advertises `view_image` to every agent regardless of
whether the session's model accepts images. When a text-only model (e.g.
DeepSeek V4 Flash) takes the bait, the image lands in session history
and every subsequent LLM request 404s with `No endpoints found that
support image input`. The error was classified as `LlmModelNotFound` and
propagated fatally out of the turn loop — history stays poisoned,
buzz-acp retries the batch with exponential backoff, and the session
burns its entire clock doing no work. In a recent trial run, **all 57
trials that called `view_image` on a text-only model died this way; none
recovered.**
## Fix
Capability-gating the advertised tool isn't reliable — there is no
image-capability metadata at the agent layer across providers. Instead,
recover at the turn loop:
- **Typed error**: new `AgentError::UnsupportedImageInput`, classified
narrowly on the exact provider phrase `No endpoints found that support
image input` on both the generic 404 path and OpenRouter's 404 path.
Unknown-model 404s and OpenRouter parameter-routing 404s keep their
existing classifications. No deterministic retry.
- **In-turn recovery**: on this error, `RunCtx::run` strips every image
block from history — keeping the tool result (and therefore
tool-call/result pairing) intact — marks the result `is_error`, appends
actionable model-facing guidance ("The current model does not support
image input. The image was removed from conversation history so this
turn can continue. Use a text-based inspection tool…"), and continues
the same turn. Base64 never replays again.
- **Loop guard**: recovery only fires when at least one image was
removed; if the provider says "image" and history has none, the error
propagates as before.
## Tests
- Unit: phrase classification (typed, not retried; unknown-model 404
unaffected), idempotent image-to-error history mutation preserving call
IDs and text.
- End-to-end (`fake_llm.rs` + `fake_mcp.rs`): tool call → MCP image
result → 404 unsupported-image → same-turn recovery. Captured requests
prove round 2 carried the image, round 3 replays no image, carries the
guidance text, preserves pairing, and ends `end_turn`.
- Loop guard: typed unsupported-image error with **no** image in history
fails after exactly one provider request instead of spinning —
mutation-testing showed deleting the `removed == 0` guard survived the
suite, and `max_rounds` defaults to unlimited in production, so this
branch needed direct coverage.
Verified at `a210305019b33d5f56677b4c82bab79e4ac52d24`: `cargo test -p
buzz-agent` (full package, 381 unit + all integration suites) green;
`clippy --all-targets -D warnings` green; `fmt --check` green; pre-push
hooks (rust-tests, desktop-tauri-checks, branch-skew) green.
**Scope of the classification guarantee**: the classifier runs in the
shared `post()` (which Anthropic and OpenAI paths route through) and in
`openrouter_post()` — i.e., every 404 path in `llm.rs`. It only runs on
404 responses; providers that reject images with a different status
(e.g. a 400) are out of scope for this PR — see the review-comment
discussion for why broadening the phrase list alone would not cover
them.
Authored by Wren, loop-guard test by Sami, reviewed by Eva.
---------
Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Signed-off-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Sami <f4a42a97e594b77bdbd8ee35191c8b28a94a4cb871d96f32921558275421fb68@buzz.block.builderlab.xyz>
|
||
|
Kalvin C
|
4a2305170e |
fix: reauthenticate databricks model discovery (#4008)
## Summary - preserve Databricks catalog 401 responses as authentication failures and retry discovery exactly once after silently refreshing the rejected bearer - preserve runtime OAuth recovery: when discovery has no usable OAuth credential, `session/new` succeeds with only the trimmed configured model so the first `session/prompt` can run the existing browser PKCE flow - reject a rejected configured `DATABRICKS_TOKEN` with actionable, non-interactive guidance; static credentials cannot recover through PKCE - use the configured-model fallback for non-auth discovery failures without caching failed or fallback catalogs, so later sessions retry discovery - keep known Databricks v2 models only for authenticated empty-catalog responses and mark their provenance - resolve discovery before MCP spawn or session registration, preventing failed discovery from leaking resources or consuming session capacity - permit serialized interactive PKCE only from the explicit saved-agent model picker; passive draft discovery never opens a browser ## Runtime flow 1. OAuth discovery attempts cached credentials and silent refresh without opening a browser. 2. If no usable OAuth bearer exists, `session/new` advertises only the configured model and succeeds. 3. The first `session/prompt` uses `TokenSource::bearer()`, which may launch browser PKCE. 4. A later session retries discovery and caches only the authenticated catalog. ## Regression coverage - rejected-but-locally-fresh OAuth bearer performs one refresh and one catalog retry - OAuth mode with no cached token allows `session/new` and returns exactly the trimmed configured model - the OAuth fallback is not cached; a later authenticated session retries discovery and caches the returned catalog - rejected static tokens still reject `session/new` - failed discovery does not consume the sole session slot or spawn the supplied MCP process - Desktop interactive/passive auth intent, static-token redaction, and authenticated empty-catalog provenance ## Verification - `cargo test -p buzz-agent` - `cargo test --manifest-path desktop/src-tauri/Cargo.toml --lib commands::agent_models` - `cargo clippy --manifest-path desktop/src-tauri/Cargo.toml --all-targets -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` - full pre-push hooks ## Review Adversarial review found and drove fixes for session/MCP resource leakage, duplicate concurrent PKCE flows, sensitive error propagation, incorrect 403 reauthentication, missing discovery-level coverage, passive browser launch, and the Desktop file-size ratchet. The final follow-up preserves the existing prompt-time OAuth flow while retaining static-token rejection and pre-allocation discovery ordering. --------- Signed-off-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> Co-authored-by: npub1dccv64krpcpse5cmkzfeh998cftungyatw3djt8jwdw6g43f7fyqzzmrf7 <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz> |
||
|
Atish Patel
|
09c86c56e5 |
fix: report agent usage per provider round, not once per turn (#4545)
## The bug buzz-agent emitted its `usage_update` notification in exactly one place: after `ctx.run()` returned. Until that moment a turn's token counters lived only in the prompt task's stack frame. **A turn killed mid-flight reported nothing at all** — the provider had already billed every round it completed, and no consumer ever saw any of it. That is not a corner case for anything that ends a turn on a clock. It is the normal case for a long-horizon benchmark run that relaunches its agent between phases. ## How big Measured against a provider's own billing ledger over one run's window: | | provider ledger | what we recorded | |---|---|---| | the relaunched lead seat | $485 / 348M tok | $98.99 / 90.3M tok | | the two seats that were not relaunched | $29.90 / 856M | $25.81 / 765M — reconciles | 97% of that run's usage rows came back all zeros, against 1–4% for comparable runs that never relaunch. In one 450-phase trial exactly 7 phases recorded any usage — and each of those carries 177k–437k input tokens, a whole session's worth landing in the one phase that happened to end gracefully. Worth being precise about what was *not* wrong, since both were plausible and both were checked: - **Not pricing.** The rates were verified against the provider's endpoints API and match what we charge. - **Not a truncation bug.** The usage files were intact and internally consistent. The tokens were never captured in the first place. ## The fix The run loop now emits a session-cumulative `usage_update` after every usage-bearing provider response, so an interrupted turn has reported everything but its single in-flight request. - **Emitting more than once per turn is already part of the contract.** buzz-acp's `UsageTracker` advances its committed baseline only at publish time, and goose behaves the same way — which is why the tracker was written to tolerate it. - **The turn-start session baseline is snapshotted into `RunCtx`** so the mid-turn figure stays *session*-cumulative. A turn-local number would be discarded by a high-water-mark consumer and lose the turn entirely; there is a test for exactly that. - **Snapshot by value, not a session handle.** The loop reports once per round, and taking the sessions lock on each would serialise concurrent sessions behind one another's provider round-trips. Nothing else advances those counters while the turn holds `busy`, so it cannot go stale. - **One shared `wire::usage_update_payload`** for both call sites, so the mid-turn and end-of-turn shapes cannot drift. A drift there would present as tokens silently vanishing, which is the failure this reporting exists to prevent. ## Why not a SIGTERM handler That was the obvious shape and it does not work. At signal time the counters are not sitting anywhere a handler could reach — they are in the turn's stack frame, and the value the handler would need has not been folded into the session yet. Making usage durable *during* the turn is what actually fixes it; once it is, a handler adds nothing beyond the in-flight request, whose cost is unknown until its response lands. ## Tests - `usage_is_reported_after_each_round_not_only_at_turn_end` — two rounds; asserts the **first** notification carries round 1's counts alone, proving it went out before round 2 returned. - `mid_turn_usage_includes_earlier_turns` — a mid-turn report must be session-cumulative, not turn-local. buzz-agent 18/18 on the `fake_llm` suite, 382 unit. `cargo fmt` / `clippy` / `cargo check --workspace --all-targets` clean. ## Scope Agent-side only, against `main`. The matching harness change — settling usage on the timeout path, which was skipped on the reasoning that an incomplete turn has nothing to flush — is **#4553**, against the benchmark branch, since that harness does not exist on `main`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Code <noreply@anthropic.com> |
||
|
Tyler
|
081f805d5e |
feat(agent): optional reply guard reminds a silent turn to publish (#3763)
## Why
A Buzz agent's assistant text and reasoning are never shown to anyone —
only what it posts through the CLI. A turn that runs fifteen tool calls
and never publishes is a silent failure: the requester waits on a result
that was produced and thrown away.
This adds an optional reminder at the end-of-turn gate, off by default.
Tyler asked for it in buzz-mesh; plan iterated to **9.5/10 with @Wren**
(Minimalness 9.7, Elegance 9.5, Correctness 9.3).
## What
`BUZZ_AGENT_REQUIRE_REPLY=1` (default off, per-agent opt-in). A turn
about to end with no recognized attempt to post gets a reminder and is
rerolled. **At most two, then the turn ends regardless** — the guard
catches accidental omission, it does not compel speech. The reminder
text explicitly licenses silence so it cannot fight the base prompt's
"silence is usually correct."
**This is not a new MCP hook.** `RunCtx::run` *is* the turn, so the two
per-turn locals need no plumbing, and every tool call already passes
through it with arguments visible. The objection is appended at the
existing `_Stop` gate and rides `push_hook_outputs_as_tool_results`, so
the model receives it as a lower-trust tool result with `{hook, server,
text}` attribution. No new trust path, no new lifecycle event, no
dev-mcp or CLI protocol change.
Earlier revisions of this plan needed four crates (a `_UserPromptSubmit`
hook, a marker file, a `buzz-cli` change, dev-mcp state). Tyler pointed
out the agent already knows both facts; that deleted all of it. Net
runtime change is ~35 lines in `agent.rs` + ~4 in `config.rs`.
### Recognition contract
A registered non-hook tool whose qualified name ends in `__shell`, whose
`command` argument contains `messages send` or `reactions add`.
- **The `__` separator is exact, not approximate.** Given `has()` +
`!is_hook()`, `ends_with("__shell")` is *provably equivalent* to a bare
name of `shell`: registration forbids `__` in server and bare names
(`mcp.rs:227,268`) and qnames are `{server}__{bare}`, so a trailing
`__shell` could only straddle the separator if the bare name began with
`_` — which `is_hook` excludes. Without the separator, `powershell` and
`noshell` would match.
- **Reads the structured `command` field**, not serialized arguments, so
a `description` that quotes a send cannot disarm the guard, and a
non-string `command` is rejected rather than coerced.
- **Detects an attempt, not a successful publish.** A failed send
already returns non-zero exit and error JSON — louder than this
reminder. The variable is named `buzz_reply_call_seen` so the code can't
pretend otherwise.
- **Checked after the per-turn tool-call cap**, since a discarded call
never ran.
- `messages send` also covers `messages send-diff`. Reactions count
because the base prompt directs agents to react rather than post a bare
acknowledgement.
**Known limits, both deliberate and documented:** a command assembled at
runtime (`$CMD`) or hidden in a wrapper script is missed; text that
merely quotes a send (`echo "buzz messages send"`) matches. Missing a
real post is the expensive direction and substring matching is the
forgiving one there. Neither edge is pinned by a test, so the matcher
stays free to improve.
### Budget
Reminders share `BUZZ_AGENT_STOP_MAX_REJECTIONS`, the existing outer cap
on every end-turn objection. Default 3 fits both; at 1 only one fits; at
0 the guard is off with the hooks. A round carrying both a hook
objection and a reminder costs one rejection and delivers both texts. An
independent budget would either violate that bound or need a second
arbitration rule.
## Prior art
- **#3467** (closed) built the same detector one layer up in `buzz-acp`
for a different remedy. None of its symbols are on main — this borrows
its permission to be coarse, but reads structured data that ACP didn't
have.
- **#3648** (open) detects turns with *no output at all*; a turn with
fifteen tool calls and no post counts as output there, so it does not
cover this case.
- **#3741** (merged) is mesh-only.
## Testing
**14 new tests.** 4 unit tests on the matcher; 10 integration tests
through the ACP wire harness: off by default, `=0` still off, opted-in
silent → exactly 2 reminders then `end_turn`, registered `fake__shell`
send → 0 reminders, hallucinated `fake__shell` → still reminded, publish
call truncated past the 64-call cap → still reminded, budget 1 → 1
reminder, budget 0 → off, combined `_Stop` hook objection + reminder →
one round both texts and after 2 reminders the hook objection continues
alone, unparseable `=true` → startup error naming the key.
**10 mutation checks, each breaking a specific named test** — neutralize
the nag cap, stop sharing the budget, neutralize `buzz_reply_call_seen`,
drop `has`/`is_hook`, ignore the flag, drop the `__`, drop `reactions
add`, read serialized args, move detection before truncation.
`tests/bin/fake_mcp.rs` gains `FAKE_MCP_SHELL_TOOL=1`: it previously
exposed no tool with a bare name of `shell`, so the satisfied-guard path
was untestable.
Full `cargo test -p buzz-agent` green at 9e0ae1f04; clippy `-D warnings`
and `cargo fmt --check` clean.
**Unrelated flake found:**
`cancelled_turn_with_usage_emits_notification_before_response`
(`tests/fake_llm.rs`) is timing-sensitive. Under 10 loaded cores it
fails **2/20 on this branch and 1/20 at unmodified
`origin/main@02be413b8`** — pre-existing, not caused by this change
(which is inert without the env var). Flagging so it isn't misattributed
to the next PR that's open when CI hits it.
## Docs
`crates/buzz-agent/README.md` is the primary home (env var, recognition
contract, limits, budget interaction). `docs/MCP_DRIVEN_HOOKS.md` gets a
short cross-reference explaining this is *not* a hook — otherwise
readers hunt for a `_ReplyGuard` tool that doesn't exist.
---------
Signed-off-by: tlongwell-block <109685178+tlongwell-block@users.noreply.github.com>
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Co-authored-by: Dawn (sprout agent) <c6237ef84fa537c78dcee78efd2d4e59f728859c7f194da42ac51ededfa0be05@sprout-oss.stage.blox.sqprod.co>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
|
||
|
Michael Neale
|
1dee39286e |
Merge origin/main into micn/buzz-agent-goose-core
Nine conflicts. The deletions resolve trivially -- goose owns those files now, so mcp.rs, llm.rs, handoff.rs and tests/fake_llm.rs stay deleted. The four content conflicts (agent.rs, config.rs, lib.rs, types.rs) all resolve to this branch's rewrite, but two of main's changes are wire-facing and buzz-acp now depends on them, so they are ported onto the goose loop rather than discarded: * #3463 `accumulatedCachedInputTokens` -- buzz-acp/src/usage.rs reads it for pricing. Sourced from goose's `cache_read_input_tokens` + `cache_write_input_tokens`, which goose documents as subsets of `input_tokens` (`token_usage.rs:72-78`), so it stays a subset here too. * #3593 `accumulatedTotalTokens` -- emitted only when exactly known. One turn without a provider total poisons the session cumulative to `None` and the field is omitted, because buzz-acp must not read a missing total as zero. Cargo.lock was regenerated rather than resolved by hand: main moved mesh-llm to v0.74.0, which requires rmcp ^1.8, and the stale lock still pinned 1.7.0. 89 tests green, fmt + clippy -D warnings clean workspace-wide; workspace, sprig and desktop/src-tauri all build. Signed-off-by: Michael Neale <michael.neale@gmail.com> |
||
|
Michael Neale
|
3a7ffc07af |
refactor(buzz-agent): dispatch load_skill as a platform extension, not a frontend tool
Adopts the shape Maple uses for its in-process tools (MapleDeveloperClient,
SkillsClient): an `McpClientTrait` impl registered via
`extension_manager.add_client` with an `ExtensionConfig::Platform`, rather than
`ExtensionConfig::Frontend`.
Goose advertises frontend tools but refuses to dispatch them. It yields a
`FrontendToolRequest` and blocks the reply stream until the embedder calls
`handle_tool_result` -- strictly sequential, no timeout, and a single result
channel with no request-id correlation, so one missed or duplicated result
wedges the session for good and the cancel token will not free it. That is a
failure mode with no upside here. A platform client goes through goose's
ordinary tool path and gets concurrency, per-request timeouts and
`notifications/cancelled` for free.
Net effect is less code: `serve_frontend_tool` is gone, and with it the
`skills` parameter threaded through run_turn -> drive_stream -> handle_event
and the `Session.skills` field. The BuiltinClient owns the skill list.
Also fixes a real mismatch the swap exposed. Goose namespaces platform tools as
`{extension}__{tool}` (`extension_manager.rs:1415`), so the model sees
`buzz__load_skill`, but the skills section of the system prompt told it to call
`load_skill`. The prompt now derives the name from the same constants the
registration uses, so the two cannot drift.
89 tests green, fmt + clippy -D warnings clean.
Signed-off-by: Michael Neale <michael.neale@gmail.com>
|
||
|
Will Pfleger
|
ab55fee818 |
feat: add first-class OpenRouter provider support (#1975)
## Summary
First-class `Provider::OpenRouter` support joining the existing
anthropic/openai/databricks providers. Reuses the Chat Completions path
with targeted mutations for OpenRouter's routing contract.
**Core (`crates/buzz-agent`):**
- `Provider::OpenRouter` enum variant with `OPENROUTER_API_KEY`,
`BUZZ_AGENT_MODEL` → `OPENROUTER_MODEL` fallback, `OPENROUTER_BASE_URL`
env convention
- Body mutator: `reasoning: {effort}` when effort is configured, and
`max_completion_tokens` translated to OpenRouter's `max_tokens`
spelling; no `provider.require_parameters` filter (it routes only to
endpoints advertising every parameter in the body, which hard-404s a
valid model id); summaries get neither. `openai_body` is always called
with `effort=None` on the OpenRouter path — the `reasoning` object is
added by the mutator directly, so `reasoning_effort` is structurally
absent.
- Attribution headers: `HTTP-Referer: https://github.com/block/buzz`,
`X-OpenRouter-Title: Buzz`
- Error-inside-200 check in shared `parse_openai` (`finish_reason ==
"error"`)
- 401 auth handling: static API keys (`refresh_now` returns the same
token) fail terminal immediately with one wire request; PKCE/minting
sources get one retry with the fresh token.
- Status+`error_type` retry matrix (4-arm collapsed form): 429 (honor
`Retry-After`), 502 (retry), 503/`provider_overloaded` (honor
`Retry-After`), everything else including untyped 503 (bounded retries →
actionable routing message). 499 included matching shared `post()`
(#2175) for turn-timeout stall surfacing. Terminal failures wrapped in
`terminal_llm_error` for duration+attempt-count context.
- `anthropic/*` `cache_control` injection (model-gated, mixed-content
safe)
- Provider-agnostic `reasoning_details` opaque round-trip on
`HistoryItem::Assistant` for tool-call continuations — captured verbatim
in `parse_openai_with_reasoning_details`, replayed verbatim in
`openai_body`, byte-accounting charged. `provider_extra` passthrough
from `make_tool_call` composes independently.
**Desktop:**
- Readiness arms checking `OPENROUTER_API_KEY` + `OPENROUTER_MODEL`
- Model discovery via `{OPENROUTER_BASE_URL}/models` filtered on
`supported_parameters` contains `tools`
- Picker entry, credential config, effort table 3-file sync
**`desktop/src/features/agents/AGENTS.md`: no rules changed** — the
scoped rule requiring an explicit note is satisfied here.
Implements the gate-cleared plan from
`PLANS/OPENROUTER_PROVIDER_PLAN.md` (rev 3).
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
|
||
|
Will Pfleger
|
f95fdc1a10 |
feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593)
## What Wires genuine provider-reported `total_tokens` through the full buzz-agent → buzz-acp publish chain so kind-44200 events carry real per-turn and cumulative totals for OpenAI-backed models, while preserving all existing behaviour for Anthropic and external harnesses (goose, claude-code). ## Why Live prod data showed 0 of 1,934 archived reports carry `totalTokens`. Both hardcoded `total_tokens: None` in `pool.rs` and the absent field in `buzz-agent`'s parser are root causes. This is the backend half of a two-track fix; the display-fallback half lands in [#2035](https://github.com/block/buzz/pull/2035). ## Changes **`crates/buzz-agent/src/types.rs`** - Added `total_tokens: Option<u64>` to `LlmResponse` with an explicit doc comment that NIP-AM forbids deriving it. - Added `TurnTotalState` enum (`Unseen | Exact(u64) | Unknown`) with `fold()` and `exact_value()` — the tri-state accumulator that distinguishes not-yet-observed from permanently poisoned. **`crates/buzz-agent/src/llm.rs`** - `parse_responses` and `parse_openai`: read `usage.total_tokens` from OpenAI Chat Completions (including Databricks routes) and the Responses API via `sum_usage`. - Anthropic: explicit `total_tokens: None` — no genuine total available; NIP-AM forbids summing categories. **`crates/buzz-agent/src/agent.rs`** - Added `turn_total_state: &'a mut TurnTotalState` to `RunCtx`. - Fold `response.total_tokens` into the accumulator after each usage-bearing response; non-usage-bearing responses (keepalive/stream frames) do not poison. **`crates/buzz-agent/src/lib.rs`** - Added `accumulated_total_state: TurnTotalState` to `Session` (default `Unseen`). - Per-turn state passed to `RunCtx`, folded into session cumulative after each turn. - Emits `accumulatedTotalTokens` in `usage_update` only when cumulative is `Exact(n)`. **`crates/buzz-acp/src/usage.rs`** - Added `accumulated_total_tokens: Option<u64>` (serde default) to `UsageUpdatePayload` — optional for goose compat. - Added `last_total: Option<u64>` to `SessionState`. - Added `turn_total_tokens` and `cumulative_total_tokens` to `TurnUsage` (field-local — never affect `delta_reliable`). - Derive turn-total delta only when prev and current are both `Some` and monotonic; absence, decrease, or no baseline leaves only the total delta null without touching input/output reliability. **`crates/buzz-acp/src/pool.rs`** - Replaced both hardcoded `total_tokens: None` in `publish_agent_turn_metric` with `usage.turn_total_tokens` and `usage.cumulative_total_tokens`. ## Tests 20 new tests across the four touched files: | File | Tests | |------|-------| | `types.rs` | `TurnTotalState` fold, accumulation, exact_value, default (7 tests) | | `llm.rs` | Chat present/absent, Responses present/absent, Anthropic always-None (5 tests) | | `usage.rs` | First turn no baseline, second-turn delta, cumulative decrease (field-local), current absent, goose-shaped deserialization, baseline absent (6 tests) | | `pool.rs` | Exact turn+cumulative mapping, null totals never derived (2 tests) | `cargo test -p buzz-acp -p buzz-agent` — all passing, 0 failures. ## Scope Boundary: `crates/buzz-agent/**` + `crates/buzz-acp/**` only. Desktop unchanged. `costUsd` explicitly out of scope. Related: [#2035](https://github.com/block/buzz/pull/2035) --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Co-authored-by: npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz> |
||
|
Atish Patel
|
4a1ebf25c7 |
feat(agent): make Gemini and MLflow-route models usable through databricks_v2 (#3569)
## Summary
Makes Gemini — and every other non-Claude, non-GPT-5 model on the
Databricks MLflow route (`databricks_v2`) — usable in an agent loop.
These are the non-`benchmarks/` changes from
`benchmark/harness-accounting-and-solo`, lifted onto a clean base off
`main` so they can land independently while the harness work continues.
Two defects made these models unusable, one fatal and one silent. Both
live only in `openai_body` / `parse_openai`, which is the
least-exercised of the three `databricks_v2` sub-routes — the
`luna`/`sol` conditions run the Responses route and the `opus`
conditions run the Anthropic route, so **this change is inert for every
model already in use** and only lights up the MLflow path.
## Why a third route at all
`databricks_v2_route_for_model` buckets by model family: `claude*` →
Anthropic Messages, `gpt-5`/code-names → OpenAI Responses, **everything
else → MLflow chat-completions**. Gemini, Qwen, gpt-oss, and friends all
fall through to that third pair — and both bugs below live only there.
## D1 — dropped thought signatures (fatal)
Gemini returns a `thoughtSignature` on every tool call and **requires it
echoed back**. `openai_body` reserialized each call as `{id, type,
function}` only, dropping the field, so the next request 400'd:
```
HTTP 400 Function call is missing a thought_signature in functionCall parts.
```
For a coding agent this fires on the **first** tool call, so the model
never completes a single turn.
**Position is load-bearing.** A four-shape replay probe against the live
gateway established that the signature must sit as a *sibling* of
`function` — nesting it inside `function{}` fails with the *same* 400 as
omitting it. A fix that "preserves the field" without preserving its
position passes a unit test and still 400s.
The fix: `ToolCall` gains `provider_extra: Map<String, Value>`.
`parse_openai` captures every top-level wire key except the three we
model (`id`, `type`, `function`); `openai_body` re-emits them beside
`function`. Keeping *whatever we did not model*, rather than naming
`thoughtSignature`, means the next provider with an opaque per-call
token needs no change here. The Responses and Anthropic replay shapes
are fully modelled, so they pass `Default::default()` and stay
**byte-identical** to before.
### D1b — duplicate tool-call ids (same root cause)
Gemini returns the **function name** as the id, so two parallel calls to
one function arrive sharing an id — and that id is what pairs a
`role:"tool"` result back to its call, making two results
indistinguishable. `dedupe_provider_ids` suffixes collisions
(`get_weather`, `get_weather-2`). Safe because both halves of the
pairing (the assistant `tool_calls[].id` and the result's
`tool_call_id`) are re-emitted from this same value; the provider never
sees its original id again.
## D2 — block-array content discarded (silent, worse than a crash)
`parse_openai` read `content` with `as_str()`, which returns `""` for
anything that isn't a JSON string. Gemini (and Qwen35, gpt-oss) send an
array of typed blocks:
```json
"content": [
{"type": "reasoning", "summary": [{"type": "summary_text", "text": "…"}]},
{"type": "text", "text": "391"}
]
```
So the model answered and the answer was thrown away — no error, no
warning, just a turn that looked like the model had said nothing. On a
benchmark this reads as "Gemini is bad at the task" rather than "buzz
dropped the reply."
`openai_content_parts` now accepts either shape — string as before, or a
block array where `text` blocks concatenate into text and `reasoning`
blocks into reasoning (Gemini nests the prose one level down under
`summary`). Message-level `reasoning_content` / `reasoning` still win
when present, so DeepSeek and vLLM-style hosts are unchanged; block
reasoning is the last fallback.
## Also: a turn-start log line (`buzz-acp` `pool.rs`)
Small, independent observability change that also rides in the
non-benchmark delta: `run_prompt_task` now emits a `pool::prompt` "turn
starting" line, labelled by the same `prompt_label` helper as
`log_stop_reason`, so a log reads as start/stop pairs. An unpaired start
is the only durable evidence that a turn was entered and never returned
— without it, a stalled agent and an agent nobody woke leave identical
(zero-completion) logs.
## Interaction with #3538
#3538 (already merged) rewrote `databricks_v2_route_for_model` to route
by boundary-aware model-family segments. That change and this one touch
**different functions** in `llm.rs` — routing vs. body/parse — and
compose cleanly; the family routing decides *which* pair runs, and this
fixes the MLflow pair it can now select.
## Testing
- `cargo fmt --all -- --check`, `cargo clippy -p buzz-agent -p buzz-acp
--all-targets -- -D warnings` — clean.
- `cargo test -p buzz-agent -p buzz-acp` — all green (304 + 632 lib
tests plus integration suites, 0 failures). Five new tests cover:
block-array text extraction, plain-string regression, passthrough
capture (and non-duplication of the modelled keys), replay position
(`thoughtSignature` beside `function`, not inside it), and id
de-duplication.
- Wire evidence: the four-shape replay table and the reasoning-effort
probe were run against `block-lakehouse-staging` (recorded in the design
doc).
## Relationship to the benchmark branch
The full design write-up (four-shape replay table, position-matters
analysis, effort verification, and open pricing item) lives in
`docs/08-gemini-provider-fixes.md` on
`benchmark/harness-accounting-and-solo`. The benchmark manifests and
endpoint-config entries that exercise these models are separable and
stay on that branch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Signed-off-by: Atish Patel <atish@squareup.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Atish Patel
|
6438dedf83 |
feat(agent): route Claude/GPT model families to their native gateway wire (#3538)
## Summary Databricks v2 chooses the gateway wire format — OpenAI Responses, Anthropic Messages, or MLflow chat — purely from substrings in the endpoint name. There is no family field on the endpoint to key off, so the substring set *is* the routing contract. The matcher only recognised `gpt-5`/`gpt5` and `claude`, which makes correct billing depend on every Claude endpoint happening to be named with the literal string "claude". ## Why this matters Getting a Claude model onto the Anthropic Messages route is exactly what lets buzz attach the `cache_control` breakpoint (the fix in #3463). If a Claude endpoint's catalog name omits "claude" — an alias, a bare `opus-5`, a `goose-opus-5` — it silently falls through to the MLflow (OpenAI-wire) path, where Anthropic prompt caching is **structurally impossible**. The result is the same failure #3463 fixed: 0% cache reads, the full ~10x read discount lost, and no error — a naming convention quietly holding up a billing-correctness invariant. ## What changed `databricks_v2_route_for_model` (`crates/buzz-agent/src/llm.rs`) now matches broader, case-insensitive marker sets: - **Claude → Anthropic Messages:** `claude`, `opus`, `sonnet`, `haiku`, `mythos`, `fable` — the Claude family names and release code names, so a Claude endpoint reaches the cache-capable route regardless of how it's named. - **GPT → OpenAI Responses:** the `gpt` family (now `gpt` on its own, not just `gpt-5`) plus the GPT-5 launch code names `sol`, `luna`, `terra`. OpenAI markers are evaluated first, preserving the prior `gpt-5`-first precedence for any name that could carry both. Names matching neither set still fall through to the MLflow chat route. ## Testing - `cargo fmt`, `cargo clippy -p buzz-agent --all-targets -- -D warnings` — clean. - `cargo test -p buzz-agent` — all green (299 lib + integration suites, 0 failures). The `databricks_v2_routes_by_model_family` test was expanded to cover each new marker, the GPT-5 code names, case-insensitivity, and the unchanged MLflow fallback (including `gemini`). ## Relationship to #3463 #3463 taught the Anthropic path to request caching; this makes sure Claude models actually land on that path. Follow-up still open: surfacing `cache_creation_input_tokens` end-to-end so a persistent `reads == 0 && writes == 0` reveals a disabled cache regardless of which wire a model takes — happy to do that next. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Atish Patel
|
c405ad1d4b |
feat(agent): fix Anthropic prompt caching with Databricks (+ MCP proxy/TLS passthrough) (#3463)
> On 8 tasks matched by name across the two runs, cost fell $8.36 → $1.77 (4.71×) and wall-clock 12,423 s → 1,085 s (11.45×). ## Summary Two independent, self-contained fixes to `buzz-agent`/`buzz-acp`, split out of the benchmark branch so they can land while the harness work continues: 1. **Request and surface Anthropic prompt caching.** buzz never sent a `cache_control` breakpoint, so on the Databricks Anthropic route `cache_read_input_tokens` was **structurally always 0** and the ~10× cache-read discount was never claimed. This teaches `anthropic_body()` to mark the cacheable prefix, and plumbs the cache split end-to-end so accounting can price it. 2. **Pass proxy + TLS-trust env into MCP tool subprocesses**, so agent tools on a proxy-only host stop reporting a live network as offline. ## Why the caching gap matters The Anthropic Messages API does **not** cache unless the request carries a `cache_control` breakpoint, and the Databricks AI Gateway — a third-party proxy in front of the model, in the same category as Bedrock/Vertex — does **not** auto-cache (only the first-party Anthropic API and Claude-on-AWS do zero-config caching). So every request was billed cold. Measured live against the Databricks gateway (`databricks-claude-opus-5`, 2026-07-28), the same call with and without a single `cache_control` marker: | Run | `input_tokens` | `cache_creation` | `cache_read` | latency | |---|---|---|---|---| | No `cache_control`, two byte-identical calls | 121,625 | 0 | **0** | ~9.3 s | | With one marker — cold (write) | 4 | 121,625 | 0 | 9.3 s | | With one marker — warm (read) | 4 | 0 | **121,625** | **4.5 s** | One marker moved 121,625 tokens from full-price input to a 0.1× cache read and roughly halved latency (a clean, isolated ~2.07× prefill speedup on this single-threaded microbenchmark). The gateway honours `cache_control`; buzz simply never sent it. At fleet scale this was a real budget item. Across matched Terminal-Bench solo sweeps (89 tasks, `-n 20`, before the fix), the two OpenAI-route models independently landed at ~86–87% cache reads — the expected shape for an agentic loop, where system + tools + append-only history repeat every turn — while the Anthropic route returned a hard 0% on every receipt: | Condition | Route | Input tokens | Cache reads | Cost | Cost if uncached | Discount | |---|---|---|---|---|---|---| | luna (`gpt-5-6`) | OpenAI | 20,320,818 | **17.7M (87.0%)** | $6.96 | $22.87 | **3.28×** | | sol (`gpt-5-6`) | OpenAI | 22,312,290 | **19.2M (85.9%)** | $37.07 | $123.35 | **3.33×** | | opus (`claude-opus-5`) | Anthropic | 12,459,822 | **0 (0.0%)** | $81.31 | $81.31 | **1.00×** | Applying luna's measured 87% read rate to the opus token counts at list prices (`input $5/M`, `cached_input $0.5/M`, `output $25/M`) puts the opus run at **~$32.53 vs the $81.31 actually paid — a ~60% overspend on those 49 trials (~$89 on a full sweep)**. That is an upper bound (it prices every cached token at the 0.1× read rate and ignores the 1.25× write premium), and the opus discount is structurally smaller than luna/sol's because opus emits ~3.5× more uncacheable output per trial, which sets a floor on what caching can recover. There is also a plausible **second-order effect**: Databricks appears to meter its per-minute rate limit on *uncached* input tokens, so the missing cache also cost rate-limit headroom — the opus endpoint lost 63% of its trials to fatal 429s while running alone at one-third of a GPT endpoint's raw throughput. This is a hypothesis, not a proven mechanism (the only zero-cache condition is also the only Anthropic endpoint), but it is the reading that explains the throttling with one rule instead of two. ## Post-fix results (provisional — first trials of an in-flight re-run) On 8 tasks matched by name across the two runs, cost fell **$8.36 → $1.77 (4.71×)** and wall-clock **12,423 s → 1,085 s (11.45×)**. | Metric | before (`4a955a858`) | after (`3bef1f6a`) | |---|---|---| | Cache reads as % of input | **0.0%** | **78.7%** (still climbing toward the ~86% steady state) | | `cost_usd_no_cache_discount / cost_usd` | **1.00×** | **2.18×** (tracking the projected ~2.5×) | | Trials with a fatal 429 (same `-n 20`) | **63%** | **15–19%** | To be clear about attribution: **~2× of that is the clean prefill saving from caching itself**; the rest is second-order — cached requests burn far less rate-limit budget, so they stall less and redo less destroyed work. The 11.45× is a system-level result specific to this throttled workspace, not a caching benchmark. Quality held (7/8 solved in each run). A controlled low-`-n` A/B (neither arm hitting a 429), which the `BUZZ_AGENT_PROMPT_CACHING` opt-out exists to enable, is still owed before this becomes a published claim. ## What changed ### 1. Request caching (`llm.rs`, `config.rs`) `anthropic_body()` emits ephemeral `cache_control` breakpoints, gated by `BUZZ_AGENT_PROMPT_CACHING` (**default on**, `=0` to opt out): - **Static prefix** — marker on the `system` block. Prefix order is `tools → system → messages`, so this single marker caches **tools + system** together. Byte-identical on every turn of a run, and survives a context handoff (system/tools come from cfg/mcp, not `self.history`). - **Rolling tail + leapfrog** — marker on the last block of the last **two** messages. The append-only history re-reads the prior turn's prefix from cache; marking two messages (not one) keeps consecutive breakpoints inside Anthropic's **20-block lookback window** even as tool parallelism rises, avoiding a silent full-price miss. An empty system prompt stays a bare string (Anthropic rejects empty text blocks), and below-threshold prefixes are silently not cached, so the flag is safe on by default. ### 2. Surface the cache split end-to-end — the plumbing (`types.rs`, `llm.rs`, `agent.rs`, `lib.rs`, `usage.rs`, `acp.rs`) This is the part that makes gaps like the one above **visible** instead of silent. A consumer that prices all of `input_tokens` at the full rate can't tell a route that's caching from one that isn't — the total looks right either way. So: - `LlmResponse` gains `cached_input_tokens` (a **subset** of `input_tokens`, never an addition); `parse_anthropic` / `parse_openai` / `parse_responses` each populate it. - A `usage_first()` helper reads the cache count wherever a provider hides it — flat `cache_read_input_tokens` (Anthropic), `prompt_tokens_details.cached_tokens` (OpenAI chat), `input_tokens_details.cached_tokens` (Responses) — taking the **first present value, never a sum**. Reading only flat keys is exactly why the OpenAI route's nested `cached_tokens` had *also* been going unclaimed: `prompt_tokens` is already inclusive, so the total looked correct while the discount silently went unreported. - The per-turn/per-session accumulators and the goose `usage_update` payload now carry `accumulatedCachedInputTokens`; `buzz-acp` deserializes it (`serde` default `0` for goose, which doesn't send it) and logs `cached=<n>`. ### 3. Fix a Databricks MLflow-route double-count (`llm.rs`) The Databricks MLflow route reports the flat Anthropic-spelled `cache_read_input_tokens` *alongside* an already-inclusive `prompt_tokens`, so the old code summed them and nearly doubled the count — inflating both the context-budget gate and cost. `openai_chat_input_tokens()` now reads `prompt_tokens` alone. Verified on a live `databricks-glm-5-2` response where `prompt_tokens + completion == total` proves inclusivity. (Anthropic's native route genuinely *excludes* the cache fields and is still summed — the two never collide, because `claude*` models route to the Anthropic path.) ### 4. Proxy + TLS-trust passthrough into MCP tools (`mcp.rs`) — independent fix `buzz-agent` `env_clear()`s each MCP child, and the allowlist carried no proxy/TLS vars. On a proxy-only host that doesn't degrade the tools, it **blinds** them: apt, curl, pip, git connect directly, the egress firewall resets the socket, and the agent reports "Connection reset by peer" — indistinguishable from a genuinely offline task. Adds both spellings of `HTTP(S)_PROXY`/`NO_PROXY`/`ALL_PROXY` (curl/git read lowercase; Go/Python read uppercase; libcurl ignores uppercase `HTTP_PROXY`) plus `SSL_CERT_FILE`/`SSL_CERT_DIR` for TLS-terminating proxies that present their own CA. ## Testing - `cargo fmt --all -- --check`, `cargo clippy -p buzz-agent -p buzz-acp --all-targets -- -D warnings` — clean. - `cargo test -p buzz-agent -p buzz-acp` — **all green** (632 + 299 lib tests plus integration suites, 0 failures). New tests cover: the three breakpoints and the disabled/empty-system/single-message edge cases; nested-vs-flat cache parsing for all three routes; the Databricks inclusive-`prompt_tokens` fix; wire deserialization of `accumulatedCachedInputTokens`; and the proxy/TLS passthrough allowlist. - Pre-push lefthook suite green (branch-skew, rust-tests, test, desktop-check/test/tauri). ## Relationship to the benchmark branch These are the non-`benchmarks/` changes from `benchmark/harness-accounting-and-solo`, lifted onto a clean base off `main` so they can merge independently. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Atish Patel <atish@squareup.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Apurva324
|
90e058ebf6 |
feat: add explicit entry for claude-opus-5 in model config (#2831)
Fixes #2787 - Added `claude-opus-5` to `config.rs` model classification and adaptive effort helpers. - Updated fixture test configurations to cover `claude-opus-5`. - Verified with `cargo test` and JS unit tests. Signed-off-by: Apurva Shaw <apurvashaw@Apurvas-MacBook-Air.local> Co-authored-by: Apurva Shaw <apurvashaw@Apurvas-MacBook-Air.local> |