mirror of
https://github.com/block/buzz.git
synced 2026-10-31 14:31:02 +00:00
33bcaa7ced
buzz-agent's `BUZZ_AGENT_LLM_TIMEOUT_SECS` defaults to 240s. MeshLLM's OpenAI frontend gives a backend call 600s (`OpenAiFrontendConfig::DEFAULT_BACKEND_TIMEOUT`), so on shared compute the client aborts work the server is still legitimately doing. Requests to a local mesh are `stream: false`, so a cold multi-ten-thousand token prefill is one silent request. Measured on an M5 Max serving Qwen3.8-27B Q4_K_M: an 88,318-token cold prompt returns HTTP 200 after 503s at 176 tok/s prefill. At the 240s default the client gives up at four minutes, then retries with an escalated budget (240 -> 480 -> 960), piling load onto a box that is already prefilling the first attempt. Seat the mesh client budget at 660s, just above the frontend's 600s, so the mesh's own error surfaces rather than a client abort racing it. Remote providers are untouched and keep buzz-agent's 240s default. Like the other mesh seeds this is a default, not policy: an explicit user value survives via `insert_default_if_unset` plus the copy-forward list in `relay_mesh_process_env`. Co-authored-by: Michael Neale <michael.neale@gmail.com> Signed-off-by: Michael Neale <michael.neale@gmail.com>