mirror of
https://github.com/block/buzz.git
synced 2026-11-03 16:01:35 +00:00
d22ba793a6
Replace buffered LLM requests with Server-Sent Events streaming. Each text delta emits an agent_message_chunk session update as it arrives, providing a natural keepalive that resets the ACP idle clock without relying solely on the 30s ticker. Design decisions: - Two reqwest::Client instances: `http` (with global timeout for summarize) and `http_stream` (no global timeout, enforces first-byte and inter-chunk timeouts via tokio::time::timeout) - Two-phase timeout: llm_timeout (120s) until first content delta, then stream_chunk_timeout (30s) between subsequent events - Anthropic: index-keyed HashMap<usize, (String, String)> for parallel tool-call accumulation via content_block index - OpenAI Chat: Vec-indexed accumulation by tool_calls[].index - Responses API: routes on JSON `type` field inside data payload, uses response.output_text.delta and response.function_call_arguments.delta - MAX_LLM_RESPONSE_BYTES caps accumulated semantic content (text + tool args), not raw SSE wire bytes - SSE parser handles : comments, multi-line data:, id:/retry: fields - Keepalive ticker (G) preserved as fallback for reasoning models that pause before producing content Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>