mirror of
https://github.com/block/buzz.git
synced 2026-11-07 01:41:06 +00:00
3fdd8c6ff1
## Why The loadgen agent polled HTTP `/query` every few seconds. A real agent harness (`buzz-acp`) never polls when idle. It works in bursts around each prompt, and those bursts are the load a fleet of active agents puts on the relay. To answer "how well does the relay handle N active users plus their agents over a sustained period", the loadgen needs to model those bursts and hold them for an hour or more. ## What - **`[agent.turn]`**: an idle agent starts a turn on a Poisson schedule and runs one turn at a time. A turn lasts 0.5–1.5 × `mean_secs`. The steps follow `buzz-acp`: - **Start**: a `limit:1` `/query` finds the message to answer (`http_query`), which stands in for the harness's subscription. Then the harness's own reads (`turn_context` op): for a threaded message, its thread context in one `/query` (root by id, newest 13 replies, the agent's own newest reply) plus a `/count` when the window is full. Then kind:39000 metadata, project home (kind:30621, kind:30617) and the author's profile. Then two kind:7 reactions on the message. - **Work**: typing every `typing_secs` and observer frames of `frame_bytes` every `frame_secs` over the socket, plus `reads` tool reads (`/query`, limit 50). - **End**: a threaded reply, under the root that NIP-10 `resolve()` gives. Then reaction removal: query own reactions, then a kind:5 deletion for each. - One HTTP action at a time runs next to the socket. Socket publishes (presence, typing, frames) don't wait for their OK: each OK is matched as it arrives, one missing after 30 s is an error, and at most 128 wait at once. So the socket and HTTP sides stay on schedule while either is slow, as in the harness. A 2xx response that doesn't decode counts as an error. No agent request starts after the stop. - The request mix approximates sustained turns: every turn reads like a first prompt. A harness already holding a thread reads 25 replies (not 13), batches profile lookups and caches project-home lookups for 30 s, so this reads profiles and project homes more often than a busy harness. - `agent.channel_subscriptions`: one live mentions REQ per channel, like the harness. `agent.presence_interval_secs`: presence heartbeats. - `agent.query_interval_secs = 0` turns polling off. - **Sustained holds**: ramp steps may now repeat a level. A 60-minute hold can be six 600 s steps, each one reported and judged, so drift shows up as a trend. - New op `turn_context`. New scenario `perf/scenarios/local-agent-turns.toml`. `perf/LOADGEN.md` documents the model. All new fields are off by default (`turn.per_hour = 0`, presence 0, no channel subscriptions), so existing scenarios run unchanged. ## What got simpler - The agent loop's per-action branches now go through one `sent` result. The observer frame build, the HTTP submit and the `/query` body read are shared helpers. The old inline frame code is gone. - "Hold for an hour, judge every 10 minutes" is now a ramp of repeated levels. No new abort-window concept was needed. - Identified debt, not touched here: abort is still judged only at step end, so a collapsed relay is not detected until the step closes. Repeated short steps reduce that gap. ## Testing - On a Blox workstation at6bb151d1: `cargo fmt --check`, `cargo clippy -p buzz-test-client --all-targets -D warnings`, and `cargo test -p buzz-test-client --lib` (103 passed). The five turn tests passed 10/10 repeated runs. Three mutants (awaiting the OK again, dropping the stop guard in `submit`, dropping it in `query`) each fail a test. - `local-agent-turns` against a local relay at6bb151d1: both steps 0 errors, 0 clients cancelled. - `agent_turns_read_react_stream_reply_and_remove_their_reactions` runs an agent against a fake socket and a fake HTTP relay. It asserts the exact thread-context filters and count, the metadata, project and profile reads, the reactions, typing, frames and presence, and that typing or frames arrive while a tool read is held. It also asserts the threaded reply under the right root and the deletion of exactly the reactions the agent added. - `agent_turns_keep_moving_while_socket_oks_lag`: the same journey with every socket OK 1.5 s late, past the 1 s heartbeat. - `a_turn_stopped_while_adding_a_reaction_sends_nothing_more` and `a_turn_stopped_while_reading_its_reactions_deletes_none`: the stop lands while a reaction POST or removal query is held, and no request follows it. - `agent_turn_reads_that_do_not_decode_are_errors`: 200 responses that are not events count as errors, and no turn reacts. - `a_timed_out_route_is_recovered_again_after_a_renewal` (desktop) fails 2 times in 10 on unmodified main as well. That flake is pre-existing and unrelated to this change. - Live: a single-relay run with 10k humans and 20k turn-taking agents (held at 1x for 60 min, then 1.5x and 2x). Results will be added here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Luke Tornquist <tornquist@squareup.com> Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>