Files
buzz/crates/buzz-acp/TESTING.md
Sanket Kanjalkar a314975ce7 perf(acp): reduce observer CPU and lock overhead (#9381)
## Summary

The Agent Client Protocol (ACP) observer records agent activity for the
owner's viewer. Sizing currently allocates JSON strings and rescans
whole frames for each trimmed field.

Count JSON bytes directly, reuse fitted sizes, and count changed fields
when cheaper. Skip raw-message copies for disabled observers and copy
replay payloads outside the mutex. Keep sequence numbers, timestamps,
replay insertion, and publication under one lock to preserve replay/live
ordering.

This adds one `Arc` allocation per replay entry, and snapshots pin
entries until copying completes. Replay retains the existing 1,000-event
limit.

### Related issue

Related to #7987. This PR targets `main` independently of #9366, which
owns the byte-budget fix. The changes overlap in observer plumbing, so
the second PR to land will need a rebase.

### Testing

The new ignored probe exercises the production publish queue. Timing
includes template clones and queue ingestion, before encryption. Five
alternating fresh release processes per arm/case compared `75d38b6c2`
with this commit on a busy host. Every pair retained identical queue
bytes and event counts.

```sh
cargo test --release -p buzz-acp --lib observer::perf_tests::publish_queue_probe -- --exact --ignored --nocapture
```

Set `BUZZ_OBSERVER_PROBE_EVENTS`, `BUZZ_OBSERVER_PROBE_BYTES`, and
`BUZZ_OBSERVER_PROBE_FIELDS` for each row. Medians:

| Events / string bytes per event / fields | Before | After |
| --- | ---: | ---: |
| 200 / 1 MiB / 64 | 5605 ms | 228 ms |
| 1,000 / 1 MiB / 1 | 714 ms | 352 ms |
| 10,000 / 1 KiB / 1 | 13.01 ms | 6.90 ms |

A separate 20-trial integration run used the same production Chromium
app build (`block/buzz-app@128cd4c2`) and four real ACP subprocesses.
Three peers emitted 225 synthetic messages over five seconds, each with
1 MiB across 64 strings. A fourth emitted 50 markers, 100 ms apart. Each
arm/mode used five fresh processes and browser contexts, with
alternating arm order.

The real observer, publisher, encryption/signing, browser broker, and
Activity UI ran through a modeled relay transport. Marker timing used
one controller clock, from release through transcript DOM commit plus
two animation frames. Observer timestamps were not the timing origin.
Observation-enabled medians:

| Metric | Before | After |
| --- | ---: | ---: |
| Rust CPU time | 4.49 s | 0.45 s |
| Marker-to-render p95 | 4471 ms | 1018 ms |
| Sampled peak RSS | 342.3 MiB | 296.9 MiB |

All 500 enabled-mode markers rendered exactly once; all 2,250 large
events were delivered with the existing payload elision.
Observation-disabled CPU stayed roughly flat (0.13 s to 0.12 s). The
controlled publication-order regression also fails original `main` and
passes this commit.

These are stress-workload measurements, not a general app speedup.
Native hosting and deployed-relay performance remain unmeasured. The
publish cadence remains one frame per second. No CI timing threshold was
added.

Generated with Codex

Signed-off-by: Sanket Kanjalkar <sanketk@squareup.com>
Co-authored-by: Codex <codex@openai.com>
2026-11-12 14:37:33 -08:00

3.1 KiB

Pi adapter integration

Buzz uses the buzz-pi-acp fork. Install Node.js 22 or newer, configure Pi, and install the latest development adapter from the fork's main branch:

npm install -g @earendil-works/pi-coding-agent
pi
npm install -g --install-links=true 'git+https://github.com/salman1993/buzz-pi-acp.git#main'

This test setup intentionally tracks main; the Desktop runtime catalog pins a reviewed adapter revision for users.

Make sure pi and buzz-pi-acp are on PATH, then restart Buzz.

Tests

cargo test -p buzz-acp

Managed agent sessions may already export harness options. Clear them when running the package suite: three CLI parsing tests assert the unset defaults, and inherited values would change the inputs those tests exercise. Running the package serially also avoids scheduling flakes in existing short-deadline tests.

env -u BUZZ_ACP_ALLOWED_RESPOND_TO \
  -u BUZZ_ACP_LAZY_POOL \
  -u BUZZ_ACP_IDLE_POOL_SLEEP \
  cargo test -p buzz-acp -- --test-threads=1

Run the ignored real-adapter test with a built fork checkout:

BUZZ_TEST_PI_ACP=/absolute/buzz-pi-acp/dist/index.js \
  cargo test -p buzz-acp real_pi_preserves -- --ignored

Git bootstrap

cargo test -p buzz-acp --test git_bootstrap starts the actual harness with a probe adapter, runs real signed commits/tags and scoped credential resolution, and verifies key cleanup on startup failure and SIGTERM. No relay is contacted.

To exercise the real runtime boundaries on Unix:

cargo build -p buzz-acp -p buzz-agent -p buzz-dev-mcp
BUZZ_TEST_BIN_DIR="$PWD/target/debug" cargo test -p buzz-acp git_runtime_tests -- --ignored --nocapture

The Buzz Agent test uses a deterministic local OpenAI-compatible response to invoke the actual MCP shell. The Goose test requires an installed, configured Goose and uses its provider to invoke the native developer shell. Both operate only on temporary local repositories, verify commit/tag signatures and identity, check unrelated-remote credential scoping, and assert keyfile removal. They do not replace authenticated relay clone/push/readback testing.

Observer performance measurement

An ignored local probe exercises the production publish queue and its JSON fitter. The default workload is 200 events, each with 1 MiB across 64 string leaves. It prints elapsed milliseconds and retained queue bytes/events; there is no timing threshold or additional CI job.

cargo test --release -p buzz-acp --lib observer::perf_tests::publish_queue_probe -- \
  --exact --ignored --nocapture

# Under-budget control:
BUZZ_OBSERVER_PROBE_EVENTS=10000 BUZZ_OBSERVER_PROBE_BYTES=1024 \
  BUZZ_OBSERVER_PROBE_FIELDS=1 \
  cargo test --release -p buzz-acp --lib observer::perf_tests::publish_queue_probe -- \
  --exact --ignored --nocapture

Elapsed time includes cloning the template payload and queue ingestion; it excludes template construction and later publication/encryption. Use release builds, fresh processes, identical workloads, and repeated alternating runs when comparing revisions. Queue bytes are serialized JSON, not heap or RSS.