jerry/claude-agent-acp
1
0
Fork 0
mirror of https://github.com/agentclientprotocol/claude-agent-acp.git synced 2026-10-27 04:30:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

test: keep the suite off the real Claude CLI (#1223)

Browse Source 
Fixes an intermittent failure of "replays a subagent from its own
transcript file", and with it a broader leak: the unit suite was running
the real Claude CLI hundreds of times per run.

## Cause

The agent runs `claude auth status --json` in the background, without
awaiting it, on every `initialize` and prompt. The scenario suite stubs
it (`CLAUDE_CODE_EXECUTABLE=/usr/bin/false`), but other files didn't.
Logging the probe's executable over one `vitest run` showed about 670
runs of the real CLI:

| File | Real CLI runs |
|---|---|
| `acp-agent.test.ts` | 375 |
| `create-session-options.test.ts` | 14 |
| `incomplete-tools.test.ts` | 12 |
| `session-titles.test.ts` | 8 |
| `file-change-audit-integration.test.ts` | 5 |

Each run uses the machine's own Claude config, unless a test points
`CLAUDE_CONFIG_DIR` elsewhere.

That flaky test points `CLAUDE_CONFIG_DIR` at a temp dir. The CLI it
started then wrote `.claude.json`, `.claude.json.lock` and `backups/`
into that dir about a second later, after the test body had finished.
Under full-suite load, that overlapped the `rm` in the test's `finally`,
which failed with `ENOTEMPTY`. Its assertions always passed.

## Change

`vitest.config.ts` sets `CLAUDE_CODE_EXECUTABLE=/usr/bin/false` in
`test.env`:
- It's the override `claudeCliPath()` already honors, and the value the
scenario suite already uses, so production code doesn't change.
- `test.env` is the base environment, so it survives the
`vi.unstubAllEnvs()` that three files call.
- Runs that exercise the real CLI keep it: with `RUN_INTEGRATION_TESTS`
(`npm run test:integration`) or `RUN_LIVE_CAPABILITY_AUDIT=true`, the
variable is left unset.

Tests that mock `execFile` or set their own executable are unaffected.

## Evidence

- After the change, the same logging shows 0 runs of the real CLI: 688
probes run `/usr/bin/false`, and 108 hit files that already mock
`execFile`.
- The flaky test's temp dir now holds only the test's own `projects/`
after the test, with no `.claude.json`.
- Before the change, it failed in about 2 of 6 full runs on my machine.
After, 8 of 8 full runs passed.
- A full run takes about 4.8 s instead of 8.4 s.
- The variable is `/usr/bin/false` in a default run and unset with
either opt-in flag.
- `npm run build`, `npm run check`, `npm run test:run`.
This commit is contained in:
Ben Brandt
2026-10-02 06:15:00 -07:00
committed by GitHub
parent 6b69852cfc
commit 3d4017b15a
1 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
vitest.config.ts
View File

@@ -1,10 +1,19 @@
import { defineConfig } from "vitest/config";
// Runs that exercise the real Claude CLI opt in with these variables.
const usesRealCli =
Boolean(process.env.RUN_INTEGRATION_TESTS) || process.env.RUN_LIVE_CAPABILITY_AUDIT === "true";
export default defineConfig({
test: {
watch: false,
globals: true,
environment: "node",
// The agent runs `claude auth status` in the background on every
// initialize and prompt. Pointing it at a CLI that exits at once keeps
// tests off the machine's real CLI and Claude config, and stops a run
// from outliving its test (it wrote into temp config dirs being deleted).
env: usesRealCli ? {} : { CLAUDE_CODE_EXECUTABLE: "/usr/bin/false" },
include: [
"src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}",
// Release tooling lives in scripts/ rather than src/ so it stays outside tsc's