Commit Graph

5 Commits

Author SHA1 Message Date
Sergey Shkredov
e8eb9d340f feat: narrow models to what the providers/set gateway lists (#1340)
### Summary

Addresses [IJAI-1783](https://youtrack.jetbrains.com/issue/IJAI-1783).

An organization can restrict the models its JetBrains gateway serves,
but a session routed through `providers/set` offered the whole Claude
Code catalogue, so the picker showed models the gateway then refused.
`providers/set` now lists the gateway's models before it responds, and
the model picker offers only the SDK models the gateway lists.

- `anthropic`: `GET {baseUrl}/v1/models?limit=1000`, paginated by
`has_more`/`last_id`, with the provider headers and `anthropic-version`.
- `bedrock`: `GET {baseUrl}/foundation-models` (`ListFoundationModels`,
`modelSummaries[].modelId`), the listing the JetBrains gateway answers
on its Bedrock route. Only `anthropic.` ids count. The cross-region
prefix `global` when listed, else that of the first listed id, becomes
the session's `ANTHROPIC_BEDROCK_REGION_PREFIX`, so the CLI calls the
models under the prefix the gateway serves; ids of other prefixes, and
SDK rows resolved to another prefix, are dropped.
- Rows match by `resolvedModel` (or `value`) without dates, `-vN:M`
suffixes, `[1m]` hints and provider prefixes; the narrowing runs after
the `availableModels` allowlist. When the gateway does not list the
model a session runs (the dropped default, or a resumed or recreated
session's last model), the session switches to the first listed one with
`setModel`, so the CLI does not keep running an unlisted model.
- The listing runs in the provider-update queue, so `providers/set` and
`providers/disable` apply in the order they arrive; logout also clears a
`providers/set` that is still listing.
- A 4xx answer means the gateway has no listing: `providers/set`
succeeds and nothing is narrowed. Any other failed listing (network
error, 5xx, unexpected body, 10 s timeout) fails `providers/set` with
`internal_error`, and the previous provider stays in effect. An invalid
header or a base URL with credentials fails without repeating the value.
A listing that matches no SDK model leaves the session with no model to
offer: there is no model config option.
- `vertex`, gateways set through `authenticate`, and native routing are
not listed. Setting the same gateway again reuses its listing;
`providers/disable` and logout drop it.
- `docs/model-configuration.md` describes the narrowing.

### Not covered

Background haiku requests, subagents, and a `/model` typed into the
prompt choose their own model and are not checked against the listing.

### Verification

```bash
npm run check
npm run build
npm run test:run -- --no-file-parallelism   # 2964 passed, 5 failed, 30 skipped; all 5 in session-index.test.ts, flaky on main too
```

Tests stub `fetch` (`src/tests/setup-no-network.ts` answers an empty
listing by default) and cover the listing requests and pagination, the
Bedrock region prefix and its env, region-aware matching, the dropped
default, the failing `providers/set`, and the empty picker. The
JetBrains Bedrock gateway's `/foundation-models` answers
`global.anthropic.claude-{haiku,opus,sonnet}-5-5` (seen through Junie,
which lists the same route); this adapter has not yet been run end to
end against it.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:03:27 +02:00
Ben Brandt
3d4017b15a test: keep the suite off the real Claude CLI (#1223)
Fixes an intermittent failure of "replays a subagent from its own
transcript file", and with it a broader leak: the unit suite was running
the real Claude CLI hundreds of times per run.

## Cause

The agent runs `claude auth status --json` in the background, without
awaiting it, on every `initialize` and prompt. The scenario suite stubs
it (`CLAUDE_CODE_EXECUTABLE=/usr/bin/false`), but other files didn't.
Logging the probe's executable over one `vitest run` showed about 670
runs of the real CLI:

| File | Real CLI runs |
|---|---|
| `acp-agent.test.ts` | 375 |
| `create-session-options.test.ts` | 14 |
| `incomplete-tools.test.ts` | 12 |
| `session-titles.test.ts` | 8 |
| `file-change-audit-integration.test.ts` | 5 |

Each run uses the machine's own Claude config, unless a test points
`CLAUDE_CONFIG_DIR` elsewhere.

That flaky test points `CLAUDE_CONFIG_DIR` at a temp dir. The CLI it
started then wrote `.claude.json`, `.claude.json.lock` and `backups/`
into that dir about a second later, after the test body had finished.
Under full-suite load, that overlapped the `rm` in the test's `finally`,
which failed with `ENOTEMPTY`. Its assertions always passed.

## Change

`vitest.config.ts` sets `CLAUDE_CODE_EXECUTABLE=/usr/bin/false` in
`test.env`:
- It's the override `claudeCliPath()` already honors, and the value the
scenario suite already uses, so production code doesn't change.
- `test.env` is the base environment, so it survives the
`vi.unstubAllEnvs()` that three files call.
- Runs that exercise the real CLI keep it: with `RUN_INTEGRATION_TESTS`
(`npm run test:integration`) or `RUN_LIVE_CAPABILITY_AUDIT=true`, the
variable is left unset.

Tests that mock `execFile` or set their own executable are unaffected.

## Evidence

- After the change, the same logging shows 0 runs of the real CLI: 688
probes run `/usr/bin/false`, and 108 hit files that already mock
`execFile`.
- The flaky test's temp dir now holds only the test's own `projects/`
after the test, with no `.claude.json`.
- Before the change, it failed in about 2 of 6 full runs on my machine.
After, 8 of 8 full runs passed.
- A full run takes about 4.8 s instead of 8.4 s.
- The variable is `/usr/bin/false` in a default run and unset with
either opt-in flag.
- `npm run build`, `npm run check`, `npm run test:run`.
2026-10-02 15:15:00 +02:00
Andrey Bragin
0dd58685b1 ci: set up preview publishing (#1084)
Today nothing reaches npm between two release-please releases, so trying
a merged
but unreleased change means building from source. This adds a
continuously
published preview channel.

Every CI-green push to `main` that is not a release merge publishes
`<package.json version, patch bumped>-preview.N` under the `preview`
dist-tag,
tags the commit `v<version>`, and dispatches the same agent registry
update a
stable release does. With `main` at 0.73.0 that gives
`0.73.1-preview.1`,
`0.73.1-preview.2`, and so on; `N` restarts at 1 whenever a release
moves the
base. `latest` is untouched.

## Approach

- **Added to `publish.yml` rather than a new workflow.** npm binds a
trusted
publisher to one repo + workflow filename, and a package gets only one.
A
  separate file would fail OIDC and cost the stable path its publisher.
- **Gated on the `CI` workflow succeeding** (`workflow_run`), so a red
commit is
never published. release-please's release merge is excluded by the head
commit's author and subject — its `releases_created` output is not
reachable
  from a different workflow run.
- **`N` is the max of the npm registry and local git tags.** npm
reserves
versions forever, even after `unpublish`, so a reused `N` is a hard
failure;
but the registry is CDN-served and lags, which the tags cover. A
registry read
  failure aborts rather than guessing.
- **Base is patch+1, not release-please's prediction**, which moves
mid-flight
  and would reset `N` under already-published previews.
- **The version is applied to the CI working tree only**, never
committed —
`package.json` and the release-please manifest stay release-please's to
own.
- **`--tag preview` is mandatory:** `npm publish` defaults to `latest`
even for a
  semver prerelease.
- **Three jobs.** `publish-npm-preview` mirrors `publish-npm` and only
publishes
(no repo write access); `publish-tag-preview` only tags, so a failed tag
is
retryable with _Re-run failed jobs_ without re-publishing an immutable
version;
`trigger-registry-update` is shared with the stable path. Both
downstream jobs
gate on a `published` output the publish step sets, not on job status —
the tag
push happens after the publish, so a red job can still mean npm has the
  version.

## Next step

The publish is `npm publish --dry-run` for now. The `workflow_run`
trigger and
the `release` environment's protected-branch policy mean this job cannot
run off
the default branch at all, so merging this is the only way to exercise
the
trigger, the gate and the version calculation — and with `published`
unset, the
rehearsal neither tags nor touches the registry.

Once one run looks right, a follow-up drops `--dry-run` and adds
`echo "published=true" >> "$GITHUB_OUTPUT"` to that step. Nothing else
changes.
2026-09-08 14:16:37 +02:00
Ben Brandt
4d9d00ed81 Turn off watch mode by default 2025-08-27 15:41:42 -07:00
Conrad Irwin
dc78ab2eb4 Tests! 2025-08-27 10:59:00 -06:00