mirror of
https://github.com/agentclientprotocol/claude-agent-acp.git
synced 2026-10-27 04:30:59 +00:00
feat: narrow models to what the providers/set gateway lists (#1340)
### Summary Addresses [IJAI-1783](https://youtrack.jetbrains.com/issue/IJAI-1783). An organization can restrict the models its JetBrains gateway serves, but a session routed through `providers/set` offered the whole Claude Code catalogue, so the picker showed models the gateway then refused. `providers/set` now lists the gateway's models before it responds, and the model picker offers only the SDK models the gateway lists. - `anthropic`: `GET {baseUrl}/v1/models?limit=1000`, paginated by `has_more`/`last_id`, with the provider headers and `anthropic-version`. - `bedrock`: `GET {baseUrl}/foundation-models` (`ListFoundationModels`, `modelSummaries[].modelId`), the listing the JetBrains gateway answers on its Bedrock route. Only `anthropic.` ids count. The cross-region prefix `global` when listed, else that of the first listed id, becomes the session's `ANTHROPIC_BEDROCK_REGION_PREFIX`, so the CLI calls the models under the prefix the gateway serves; ids of other prefixes, and SDK rows resolved to another prefix, are dropped. - Rows match by `resolvedModel` (or `value`) without dates, `-vN:M` suffixes, `[1m]` hints and provider prefixes; the narrowing runs after the `availableModels` allowlist. When the gateway does not list the model a session runs (the dropped default, or a resumed or recreated session's last model), the session switches to the first listed one with `setModel`, so the CLI does not keep running an unlisted model. - The listing runs in the provider-update queue, so `providers/set` and `providers/disable` apply in the order they arrive; logout also clears a `providers/set` that is still listing. - A 4xx answer means the gateway has no listing: `providers/set` succeeds and nothing is narrowed. Any other failed listing (network error, 5xx, unexpected body, 10 s timeout) fails `providers/set` with `internal_error`, and the previous provider stays in effect. An invalid header or a base URL with credentials fails without repeating the value. A listing that matches no SDK model leaves the session with no model to offer: there is no model config option. - `vertex`, gateways set through `authenticate`, and native routing are not listed. Setting the same gateway again reuses its listing; `providers/disable` and logout drop it. - `docs/model-configuration.md` describes the narrowing. ### Not covered Background haiku requests, subagents, and a `/model` typed into the prompt choose their own model and are not checked against the listing. ### Verification ```bash npm run check npm run build npm run test:run -- --no-file-parallelism # 2964 passed, 5 failed, 30 skipped; all 5 in session-index.test.ts, flaky on main too ``` Tests stub `fetch` (`src/tests/setup-no-network.ts` answers an empty listing by default) and cover the listing requests and pagination, the Bedrock region prefix and its env, region-aware matching, the dropped default, the failing `providers/set`, and the empty picker. The JetBrains Bedrock gateway's `/foundation-models` answers `global.anthropic.claude-{haiku,opus,sonnet}-5-5` (seen through Junie, which lists the same route); this adapter has not yet been run end to end against it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -54,3 +54,14 @@ An ACP caller can provide `settings` via `_meta.claudeCode.options.settings` in
|
|||||||
- The value must be valid JSON. Invalid JSON will cause session creation to fail with a parse error.
|
- The value must be valid JSON. Invalid JSON will cause session creation to fail with a parse error.
|
||||||
- Only `modelOverrides` and `availableModels` keys are read; other keys in the JSON are ignored.
|
- Only `modelOverrides` and `availableModels` keys are read; other keys in the JSON are ignored.
|
||||||
- Both fields map directly to the Claude Agent SDK's `Settings` type.
|
- Both fields map directly to the Claude Agent SDK's `Settings` type.
|
||||||
|
|
||||||
|
## Models of a `providers/set` gateway
|
||||||
|
|
||||||
|
When an ACP client routes the agent through a gateway with `providers/set`, the adapter lists the gateway's models before `providers/set` returns, with the provider headers:
|
||||||
|
|
||||||
|
- `anthropic`: `GET {baseUrl}/v1/models`, paginated by `has_more`/`last_id`.
|
||||||
|
- `bedrock`: `GET {baseUrl}/foundation-models` (Bedrock `ListFoundationModels`, `modelSummaries[].modelId`). Only `anthropic.` ids count. The cross-region prefix `global` when listed, else that of the first listed id with one (`eu` for `eu.anthropic.claude-sonnet-5`), becomes the session's `ANTHROPIC_BEDROCK_REGION_PREFIX`, and ids with other prefixes, or without one, are dropped.
|
||||||
|
|
||||||
|
The model picker then offers only the SDK models whose resolved model the gateway lists, matched without dates, version suffixes (`-v1:0`), `-fast`, `@` suffixes, context hints (`[1m]`) and provider prefixes, after the `availableModels` allowlist above. On Bedrock, a model the CLI resolved to another region prefix is dropped as well. When the gateway does not list the model a session runs, that is the default model or a resumed session's last model, the session switches to the first remaining picker row.
|
||||||
|
|
||||||
|
A 4xx answer means the gateway has no listing: `providers/set` succeeds and nothing is narrowed. A failed listing (network error, a status other than 2xx or 4xx, unexpected body, the 10 s timeout, an invalid header, or a base URL with credentials) fails `providers/set` with a JSON-RPC internal error, and the previous provider stays in effect. A listing that matches no SDK model leaves the session with no model to offer: there is no model config option. Gateways set through `authenticate`, `vertex` providers, and native routing are not listed. Setting the same gateway again (the same protocol, base URL and headers) reuses its listing, unless it answered 4xx; `providers/set` and `providers/disable` apply in the order they arrive, and logout clears the routing after `claude auth logout`, including a `providers/set` still listing; `providers/disable` and logout drop it.
|
||||||
|
|||||||
130
src/acp-agent.ts
130
src/acp-agent.ts
@@ -320,9 +320,17 @@ import {
|
|||||||
buildModelConfigOption,
|
buildModelConfigOption,
|
||||||
getAvailableModels,
|
getAvailableModels,
|
||||||
MODEL_CONFIG_ID,
|
MODEL_CONFIG_ID,
|
||||||
|
narrowToGatewayModels,
|
||||||
resolveModelPreference,
|
resolveModelPreference,
|
||||||
type SessionModelState,
|
type SessionModelState,
|
||||||
} from "./session-model.js";
|
} from "./session-model.js";
|
||||||
|
import {
|
||||||
|
GatewayListingError,
|
||||||
|
listBedrockModels,
|
||||||
|
listGatewayModels,
|
||||||
|
PROVIDER_AUTH_TOKEN_PLACEHOLDER,
|
||||||
|
selectBedrockRegion,
|
||||||
|
} from "./gateway-models.js";
|
||||||
import {
|
import {
|
||||||
buildEffortConfigOption,
|
buildEffortConfigOption,
|
||||||
EFFORT_CONFIG_ID,
|
EFFORT_CONFIG_ID,
|
||||||
@@ -1730,6 +1738,19 @@ type ProviderConfig = {
|
|||||||
projectId: string;
|
projectId: string;
|
||||||
region: string;
|
region: string;
|
||||||
};
|
};
|
||||||
|
/**
|
||||||
|
* Model ids an `anthropic` or `bedrock` gateway listed when it was set via
|
||||||
|
* `providers/set`; the model picker offers only these. Absent when the
|
||||||
|
* listing failed or the config came from elsewhere (gateway `authenticate`),
|
||||||
|
* so nothing is narrowed.
|
||||||
|
*/
|
||||||
|
listedModels?: string[];
|
||||||
|
/**
|
||||||
|
* The cross-region prefix of the model ids a `bedrock` gateway
|
||||||
|
* listed (`eu` for `eu.anthropic.claude-sonnet-5`), passed to the CLI as
|
||||||
|
* `ANTHROPIC_BEDROCK_REGION_PREFIX` so it calls the models under that prefix.
|
||||||
|
*/
|
||||||
|
bedrockRegionPrefix?: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
export type { ToolUpdateMeta } from "./tool-calls/renderer.js";
|
export type { ToolUpdateMeta } from "./tool-calls/renderer.js";
|
||||||
@@ -2922,7 +2943,7 @@ export class ClaudeAcpAgent {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async newSession(params: NewSessionRequest): Promise<NewSessionResponse> {
|
async newSession(params: NewSessionRequest): Promise<NewSessionResponse> {
|
||||||
if (this.providerUpdate) await this.providerUpdate;
|
while (this.providerUpdate) await this.providerUpdate;
|
||||||
const response = await this.createSession(params, {
|
const response = await this.createSession(params, {
|
||||||
// Revisit these meta values once we support resume
|
// Revisit these meta values once we support resume
|
||||||
resume: (params._meta as NewSessionMeta | undefined)?.claudeCode?.options?.resume,
|
resume: (params._meta as NewSessionMeta | undefined)?.claudeCode?.options?.resume,
|
||||||
@@ -2997,7 +3018,7 @@ export class ClaudeAcpAgent {
|
|||||||
* later load or resume with the same settings reuses it.
|
* later load or resume with the same settings reuses it.
|
||||||
*/
|
*/
|
||||||
async unstable_forkSession(params: ForkSessionRequest): Promise<ForkSessionResponse> {
|
async unstable_forkSession(params: ForkSessionRequest): Promise<ForkSessionResponse> {
|
||||||
if (this.providerUpdate) await this.providerUpdate;
|
while (this.providerUpdate) await this.providerUpdate;
|
||||||
const { sessionId } = await forkSession(params, {
|
const { sessionId } = await forkSession(params, {
|
||||||
liveMessageIdToUuid: this.sessions[params.sessionId]?.messageIdToUuid,
|
liveMessageIdToUuid: this.sessions[params.sessionId]?.messageIdToUuid,
|
||||||
logger: this.logger,
|
logger: this.logger,
|
||||||
@@ -3015,7 +3036,7 @@ export class ClaudeAcpAgent {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async resumeSession(params: ResumeSessionRequest): Promise<ResumeSessionResponse> {
|
async resumeSession(params: ResumeSessionRequest): Promise<ResumeSessionResponse> {
|
||||||
if (this.providerUpdate) await this.providerUpdate;
|
while (this.providerUpdate) await this.providerUpdate;
|
||||||
const result = await this.getOrCreateSession(params);
|
const result = await this.getOrCreateSession(params);
|
||||||
|
|
||||||
this.afterSetupResponse(params.sessionId, params.mcpServers ?? []);
|
this.afterSetupResponse(params.sessionId, params.mcpServers ?? []);
|
||||||
@@ -3024,7 +3045,7 @@ export class ClaudeAcpAgent {
|
|||||||
|
|
||||||
async loadSession(params: LoadSessionRequest): Promise<LoadSessionResponse> {
|
async loadSession(params: LoadSessionRequest): Promise<LoadSessionResponse> {
|
||||||
const timing = new SessionTiming(this.logger, "load", params.sessionId);
|
const timing = new SessionTiming(this.logger, "load", params.sessionId);
|
||||||
if (this.providerUpdate) await this.providerUpdate;
|
while (this.providerUpdate) await this.providerUpdate;
|
||||||
let result: NewSessionResponse;
|
let result: NewSessionResponse;
|
||||||
if (this.sessions[params.sessionId]) {
|
if (this.sessions[params.sessionId]) {
|
||||||
const resumedSession = await readResumedSession(params.sessionId, this.logger);
|
const resumedSession = await readResumedSession(params.sessionId, this.logger);
|
||||||
@@ -3458,6 +3479,48 @@ export class ClaudeAcpAgent {
|
|||||||
return {};
|
return {};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The model ids the gateway in `config` lists, read before `providers/set`
|
||||||
|
* returns so the sessions it recreates already offer only those, and for
|
||||||
|
* `bedrock` the region prefix picked from them. Setting the same gateway
|
||||||
|
* again reuses the previous listing. `vertex` gateways are not listed.
|
||||||
|
* Throws a {@link GatewayListingError} when the listing fails.
|
||||||
|
*/
|
||||||
|
private async listProviderModels(
|
||||||
|
config: ProviderConfig,
|
||||||
|
): Promise<Pick<ProviderConfig, "listedModels" | "bedrockRegionPrefix">> {
|
||||||
|
// Reuse needs the very same headers, so a rotated token lists again, and a
|
||||||
|
// failed listing then fails that `providers/set`. Deliberate: AIR keeps its
|
||||||
|
// proxy key for the IDE's lifetime and does not rotate it.
|
||||||
|
const current = this.providerConfig;
|
||||||
|
if (
|
||||||
|
current?.listedModels &&
|
||||||
|
current.apiType === config.apiType &&
|
||||||
|
current.baseUrl === config.baseUrl &&
|
||||||
|
sameHeaders(current.headers, config.headers)
|
||||||
|
) {
|
||||||
|
return {
|
||||||
|
listedModels: current.listedModels,
|
||||||
|
bedrockRegionPrefix: current.bedrockRegionPrefix,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (config.apiType === "anthropic") {
|
||||||
|
return { listedModels: await listGatewayModels(config.baseUrl, config.headers, this.logger) };
|
||||||
|
}
|
||||||
|
if (config.apiType === "bedrock") {
|
||||||
|
const modelIds = await listBedrockModels(config.baseUrl, config.headers, this.logger);
|
||||||
|
if (!modelIds) return {};
|
||||||
|
const region = selectBedrockRegion(modelIds);
|
||||||
|
if (region.modelIds.length < modelIds.length) {
|
||||||
|
this.logger.log(
|
||||||
|
`[providers/set] Bedrock models span several region prefixes; using ${region.regionPrefix} and dropping ${modelIds.length - region.modelIds.length} id(s) of other prefixes`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return { listedModels: region.modelIds, bedrockRegionPrefix: region.regionPrefix };
|
||||||
|
}
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
|
||||||
resolveProviderConfig(): ProviderConfig | null {
|
resolveProviderConfig(): ProviderConfig | null {
|
||||||
return this.providerConfig ?? gatewayRequestToProviderConfig(this.gatewayAuthRequest);
|
return this.providerConfig ?? gatewayRequestToProviderConfig(this.gatewayAuthRequest);
|
||||||
}
|
}
|
||||||
@@ -3493,7 +3556,10 @@ export class ClaudeAcpAgent {
|
|||||||
// provider routing set via `providers/set`. Neither touches the on-disk
|
// provider routing set via `providers/set`. Neither touches the on-disk
|
||||||
// credential store, so dropping these references is the whole logout for
|
// credential store, so dropping these references is the whole logout for
|
||||||
// those paths.
|
// those paths.
|
||||||
const hadRoutingOverride = this.resolveProviderConfig() !== null;
|
// A queued `providers/set` may still be listing and apply after this point,
|
||||||
|
// so a busy queue counts as an override to clear.
|
||||||
|
const hadRoutingOverride =
|
||||||
|
this.resolveProviderConfig() !== null || this.providerUpdate !== null;
|
||||||
this.gatewayAuthRequest = undefined;
|
this.gatewayAuthRequest = undefined;
|
||||||
this.providerConfig = undefined;
|
this.providerConfig = undefined;
|
||||||
// Any probe already running read the pre-logout world; the bump makes its
|
// Any probe already running read the pre-logout world; the bump makes its
|
||||||
@@ -3606,7 +3672,7 @@ export class ClaudeAcpAgent {
|
|||||||
* before this resolves.
|
* before this resolves.
|
||||||
*/
|
*/
|
||||||
async startTurn(params: PromptRequest, events: TurnEvents): Promise<void> {
|
async startTurn(params: PromptRequest, events: TurnEvents): Promise<void> {
|
||||||
if (this.providerUpdate) await this.providerUpdate;
|
while (this.providerUpdate) await this.providerUpdate;
|
||||||
let session = this.sessions[params.sessionId];
|
let session = this.sessions[params.sessionId];
|
||||||
if (!session) {
|
if (!session) {
|
||||||
throw new Error("Session not found");
|
throw new Error("Session not found");
|
||||||
@@ -10980,7 +11046,7 @@ export class ClaudeAcpAgent {
|
|||||||
// configured. Managed `deniedModels` drops entries the CLI would refuse.
|
// configured. Managed `deniedModels` drops entries the CLI would refuse.
|
||||||
const settingsAvailableModels = settingsManager.getSettings().availableModels;
|
const settingsAvailableModels = settingsManager.getSettings().availableModels;
|
||||||
const settingsModelOverrides = settingsManager.getSettings().modelOverrides;
|
const settingsModelOverrides = settingsManager.getSettings().modelOverrides;
|
||||||
const allowedModels = Array.isArray(settingsAvailableModels)
|
const settingsAllowedModels = Array.isArray(settingsAvailableModels)
|
||||||
? applyAvailableModelsAllowlist(
|
? applyAvailableModelsAllowlist(
|
||||||
initializationResult.models,
|
initializationResult.models,
|
||||||
settingsAvailableModels,
|
settingsAvailableModels,
|
||||||
@@ -10988,6 +11054,14 @@ export class ClaudeAcpAgent {
|
|||||||
settingsManager.getManagedDeniedModels(),
|
settingsManager.getManagedDeniedModels(),
|
||||||
)
|
)
|
||||||
: initializationResult.models;
|
: initializationResult.models;
|
||||||
|
// Then keep only the models the `providers/set` gateway listed.
|
||||||
|
const allowedModels = resolvedProvider?.listedModels
|
||||||
|
? narrowToGatewayModels(
|
||||||
|
settingsAllowedModels,
|
||||||
|
resolvedProvider.listedModels,
|
||||||
|
resolvedProvider.bedrockRegionPrefix,
|
||||||
|
)
|
||||||
|
: settingsAllowedModels;
|
||||||
|
|
||||||
const models = await getAvailableModels(
|
const models = await getAvailableModels(
|
||||||
q,
|
q,
|
||||||
@@ -10998,6 +11072,7 @@ export class ClaudeAcpAgent {
|
|||||||
creationOpts.resume !== undefined,
|
creationOpts.resume !== undefined,
|
||||||
sessionId,
|
sessionId,
|
||||||
await resumedModelHint,
|
await resumedModelHint,
|
||||||
|
resolvedProvider?.listedModels !== undefined,
|
||||||
);
|
);
|
||||||
timing.phase("models");
|
timing.phase("models");
|
||||||
|
|
||||||
@@ -11218,6 +11293,25 @@ export class ClaudeAcpAgent {
|
|||||||
private async enqueueProviderUpdate(config: ProviderConfig | undefined): Promise<void> {
|
private async enqueueProviderUpdate(config: ProviderConfig | undefined): Promise<void> {
|
||||||
const previous = this.providerUpdate?.catch(() => undefined) ?? Promise.resolve();
|
const previous = this.providerUpdate?.catch(() => undefined) ?? Promise.resolve();
|
||||||
const update = previous.then(async () => {
|
const update = previous.then(async () => {
|
||||||
|
// The listing runs in the queue, so `providers/set`, `providers/disable`
|
||||||
|
// and logout apply in the order they arrived. A failed listing fails this
|
||||||
|
// `providers/set` before the provider replaces the current one.
|
||||||
|
if (config) {
|
||||||
|
try {
|
||||||
|
Object.assign(config, await this.listProviderModels(config));
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof GatewayListingError) {
|
||||||
|
throw RequestError.internalError(
|
||||||
|
{ providerId: PROVIDER_ID, apiType: config.apiType, baseUrl: config.baseUrl },
|
||||||
|
`Model discovery for provider "${PROVIDER_ID}" failed: ${error.message}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
this.logger.log(
|
||||||
|
`[providers/set] listedModels=${config.listedModels?.length ?? "none"} bedrockRegionPrefix=${config.bedrockRegionPrefix ?? "none"}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
const sessions = Object.entries(this.sessions);
|
const sessions = Object.entries(this.sessions);
|
||||||
const activeTurns = sessions.flatMap(([, session]) =>
|
const activeTurns = sessions.flatMap(([, session]) =>
|
||||||
(session.turnQueue ?? []).flatMap((turn) => (turn.completion ? [turn.completion] : [])),
|
(session.turnQueue ?? []).flatMap((turn) => (turn.completion ? [turn.completion] : [])),
|
||||||
@@ -11733,9 +11827,12 @@ function createEnvForProvider(config: ProviderConfig | null): Record<string, str
|
|||||||
return {
|
return {
|
||||||
...resetRouting,
|
...resetRouting,
|
||||||
CLAUDE_CODE_USE_BEDROCK: "1",
|
CLAUDE_CODE_USE_BEDROCK: "1",
|
||||||
AWS_BEARER_TOKEN_BEDROCK: "acp-proxy", // Bypass local AWS credential checks
|
AWS_BEARER_TOKEN_BEDROCK: PROVIDER_AUTH_TOKEN_PLACEHOLDER, // Bypass local AWS credential checks
|
||||||
ANTHROPIC_BEDROCK_BASE_URL: config.baseUrl,
|
ANTHROPIC_BEDROCK_BASE_URL: config.baseUrl,
|
||||||
ANTHROPIC_CUSTOM_HEADERS: customHeaders,
|
ANTHROPIC_CUSTOM_HEADERS: customHeaders,
|
||||||
|
...(config.bedrockRegionPrefix
|
||||||
|
? { ANTHROPIC_BEDROCK_REGION_PREFIX: config.bedrockRegionPrefix }
|
||||||
|
: {}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -11762,10 +11859,15 @@ function createEnvForProvider(config: ProviderConfig | null): Record<string, str
|
|||||||
...resetRouting,
|
...resetRouting,
|
||||||
ANTHROPIC_BASE_URL: config.baseUrl,
|
ANTHROPIC_BASE_URL: config.baseUrl,
|
||||||
ANTHROPIC_CUSTOM_HEADERS: customHeaders,
|
ANTHROPIC_CUSTOM_HEADERS: customHeaders,
|
||||||
ANTHROPIC_AUTH_TOKEN: "acp-proxy", // Bypass local Claude login checks
|
ANTHROPIC_AUTH_TOKEN: PROVIDER_AUTH_TOKEN_PLACEHOLDER, // Bypass local Claude login checks
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function sameHeaders(a: Record<string, string>, b: Record<string, string>): boolean {
|
||||||
|
const aKeys = Object.keys(a);
|
||||||
|
return aKeys.length === Object.keys(b).length && aKeys.every((key) => b[key] === a[key]);
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Validate a provider base URL: must be a non-empty absolute http(s) URL.
|
* Validate a provider base URL: must be a non-empty absolute http(s) URL.
|
||||||
*/
|
*/
|
||||||
@@ -11970,10 +12072,14 @@ export function buildConfigOptions(
|
|||||||
fastMode?: FastModeOptionState,
|
fastMode?: FastModeOptionState,
|
||||||
presentation?: ConfigOptionPresentation,
|
presentation?: ConfigOptionPresentation,
|
||||||
): SessionConfigOption[] {
|
): SessionConfigOption[] {
|
||||||
const options: SessionConfigOption[] = [
|
const options: SessionConfigOption[] = [SessionModeManager.configOption(modes)];
|
||||||
SessionModeManager.configOption(modes),
|
// A gateway that lists none of the SDK's models leaves no model to offer, so
|
||||||
|
// there is no model option at all.
|
||||||
|
if (models.availableModels.length > 0) {
|
||||||
|
options.push(
|
||||||
buildModelConfigOption(models, modelInfos, presentation?.useRecommendedValue === true),
|
buildModelConfigOption(models, modelInfos, presentation?.useRecommendedValue === true),
|
||||||
];
|
);
|
||||||
|
}
|
||||||
const effort = buildEffortConfigOption(
|
const effort = buildEffortConfigOption(
|
||||||
modelInfos,
|
modelInfos,
|
||||||
models.currentModelId,
|
models.currentModelId,
|
||||||
|
|||||||
223
src/gateway-models.ts
Normal file
223
src/gateway-models.ts
Normal file
@@ -0,0 +1,223 @@
|
|||||||
|
/**
|
||||||
|
* Model discovery for a gateway set via `providers/set`.
|
||||||
|
*
|
||||||
|
* An organization can restrict which models its gateway serves, and the
|
||||||
|
* gateway's listing returns only those: `GET /v1/models` for an `anthropic`
|
||||||
|
* gateway, `GET /foundation-models` (Bedrock `ListFoundationModels`) for a
|
||||||
|
* `bedrock` one. The adapter lists them once when the provider is set, and the
|
||||||
|
* model picker then offers only the SDK models the gateway listed (see
|
||||||
|
* `narrowToGatewayModels` in `session-model.ts`). A listing that fails fails
|
||||||
|
* `providers/set` with a {@link GatewayListingError}.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const LISTING_TIMEOUT_MS = 10_000;
|
||||||
|
const ANTHROPIC_PAGE_LIMIT = "1000";
|
||||||
|
const MAX_LISTING_PAGES = 20;
|
||||||
|
const MAX_LOGGED_BODY_CHARS = 500;
|
||||||
|
const ANTHROPIC_VERSION = "2023-06-01";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The bearer token `createEnvForProvider` sets as `ANTHROPIC_AUTH_TOKEN` and
|
||||||
|
* `AWS_BEARER_TOKEN_BEDROCK`, so the CLI sends `Authorization: Bearer acp-proxy`
|
||||||
|
* unless the provider headers replace it. The listing sends the same, so the
|
||||||
|
* gateway sees the same auth as on chat requests.
|
||||||
|
*/
|
||||||
|
export const PROVIDER_AUTH_TOKEN_PLACEHOLDER = "acp-proxy";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The cross-region inference profile prefixes the CLI knows:
|
||||||
|
* `eu.anthropic.claude-sonnet-5`. `ANTHROPIC_BEDROCK_REGION_PREFIX` accepts all
|
||||||
|
* of them but `us-gov`, which the CLI takes only from a `us-gov-*` AWS region;
|
||||||
|
* a gateway listing only `us-gov.*` ids would then get an empty picker. That
|
||||||
|
* case is theoretical, so it is left as is.
|
||||||
|
*/
|
||||||
|
const BEDROCK_REGION_PREFIXES = ["us", "eu", "apac", "jp", "au", "us-gov", "global"];
|
||||||
|
|
||||||
|
type ListingLogger = { log: (...args: any[]) => void; error: (...args: any[]) => void };
|
||||||
|
|
||||||
|
type ListingPage = { ids: string[]; nextQuery?: Record<string, string> };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A gateway listing that failed: a network error, a status other than 2xx or
|
||||||
|
* 4xx, a body that is not the expected listing, or the 10 s timeout. The
|
||||||
|
* message names the request path and the reason, never a header value.
|
||||||
|
*/
|
||||||
|
export class GatewayListingError extends Error {
|
||||||
|
override name = "GatewayListingError";
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Lists the model ids an `anthropic` gateway at `baseUrl` serves, following
|
||||||
|
* `has_more` / `last_id` pagination. Returns `undefined` when the gateway
|
||||||
|
* answers 4xx: it has no listing, and nothing is narrowed. Throws a
|
||||||
|
* {@link GatewayListingError} when the listing fails. Header values are never
|
||||||
|
* logged.
|
||||||
|
*/
|
||||||
|
export async function listGatewayModels(
|
||||||
|
baseUrl: string,
|
||||||
|
providerHeaders: Record<string, string>,
|
||||||
|
logger: ListingLogger,
|
||||||
|
): Promise<string[] | undefined> {
|
||||||
|
return listPages(
|
||||||
|
`${baseUrl.replace(/\/+$/, "")}/v1/models`,
|
||||||
|
{ limit: ANTHROPIC_PAGE_LIMIT },
|
||||||
|
listingHeaders(providerHeaders, { "anthropic-version": ANTHROPIC_VERSION }),
|
||||||
|
logger,
|
||||||
|
(body) => {
|
||||||
|
const page = body as { data?: unknown; has_more?: unknown; last_id?: unknown };
|
||||||
|
if (!Array.isArray(page.data)) return undefined;
|
||||||
|
return {
|
||||||
|
ids: stringField(page.data, "id"),
|
||||||
|
nextQuery:
|
||||||
|
page.has_more === true && typeof page.last_id === "string"
|
||||||
|
? { limit: ANTHROPIC_PAGE_LIMIT, after_id: page.last_id }
|
||||||
|
: undefined,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Lists the Anthropic models a `bedrock` gateway at `baseUrl` serves
|
||||||
|
* (`GET /foundation-models`, Bedrock `ListFoundationModels`, one page), the
|
||||||
|
* listing AI Router answers on its Bedrock route. Returns the listed ids, such
|
||||||
|
* as `global.anthropic.claude-sonnet-5-5`; `undefined` and the errors are
|
||||||
|
* those of {@link listGatewayModels}.
|
||||||
|
*/
|
||||||
|
export async function listBedrockModels(
|
||||||
|
baseUrl: string,
|
||||||
|
providerHeaders: Record<string, string>,
|
||||||
|
logger: ListingLogger,
|
||||||
|
): Promise<string[] | undefined> {
|
||||||
|
const ids = await listPages(
|
||||||
|
`${baseUrl.replace(/\/+$/, "")}/foundation-models`,
|
||||||
|
{},
|
||||||
|
listingHeaders(providerHeaders, {}),
|
||||||
|
logger,
|
||||||
|
(body) => {
|
||||||
|
const page = body as { modelSummaries?: unknown };
|
||||||
|
if (!Array.isArray(page.modelSummaries)) return undefined;
|
||||||
|
return { ids: stringField(page.modelSummaries, "modelId") };
|
||||||
|
},
|
||||||
|
);
|
||||||
|
return ids?.filter((id) => id.includes("anthropic."));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The cross-region prefix of a Bedrock model id or ARN, if it has one. */
|
||||||
|
export function bedrockRegionPrefix(modelId: string): string | undefined {
|
||||||
|
const id = modelId.slice(modelId.lastIndexOf("/") + 1).toLowerCase();
|
||||||
|
return BEDROCK_REGION_PREFIXES.find((prefix) => id.startsWith(`${prefix}.anthropic.`));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Picks the region of the listed Bedrock models: `global` when the gateway
|
||||||
|
* lists it, else the prefix of the first id that has one. A gateway that lists
|
||||||
|
* `eu.*` and `us.*` ids gets the prefix that comes first, and the ids of the
|
||||||
|
* other prefixes are dropped, so the picker offers only models the CLI can
|
||||||
|
* call under that prefix. Ids without a prefix (`anthropic.claude-…`) are
|
||||||
|
* dropped too once a prefix is picked; without any prefix every id is kept.
|
||||||
|
*/
|
||||||
|
export function selectBedrockRegion(modelIds: readonly string[]): {
|
||||||
|
regionPrefix?: string;
|
||||||
|
modelIds: string[];
|
||||||
|
} {
|
||||||
|
const prefixes = modelIds.map(bedrockRegionPrefix);
|
||||||
|
const regionPrefix = prefixes.includes("global")
|
||||||
|
? "global"
|
||||||
|
: prefixes.find((prefix) => prefix !== undefined);
|
||||||
|
if (!regionPrefix) return { modelIds: [...modelIds] };
|
||||||
|
return {
|
||||||
|
regionPrefix,
|
||||||
|
modelIds: modelIds.filter((id) => bedrockRegionPrefix(id) === regionPrefix),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// Provider headers win over the defaults, matched case-insensitively, the way
|
||||||
|
// `ANTHROPIC_CUSTOM_HEADERS` overrides the CLI's own headers.
|
||||||
|
function listingHeaders(
|
||||||
|
providerHeaders: Record<string, string>,
|
||||||
|
protocolHeaders: Record<string, string>,
|
||||||
|
): Headers {
|
||||||
|
const headers = new Headers({
|
||||||
|
authorization: `Bearer ${PROVIDER_AUTH_TOKEN_PLACEHOLDER}`,
|
||||||
|
accept: "application/json",
|
||||||
|
...protocolHeaders,
|
||||||
|
});
|
||||||
|
for (const [name, value] of Object.entries(providerHeaders)) {
|
||||||
|
// `Headers.set` puts an invalid value into its error message, so name only the header.
|
||||||
|
try {
|
||||||
|
headers.set(name, value);
|
||||||
|
} catch {
|
||||||
|
throw new GatewayListingError(`Model listing cannot send the invalid header "${name}"`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return headers;
|
||||||
|
}
|
||||||
|
|
||||||
|
function stringField(entries: unknown[], field: string): string[] {
|
||||||
|
return entries.flatMap((entry) => {
|
||||||
|
const value = (entry as Record<string, unknown> | null)?.[field];
|
||||||
|
return typeof value === "string" && value.trim() !== "" ? [value] : [];
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reads `endpoint` page by page; `parse` returns `undefined` for a body it does
|
||||||
|
* not understand. A 4xx answer on any page means the gateway has no listing:
|
||||||
|
* the result is `undefined`, and nothing is narrowed.
|
||||||
|
*/
|
||||||
|
async function listPages(
|
||||||
|
endpoint: string,
|
||||||
|
firstQuery: Record<string, string>,
|
||||||
|
headers: Headers,
|
||||||
|
logger: ListingLogger,
|
||||||
|
parse: (body: unknown) => ListingPage | undefined,
|
||||||
|
): Promise<string[] | undefined> {
|
||||||
|
const { pathname: path, username, password } = new URL(endpoint);
|
||||||
|
const failure = (reason: string): GatewayListingError => {
|
||||||
|
const error = new GatewayListingError(`Model listing GET ${path} ${reason}`);
|
||||||
|
logger.error(`[providers/set] ${error.message}`);
|
||||||
|
return error;
|
||||||
|
};
|
||||||
|
// fetch refuses such a URL and repeats it, credentials included, in its error.
|
||||||
|
if (username || password) throw failure("cannot be sent: the base URL carries credentials");
|
||||||
|
const signal = AbortSignal.timeout(LISTING_TIMEOUT_MS);
|
||||||
|
const ids: string[] = [];
|
||||||
|
let query: Record<string, string> | undefined = firstQuery;
|
||||||
|
for (let page = 0; page < MAX_LISTING_PAGES && query; page++) {
|
||||||
|
const url = new URL(endpoint);
|
||||||
|
for (const [name, value] of Object.entries(query)) url.searchParams.set(name, value);
|
||||||
|
let response: Response;
|
||||||
|
let body: string;
|
||||||
|
try {
|
||||||
|
response = await fetch(url, { headers, signal });
|
||||||
|
body = await response.text();
|
||||||
|
} catch (error) {
|
||||||
|
throw failure(`failed: ${error instanceof Error ? error.message : String(error)}`);
|
||||||
|
}
|
||||||
|
if (response.status >= 400 && response.status < 500) {
|
||||||
|
logger.log(
|
||||||
|
`[providers/set] Model listing GET ${path} returned ${response.status}: the gateway has no listing, models are not narrowed`,
|
||||||
|
);
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
if (!response.ok) {
|
||||||
|
throw failure(`returned ${response.status}: ${body.slice(0, MAX_LOGGED_BODY_CHARS)}`);
|
||||||
|
}
|
||||||
|
let parsed: ListingPage | undefined;
|
||||||
|
try {
|
||||||
|
parsed = parse(JSON.parse(body));
|
||||||
|
} catch {
|
||||||
|
parsed = undefined;
|
||||||
|
}
|
||||||
|
if (!parsed) throw failure("returned an unexpected body");
|
||||||
|
ids.push(...parsed.ids);
|
||||||
|
query = parsed.nextQuery;
|
||||||
|
}
|
||||||
|
if (query) {
|
||||||
|
logger.error(
|
||||||
|
`[providers/set] Model listing GET ${path} still had more pages after ${MAX_LISTING_PAGES}; using what was read`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return ids;
|
||||||
|
}
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
import type { SessionConfigOption } from "@agentclientprotocol/sdk";
|
import type { SessionConfigOption } from "@agentclientprotocol/sdk";
|
||||||
import type { ModelInfo, Query, Settings } from "@anthropic-ai/claude-agent-sdk";
|
import type { ModelInfo, Query, Settings } from "@anthropic-ai/claude-agent-sdk";
|
||||||
import { AIR_RECOMMENDED_CONFIG_VALUE_CAPABILITY, withAirMeta } from "./air-extension.js";
|
import { AIR_RECOMMENDED_CONFIG_VALUE_CAPABILITY, withAirMeta } from "./air-extension.js";
|
||||||
|
import { bedrockRegionPrefix } from "./gateway-models.js";
|
||||||
|
|
||||||
export const MODEL_CONFIG_ID = "model";
|
export const MODEL_CONFIG_ID = "model";
|
||||||
|
|
||||||
@@ -262,9 +263,10 @@ function resolveSettingsModel(
|
|||||||
|
|
||||||
const MODEL_FAMILY_ALIASES = new Set(["opus", "sonnet", "haiku", "fable"]);
|
const MODEL_FAMILY_ALIASES = new Set(["opus", "sonnet", "haiku", "fable"]);
|
||||||
|
|
||||||
/** A model id without the spellings `deniedModels` ignores: provider
|
/** A model id without the spellings that name the same model: provider
|
||||||
* prefixes, dates, version suffixes, `-fast` and context hints. */
|
* prefixes, dates, version suffixes, `-fast` and context hints. `deniedModels`
|
||||||
function normalizeDeniedModelId(value: string): string {
|
* and gateway listings both match on it. */
|
||||||
|
function normalizeModelId(value: string): string {
|
||||||
let id = stripContextHints(value.trim().toLowerCase());
|
let id = stripContextHints(value.trim().toLowerCase());
|
||||||
const providerPrefix = id.lastIndexOf("anthropic.");
|
const providerPrefix = id.lastIndexOf("anthropic.");
|
||||||
if (providerPrefix !== -1) id = id.slice(providerPrefix + "anthropic.".length);
|
if (providerPrefix !== -1) id = id.slice(providerPrefix + "anthropic.".length);
|
||||||
@@ -283,10 +285,10 @@ function normalizeDeniedModelId(value: string): string {
|
|||||||
* the enforcer; this only keeps the picker from offering a refused row.
|
* the enforcer; this only keeps the picker from offering a refused row.
|
||||||
*/
|
*/
|
||||||
export function isDeniedModel(modelId: string, deniedModels: readonly string[]): boolean {
|
export function isDeniedModel(modelId: string, deniedModels: readonly string[]): boolean {
|
||||||
const id = normalizeDeniedModelId(modelId);
|
const id = normalizeModelId(modelId);
|
||||||
if (!id) return false;
|
if (!id) return false;
|
||||||
return deniedModels.some((entry) => {
|
return deniedModels.some((entry) => {
|
||||||
const denied = normalizeDeniedModelId(entry);
|
const denied = normalizeModelId(entry);
|
||||||
if (!denied) return false;
|
if (!denied) return false;
|
||||||
if (MODEL_FAMILY_ALIASES.has(denied)) return id === denied || id.includes(`-${denied}`);
|
if (MODEL_FAMILY_ALIASES.has(denied)) return id === denied || id.includes(`-${denied}`);
|
||||||
return id === denied || id.startsWith(`${denied}-`);
|
return id === denied || id.startsWith(`${denied}-`);
|
||||||
@@ -379,6 +381,48 @@ export function applyAvailableModelsAllowlist(
|
|||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keep the SDK models the `providers/set` gateway listed (`listGatewayModels`,
|
||||||
|
* `listBedrockModels`). A row matches when its `resolvedModel` (or
|
||||||
|
* its `value` when the SDK gives none) names a listed model in any spelling:
|
||||||
|
* `claude-sonnet-5[1m]` matches a listed `claude-sonnet-5-20260101`, and
|
||||||
|
* `eu.anthropic.claude-sonnet-5` matches a listed `eu.anthropic.claude-sonnet-5-v1:0`.
|
||||||
|
*
|
||||||
|
* With a Bedrock `regionPrefix`, a row resolved to another cross-region prefix
|
||||||
|
* (`us.anthropic.claude-sonnet-5` under `eu`) is dropped even when the same
|
||||||
|
* model is listed under `eu`: the CLI would call it in the other region.
|
||||||
|
*
|
||||||
|
* A row the SDK gives no `resolvedModel` names its model only by an alias, so
|
||||||
|
* it is checked as far as the alias allows: `default` is kept, and a family
|
||||||
|
* alias such as `opus` is kept when the gateway lists a model of that family.
|
||||||
|
*
|
||||||
|
* When no row matches by its model (a `default` row kept only for lacking a
|
||||||
|
* `resolvedModel` does not count), the gateway serves none of the SDK's
|
||||||
|
* models, and nothing is left: the session offers no model.
|
||||||
|
*/
|
||||||
|
export function narrowToGatewayModels(
|
||||||
|
models: ModelInfo[],
|
||||||
|
listedModelIds: readonly string[],
|
||||||
|
regionPrefix?: string,
|
||||||
|
): ModelInfo[] {
|
||||||
|
const listed = new Set(listedModelIds.map(normalizeModelId).filter(Boolean));
|
||||||
|
const narrowed = models.filter((model) => {
|
||||||
|
const modelId = model.resolvedModel ?? model.value;
|
||||||
|
const prefix = bedrockRegionPrefix(modelId);
|
||||||
|
if (regionPrefix !== undefined && prefix !== undefined && prefix !== regionPrefix) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
const id = normalizeModelId(modelId);
|
||||||
|
if (listed.has(id)) return true;
|
||||||
|
if (model.resolvedModel !== undefined) return false;
|
||||||
|
if (model.value === "default") return true;
|
||||||
|
return MODEL_FAMILY_ALIASES.has(id) && [...listed].some((entry) => entry.includes(`-${id}`));
|
||||||
|
});
|
||||||
|
return narrowed.some((model) => listed.has(normalizeModelId(model.resolvedModel ?? model.value)))
|
||||||
|
? narrowed
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
|
||||||
/** Whether a rejected `setModel` was vetoed by a user-configured
|
/** Whether a rejected `setModel` was vetoed by a user-configured
|
||||||
* PreModelSwitch hook. The CLI's rejection message is the stable,
|
* PreModelSwitch hook. The CLI's rejection message is the stable,
|
||||||
* distinguishable marker ("Model switch blocked by a PreModelSwitch hook:
|
* distinguishable marker ("Model switch blocked by a PreModelSwitch hook:
|
||||||
@@ -397,7 +441,18 @@ export async function getAvailableModels(
|
|||||||
isResumedSession: boolean,
|
isResumedSession: boolean,
|
||||||
sessionId: string,
|
sessionId: string,
|
||||||
resumedModelHint?: string,
|
resumedModelHint?: string,
|
||||||
|
narrowedByGateway = false,
|
||||||
): Promise<SessionModelState> {
|
): Promise<SessionModelState> {
|
||||||
|
// A gateway that lists none of the SDK's models leaves none to offer
|
||||||
|
// (`narrowToGatewayModels`). The session stays on the model the SDK runs,
|
||||||
|
// and the gateway refuses its requests.
|
||||||
|
if (models.length === 0) {
|
||||||
|
return {
|
||||||
|
availableModels: [],
|
||||||
|
currentModelId: (isResumedSession && resumedModelHint) || sdkModels[0]?.value || "default",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
const settings = settingsManager.getSettings();
|
const settings = settingsManager.getSettings();
|
||||||
|
|
||||||
let currentModel = models[0];
|
let currentModel = models[0];
|
||||||
@@ -444,9 +499,34 @@ export async function getAvailableModels(
|
|||||||
// be re-asserted to keep the reported model truthful.
|
// be re-asserted to keep the reported model truthful.
|
||||||
// Anything else (fuzzy match, allowlist-synthesized value, alias) gets a
|
// Anything else (fuzzy match, allowlist-synthesized value, alias) gets a
|
||||||
// setModel call so we don't drift from the user's intended pin.
|
// setModel call so we don't drift from the user's intended pin.
|
||||||
|
// Case (a) does not hold when `narrowToGatewayModels` left the SDK on a
|
||||||
|
// model the gateway does not list: the dropped `default` row of a fresh or
|
||||||
|
// transcript-less session, or a resumed transcript model outside the list.
|
||||||
|
// Such a session switches to models[0].
|
||||||
const sdkSawSameValue = sdkModels.some((m) => m.value === currentModel.value);
|
const sdkSawSameValue = sdkModels.some((m) => m.value === currentModel.value);
|
||||||
|
const sdkDefault = sdkModels.find((m) => m.value === "default");
|
||||||
|
const sdkDefaultDropped = sdkDefault !== undefined && !models.some((m) => m.value === "default");
|
||||||
|
let switchToListed = false;
|
||||||
|
if (resolvedFromInput === undefined) {
|
||||||
|
if (!isResumedSession || !resumedModelHint) {
|
||||||
|
switchToListed = sdkDefaultDropped;
|
||||||
|
} else if (
|
||||||
|
narrowedByGateway &&
|
||||||
|
(!models.some((m) => m.value === currentModel.value) ||
|
||||||
|
// A loose match can map the transcript's model onto a listed row of
|
||||||
|
// another version, which the CLI does not resume on.
|
||||||
|
(currentModel.resolvedModel !== undefined &&
|
||||||
|
normalizeModelId(currentModel.resolvedModel) !== normalizeModelId(resumedModelHint)))
|
||||||
|
) {
|
||||||
|
currentModel = models[0];
|
||||||
|
switchToListed = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// When the switch fails, the SDK stays on the model it ran before it.
|
||||||
|
const modelBeforeSwitch = (): ModelInfo =>
|
||||||
|
switchToListed && sdkDefault !== undefined ? sdkDefault : models[0];
|
||||||
const skipSetModel =
|
const skipSetModel =
|
||||||
resolvedFromInput === undefined ||
|
(resolvedFromInput === undefined && !switchToListed) ||
|
||||||
(!isResumedSession && currentModel.value === resolvedFromInput && sdkSawSameValue);
|
(!isResumedSession && currentModel.value === resolvedFromInput && sdkSawSameValue);
|
||||||
if (!skipSetModel) {
|
if (!skipSetModel) {
|
||||||
const setModelStartedAt = performance.now();
|
const setModelStartedAt = performance.now();
|
||||||
@@ -478,15 +558,19 @@ export async function getAvailableModels(
|
|||||||
// live model back on this path: getContextUsage isn't serviced on a
|
// live model back on this path: getContextUsage isn't serviced on a
|
||||||
// fresh session until the first prompt turn (issues #886/#880).
|
// fresh session until the first prompt turn (issues #886/#880).
|
||||||
if (!isResumedSession) {
|
if (!isResumedSession) {
|
||||||
if (!isPreModelSwitchHookBlock(error)) throw error;
|
// A switch onto a listed model is advisory too: the session stays on
|
||||||
|
// the model the SDK runs and reports it.
|
||||||
|
if (!switchToListed && !isPreModelSwitchHookBlock(error)) throw error;
|
||||||
logger.error(
|
logger.error(
|
||||||
`Model pin "${currentModel.value}" was vetoed by a PreModelSwitch hook; staying on the default model:`,
|
`Model switch to "${currentModel.value}" failed or was vetoed by a PreModelSwitch hook; staying on the default model:`,
|
||||||
error,
|
error,
|
||||||
);
|
);
|
||||||
currentModel = models[0];
|
currentModel = modelBeforeSwitch();
|
||||||
} else {
|
} else {
|
||||||
logger.error(`Failed to re-assert model "${currentModel.value}" on resume:`, error);
|
logger.error(`Failed to re-assert model "${currentModel.value}" on resume:`, error);
|
||||||
currentModel = resumedModelHint ? matchResumedModel(models, resumedModelHint) : models[0];
|
currentModel = resumedModelHint
|
||||||
|
? matchResumedModel(models, resumedModelHint)
|
||||||
|
: modelBeforeSwitch();
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
391
src/tests/gateway-models.test.ts
Normal file
391
src/tests/gateway-models.test.ts
Normal file
@@ -0,0 +1,391 @@
|
|||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import type { ModelInfo, Query } from "@anthropic-ai/claude-agent-sdk";
|
||||||
|
import {
|
||||||
|
bedrockRegionPrefix,
|
||||||
|
GatewayListingError,
|
||||||
|
listBedrockModels,
|
||||||
|
listGatewayModels,
|
||||||
|
selectBedrockRegion,
|
||||||
|
} from "../gateway-models.js";
|
||||||
|
import { getAvailableModels, narrowToGatewayModels } from "../session-model.js";
|
||||||
|
|
||||||
|
const logger = { log: vi.fn(), error: vi.fn() };
|
||||||
|
|
||||||
|
function jsonResponse(body: unknown, status = 200): Response {
|
||||||
|
return new Response(JSON.stringify(body), {
|
||||||
|
status,
|
||||||
|
headers: { "content-type": "application/json" },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("listGatewayModels", () => {
|
||||||
|
it("lists /v1/models under the base URL with the chat request's auth", async () => {
|
||||||
|
const fetchMock = vi.fn(async () => jsonResponse({ data: [{ id: "claude-sonnet-5" }] }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
const ids = await listGatewayModels(
|
||||||
|
"https://gateway.example/anthropic/",
|
||||||
|
{ Authorization: "Bearer real", "X-Route": "a" },
|
||||||
|
logger,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(ids).toEqual(["claude-sonnet-5"]);
|
||||||
|
const [url, init] = fetchMock.mock.calls[0] as unknown as [URL, { headers: Headers }];
|
||||||
|
expect(url.toString()).toBe("https://gateway.example/anthropic/v1/models?limit=1000");
|
||||||
|
const headers = init.headers;
|
||||||
|
expect(headers.get("authorization")).toBe("Bearer real");
|
||||||
|
expect(headers.get("x-route")).toBe("a");
|
||||||
|
expect(headers.get("anthropic-version")).toBe("2023-06-01");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sends the placeholder bearer token when the provider headers carry no Authorization", async () => {
|
||||||
|
const fetchMock = vi.fn(async () => jsonResponse({ data: [] }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
await listGatewayModels("https://gateway.example", { "x-api-key": "key" }, logger);
|
||||||
|
|
||||||
|
const headers = (fetchMock.mock.calls[0] as unknown as [URL, { headers: Headers }])[1].headers;
|
||||||
|
expect(headers.get("authorization")).toBe("Bearer acp-proxy");
|
||||||
|
expect(headers.get("x-api-key")).toBe("key");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("follows has_more / last_id pagination", async () => {
|
||||||
|
const fetchMock = vi
|
||||||
|
.fn()
|
||||||
|
.mockResolvedValueOnce(
|
||||||
|
jsonResponse({ data: [{ id: "a" }, { id: "b" }], has_more: true, last_id: "b" }),
|
||||||
|
)
|
||||||
|
.mockResolvedValueOnce(jsonResponse({ data: [{ id: "c" }], has_more: false, last_id: "c" }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
const ids = await listGatewayModels("https://gateway.example", {}, logger);
|
||||||
|
|
||||||
|
expect(ids).toEqual(["a", "b", "c"]);
|
||||||
|
expect((fetchMock.mock.calls[1][0] as URL).searchParams.get("after_id")).toBe("b");
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
["a 5xx status", async () => jsonResponse({ error: "unavailable" }, 503)],
|
||||||
|
["a body without data", async () => jsonResponse({ models: [] })],
|
||||||
|
["a non-JSON body", async () => new Response("<html>")],
|
||||||
|
[
|
||||||
|
"a network error",
|
||||||
|
async () => {
|
||||||
|
throw new TypeError("fetch failed");
|
||||||
|
},
|
||||||
|
],
|
||||||
|
])("throws a GatewayListingError on %s", async (_name, respond) => {
|
||||||
|
vi.stubGlobal("fetch", vi.fn(respond));
|
||||||
|
|
||||||
|
await expect(listGatewayModels("https://gateway.example", {}, logger)).rejects.toBeInstanceOf(
|
||||||
|
GatewayListingError,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("gateways without a listing", () => {
|
||||||
|
it.each([400, 401, 403, 404, 405, 429])("returns undefined for a %s answer", async (status) => {
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => new Response("no", { status })),
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(listGatewayModels("https://gateway.example", {}, logger)).resolves.toBe(undefined);
|
||||||
|
await expect(listBedrockModels("https://gateway.example", {}, logger)).resolves.toBe(undefined);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("listing request guards", () => {
|
||||||
|
it("names an invalid provider header without its value", async () => {
|
||||||
|
const fetchMock = vi.fn();
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
const listing = listGatewayModels(
|
||||||
|
"https://gateway.example",
|
||||||
|
{ authorization: "Bearer sk-SECRET\r\nx: y" },
|
||||||
|
logger,
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(listing).rejects.toThrow(
|
||||||
|
'Model listing cannot send the invalid header "authorization"',
|
||||||
|
);
|
||||||
|
await expect(listing).rejects.not.toThrow("sk-SECRET");
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses a base URL with credentials without repeating them", async () => {
|
||||||
|
const fetchMock = vi.fn();
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
const listing = listGatewayModels("https://user:secret@gateway.example", {}, logger);
|
||||||
|
|
||||||
|
await expect(listing).rejects.toThrow(
|
||||||
|
"Model listing GET /v1/models cannot be sent: the base URL carries credentials",
|
||||||
|
);
|
||||||
|
await expect(listing).rejects.not.toThrow("secret");
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("listBedrockModels", () => {
|
||||||
|
it("lists the Anthropic ids of /foundation-models under the base URL", async () => {
|
||||||
|
const fetchMock = vi.fn(async () =>
|
||||||
|
jsonResponse({
|
||||||
|
modelSummaries: [
|
||||||
|
{ modelId: "global.anthropic.claude-sonnet-5-5" },
|
||||||
|
{ modelId: "global.meta.llama3-3-70b-instruct-v1:0" },
|
||||||
|
{ modelId: "global.anthropic.claude-opus-5-5" },
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
const ids = await listBedrockModels("https://gateway.example/bedrock/", {}, logger);
|
||||||
|
|
||||||
|
expect(ids).toEqual(["global.anthropic.claude-sonnet-5-5", "global.anthropic.claude-opus-5-5"]);
|
||||||
|
expect((fetchMock.mock.calls[0] as unknown as [URL])[0].toString()).toBe(
|
||||||
|
"https://gateway.example/bedrock/foundation-models",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("throws a GatewayListingError on a body without modelSummaries", async () => {
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => jsonResponse({ data: [] })),
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(listBedrockModels("https://gateway.example", {}, logger)).rejects.toThrow(
|
||||||
|
"Model listing GET /foundation-models returned an unexpected body",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("selectBedrockRegion", () => {
|
||||||
|
it("reads the prefix of an id or an inference-profile ARN", () => {
|
||||||
|
expect(bedrockRegionPrefix("eu.anthropic.claude-sonnet-5")).toBe("eu");
|
||||||
|
expect(bedrockRegionPrefix("us-gov.anthropic.claude-sonnet-5")).toBe("us-gov");
|
||||||
|
expect(
|
||||||
|
bedrockRegionPrefix(
|
||||||
|
"arn:aws:bedrock:eu-west-1:123456789012:inference-profile/global.anthropic.claude-opus-5",
|
||||||
|
),
|
||||||
|
).toBe("global");
|
||||||
|
expect(bedrockRegionPrefix("anthropic.claude-sonnet-5")).toBe(undefined);
|
||||||
|
expect(bedrockRegionPrefix("claude-sonnet-5")).toBe(undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("prefers global over a prefix listed before it", () => {
|
||||||
|
expect(
|
||||||
|
selectBedrockRegion([
|
||||||
|
"eu.anthropic.claude-sonnet-5",
|
||||||
|
"global.anthropic.claude-sonnet-5",
|
||||||
|
"us.anthropic.claude-opus-5",
|
||||||
|
]),
|
||||||
|
).toEqual({ regionPrefix: "global", modelIds: ["global.anthropic.claude-sonnet-5"] });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the ids of the first listed prefix without global", () => {
|
||||||
|
expect(
|
||||||
|
selectBedrockRegion(["eu.anthropic.claude-sonnet-5", "us.anthropic.claude-opus-5"]),
|
||||||
|
).toEqual({ regionPrefix: "eu", modelIds: ["eu.anthropic.claude-sonnet-5"] });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the ids of global when listed", () => {
|
||||||
|
expect(
|
||||||
|
selectBedrockRegion([
|
||||||
|
"anthropic.claude-haiku-4-5",
|
||||||
|
"global.anthropic.claude-opus-5",
|
||||||
|
"eu.anthropic.claude-sonnet-5",
|
||||||
|
"global.anthropic.claude-sonnet-5",
|
||||||
|
]),
|
||||||
|
).toEqual({
|
||||||
|
regionPrefix: "global",
|
||||||
|
modelIds: ["global.anthropic.claude-opus-5", "global.anthropic.claude-sonnet-5"],
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps every id when none has a prefix", () => {
|
||||||
|
expect(selectBedrockRegion(["anthropic.claude-sonnet-5"])).toEqual({
|
||||||
|
modelIds: ["anthropic.claude-sonnet-5"],
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("narrowToGatewayModels", () => {
|
||||||
|
const MODELS: ModelInfo[] = [
|
||||||
|
{
|
||||||
|
value: "default",
|
||||||
|
resolvedModel: "claude-opus-5[1m]",
|
||||||
|
displayName: "Default",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{ value: "sonnet", resolvedModel: "claude-sonnet-5", displayName: "Sonnet", description: "" },
|
||||||
|
{
|
||||||
|
value: "sonnet[1m]",
|
||||||
|
resolvedModel: "claude-sonnet-5[1m]",
|
||||||
|
displayName: "Sonnet (1M context)",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "haiku",
|
||||||
|
resolvedModel: "claude-haiku-4-5-20251001",
|
||||||
|
displayName: "Haiku",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
it("keeps the rows whose resolved model is listed in any spelling", () => {
|
||||||
|
const narrowed = narrowToGatewayModels(MODELS, [
|
||||||
|
"claude-sonnet-5-20260101",
|
||||||
|
"claude-haiku-4-5",
|
||||||
|
]);
|
||||||
|
|
||||||
|
expect(narrowed.map((m) => m.value)).toEqual(["sonnet", "sonnet[1m]", "haiku"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a default row without resolvedModel and checks a bare family alias by family", () => {
|
||||||
|
const models: ModelInfo[] = [
|
||||||
|
{ value: "default", displayName: "Default", description: "" },
|
||||||
|
{ value: "opus", displayName: "Opus", description: "" },
|
||||||
|
{ value: "haiku", displayName: "Haiku", description: "" },
|
||||||
|
{ value: "claude-sonnet-5", displayName: "Sonnet 5", description: "" },
|
||||||
|
];
|
||||||
|
|
||||||
|
const narrowed = narrowToGatewayModels(models, ["claude-opus-5", "claude-sonnet-5"]);
|
||||||
|
|
||||||
|
expect(narrowed.map((m) => m.value)).toEqual(["default", "opus", "claude-sonnet-5"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops rows resolved to another Bedrock region prefix", () => {
|
||||||
|
const models: ModelInfo[] = [
|
||||||
|
{
|
||||||
|
value: "sonnet",
|
||||||
|
resolvedModel: "eu.anthropic.claude-sonnet-5",
|
||||||
|
displayName: "Sonnet",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "opus",
|
||||||
|
resolvedModel: "us.anthropic.claude-opus-5",
|
||||||
|
displayName: "Opus",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
const narrowed = narrowToGatewayModels(
|
||||||
|
models,
|
||||||
|
["eu.anthropic.claude-sonnet-5", "eu.anthropic.claude-opus-5"],
|
||||||
|
"eu",
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(narrowed.map((m) => m.value)).toEqual(["sonnet"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaves no model when the listing matches none", () => {
|
||||||
|
expect(narrowToGatewayModels(MODELS, ["gpt-5", "gemini-3-pro"])).toEqual([]);
|
||||||
|
expect(narrowToGatewayModels(MODELS, [])).toEqual([]);
|
||||||
|
// A default row without resolvedModel is not a match on its own.
|
||||||
|
expect(
|
||||||
|
narrowToGatewayModels(
|
||||||
|
[{ value: "default", displayName: "Default", description: "" }, ...MODELS.slice(1)],
|
||||||
|
["gpt-5"],
|
||||||
|
),
|
||||||
|
).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("getAvailableModels under a gateway listing", () => {
|
||||||
|
const SDK_MODELS: ModelInfo[] = [
|
||||||
|
{ value: "default", resolvedModel: "claude-opus-5", displayName: "Default", description: "" },
|
||||||
|
{ value: "sonnet", resolvedModel: "claude-sonnet-5", displayName: "Sonnet", description: "" },
|
||||||
|
{ value: "haiku", resolvedModel: "claude-haiku-4-5", displayName: "Haiku", description: "" },
|
||||||
|
];
|
||||||
|
const LISTED = narrowToGatewayModels(SDK_MODELS, ["claude-sonnet-5", "claude-haiku-4-5"]);
|
||||||
|
|
||||||
|
function available(
|
||||||
|
setModel: ReturnType<typeof vi.fn>,
|
||||||
|
isResumedSession: boolean,
|
||||||
|
resumedModelHint?: string,
|
||||||
|
narrowedByGateway = true,
|
||||||
|
) {
|
||||||
|
return getAvailableModels(
|
||||||
|
{ setModel } as unknown as Query,
|
||||||
|
LISTED,
|
||||||
|
SDK_MODELS,
|
||||||
|
{ getSettings: () => ({}) },
|
||||||
|
{ log: vi.fn(), error: vi.fn() },
|
||||||
|
isResumedSession,
|
||||||
|
"gateway-test",
|
||||||
|
resumedModelHint,
|
||||||
|
narrowedByGateway,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
it("switches a resumed session off a transcript model the gateway does not list", async () => {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
|
||||||
|
const state = await available(setModel, true, "claude-opus-5");
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("sonnet");
|
||||||
|
expect(setModel).toHaveBeenCalledWith("sonnet");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("switches a resumed session without a transcript off the dropped default", async () => {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
|
||||||
|
const state = await available(setModel, true);
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("sonnet");
|
||||||
|
expect(setModel).toHaveBeenCalledWith("sonnet");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a resumed session on a listed transcript model", async () => {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
|
||||||
|
const state = await available(setModel, true, "claude-haiku-4-5");
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("haiku");
|
||||||
|
expect(setModel).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a resumed transcript model outside an allowlist when no gateway narrowed the list", async () => {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
|
||||||
|
const state = await available(setModel, true, "claude-opus-5", false);
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("claude-opus-5");
|
||||||
|
expect(setModel).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports the SDK default when a PreModelSwitch hook vetoes the switch", async () => {
|
||||||
|
const setModel = vi
|
||||||
|
.fn()
|
||||||
|
.mockRejectedValue(new Error("Model switch blocked by a PreModelSwitch hook: no"));
|
||||||
|
|
||||||
|
const state = await available(setModel, false);
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("default");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports the SDK default when the switch onto a listed model fails", async () => {
|
||||||
|
const setModel = vi.fn().mockRejectedValue(new Error("boom"));
|
||||||
|
|
||||||
|
const state = await available(setModel, false);
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("default");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("switches a resumed session whose transcript model only loosely matches a listed row", async () => {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
|
||||||
|
const state = await available(setModel, true, "claude-sonnet-4-5-20250929");
|
||||||
|
|
||||||
|
expect(state.currentModelId).toBe("sonnet");
|
||||||
|
expect(setModel).toHaveBeenCalledWith("sonnet");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps a default row the gateway lists by its resolved model", () => {
|
||||||
|
expect(narrowToGatewayModels(SDK_MODELS, ["claude-opus-5"]).map((m) => m.value)).toEqual([
|
||||||
|
"default",
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -528,6 +528,34 @@ describe("providers", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("clears a providers/set that is still listing when logout arrives", async () => {
|
||||||
|
const [agent] = await createAgentMock();
|
||||||
|
vi.stubEnv("CLAUDE_CODE_EXECUTABLE", "/bin/true");
|
||||||
|
let releaseListing!: () => void;
|
||||||
|
const held = new Promise<void>((resolve) => (releaseListing = resolve));
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => {
|
||||||
|
await held;
|
||||||
|
return new Response(JSON.stringify({ data: [{ id: "claude-sonnet-5" }] }));
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
|
const set = agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://slow.example",
|
||||||
|
});
|
||||||
|
const logout = agent.logout({});
|
||||||
|
releaseListing();
|
||||||
|
await Promise.all([set, logout]);
|
||||||
|
|
||||||
|
expect((await agent.unstable_listProviders({})).providers[0].current).toEqual({
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://api.anthropic.com",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it("recreates live sessions without the cleared provider on logout", async () => {
|
it("recreates live sessions without the cleared provider on logout", async () => {
|
||||||
const [agent, mockQuery] = await createAgentMock();
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
vi.stubEnv("CLAUDE_CODE_EXECUTABLE", "/bin/true");
|
vi.stubEnv("CLAUDE_CODE_EXECUTABLE", "/bin/true");
|
||||||
@@ -562,4 +590,335 @@ describe("providers", () => {
|
|||||||
expect(originalQuery.close).not.toHaveBeenCalled();
|
expect(originalQuery.close).not.toHaveBeenCalled();
|
||||||
expect(mockQuery).toHaveBeenCalledOnce();
|
expect(mockQuery).toHaveBeenCalledOnce();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("gateway model discovery", () => {
|
||||||
|
const GATEWAY_MODELS = [
|
||||||
|
{ value: "default", resolvedModel: "claude-opus-5", displayName: "Default", description: "" },
|
||||||
|
{ value: "opus", resolvedModel: "claude-opus-5", displayName: "Opus", description: "" },
|
||||||
|
{ value: "sonnet", resolvedModel: "claude-sonnet-5", displayName: "Sonnet", description: "" },
|
||||||
|
{ value: "haiku", resolvedModel: "claude-haiku-4-5", displayName: "Haiku", description: "" },
|
||||||
|
];
|
||||||
|
|
||||||
|
function mockQueryWithModels(mockQuery: Mock, models: unknown[] = GATEWAY_MODELS): Mock {
|
||||||
|
const setModel = vi.fn();
|
||||||
|
mockQuery.mockImplementation(() => ({
|
||||||
|
initializationResult: vi.fn().mockResolvedValue({ models }),
|
||||||
|
setModel,
|
||||||
|
setPermissionMode: vi.fn(),
|
||||||
|
supportedCommands: vi.fn().mockResolvedValue([]),
|
||||||
|
close: vi.fn(),
|
||||||
|
interrupt: vi.fn().mockResolvedValue(undefined),
|
||||||
|
getContextUsage: vi.fn().mockResolvedValue({ totalTokens: 0, rawMaxTokens: 200000 }),
|
||||||
|
[Symbol.asyncIterator]: async function* () {},
|
||||||
|
}));
|
||||||
|
return setModel;
|
||||||
|
}
|
||||||
|
|
||||||
|
function stubListing(ids: string[]): Mock {
|
||||||
|
const fetchMock = vi.fn(
|
||||||
|
async () => new Response(JSON.stringify({ data: ids.map((id) => ({ id })) })),
|
||||||
|
);
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
return fetchMock;
|
||||||
|
}
|
||||||
|
|
||||||
|
function modelOptionValues(response: { configOptions?: any[] | null }): string[] {
|
||||||
|
const option = response.configOptions?.find((o: any) => o.id === "model");
|
||||||
|
return option.options.map((o: any) => o.value);
|
||||||
|
}
|
||||||
|
|
||||||
|
it("offers only the models the gateway lists and switches off a dropped default", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
const setModel = mockQueryWithModels(mockQuery);
|
||||||
|
const fetchMock = stubListing(["claude-sonnet-5-20260101", "claude-haiku-4-5"]);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
await agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://gateway.example",
|
||||||
|
headers: { authorization: "Bearer real" },
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
|
||||||
|
expect(fetchMock).toHaveBeenCalledOnce();
|
||||||
|
expect(modelOptionValues(response)).toEqual(["sonnet", "haiku"]);
|
||||||
|
expect(response.configOptions?.find((o: any) => o.id === "model")?.currentValue).toBe(
|
||||||
|
"sonnet",
|
||||||
|
);
|
||||||
|
expect(setModel).toHaveBeenCalledWith("sonnet");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fails providers/set and keeps the current provider when the listing fails", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
mockQueryWithModels(mockQuery);
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => new Response("unavailable", { status: 503 })),
|
||||||
|
);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://gateway.example",
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({
|
||||||
|
code: -32603,
|
||||||
|
message: expect.stringContaining("Model listing GET /v1/models returned 503: unavailable"),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect((await agent.unstable_listProviders({})).providers[0].current).toEqual({
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://api.anthropic.com",
|
||||||
|
});
|
||||||
|
const response = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
expect(modelOptionValues(response)).toEqual(["default", "opus", "sonnet", "haiku"]);
|
||||||
|
expect(mockQuery.mock.calls.at(-1)![0].options.env).not.toHaveProperty(
|
||||||
|
"ANTHROPIC_BASE_URL",
|
||||||
|
"https://gateway.example",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("lists nothing without providers/set or for a vertex provider", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
mockQueryWithModels(mockQuery);
|
||||||
|
const fetchMock = stubListing(["claude-sonnet-5"]);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
const native = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
await agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "vertex",
|
||||||
|
baseUrl: "https://vertex.example",
|
||||||
|
_meta: { claudeCode: { vertex: { projectId: "p", region: "europe-west1" } } },
|
||||||
|
});
|
||||||
|
const vertex = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
|
||||||
|
expect(fetchMock).not.toHaveBeenCalled();
|
||||||
|
expect(modelOptionValues(native)).toEqual(["default", "opus", "sonnet", "haiku"]);
|
||||||
|
expect(modelOptionValues(vertex)).toEqual(["default", "opus", "sonnet", "haiku"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("narrows a bedrock gateway to the region prefix of its first listed model", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
mockQueryWithModels(mockQuery, [
|
||||||
|
{
|
||||||
|
value: "default",
|
||||||
|
resolvedModel: "eu.anthropic.claude-opus-5",
|
||||||
|
displayName: "Default",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "sonnet",
|
||||||
|
resolvedModel: "eu.anthropic.claude-sonnet-5",
|
||||||
|
displayName: "Sonnet",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "haiku",
|
||||||
|
resolvedModel: "eu.anthropic.claude-haiku-4-5-20251001-v1:0",
|
||||||
|
displayName: "Haiku",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
value: "us.anthropic.claude-sonnet-5",
|
||||||
|
displayName: "Sonnet (US)",
|
||||||
|
description: "",
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
const fetchMock = vi.fn(
|
||||||
|
async () =>
|
||||||
|
new Response(
|
||||||
|
JSON.stringify({
|
||||||
|
modelSummaries: [
|
||||||
|
{ modelId: "eu.anthropic.claude-sonnet-5" },
|
||||||
|
{ modelId: "us.anthropic.claude-opus-5" },
|
||||||
|
{ modelId: "us.anthropic.claude-sonnet-5" },
|
||||||
|
{ modelId: "eu.anthropic.claude-haiku-4-5-20251001-v1:0" },
|
||||||
|
{ modelId: "eu.meta.llama3-3-70b-instruct-v1:0" },
|
||||||
|
],
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
await agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "bedrock",
|
||||||
|
baseUrl: "https://bedrock.example/llm/bedrock",
|
||||||
|
headers: { authorization: "Bearer real" },
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
|
||||||
|
const [url, init] = fetchMock.mock.calls[0] as unknown as [URL, { headers: Headers }];
|
||||||
|
expect(url.toString()).toBe("https://bedrock.example/llm/bedrock/foundation-models");
|
||||||
|
expect(init.headers.get("authorization")).toBe("Bearer real");
|
||||||
|
// us.* ids are dropped: opus is listed only under us, and the us
|
||||||
|
// sonnet row would call another region.
|
||||||
|
expect(modelOptionValues(response)).toEqual(["sonnet", "haiku"]);
|
||||||
|
expect(mockQuery).toHaveBeenLastCalledWith(
|
||||||
|
expect.objectContaining({
|
||||||
|
options: expect.objectContaining({
|
||||||
|
env: expect.objectContaining({
|
||||||
|
CLAUDE_CODE_USE_BEDROCK: "1",
|
||||||
|
ANTHROPIC_BEDROCK_REGION_PREFIX: "eu",
|
||||||
|
}),
|
||||||
|
}),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fails providers/set for a bedrock gateway whose listing fails", async () => {
|
||||||
|
const [agent] = await createAgentMock();
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => {
|
||||||
|
throw new TypeError("fetch failed");
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "bedrock",
|
||||||
|
baseUrl: "https://bedrock.example",
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({
|
||||||
|
message: expect.stringContaining(
|
||||||
|
"Model listing GET /foundation-models failed: fetch failed",
|
||||||
|
),
|
||||||
|
});
|
||||||
|
expect((await agent.unstable_listProviders({})).providers[0].current?.apiType).toBe(
|
||||||
|
"anthropic",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("offers no model when the listing succeeds but matches none", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
const setModel = mockQueryWithModels(mockQuery);
|
||||||
|
stubListing(["gpt-5"]);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
await agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://gateway.example",
|
||||||
|
});
|
||||||
|
|
||||||
|
const response = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
|
||||||
|
expect(response.configOptions?.find((o: any) => o.id === "model")).toBeUndefined();
|
||||||
|
expect(setModel).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sets a gateway that has no listing and narrows nothing", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
mockQueryWithModels(mockQuery);
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => new Response("not found", { status: 404 })),
|
||||||
|
);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "bedrock",
|
||||||
|
baseUrl: "https://bedrock.example",
|
||||||
|
}),
|
||||||
|
).resolves.toEqual({});
|
||||||
|
|
||||||
|
const response = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
expect(modelOptionValues(response)).toEqual(["default", "opus", "sonnet", "haiku"]);
|
||||||
|
expect(mockQuery.mock.calls.at(-1)![0].options.env).not.toHaveProperty(
|
||||||
|
"ANTHROPIC_BEDROCK_REGION_PREFIX",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("applies providers/set and providers/disable in the order they arrive while a listing runs", async () => {
|
||||||
|
const [agent] = await createAgentMock();
|
||||||
|
let releaseListing!: () => void;
|
||||||
|
const held = new Promise<void>((resolve) => (releaseListing = resolve));
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => {
|
||||||
|
await held;
|
||||||
|
return new Response(JSON.stringify({ data: [{ id: "claude-sonnet-5" }] }));
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
const set = agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://slow.example",
|
||||||
|
});
|
||||||
|
const disable = agent.unstable_disableProvider({ providerId: "main" });
|
||||||
|
releaseListing();
|
||||||
|
await Promise.all([set, disable]);
|
||||||
|
|
||||||
|
expect((await agent.unstable_listProviders({})).providers[0].current).toEqual({
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://api.anthropic.com",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the later providers/set when an earlier one lists slower", async () => {
|
||||||
|
const [agent] = await createAgentMock();
|
||||||
|
let releaseSlow!: () => void;
|
||||||
|
const slow = new Promise<void>((resolve) => (releaseSlow = resolve));
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async (url: URL) => {
|
||||||
|
if (url.hostname === "slow.example") await slow;
|
||||||
|
return new Response(JSON.stringify({ data: [{ id: "claude-sonnet-5" }] }));
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
|
||||||
|
const first = agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://slow.example",
|
||||||
|
});
|
||||||
|
const second = agent.unstable_setProvider({
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://fast.example",
|
||||||
|
});
|
||||||
|
releaseSlow();
|
||||||
|
await Promise.all([first, second]);
|
||||||
|
|
||||||
|
expect((await agent.unstable_listProviders({})).providers[0].current?.baseUrl).toBe(
|
||||||
|
"https://fast.example",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops the listing on providers/disable and reuses it when the same gateway is set again", async () => {
|
||||||
|
const [agent, mockQuery] = await createAgentMock();
|
||||||
|
mockQueryWithModels(mockQuery);
|
||||||
|
const fetchMock = stubListing(["claude-sonnet-5"]);
|
||||||
|
await agent.initialize({ protocolVersion: 1, clientCapabilities: {} });
|
||||||
|
const provider = {
|
||||||
|
providerId: "main",
|
||||||
|
apiType: "anthropic",
|
||||||
|
baseUrl: "https://gateway.example",
|
||||||
|
headers: { authorization: "Bearer real" },
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
await agent.unstable_setProvider(provider);
|
||||||
|
await agent.unstable_setProvider({ ...provider, headers: { ...provider.headers } });
|
||||||
|
expect(fetchMock).toHaveBeenCalledOnce();
|
||||||
|
|
||||||
|
await agent.unstable_disableProvider({ providerId: "main" });
|
||||||
|
const native = await agent.newSession({ cwd: process.cwd(), mcpServers: [] });
|
||||||
|
expect(modelOptionValues(native)).toEqual(["default", "opus", "sonnet", "haiku"]);
|
||||||
|
|
||||||
|
await agent.unstable_setProvider(provider);
|
||||||
|
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
16
src/tests/setup-no-network.ts
Normal file
16
src/tests/setup-no-network.ts
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
import { beforeEach, vi } from "vitest";
|
||||||
|
|
||||||
|
// `providers/set` lists the gateway's models with `fetch` (gateway-models.ts),
|
||||||
|
// and a failed listing fails the call. Tests that set an `anthropic` or
|
||||||
|
// `bedrock` provider would otherwise send that request to their example
|
||||||
|
// gateway URLs, so every test gets an empty listing instead. An empty listing
|
||||||
|
// narrows the picker to no model: a test that checks the picker after
|
||||||
|
// `providers/set` stubs `fetch` with the models it expects. This default is
|
||||||
|
// deliberate: listing fixture models for every test would hide which tests
|
||||||
|
// depend on a listing.
|
||||||
|
beforeEach(() => {
|
||||||
|
vi.stubGlobal(
|
||||||
|
"fetch",
|
||||||
|
vi.fn(async () => new Response(JSON.stringify({ data: [], modelSummaries: [] }))),
|
||||||
|
);
|
||||||
|
});
|
||||||
@@ -14,6 +14,8 @@ export default defineConfig({
|
|||||||
// tests off the machine's real CLI and Claude config, and stops a run
|
// tests off the machine's real CLI and Claude config, and stops a run
|
||||||
// from outliving its test (it wrote into temp config dirs being deleted).
|
// from outliving its test (it wrote into temp config dirs being deleted).
|
||||||
env: usesRealCli ? {} : { CLAUDE_CODE_EXECUTABLE: "/usr/bin/false" },
|
env: usesRealCli ? {} : { CLAUDE_CODE_EXECUTABLE: "/usr/bin/false" },
|
||||||
|
// Real-CLI runs may want a real gateway listing.
|
||||||
|
setupFiles: usesRealCli ? [] : ["src/tests/setup-no-network.ts"],
|
||||||
include: [
|
include: [
|
||||||
"src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}",
|
"src/**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}",
|
||||||
// Release tooling lives in scripts/ rather than src/ so it stays outside tsc's
|
// Release tooling lives in scripts/ rather than src/ so it stays outside tsc's
|
||||||
|
|||||||
Reference in New Issue
Block a user