mirror of
https://github.com/agentclientprotocol/claude-agent-acp.git
synced 2026-11-21 16:51:28 +00:00
## Summary Follow-up to #1612. The context window that `usage_update.size` reports before Claude Code has said what the window is changes from 200K to 1M, and the "1m" text heuristic that upgraded it is removed. ## Why The adapter takes the window from Claude Code: a background `getContextUsage` at session creation and at every model switch, then each result's `modelUsage`. Until the first of those lands it reports a placeholder, `DEFAULT_CONTEXT_WINDOW`. Since Claude Code 2.1.323 that placeholder is the less likely value. Asked directly (`getContextUsage`, no prompt sent): | Model | Window | | --- | --- | | An id Claude Code does not know (`my-gateway-model`, `gpt-5`) | 1,000,000 | | `opus`, `sonnet`, `haiku` | 1,000,000 | | `claude-haiku-4-5`, `claude-sonnet-4-5` | 200,000 | | An unknown id or `sonnet` with `CLAUDE_CODE_DISABLE_1M_CONTEXT=1` | 200,000 | So 1M is what Claude Code itself assumes when it knows nothing about a model, and what it reports for its current aliases. ## What changes - `DEFAULT_CONTEXT_WINDOW` is 1,000,000. - `inferContextWindowFromModel` is gone, with its two callers: the seed in `immediateContextWindow` and the `message_start` upgrade in the v1 and v2 cores. It only ever turned 200K into 1M for a model whose id, resolved id, name or description held a "1m" token. With a 1M default it has nothing left to do, and the table shows that a model's names do not give its window (`sonnet` is 1M, or 200K under the environment variable). - Nothing changes in how the real window is learned: the background read, the per-result update and the cross-session cache are as before. For a client: the first `usage_update` of a session or after a model switch can carry `size: 1000000` where it carried `200000`, until the read lands (0.2 to 0.6 s in the probe above) or, if the read fails, until the first result. A model with a 200K window shows 1M for that moment; before, a model with a 1M window showed 200K. ## Tests - The session double (`mockSessionState`) starts with the default window, as a real session does. It held 200K with `contextWindowAuthoritative: false`, which can no longer occur. - Tests of the heuristic are removed. Tests that tell a learned window from the default now use values that differ from 1M. - The scenario recordings are unchanged: their scripted Claude Code reports a 200K window, and the read lands before the first turn. ## Testing - `npm run build`, `npm run check`, `npm run test:run` (4,115 tests pass; 6 tests of the removed heuristic are gone) - Real CLI: the `getContextUsage` reads in the table above
160 lines
8.0 KiB
TypeScript
160 lines
8.0 KiB
TypeScript
/**
|
|
* The context window of a session's model, as both the v1 core and the
|
|
* v2 core report it in `usage_update.size`: seeded without IPC when the
|
|
* session starts or switches models, refined by a background
|
|
* `getContextUsage`, and learned from each result's `modelUsage`.
|
|
*/
|
|
import type { ModelInfo, Query } from "@anthropic-ai/claude-agent-sdk";
|
|
import { DEFAULT_CONTEXT_WINDOW } from "./session-usage.js";
|
|
|
|
/** Cross-session cache of authoritative context windows, keyed by
|
|
* `${providerCacheKey}\0${modelId}` (see {@link contextWindowCacheKey}).
|
|
* The window is a property of (model id, backend): the same resolved model id
|
|
* (e.g. "claude-sonnet-5[1m]", the spelling of the `result.modelUsage` keys)
|
|
* can name different context lanes behind different base URLs, routing
|
|
* headers, or credentials, so the key carries both. Caching it module-level
|
|
* lets a later session/new or switch that resolves to the same (backend,
|
|
* model) — in this session or any other, within the adapter's lifetime — seed
|
|
* the correct window synchronously with no IPC. Keying on the resolved id
|
|
* (rather than the picker value) means aliases that resolve to the same
|
|
* concrete model share one entry; the result handler additionally writes the
|
|
* bare assistant-message spelling so seed-time reads that fall back to a
|
|
* verbatim live id (rows without `resolvedModel`) can hit too.
|
|
*
|
|
* Populated authoritatively by each `result.modelUsage` a turn confirms (see
|
|
* the consumer's result handler). We deliberately never populate it from
|
|
* `getContextUsage`: its `model` spelling and `rawMaxTokens` can differ from
|
|
* the `result.modelUsage` key and window (e.g. 967000 vs 1000000 for sonnet),
|
|
* so it only refines the live session (`refreshContextWindowInBackground`).
|
|
* Cleared on `logout`: 1M-context entitlement can differ per account/tier, so
|
|
* windows learned under one login must not seed sessions under the next. */
|
|
const contextWindowCache = new Map<string, number>();
|
|
|
|
/** The env vars that determine which LLM backend — and which context lane on
|
|
* it — a query's API traffic reaches: endpoint selection (base URLs and the
|
|
* Bedrock/Vertex switches with their project/region), routing/beta headers
|
|
* (an `anthropic-beta: context-1m-…` header flips the same model id at the
|
|
* same endpoint between context lanes), and credential identity (extended
|
|
* context is entitlement-gated per account). Used to derive the
|
|
* provider-cache key from the exact env a query is created with, so
|
|
* `providers/set` config, per-session `_meta` env overrides, and ambient
|
|
* process env are all distinguished exactly as the CLI will see them. */
|
|
const PROVIDER_ROUTING_ENV_VARS = [
|
|
"ANTHROPIC_BASE_URL",
|
|
"ANTHROPIC_BEDROCK_BASE_URL",
|
|
"ANTHROPIC_VERTEX_BASE_URL",
|
|
"CLAUDE_CODE_USE_BEDROCK",
|
|
"CLAUDE_CODE_USE_VERTEX",
|
|
"ANTHROPIC_VERTEX_PROJECT_ID",
|
|
"CLOUD_ML_REGION",
|
|
"AWS_REGION",
|
|
"ANTHROPIC_CUSTOM_HEADERS",
|
|
"ANTHROPIC_API_KEY",
|
|
"ANTHROPIC_AUTH_TOKEN",
|
|
"CLAUDE_CODE_OAUTH_TOKEN",
|
|
] as const;
|
|
|
|
/** Stable identifier for the LLM backend a session's query is created against,
|
|
* used to scope {@link contextWindowCache} per backend. Positional `\0`-join
|
|
* of {@link PROVIDER_ROUTING_ENV_VARS} values, so no segment can masquerade
|
|
* as another and unset vars everywhere yield one stable "default" bucket.
|
|
* Header/credential values can be secrets; the key only ever lives as an
|
|
* in-memory Map key and is never logged or surfaced. Over-keying is the safe
|
|
* side: a var change that didn't really change the backend costs one cache
|
|
* miss (the default window until Claude Code reports it), while under-keying
|
|
* would serve one backend's window for another's. */
|
|
export function providerCacheKeyFor(env: Record<string, string | undefined>): string {
|
|
return PROVIDER_ROUTING_ENV_VARS.map((name) => env[name] ?? "").join("\0");
|
|
}
|
|
|
|
/** Compose the `contextWindowCache` key from a session's provider key and a
|
|
* model id. `\0`-joined so the model segment can't collide with a provider
|
|
* segment. */
|
|
export function contextWindowCacheKey(providerCacheKey: string, modelId: string): string {
|
|
return `${providerCacheKey}\0${modelId}`;
|
|
}
|
|
|
|
function cacheContextWindow(modelKey: string, window: number): void {
|
|
if (window > 0) {
|
|
contextWindowCache.set(modelKey, window);
|
|
}
|
|
}
|
|
|
|
/** Caches the window that a result's `modelUsage` reported, keyed on (the
|
|
* session's provider, the resolved model id — the `modelUsage` key, e.g.
|
|
* "claude-sonnet-5[1m]") so a later session/new or switch on the same
|
|
* provider that resolves to this model seeds the correct window
|
|
* synchronously, with no getContextUsage IPC.
|
|
*
|
|
* Also caches it under the assistant message's own (bare) spelling. Seed-time
|
|
* reads fall back to a picker value / verbatim live id when a row carries no
|
|
* resolvedModel (the synthesized out-of-allowlist resume row sets it
|
|
* undefined on purpose), and those spellings match `.model` from the
|
|
* assistant message, not the decorated modelUsage key — without this entry
|
|
* such rows could never hit the cache. A non-positive window is not cached. */
|
|
export function cacheResultContextWindow(
|
|
providerCacheKey: string,
|
|
modelUsageKey: string,
|
|
assistantModel: string | null | undefined,
|
|
window: number,
|
|
): void {
|
|
cacheContextWindow(contextWindowCacheKey(providerCacheKey, modelUsageKey), window);
|
|
if (assistantModel && assistantModel !== modelUsageKey) {
|
|
cacheContextWindow(contextWindowCacheKey(providerCacheKey, assistantModel), window);
|
|
}
|
|
}
|
|
|
|
/** The context window to report *right now* for a model, with NO IPC on the
|
|
* critical path: the cached authoritative value if we've learned it (from a
|
|
* prior turn's `result.modelUsage`, this or any session on the same backend),
|
|
* else DEFAULT_CONTEXT_WINDOW. The model's names do not give the window:
|
|
* Claude Code 2.1.323 reports 1M for `sonnet`, and 200K for it under
|
|
* `CLAUDE_CODE_DISABLE_1M_CONTEXT=1`. Derives the cache key itself —
|
|
* `modelInfo?.resolvedModel ?? modelId`, the same rule at every seed site — so
|
|
* read keys can't drift from the write site's spelling. `authoritative`
|
|
* reports whether the value came from the cache: an authoritative window can
|
|
* legitimately equal DEFAULT_CONTEXT_WINDOW, so the value alone can't tell the
|
|
* caller. */
|
|
export function immediateContextWindow(
|
|
providerCacheKey: string,
|
|
modelId: string,
|
|
modelInfo?: Pick<ModelInfo, "resolvedModel">,
|
|
): { size: number; authoritative: boolean } {
|
|
const cached = contextWindowCache.get(
|
|
contextWindowCacheKey(providerCacheKey, modelInfo?.resolvedModel ?? modelId),
|
|
);
|
|
if (cached !== undefined) return { size: cached, authoritative: true };
|
|
return { size: DEFAULT_CONTEXT_WINDOW, authoritative: false };
|
|
}
|
|
|
|
/** Forgets the learned windows, as a logout asks: 1M-context entitlement is
|
|
* gated per org/tier, and an OAuth re-login is invisible to the env-derived
|
|
* provider cache key, so windows learned under the old login must not seed
|
|
* sessions under the next. */
|
|
export function clearContextWindowCache(): void {
|
|
contextWindowCache.clear();
|
|
}
|
|
|
|
/**
|
|
* Reads the window of the query's current model with
|
|
* `getContextUsage().rawMaxTokens`. Undefined when Claude Code reports none.
|
|
*
|
|
* Requests the `summary` detail instead of the default `full` breakdown,
|
|
* which issues one `messages/count_tokens` call per category (~18 for a bare
|
|
* session) and hit rate limits on every model switch; `summary` answers
|
|
* locally with the same `rawMaxTokens`. Callers never await it on the setup
|
|
* or switch path: SDK control requests are serialized, so awaiting would
|
|
* delay session/new or a model switch (~15s on older CLIs, issues
|
|
* #886/#880; ~0.5s on 2.1.283).
|
|
*
|
|
* The result does not go into the cross-session cache: its `rawMaxTokens` can
|
|
* differ from the `result.modelUsage` window (e.g. 967000 vs 1000000 for
|
|
* sonnet), so it only refines the live session.
|
|
*/
|
|
export async function readContextWindow(
|
|
query: Pick<Query, "getContextUsage">,
|
|
): Promise<number | undefined> {
|
|
const usage = await query.getContextUsage({ detail: "summary" });
|
|
return usage.rawMaxTokens > 0 ? usage.rawMaxTokens : undefined;
|
|
}
|