jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-11-03 16:01:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
cc00060ea39f46d63ac8f8569a567bd6c294eda2
buzz/desktop/src/features/agents/ui/effortTable.fixture.json
Will Pfleger cc00060ea3 feat(agent): Phase 2 — wire Rust and TS consumers to generated model-capabilities module (#3958) 
## What

Phase 2 consumer cutover targeting the
`duncan/databricks-model-label-registry` umbrella branch. Wires
`crates/**` and `desktop/**` consumers to the generated capability
module introduced in Phase 1 (#3821), while keeping old and new paths
both live for differential testing. Phase 3 removes the old paths.

## Commits (boundary-separated)

### feat(agent): Phase 2a — wire Rust consumers to generated capability
module (`crates/**`, `scripts/**`)

- `catalog.rs`: `DATABRICKS_V2_KNOWN_MODELS` re-exported from the
generated module — single source of truth.
- `llm.rs`: `databricks_v2_route_for_model` delegates to
`resolve_model_capabilities("databricks_v2", model)`. Old segment-based
classifier preserved as `#[cfg(test)] _old_*` for the differential
harness. New `databricks_v2_route_differential_old_vs_new` test confirms
100% agreement on all 20 route vectors.
- `config.rs`: new `effort_table_fixture_differential_old_vs_new` test
runs `resolve_model_capabilities` over the 36-entry
`effortTable.fixture.json` and asserts old/new agree modulo a doc-cited
allowlist (4 F1 corrections).
- `scripts/run-differential.mjs`: JS differential harness over
effortTable fixture + normative corpus + catalog-sample fixture. 85
checks, 0 unexpected divergences (5 allowlisted: 4 F1 corrections +
goose-opus-5 anthropic route correction).
- `scripts/MODELS_DEV_RECONCILIATION.md`: deferred MINOR from Phase 1 —
8 trailing-double-space line breaks replaced with `<br>`.

### feat(desktop): Phase 2b — cut TS consumers to generated
model-capabilities module (`desktop/**`)

- `buzzAgentConfig.ts`: adds
`getProviderEffortConfigFromManifest(provider, model?)` — thin wrapper
over `resolveModelCapabilities()` from `modelCapabilities.ts`. Maps
`supportedEfforts → validValues` and `defaultEffort → defaultValue`
(null preserved for manual-budget/Inherit). Old
`getProviderEffortConfig()` and all hand-tables stay live for the
differential harness; Phase 3 retires them.
- `formatAgentModelLabel.ts`: registry-label lookup re-pointed from
hand-maintained `databricksModelNames.ts` import to generated
`DATABRICKS_MODEL_NAMES` exported from `modelCapabilities.ts`. Same Map
shape, identical contents, behavior unchanged.

### fix(scripts): add ts-esm-loader and fix allowlist coverage in
run-differential (`scripts/**`)

- `scripts/ts-esm-loader.mjs`: minimal ESM custom loader that resolves
extensionless relative TS imports. Required because Phase 2b's
`buzzAgentConfig.ts` imports `modelCapabilities` without `.ts` extension
— which Node's `--experimental-strip-types` runner cannot resolve
without a hook.
- `scripts/run-differential.mjs`: shebang updated to self-bootstrap with
the loader; fixes the `totalAllowlisted` counter (was declared but never
incremented — always printed `0 allowlisted`). Replaced with per-axis
hit tracking: reports exercised slot count (`N/total`) in summary; fails
with `STALE_ALLOWLIST` if any declared entry fires zero divergences,
preventing stale entries from silently masking future regressions.

## Verification

- `cargo test -p buzz-agent --lib`: 426/426
- Corpus: 45/45 · schema-negative: 24/24 · `--check` byte-clean
- Differential: 85 checks, 0 unexpected divergences, 6/6 allowlist slots
exercised
- Desktop: 3847/3847 · typecheck clean · biome clean
- Mobile: 1019 pass, 1 skipped — same 5 flaky tests in
`mobile/test/features/channels/` that reproduce at the umbrella base;
zero mobile files in this branch range
- `git diff --check`: clean

## What Remains (Phase 3)

Remove old hand-maintained paths: `_old_*` functions in
`llm.rs`/`config.rs`, old `getProviderEffortConfig` tables in
`buzzAgentConfig.ts`, old `databricksModelNames.ts` import in
`formatAgentModelLabel.ts`, old `databricks_model_names.rs` module.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz>
Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw <41ea58f1e64c243627e8acde7c89be667052ee6e17d8f021c1195be4324ebf04@buzz.block.builderlab.xyz>
2026-08-03 14:58:34 -04:00

262 lines
8.3 KiB
JSON
Raw Blame History

[
{
"note": "Anthropic manual-budget: claude-3 family",
"provider": "anthropic",
"model": "claude-3-7-sonnet-20250219",
"validValues": ["low", "medium", "high"],
"defaultValue": null
},
{
"note": "Anthropic manual-budget: claude-opus-4-5",
"provider": "anthropic",
"model": "claude-opus-4-5",
"validValues": ["low", "medium", "high"],
"defaultValue": null
},
{
"note": "Anthropic adaptive xhigh-capable: claude-opus-4-7",
"provider": "anthropic",
"model": "claude-opus-4-7",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive xhigh-capable: claude-opus-4-8",
"provider": "anthropic",
"model": "claude-opus-4-8",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive xhigh-capable: claude-sonnet-5",
"provider": "anthropic",
"model": "claude-sonnet-5-20260101",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive xhigh-capable: claude-fable-5",
"provider": "anthropic",
"model": "claude-fable-5",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive xhigh-capable: claude-opus-5",
"provider": "anthropic",
"model": "claude-opus-5",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive xhigh-capable: claude-mythos-5",
"provider": "anthropic",
"model": "claude-mythos-5",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive no-xhigh: claude-opus-4-6",
"provider": "anthropic",
"model": "claude-opus-4-6",
"validValues": ["low", "medium", "high", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive no-xhigh: claude-sonnet-4-6",
"provider": "anthropic",
"model": "claude-sonnet-4-6",
"validValues": ["low", "medium", "high", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic adaptive no-xhigh: claude-mythos-preview",
"provider": "anthropic",
"model": "claude-mythos-preview",
"validValues": ["low", "medium", "high", "max"],
"defaultValue": "high"
},
{
"note": "Anthropic unknown model: blank \u2014 assume full adaptive",
"provider": "anthropic",
"model": "",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "OpenAI gpt-5-pro: high only",
"provider": "openai",
"model": "gpt-5-pro",
"validValues": ["high"],
"defaultValue": "high"
},
{
"note": "OpenAI gpt-5.6: none/low/medium/high/xhigh/max",
"provider": "openai",
"model": "gpt-5.6",
"validValues": ["none", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "OpenAI gpt-5.5: none/low/medium/high/xhigh",
"provider": "openai",
"model": "gpt-5.5",
"validValues": ["none", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "OpenAI gpt-5.4: same table as gpt-5.5",
"provider": "openai",
"model": "gpt-5.4",
"validValues": ["none", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "OpenAI gpt-5.1: none/low/medium/high",
"provider": "openai",
"model": "gpt-5.1",
"validValues": ["none", "low", "medium", "high"],
"defaultValue": "none"
},
{
"note": "OpenAI gpt-5 base: minimal/low/medium/high",
"provider": "openai",
"model": "gpt-5",
"validValues": ["minimal", "low", "medium", "high"],
"defaultValue": "medium"
},
{
"note": "OpenAI unknown model (gpt-4o): all-except-max",
"provider": "openai",
"model": "gpt-4o",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "OpenAI empty model: all-except-max",
"provider": "openai",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "DatabricksV2 claude route (claude-opus-4-7): xhigh-capable anthropic table",
"provider": "databricks_v2",
"model": "claude-opus-4-7",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "DatabricksV2 claude route with databricks- prefix stripped",
"provider": "databricks_v2",
"model": "databricks-claude-opus-4-7",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "DatabricksV2 gpt-5.6-sol route: OpenAI max-capable table",
"provider": "databricks_v2",
"model": "gpt-5.6-sol",
"validValues": ["none", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "DatabricksV2 gpt-5-6-sol route: dashed OpenAI max-capable table",
"provider": "databricks_v2",
"model": "gpt-5-6-sol",
"validValues": ["none", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "DatabricksV2 gpt-5.4 route: OpenAI gpt-5.5/5.4 table",
"provider": "databricks_v2",
"model": "gpt-5.4",
"validValues": ["none", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "DatabricksV2 gpt-5.1 with databricks- prefix: OpenAI gpt-5.1 table",
"provider": "databricks_v2",
"model": "databricks-gpt-5.1",
"validValues": ["none", "low", "medium", "high"],
"defaultValue": "none"
},
{
"note": "DatabricksV2 concrete non-claude non-gpt5 (llama-3): MLflow path, all-except-max",
"provider": "databricks_v2",
"model": "llama-3",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "DatabricksV2 blank model: route unknown, all-7",
"provider": "databricks_v2",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "databricks v1: routes like openai unknown, all-except-max",
"provider": "databricks",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "openai-compat: canonicalizes to openai, empty model → all-except-max with medium default",
"provider": "openai-compat",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "openai-compat/gpt-5-pro: canonicalizes to openai, gpt-5-pro → [high] only",
"provider": "openai-compat",
"model": "gpt-5-pro",
"validValues": ["high"],
"defaultValue": "high"
},
{
"note": "openrouter: all-7 with medium default",
"provider": "openrouter",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "empty provider: all-7 with medium default",
"provider": "",
"model": "",
"validValues": ["none", "minimal", "low", "medium", "high", "xhigh", "max"],
"defaultValue": "medium"
},
{
"note": "databricks_v2 goose-claude-fable-5: strips goose- prefix, routes anthropic adaptive+xhigh, max valid",
"provider": "databricks_v2",
"model": "goose-claude-fable-5",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "databricks_v2 goose-gpt-5.5: strips goose- prefix, routes openai gpt-5.5 table (none+low-xhigh, no minimal)",
"provider": "databricks_v2",
"model": "goose-gpt-5.5",
"validValues": ["none", "low", "medium", "high", "xhigh"],
"defaultValue": "medium"
},
{
"note": "databricks_v2 goose-claude-sonnet-5: strips goose- prefix, routes anthropic adaptive+xhigh",
"provider": "databricks_v2",
"model": "goose-claude-sonnet-5",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
},
{
"note": "databricks_v2 arbitrary prefix team-x-claude-opus-4-7: strips to claude-opus-4-7, routes anthropic adaptive+xhigh, max valid",
"provider": "databricks_v2",
"model": "team-x-claude-opus-4-7",
"validValues": ["low", "medium", "high", "xhigh", "max"],
"defaultValue": "high"
}
]
Reference in New Issue View Git Blame Copy Permalink