Commit Graph

4 Commits

Author SHA1 Message Date
Kalvin C
0affe527e8 fix(model-capabilities): humanize databricks goose model names (#7135)
🤖
## Summary
- add curated human-readable labels for Databricks Goose models that
otherwise render as fully qualified identifiers
- render `data_workflow_tools.goose.goose-glm-5-3` as `GLM-5.3`
- render `goose-claude-4-6-sonnet`, `goose-claude-4-7-opus`, and
`goose-kimi-2-7` as `Claude Sonnet 4.6`, `Claude Opus 4.7`, and `Kimi
2.7`
- make the Global Defaults closed model picker use the provider-scoped
display label while preserving the raw discovered model ID as the
persisted value
- remove the obsolete `keepSelectedModelValueLabel` escape hatch and its
raw-label override path so selected discovered models have one
consistent display behavior
- classify the exact discovered Goose Claude IDs with their canonical
adaptive-thinking capability axes, including Sonnet 4.6's exclusion of
`xhigh`
- expand Rust and TypeScript alias coverage and regenerate the shared
139-vector capability corpus

## Test plan
- `cargo test -p buzz-agent --lib` — 517 passed, 1 ignored
- `cd desktop && pnpm test` — 5,821 passed
- Desktop TypeScript typecheck — passed
- Biome on the changed component — passed
- `git diff --check` — passed
- targeted Playwright Global Defaults regression — passed on the
preceding implementation head; the subsequent commit only removes dead
picker-prop plumbing

Verified at `b9609d12696173aa309d2dbaf4f093a502756c36`. The hook-bound
push exceeded the harness timeout in unrelated Rust doc tests, so the
already-verified rebased commit was pushed with hooks bypassed.

Follow-up to #6955.

---------

Signed-off-by: Kalvin Chau <kalvin@block.xyz>
Co-authored-by: am <6e30cd56c30e030cd31bb0939b94a7c257c9a09d5ba2d92cf2735da45629f248@buzz.block.builderlab.xyz>
2026-08-31 17:13:59 -07:00
Will Pfleger
ae0ecd9a67 feat(model-capabilities): humanize Databricks UC model families (#6955)
Chat-eligible Unity Catalog models from the Gemini, DeepSeek, GLM, Grok,
Llama, Qwen, Gemma, and Inkling families were rendering as raw FQNs such
as `system.ai.gemini-3-5-flash` in the agent model picker. #6918
surfaces these models, but the capability manifest carried no records
for most of them and the registry-label path could not strip their Unity
Catalog or `goose-` prefixes.

I added a display-only `label_family_tokens` list and `databricks_v2`
exact records for the sixteen previously unlabeled endpoint stems. All
26 target FQNs and their `goose-` aliases now resolve to curated labels
through `databricks_registry_label`, while capability resolution
continues to use only the existing `family_tokens` (`claude-`, `gpt-`,
and `kimi-`). This keeps label discovery from changing capability
profiles for unrelated model IDs.

Each new record includes reconciliation metadata against the pinned
[models.dev catalog](https://models.dev/api.json). Nine records adopt
first-party effort evidence that the Databricks MLflow Chat transport
can express: Gemini 3.5 Flash, Gemini 3.5 Flash Lite, Gemini 3.6 Flash,
Gemini 3 Pro Image, DeepSeek V4 Flash, DeepSeek V4 Pro, GLM-5.3 Flash,
Grok 4.6, and Inkling. The remaining seven deliberately retain the
`databricks_v2/concrete_unknown` fallback because upstream evidence is
absent, identifies a non-reasoning model, or provides only
toggle/token-budget controls that `reasoning_effort` cannot represent.

The normative corpus adds the sixteen exact records, four Unity Catalog
alias probes, and two capability-isolation probes. Existing base vectors
remain unchanged, and Rust and TypeScript consume the same strict
manifest and corpus.

Related: #6918

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Thufir <7ebdb0b67dab08a570b9faf7bbada97535673b4ccaba2cbd546ad3ba84c87fa6@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Thufir <7ebdb0b67dab08a570b9faf7bbada97535673b4ccaba2cbd546ad3ba84c87fa6@buzz.block.builderlab.xyz>
2026-08-27 17:25:19 -04:00
Will Pfleger
2ce8df8533 fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360)
Curates Databricks model labels through prefix-stripping aliasing and
materializes exact records for five endpoints missing from the
Databricks catalog.

**Alias-aware label resolution** (Databricks v2)
- Strip workspace-specific prefixes (`goose-`, `team-x-`, etc.) before
exact-record lookup so any prefixed alias resolves to its canonical
label without enumerating every alias variant.
- Return `None` for uncurated ids instead of falling back to a raw
string; callers control the fallback display.

**New exact records — label-only (Anthropic Messages route, axes from
family rules)**
- `databricks-claude-fable-5` → "Claude Fable 5" (materialized from
`anthropic-adaptive-xhigh-fable-5`)
- `databricks-claude-opus-4-8` → "Claude Opus 4.8" (materialized from
`anthropic-adaptive-xhigh-opus-4-8`)
- `databricks-claude-opus-5` → "Claude Opus 5" (materialized from
`anthropic-adaptive-xhigh-opus-5`)
- `databricks-claude-sonnet-5` → "Claude Sonnet 5" (materialized from
`anthropic-adaptive-xhigh-sonnet-5`)

**New capability-bearing record — Kimi K3 (MLflow Chat route)**
- `databricks-kimi-k3` → "Kimi K3" with axes from models.dev Moonshot
catalog: reasoning toggle + effort `[low, high, max]`, no default;
`_reconciliation_doc` records the source and reconciliation policy.

**Corpus and tests**
- 6 new normative-corpus vectors (canonical + one `goose-*` alias per
new id); executable vector count updated to 113 in both Rust and TS
gates.
- Rust and TS label tests extended to cover all 3 new canonical ids and
their aliases via the prefix stripper.
- Sentinel variable fix in discovery-provider test to prevent false
failure in Databricks dev environments where `BUZZ_AGENT_PROVIDER` is
set.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
2026-08-20 13:46:10 -04:00
Will Pfleger
1b7e5ac1be feat(model-capabilities): drive model capabilities and labels from one manifest (#5597)
## Summary

Centralizes model capability knowledge — thinking mode, supported effort
levels, wire routes, and human-readable labels — into a single manifest,
`scripts/model-capabilities.json`. Rust and TypeScript each get a small
interpreter that reads the same manifest, replacing hand-maintained
tables scattered across both languages that had already drifted apart. A
capability change is now a data edit, not parallel edits to two code
paths. Supersedes the codegen approach explored in #3603.

A cross-language contract keeps the two interpreters honest:
`scripts/normative-corpus.json` is a golden snapshot generated from the
Rust resolver (103 vectors covering all six capability axes) and
replayed natively in TS. CI fails if either language disagrees with the
corpus or the corpus drifts from the resolver. Regenerate with `just
regen-model-corpus`.

## Behavior changes

- **Effort dropdown for `openai-compat` providers** no longer offers
`max`. The request path always clamped `max` to `xhigh` on the wire, so
the UI stops offering a value that was silently rewritten. UI-only,
wire-identical.
- **Databricks v2 routing (wire-visible):** uncurated endpoint names
carrying a bare Claude code-name segment (e.g. `goose-opus-5`) now route
to the MLflow chat wire instead of Anthropic Messages — they lose
Anthropic prompt caching but still succeed on a valid OpenAI-compatible
wire. Curated `databricks-claude-*` records and any name starting with
`claude` are unchanged. A handful of other uncurated/adversarial name
shapes similarly fall back to MLflow chat instead of pattern-matched
routes; every curated model resolves identically to before, all axes.
- **Curated model labels on the real discovery path.** The Databricks
API returns no display name, so discovery emits the raw endpoint id as
the model `name` (`{id, name: id}`) on every path. `ModelEntry.name` is
now curated at all four construction seams in `buzz-agent` — v2
discovery, v1 parse, the auth-empty default catalog, and the
configured-model fallback — via a read-only `databricks_registry_label`
lookup over the manifest's `databricks_v2` exact records; `id` stays the
raw wire/config value. A known id renders its curated label
(`databricks-gpt-5-5` → `GPT-5.5`), an unknown id passes through
unchanged, and the default-catalog row reads `GPT-5.5 (default
catalog)`. As a defense against older `buzz-agent` binaries and any
harness that echoes ids, `resolveModelLabel` treats a discovered name
equal to the trimmed id as absent and falls through to the registry
tier; a genuinely distinct name (including the suffixed default-catalog
label) still wins.

## Cleanup

Deletes the duplicated capability tables and their tests: the
`config.rs` gpt5 matchers, effort tables, and clamp logic; the legacy
segment-based Databricks v2 route classifier in `llm.rs`; and the TS
hand tables plus `effortTable.fixture.json`. All are replaced by
manifest lookups through the shared resolver — no line of capability
data exists in two places.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
2026-08-17 11:23:28 -04:00