jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 12:40:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360)

Browse Source 
Curates Databricks model labels through prefix-stripping aliasing and
materializes exact records for five endpoints missing from the
Databricks catalog.

**Alias-aware label resolution** (Databricks v2)
- Strip workspace-specific prefixes (`goose-`, `team-x-`, etc.) before
exact-record lookup so any prefixed alias resolves to its canonical
label without enumerating every alias variant.
- Return `None` for uncurated ids instead of falling back to a raw
string; callers control the fallback display.

**New exact records — label-only (Anthropic Messages route, axes from
family rules)**
- `databricks-claude-fable-5` → "Claude Fable 5" (materialized from
`anthropic-adaptive-xhigh-fable-5`)
- `databricks-claude-opus-4-8` → "Claude Opus 4.8" (materialized from
`anthropic-adaptive-xhigh-opus-4-8`)
- `databricks-claude-opus-5` → "Claude Opus 5" (materialized from
`anthropic-adaptive-xhigh-opus-5`)
- `databricks-claude-sonnet-5` → "Claude Sonnet 5" (materialized from
`anthropic-adaptive-xhigh-sonnet-5`)

**New capability-bearing record — Kimi K3 (MLflow Chat route)**
- `databricks-kimi-k3` → "Kimi K3" with axes from models.dev Moonshot
catalog: reasoning toggle + effort `[low, high, max]`, no default;
`_reconciliation_doc` records the source and reconciliation policy.

**Corpus and tests**
- 6 new normative-corpus vectors (canonical + one `goose-*` alias per
new id); executable vector count updated to 113 in both Rust and TS
gates.
- Rust and TS label tests extended to cover all 3 new canonical ids and
their aliases via the prefix stripper.
- Sentinel variable fix in discovery-provider test to prevent false
failure in Databricks dev environments where `BUZZ_AGENT_PROVIDER` is
set.

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
This commit is contained in:
Will Pfleger
2026-08-20 13:46:10 -04:00
committed by GitHub
parent d274a6e949
commit 2ce8df8533
10 changed files with 578 additions and 59 deletions
Download Patch File Download Diff File Expand all files Collapse all files

112
scripts/model-capabilities.json
View File

@@ -8,7 +8,8 @@
},
"family_tokens": [
"claude-",
"gpt-"
"gpt-",
"kimi-"
],
"family_rules": [
{
@@ -371,6 +372,24 @@
"databricks_v2_wire_route": "openai-responses",
"normalization_policy": "openai-standard"
},
{
"id": "dbv2-kimi-k3-exact",
"_comment": "Databricks Kimi K3 aliases share the exact record's upstream Moonshot capabilities. Kimi is absent from the Databricks models.dev catalog, so retain the established MLflow Chat route; upstream models.dev advertises a reasoning toggle and effort values low|high|max, with no documented default. The toggle is not representable on the MLflow Chat request schema (only reasoning_effort is), so thinking_mode stays none, matching the kimi-k2-7-code precedent.",
"match_kind": "exact",
"match_value": "kimi-k3",
"providers": [
"databricks_v2"
],
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard"
},
{
"id": "dbv2-claude-prefix",
"_comment": "DBv2-only broad Claude prefix. Replaces the routing role of the dropped dbv2-claude-code-names-segment 'claude' token: uncurated databricks-claude-* endpoints still route via Anthropic Messages with conservative (omit-fields) effort classification. Bare code-name segments without a leading 'claude-' (e.g. opus-5, goose-opus-5) are deliberately dropped and fall through to the databricks_v2 concrete-unknown fallback (mlflow-chat) — segment-anywhere matching is not expressible as a prefix.",
@@ -436,6 +455,78 @@
"_reconciliation_note": "models.dev advertises low|medium|high. Same as gpt-5-4-mini. Adopt.",
"_reconciliation_doc": "https://models.dev/api.json (retrieved 2026-07-31, SHA-256 d5a4974cd69f19b0f67713acaa6bb3b16e920defdc07ecbdf6b0a936181bb0e0): providers.databricks.models[\"databricks-gpt-5-4-nano\"].reasoning_options=[{\"type\":\"effort\",\"values\":[\"low\",\"medium\",\"high\"]}]"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-fable-5",
"registry_label": "Claude Fable 5",
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"_provenance": "all axes materialized from family:anthropic-adaptive-xhigh-fable-5",
"_source": "models.dev anthropic catalog label; Databricks workspace endpoint is absent from its provider catalog"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-opus-4-8",
"registry_label": "Claude Opus 4.8",
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"_provenance": "all axes materialized from family:anthropic-adaptive-xhigh-opus-4-8",
"_source": "models.dev anthropic catalog label; Databricks workspace endpoint is absent from its provider catalog"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-opus-5",
"registry_label": "Claude Opus 5",
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"_provenance": "all axes materialized from family:anthropic-adaptive-xhigh-opus-5",
"_source": "models.dev anthropic catalog label; Databricks workspace endpoint is absent from its provider catalog"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-sonnet-5",
"registry_label": "Claude Sonnet 5",
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"_provenance": "all axes materialized from family:anthropic-adaptive-xhigh-sonnet-5",
"_source": "models.dev anthropic catalog label; Databricks workspace endpoint is absent from its provider catalog"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-gpt-5-6-sol",
@@ -932,6 +1023,25 @@
"_provenance": "all axes materialized from provider fallback (databricks_v2/concrete_unknown)",
"_source": "registry_labels"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-kimi-k3",
"registry_label": "Kimi K3",
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"_provenance": "databricks_v2_wire_route: provider fallback (databricks_v2/concrete_unknown); thinking_mode+supported_efforts: models.dev Moonshot Kimi K3 catalog; default_effort: no documented default",
"_source": "models.dev Moonshot Kimi K3 reasoning_options: toggle + low|high|max",
"_reconciliation": "adopt",
"_reconciliation_note": "models.dev advertises toggleable reasoning with [low, high, max], but no default. The Databricks endpoint is absent from its provider catalog, so retain the established MLflow Chat route, adopt the upstream capability set, and leave the effort unset rather than invent a Databricks default. The MLflow Chat request schema cannot express a reasoning toggle (only reasoning_effort is representable), so thinking_mode maps to none rather than the upstream toggle, matching the kimi-k2-7-code precedent.",
"_reconciliation_doc": "https://models.dev/api.json (retrieved 2026-08-20, SHA-256 7ccb5635f682e4248ad8d39f515fbe3f2bb10e67bbc7fa1b94e66dddb00779e5): providers.moonshotai.models[\"kimi-k3\"].reasoning_options=[{\"type\":\"toggle\"},{\"type\":\"effort\",\"values\":[\"low\",\"high\",\"max\"]}]; Databricks endpoint absent from providers.databricks.models"
},
{
"provider": "databricks_v2",
"raw_model_id": "databricks-kimi-k2-7-code",

196
scripts/normative-corpus.json
View File

@@ -680,6 +680,202 @@
"registry_label": "GPT-5.4 mini"
}
},
{
"id": "dbv2-claude-fable-5-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-fable-5",
"_note": "Probes the canonical Databricks Fable 5 endpoint record.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": "Claude Fable 5"
}
},
{
"id": "dbv2-goose-claude-fable-5-alias-probe",
"provider": "databricks_v2",
"raw_model_id": "goose-claude-fable-5",
"_note": "Probes a prefixed alias of the Databricks Fable 5 endpoint.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": null
}
},
{
"id": "dbv2-claude-opus-4-8-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-opus-4-8",
"_note": "Probes the canonical Databricks Opus 4.8 endpoint record.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": "Claude Opus 4.8"
}
},
{
"id": "dbv2-goose-claude-opus-4-8-alias-probe",
"provider": "databricks_v2",
"raw_model_id": "goose-claude-opus-4-8",
"_note": "Probes a prefixed alias of the Databricks Opus 4.8 endpoint.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": null
}
},
{
"id": "dbv2-claude-opus-5-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-opus-5",
"_note": "Probes the canonical Databricks Opus 5 endpoint record.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": "Claude Opus 5"
}
},
{
"id": "dbv2-goose-claude-opus-5-alias-probe",
"provider": "databricks_v2",
"raw_model_id": "goose-claude-opus-5",
"_note": "Probes a prefixed alias of the Databricks Opus 5 endpoint.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": null
}
},
{
"id": "dbv2-claude-sonnet-5-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-claude-sonnet-5",
"_note": "Probes the canonical Databricks Sonnet 5 endpoint record.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": "Claude Sonnet 5"
}
},
{
"id": "dbv2-goose-claude-sonnet-5-alias-probe",
"provider": "databricks_v2",
"raw_model_id": "goose-claude-sonnet-5",
"_note": "Probes a prefixed alias of the Databricks Sonnet 5 endpoint.",
"expect": {
"thinking_mode": "adaptive",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": "high",
"databricks_v2_wire_route": "anthropic-messages",
"normalization_policy": "none",
"registry_label": null
}
},
{
"id": "dbv2-kimi-k3-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-kimi-k3",
"_note": "Probes the canonical Databricks Kimi K3 endpoint record.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Kimi K3"
}
},
{
"id": "dbv2-goose-kimi-k3-alias-probe",
"provider": "databricks_v2",
"raw_model_id": "goose-kimi-k3",
"_note": "Probes a prefixed alias of the Databricks Kimi K3 endpoint.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": null
}
},
{
"id": "resolver-prefixed-alias-probe",
"provider": "databricks_v2",

4
scripts/run-tests.sh
View File

@@ -115,8 +115,8 @@ run_unit_tests() {
# buzz-agent model-capabilities corpus: the Rust half of the cross-language
# drift guard. model_capabilities.rs embeds scripts/model-capabilities.json +
# scripts/normative-corpus.json via include_str! and replays all 103 vectors
# as pure in-process tests (no infra). Mirrors the nextest path in
# scripts/normative-corpus.json via include_str! and replays the full locked
# corpus as pure in-process tests (no infra). Mirrors the nextest path in
# `just test-unit` — the two lists must stay in step.
run_test_step "buzz-agent unit tests" \
cargo test -p buzz-agent --lib -- --nocapture