mirror of
https://github.com/block/buzz.git
synced 2026-10-27 20:41:57 +00:00
d01e5f8205
New Databricks endpoints (Claude Opus 5.5, GPT-6 Astra/Sol, etc.) showed as raw FQNs in the model picker until someone added a curated corpus record for each. This adds a strict token-class grammar as a third label tier inside the Databricks label helpers (Rust `model_capabilities.rs` + TS `modelCapabilities.ts`): exact record → unique alias → **generated label** → raw id. In TS it fires only for an explicit `databricks_v2` provider; providerless ids stay raw. ## Grammar Takes the last component of a UC FQN (or strips one wrapper: `databricks-`, `goose-`, `kgoose-`, `builderbot-`; an id that is neither stays raw), splits on `-`, and classifies each token as version, letter+digit version, size, trailing date (dropped), or word. Labels are built only from the id's own text, and the grammar never reinterprets a version. When in doubt it returns raw: - **Known families only.** The family (leading letters of the first token) must be in the manifest's existing `label_family_tokens`. Custom endpoints such as `prod.ai.rag-agent-3` or `builderbot-pr-reviews-v2` stay raw. Supporting a new vendor means adding one entry to that list, not a record per model. - **Attached digits stay whole** (`gpt10` → `GPT10`). If the family already carries digits and the next token is a version (`goose-qwen3-5-122b-a10b`, `system.ai.llama3-1-8b`, `system.ai.claude3-5-sonnet`), the id stays raw; this is checked before any reorder. - A third consecutive number (`gemini-3-7-1-flash`) returns raw rather than splitting off a patch version. - Any non-ASCII input returns raw, checked before trimming or case folding. - Any unclassifiable token, or no version-bearing token (`builderbot-pr-reviews`), returns raw. `(latest)`/`Preview` never appear unless an exact record supplies them. Special cases, each justified by an unseen-id shape rather than the corpus score: - `meta-llama-` → `Llama`: a namespace convention. - Claude numeric-first reorder: Goose names Claude endpoints `claude-4-7-opus`. Only 1–2 version tokens followed by `opus`/`sonnet`/`haiku` reorder (`claude-5-6-opus` → `Claude Opus 5.6`); anything else keeps its order (`claude-5-6-thinking` → `Claude 5.6 Thinking`). Both current numeric-first ids also have exact records. - Qwen compact decimal: Qwen's own naming `qwen35` → `Qwen3.5` when the second digit is nonzero. ## Colliding labels in the picker The same model is often served from several sources, so humanized labels collide (four rows are all `GPT-5.5`). `disambiguateModelLabels` in `formatAgentModelLabel.ts` suffixes only colliding Databricks labels with their source: - UC FQN: its `catalog.schema`, e.g. `GPT-6 Astra (system.ai)` / `GPT-6 Astra (sandbox.public)`. - Workspace endpoint: its wrapper, e.g. `GPT-5.5 (databricks)` / `GPT-5.5 (goose)` / `GPT-5.5 (kgoose)`. - Still colliding after that (e.g. a date variant in the same schema): the full id, e.g. `Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925)`. Unique labels, raw-id rows, the default row, and other providers are unchanged. It runs in `getDiscoveredPersonaModelOptions` (covering `AgentDropdownSelect`, `PersonaModelCombobox`, `PersonaDropdownField`, whose closed triggers read the option label) and in `ModelPicker`. Model search in `AgentDropdownSelect` and `PersonaModelCombobox` also matches the raw id, so searching `system.ai` finds those rows. Rust `ModelEntry.name` is unchanged. ## What labels drive Picker values, persistence, routing, catalog dedup, `resolve()` and capabilities all stay keyed on the raw id; two ids sharing a label both remain selectable. Labels do drive picker search (alongside the raw id), the workspace catalog's equal-time sort, and `TeamIdentityCard`'s "Mixed models" summary, which compares lowercased base labels, so `goose-` and `system.ai` GPT-6 Astra read as one model there (correct, since they are the same model). ## Tests Shared fixtures in `scripts/databricks-label-fixtures.json` are replayed by both Rust and TS. Each row asserts its tier: `exact` rows match a record, `alias` rows resolve with generation off, `generated` rows return nothing with generation off (so a future alias or record cannot mask them), and `raw` rows return nothing either way. A curator guard requires that across all 57 exact records, the ids the grammar does not reproduce equal exactly the 10 curator-only ids. `disambiguateModelLabels` has unit tests for namespace, wrapper, and full-id-fallback collisions, untouched rows, other providers, and case-insensitive grouping. The defaults-picker e2e checks the suffixed rows and trigger, raw-id search, and that choosing the second uncurated FQN by its visible name persists its raw id and restores its label. An edit-agent e2e covers the existing-agent model dropdown (`#edit-agent-model`): both suffixed rows render, the `system.ai` row is chosen by name, and after save and reopen the agent stores the `system.ai` id and shows `GPT-6 Astra (system.ai)`. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
81 lines
5.9 KiB
JSON
81 lines
5.9 KiB
JSON
{
|
|
"_comment": "Shared Databricks display-label fixtures, replayed by crates/buzz-agent/src/databricks_label_grammar.rs and desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs. `lookup` rows run through the full label helper (exact record -> unique alias -> generative grammar -> raw); a null label means the raw id is shown. Every `generated` row must stay absent from exact records so a curated addition cannot mask a broken grammar. `curator_only_labels` lists the exact records whose curated label intentionally differs from what the grammar derives from the id: curator suffixes like `(latest)` / `Preview`, a version alias, or a versionless name the grammar deliberately refuses.",
|
|
"lookup": [
|
|
{ "id": "databricks-gpt-5-5", "tier": "exact", "label": "GPT-5.5" },
|
|
{ "id": "DATABRICKS-GPT-5-5", "tier": "exact", "label": "GPT-5.5" },
|
|
{ "id": "databricks-claude-sonnet-4", "tier": "exact", "label": "Claude Sonnet 4.5" },
|
|
{ "id": "databricks-gemini-3-pro", "tier": "exact", "label": "Gemini 3 Pro Preview" },
|
|
{ "id": "goose-claude-4-7-opus", "tier": "exact", "label": "Claude Opus 4.7" },
|
|
{ "id": "data_workflow_tools.goose.goose-claude-opus-5-5", "tier": "exact", "label": "Claude Opus 5.5" },
|
|
{ "id": "system.ai.claude-opus-5-5", "tier": "alias", "label": "Claude Opus 5.5" },
|
|
{ "id": "goose-gpt-5-6-sol", "tier": "alias", "label": "GPT-5.6 Sol" },
|
|
{ "id": "goose-claude-haiku-4-5", "tier": "alias", "label": "Claude Haiku 4.5 (latest)" },
|
|
{ "id": "system.ai.deepseek-v4-pro-0813", "tier": "alias", "label": "DeepSeek V4 Pro" },
|
|
{ "id": "builderbot-glm-5-2", "tier": "alias", "label": "GLM-5.2" },
|
|
{ "id": "system.ai.gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" },
|
|
{ "id": "data_workflow_tools.goose.goose-gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" },
|
|
{ "id": "data_workflow_tools.goose.goose-gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" },
|
|
{ "id": "data_workflow_tools.goose.goose-gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" },
|
|
{ "id": "system.ai.gemini-3-7-flash", "tier": "generated", "label": "Gemini 3.7 Flash" },
|
|
{ "id": "system.ai.gemini-3-8-flash", "tier": "generated", "label": "Gemini 3.8 Flash" },
|
|
{ "id": "system.ai.deepseek-v4-1-flash", "tier": "generated", "label": "DeepSeek V4.1 Flash" },
|
|
{ "id": "databricks-gpt-5-3-codex", "tier": "generated", "label": "GPT-5.3 Codex" },
|
|
{ "id": "databricks-gpt-5-5-pro", "tier": "generated", "label": "GPT-5.5 Pro" },
|
|
{ "id": "system.ai.mistral-large-3", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.grok-5-mini", "tier": "generated", "label": "Grok 5 Mini" },
|
|
{ "id": "system.ai.claude-opus-5-5-thinking", "tier": "generated", "label": "Claude Opus 5.5 Thinking" },
|
|
{ "id": "system.ai.gemini-3-7-flash-0925", "tier": "generated", "label": "Gemini 3.7 Flash" },
|
|
{ "id": "system.ai.phi-5-mini", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.minimax-m3", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.qwen36-35b-a3b", "tier": "generated", "label": "Qwen3.6 35B A3B" },
|
|
{ "id": "system.ai.gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" },
|
|
{ "id": "system.ai.gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" },
|
|
{ "id": "system.ai.nova12-pro", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.qwen10-32b", "tier": "generated", "label": "Qwen10 32B" },
|
|
{ "id": "system.ai.gpt10", "tier": "generated", "label": "GPT10" },
|
|
{ "id": "system.ai.qwen35-122b-a10b-instruct", "tier": "generated", "label": "Qwen3.5 122B A10B Instruct" },
|
|
{ "id": "system.ai.claude-5-6-opus", "tier": "generated", "label": "Claude Opus 5.6" },
|
|
{ "id": "system.ai.claude-5-6-v2", "tier": "generated", "label": "Claude 5.6 V2" },
|
|
{ "id": "system.ai.claude-5-6-thinking", "tier": "generated", "label": "Claude 5.6 Thinking" },
|
|
{ "id": "system.ai.gemini-3-7-1-flash", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.deepseek-v4-1-2-flash", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.\u212aimi-k4", "tier": "raw", "label": null },
|
|
{ "id": "databricks-\u212aimi-k4", "tier": "raw", "label": null },
|
|
{ "id": "\ufeffsystem.ai.nova-3\ufeff", "tier": "raw", "label": null },
|
|
{ "id": "\u0085system.ai.nova-3\u0085", "tier": "raw", "label": null },
|
|
{ "id": "goose-qwen3-5-122b-a10b", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.llama3-1-8b", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.claude3-5-sonnet", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.nova12-3-4", "tier": "raw", "label": null },
|
|
{ "id": "builderbot-pr-reviews-v2", "tier": "raw", "label": null },
|
|
{ "id": "prod.ai.rag-agent-3", "tier": "raw", "label": null },
|
|
{ "id": "dev.ai.rag-agent-3", "tier": "raw", "label": null },
|
|
{ "id": "builderbot-pr-reviews", "tier": "raw", "label": null },
|
|
{ "id": "gpt-5", "tier": "raw", "label": null },
|
|
{ "id": "gpt-5.5", "tier": "raw", "label": null },
|
|
{ "id": "qwen3:32b", "tier": "raw", "label": null },
|
|
{ "id": "claude-opus-4.7", "tier": "raw", "label": null },
|
|
{ "id": "command-r-plus", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.qwen3:32b", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.claude-opus-4.7", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.command-r-plus", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.gpt", "tier": "raw", "label": null },
|
|
{ "id": "system.ai.gpt-05", "tier": "raw", "label": null },
|
|
{ "id": "databricks-team-2025-01", "tier": "raw", "label": null },
|
|
{ "id": "custom-unlisted-endpoint", "tier": "raw", "label": null },
|
|
{ "id": " ", "tier": "raw", "label": null }
|
|
],
|
|
"curator_only_labels": [
|
|
"databricks-claude-haiku-4-5",
|
|
"databricks-claude-opus-4-1",
|
|
"databricks-claude-opus-4-5",
|
|
"databricks-claude-sonnet-4",
|
|
"databricks-claude-sonnet-4-5",
|
|
"databricks-gemini-3-1-flash-lite",
|
|
"databricks-gemini-3-1-pro",
|
|
"databricks-gemini-3-flash",
|
|
"databricks-gemini-3-pro",
|
|
"databricks-inkling"
|
|
]
|
|
}
|