From d01e5f82058463709a22e93bb4cd795da5f53e10 Mon Sep 17 00:00:00 2001 From: Will Pfleger Date: Wed, 23 Sep 2026 17:12:02 -0400 Subject: [PATCH] feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New Databricks endpoints (Claude Opus 5.5, GPT-6 Astra/Sol, etc.) showed as raw FQNs in the model picker until someone added a curated corpus record for each. This adds a strict token-class grammar as a third label tier inside the Databricks label helpers (Rust `model_capabilities.rs` + TS `modelCapabilities.ts`): exact record → unique alias → **generated label** → raw id. In TS it fires only for an explicit `databricks_v2` provider; providerless ids stay raw. ## Grammar Takes the last component of a UC FQN (or strips one wrapper: `databricks-`, `goose-`, `kgoose-`, `builderbot-`; an id that is neither stays raw), splits on `-`, and classifies each token as version, letter+digit version, size, trailing date (dropped), or word. Labels are built only from the id's own text, and the grammar never reinterprets a version. When in doubt it returns raw: - **Known families only.** The family (leading letters of the first token) must be in the manifest's existing `label_family_tokens`. Custom endpoints such as `prod.ai.rag-agent-3` or `builderbot-pr-reviews-v2` stay raw. Supporting a new vendor means adding one entry to that list, not a record per model. - **Attached digits stay whole** (`gpt10` → `GPT10`). If the family already carries digits and the next token is a version (`goose-qwen3-5-122b-a10b`, `system.ai.llama3-1-8b`, `system.ai.claude3-5-sonnet`), the id stays raw; this is checked before any reorder. - A third consecutive number (`gemini-3-7-1-flash`) returns raw rather than splitting off a patch version. - Any non-ASCII input returns raw, checked before trimming or case folding. - Any unclassifiable token, or no version-bearing token (`builderbot-pr-reviews`), returns raw. `(latest)`/`Preview` never appear unless an exact record supplies them. Special cases, each justified by an unseen-id shape rather than the corpus score: - `meta-llama-` → `Llama`: a namespace convention. - Claude numeric-first reorder: Goose names Claude endpoints `claude-4-7-opus`. Only 1–2 version tokens followed by `opus`/`sonnet`/`haiku` reorder (`claude-5-6-opus` → `Claude Opus 5.6`); anything else keeps its order (`claude-5-6-thinking` → `Claude 5.6 Thinking`). Both current numeric-first ids also have exact records. - Qwen compact decimal: Qwen's own naming `qwen35` → `Qwen3.5` when the second digit is nonzero. ## Colliding labels in the picker The same model is often served from several sources, so humanized labels collide (four rows are all `GPT-5.5`). `disambiguateModelLabels` in `formatAgentModelLabel.ts` suffixes only colliding Databricks labels with their source: - UC FQN: its `catalog.schema`, e.g. `GPT-6 Astra (system.ai)` / `GPT-6 Astra (sandbox.public)`. - Workspace endpoint: its wrapper, e.g. `GPT-5.5 (databricks)` / `GPT-5.5 (goose)` / `GPT-5.5 (kgoose)`. - Still colliding after that (e.g. a date variant in the same schema): the full id, e.g. `Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925)`. Unique labels, raw-id rows, the default row, and other providers are unchanged. It runs in `getDiscoveredPersonaModelOptions` (covering `AgentDropdownSelect`, `PersonaModelCombobox`, `PersonaDropdownField`, whose closed triggers read the option label) and in `ModelPicker`. Model search in `AgentDropdownSelect` and `PersonaModelCombobox` also matches the raw id, so searching `system.ai` finds those rows. Rust `ModelEntry.name` is unchanged. ## What labels drive Picker values, persistence, routing, catalog dedup, `resolve()` and capabilities all stay keyed on the raw id; two ids sharing a label both remain selectable. Labels do drive picker search (alongside the raw id), the workspace catalog's equal-time sort, and `TeamIdentityCard`'s "Mixed models" summary, which compares lowercased base labels, so `goose-` and `system.ai` GPT-6 Astra read as one model there (correct, since they are the same model). ## Tests Shared fixtures in `scripts/databricks-label-fixtures.json` are replayed by both Rust and TS. Each row asserts its tier: `exact` rows match a record, `alias` rows resolve with generation off, `generated` rows return nothing with generation off (so a future alias or record cannot mask them), and `raw` rows return nothing either way. A curator guard requires that across all 57 exact records, the ids the grammar does not reproduce equal exactly the 10 curator-only ids. `disambiguateModelLabels` has unit tests for namespace, wrapper, and full-id-fallback collisions, untouched rows, other providers, and case-insensitive grouping. The defaults-picker e2e checks the suffixed rows and trigger, raw-id search, and that choosing the second uncurated FQN by its visible name persists its raw id and restores its label. An edit-agent e2e covers the existing-agent model dropdown (`#edit-agent-model`): both suffixed rows render, the `system.ai` row is chosen by name, and after save and reopen the agent stores the `system.ai` id and shows `GPT-6 Astra (system.ai)`. --------- Signed-off-by: Will Pfleger Signed-off-by: Duncan Co-authored-by: Duncan --- crates/buzz-agent/src/catalog.rs | 15 +- .../src/databricks_label_grammar.rs | 281 ++++++++++++++++++ crates/buzz-agent/src/lib.rs | 4 +- crates/buzz-agent/src/model_capabilities.rs | 76 +++-- .../agents/lib/agentCardModelLabel.test.mjs | 11 + .../lib/disambiguateModelLabels.test.mjs | 82 +++++ .../agents/lib/formatAgentModelLabel.ts | 70 ++++- .../src/features/agents/ui/ModelPicker.tsx | 29 +- .../agents/ui/PersonaModelCombobox.tsx | 7 +- .../agents/ui/agentConfigControls.tsx | 10 +- .../ui/databricksLabelFixtures.test.mjs | 56 ++++ .../features/agents/ui/modelCapabilities.ts | 158 +++++++++- .../agents/ui/usePersonaModelDiscovery.ts | 16 +- desktop/tests/e2e/edit-agent.spec.ts | 76 +++++ .../global-agent-config-screenshots.spec.ts | 87 ++++++ scripts/databricks-label-fixtures.json | 80 +++++ 16 files changed, 998 insertions(+), 60 deletions(-) create mode 100644 crates/buzz-agent/src/databricks_label_grammar.rs create mode 100644 desktop/src/features/agents/lib/disambiguateModelLabels.test.mjs create mode 100644 desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs create mode 100644 scripts/databricks-label-fixtures.json diff --git a/crates/buzz-agent/src/catalog.rs b/crates/buzz-agent/src/catalog.rs index 8f90c3e445..38152b3f90 100644 --- a/crates/buzz-agent/src/catalog.rs +++ b/crates/buzz-agent/src/catalog.rs @@ -27,8 +27,9 @@ use crate::{ /// A discovered model entry: `id` is the picker value (the raw endpoint id or /// Unity Catalog model-service FQN, and the wire/config value), `name` is the /// display label. Databricks catalog APIs do not provide a consistently useful -/// picker label, so discovery curates names from the capability manifest when -/// an exact known id exists and otherwise uses the raw id. +/// picker label, so discovery derives `name` as exact record → unique alias → +/// generated label grammar (known `label_family_tokens` families only) → raw +/// id. Adding a vendor means adding one `label_family_tokens` entry. #[derive(Debug, Clone, PartialEq, Eq)] pub struct ModelEntry { pub id: String, @@ -80,12 +81,10 @@ const UNITY_CATALOG_DESCRIPTOR: CatalogDescriptor = CatalogDescripto parse_page: parse_uc_model_services_page, }; -/// Curated display label for a discovered Databricks endpoint or model-service -/// id. Unknown ids deliberately pass through unchanged. +/// Display label for a discovered Databricks endpoint or model-service id. Ids +/// the label grammar cannot fully parse deliberately pass through unchanged. fn curated_model_name(id: &str) -> String { - crate::model_capabilities::databricks_registry_label(id) - .unwrap_or(id) - .to_string() + crate::model_capabilities::databricks_registry_label(id).unwrap_or_else(|| id.to_string()) } /// Fallback catalog used only when both authenticated Databricks v2 catalogs @@ -1903,7 +1902,7 @@ mod tests { // `name` is the curated label + provenance suffix, not the raw id. assert!(models.iter().all(|model| { let label = crate::model_capabilities::databricks_registry_label(&model.id) - .unwrap_or(model.id.as_str()); + .unwrap_or_else(|| model.id.clone()); model.name == format!("{label}{AUTHENTICATED_EMPTY_CATALOG_SUFFIX}") })); } diff --git a/crates/buzz-agent/src/databricks_label_grammar.rs b/crates/buzz-agent/src/databricks_label_grammar.rs new file mode 100644 index 0000000000..0a30b9b9f7 --- /dev/null +++ b/crates/buzz-agent/src/databricks_label_grammar.rs @@ -0,0 +1,281 @@ +//! Deterministic display-label grammar for Databricks endpoint ids. +//! +//! The last tier of [`crate::model_capabilities::databricks_registry_label`], +//! reached only after an exact-record miss and a unique-alias miss. The result +//! is either a label built solely from the id's own tokens or `None`, in which +//! case callers show the raw id. It is presentation-only: capabilities, wire +//! routing, and the saved model id never depend on it. Ids from families not +//! in the manifest's `label_family_tokens` stay raw; supporting a new vendor +//! means adding one entry to that list, not a record per model. +//! +//! Mirrored by `desktop/src/features/agents/ui/modelCapabilities.ts`; both +//! replay `scripts/databricks-label-fixtures.json`. + +use crate::model_capabilities::is_databricks_model_service_fqn; + +/// Endpoint-name wrappers that precede the model name in Databricks ids. +const WRAPPERS: [&str; 4] = ["databricks-", "goose-", "kgoose-", "builderbot-"]; + +enum Part { + /// Numeric version (`5`, `5.5`) — the one part GPT/GLM join with a hyphen. + Version(String), + Word(String), +} + +/// Parse a Databricks endpoint id into a display label, or `None` when any part +/// of the id falls outside the grammar. Only families listed in the manifest's +/// `label_family_tokens` are named, so custom endpoints (`rag-agent-3`) stay raw. +pub(crate) fn generate_databricks_label( + raw_model_id: &str, + family_tokens: &[String], +) -> Option { + // Refuse non-ASCII before trimming or folding, so no Unicode case or + // whitespace rule can turn an unsupported id into an ASCII-looking one. + if !raw_model_id.is_ascii() { + return None; + } + let id = raw_model_id.trim().to_ascii_lowercase(); + let is_fqn = is_databricks_model_service_fqn(&id); + let service = if is_fqn { + id.rsplit('.').next()? + } else { + id.as_str() + }; + // Only a UC FQN or a wrapped endpoint name is attributable to Databricks; a + // bare id such as `gpt-5` stays raw. + let body = match WRAPPERS.iter().find_map(|w| service.strip_prefix(w)) { + Some(body) => body, + None if is_fqn => service, + None => return None, + }; + let body = body + .strip_prefix("meta-") + .filter(|rest| rest.starts_with("llama-")) + .unwrap_or(body); + + let mut tokens = body.split('-'); + let (family, stem) = split_family(tokens.next()?)?; + if !family_tokens + .iter() + .any(|t| t.strip_suffix('-').unwrap_or(t) == family) + { + return None; + } + let mut rest: Vec<&str> = tokens.collect(); + // Attached digits followed by a version (`qwen3-5`, `llama3-1`) cannot be + // read without guessing; checked before any reorder. + if stem.is_some() && rest.first().is_some_and(|t| is_version(t)) { + return None; + } + if family == "claude" { + // Goose's numeric-first Claude ids (`claude-4-7-opus`) name the tier + // after the version. Reorder only that exact shape; any other token + // after the version keeps its position. + let digits = rest.iter().take_while(|t| is_version(t)).count(); + if (1..=2).contains(&digits) + && rest + .get(digits) + .is_some_and(|t| matches!(*t, "opus" | "sonnet" | "haiku")) + { + rest[..=digits].rotate_right(1); + } + } + + let mut parts = Vec::new(); + let mut has_number = stem.is_some(); + let mut after_minor = false; + let mut rest = rest.into_iter().peekable(); + while let Some(tok) = rest.next() { + if is_date(tok) && rest.peek().is_none() { + break; + } + let part = if is_version(tok) { + let minor = rest.next_if(|t| is_version(t)); + after_minor = minor.is_some(); + if after_minor && rest.peek().is_some_and(|t| is_version(t)) { + // A third number (`3-7-1`) would read as a separate version. + return None; + } + Part::Version(minor.map_or_else(|| tok.to_string(), |m| format!("{tok}.{m}"))) + } else if is_letter_version(tok) { + let minor = rest.next_if(|t| is_version(t)); + if minor.is_some() && rest.peek().is_some_and(|t| is_version(t)) { + return None; + } + let major = capitalize(tok); + Part::Word(minor.map_or_else(|| major.clone(), |m| format!("{major}.{m}"))) + } else if is_size(tok) { + Part::Word(tok.to_ascii_uppercase()) + } else if !tok.is_empty() && tok.bytes().all(|b| b.is_ascii_lowercase()) { + Part::Word(match tok { + "oss" => "OSS".to_string(), + "mini" | "nano" if family == "gpt" && after_minor => tok.to_string(), + _ => capitalize(tok), + }) + } else { + return None; + }; + has_number |= match &part { + Part::Version(_) => true, + Part::Word(word) => word.bytes().any(|b| b.is_ascii_digit()), + }; + parts.push(part); + } + // A versionless id (`builderbot-pr-reviews`) is a named endpoint, not a model. + if !has_number { + return None; + } + + let (brand, hyphenated) = match family { + "gpt" => ("GPT".to_string(), true), + "glm" => ("GLM".to_string(), true), + "deepseek" => ("DeepSeek".to_string(), false), + _ => (capitalize(family), false), + }; + let mut label = brand + &stem.unwrap_or_default(); + for (index, part) in parts.iter().enumerate() { + let (sep, text) = match part { + Part::Version(v) if index == 0 && hyphenated => ('-', v), + Part::Version(text) | Part::Word(text) => (' ', text), + }; + label.push(sep); + label.push_str(text); + } + Some(label) +} + +/// Split the leading token into its family name and an optional in-name +/// version, kept whole (`nova12` → `12`). The one exception is Qwen's own +/// compact-decimal naming: `qwen35` → `3.5` when the second digit is nonzero. +fn split_family(tok: &str) -> Option<(&str, Option)> { + let alpha = tok.bytes().take_while(u8::is_ascii_lowercase).count(); + let (family, digits) = tok.split_at(alpha); + if family.is_empty() || digits.len() > 2 || !digits.bytes().all(|b| b.is_ascii_digit()) { + return None; + } + let stem = match digits.as_bytes() { + [] => None, + [major, minor] if family == "qwen" && *minor != b'0' => { + Some(format!("{}.{}", *major as char, *minor as char)) + } + _ => Some(digits.to_string()), + }; + Some((family, stem)) +} + +/// `0` or a 1–2-digit number without a leading zero. +fn is_version(tok: &str) -> bool { + match tok.as_bytes() { + [d] => d.is_ascii_digit(), + [a, b] => (b'1'..=b'9').contains(a) && b.is_ascii_digit(), + _ => false, + } +} + +/// A 4- or 8-digit date stamp (`0731`, `20260101`). +fn is_date(tok: &str) -> bool { + matches!(tok.len(), 4 | 8) && tok.bytes().all(|b| b.is_ascii_digit()) +} + +/// A letter-prefixed version such as `v4` or `k2`. +fn is_letter_version(tok: &str) -> bool { + matches!(tok.as_bytes(), [l, digits @ ..] if l.is_ascii_lowercase() + && !digits.is_empty() + && digits.iter().all(u8::is_ascii_digit)) +} + +/// A parameter-size token: `120b`, or letter-prefixed `a3b`. +fn is_size(tok: &str) -> bool { + let Some(body) = tok.strip_suffix('b') else { + return false; + }; + let digits = body + .strip_prefix(|c: char| c.is_ascii_lowercase()) + .unwrap_or(body); + !digits.is_empty() && digits.bytes().all(|b| b.is_ascii_digit()) +} + +fn capitalize(word: &str) -> String { + let mut chars = word.chars(); + chars.next().map_or_else(String::new, |first| { + first.to_ascii_uppercase().to_string() + chars.as_str() + }) +} + +#[cfg(test)] +mod tests { + use crate::model_capabilities::{databricks_curated_label, databricks_registry_label}; + use serde_json::Value; + + const FIXTURES_JSON: &str = include_str!("../../../scripts/databricks-label-fixtures.json"); + const MANIFEST_JSON: &str = include_str!("../../../scripts/model-capabilities.json"); + + fn generate(id: &str) -> Option { + let family_tokens: Vec = + serde_json::from_value(json(MANIFEST_JSON)["label_family_tokens"].clone()) + .expect("label_family_tokens"); + super::generate_databricks_label(id, &family_tokens) + } + + fn json(source: &str) -> Value { + serde_json::from_str(source).expect("fixture JSON must parse") + } + + /// `(raw_model_id, registry_label)` for every `databricks_v2` exact record. + fn exact_records() -> Vec<(String, String)> { + json(MANIFEST_JSON)["exact_records"] + .as_array() + .expect("exact_records") + .iter() + .filter(|rec| rec["provider"] == "databricks_v2") + .map(|rec| { + let field = |key: &str| rec[key].as_str().expect(key).to_string(); + (field("raw_model_id"), field("registry_label")) + }) + .collect() + } + + #[test] + fn shared_fixtures_replay_through_the_label_helper() { + let fixtures = json(FIXTURES_JSON); + let records = exact_records(); + for row in fixtures["lookup"].as_array().expect("lookup") { + let id = row["id"].as_str().expect("id"); + let label = row["label"].as_str(); + assert_eq!(databricks_registry_label(id).as_deref(), label, "id={id:?}"); + let has_exact = records.iter().any(|(raw, _)| raw.eq_ignore_ascii_case(id)); + let curated = databricks_curated_label(id); + let tier = row["tier"].as_str().expect("tier"); + assert_eq!(has_exact, tier == "exact", "tier={tier} id={id:?}"); + match tier { + "exact" | "alias" => assert_eq!(curated.as_deref(), label, "id={id:?}"), + "generated" => { + assert_eq!(curated, None, "generated fixture is masked: {id:?}"); + assert_eq!(generate(id).as_deref(), label, "id={id:?}"); + } + "raw" => { + assert_eq!(label, None, "raw fixture has a label: {id:?}"); + assert_eq!(curated, None, "raw fixture is curated: {id:?}"); + } + tier => panic!("unknown tier {tier:?} for {id}"), + } + } + } + + #[test] + fn grammar_reproduces_every_curated_label_except_curator_only_ones() { + let fixtures = json(FIXTURES_JSON); + let curator_only: Vec<&str> = fixtures["curator_only_labels"] + .as_array() + .expect("curator_only_labels") + .iter() + .map(|id| id.as_str().expect("id")) + .collect(); + let misses: Vec = exact_records() + .into_iter() + .filter(|(raw, label)| generate(raw).as_deref() != Some(label)) + .map(|(raw, _)| raw) + .collect(); + assert_eq!(misses, curator_only); + } +} diff --git a/crates/buzz-agent/src/lib.rs b/crates/buzz-agent/src/lib.rs index 01ee90a1d9..2c9c0f2036 100644 --- a/crates/buzz-agent/src/lib.rs +++ b/crates/buzz-agent/src/lib.rs @@ -6,6 +6,7 @@ mod builtin; pub mod catalog; pub mod config; pub mod databricks; +mod databricks_label_grammar; mod handoff; mod hints; mod llm; @@ -413,8 +414,7 @@ async fn resolve_models_catalog( fn configured_model_fallback(model: &str) -> Vec { let model = model.trim().to_string(); let name = crate::model_capabilities::databricks_registry_label(&model) - .unwrap_or(&model) - .to_string(); + .unwrap_or_else(|| model.clone()); vec![ModelEntry { id: model, name }] } diff --git a/crates/buzz-agent/src/model_capabilities.rs b/crates/buzz-agent/src/model_capabilities.rs index 207544a338..08be4f020f 100644 --- a/crates/buzz-agent/src/model_capabilities.rs +++ b/crates/buzz-agent/src/model_capabilities.rs @@ -459,23 +459,41 @@ pub fn databricks_v2_known_models() -> &'static [String] { &manifest().databricks_v2_known_models } -/// Curated display label for a Databricks endpoint id, or `None` when no exact -/// record covers it. Exact raw-id hits preserve the resolver's current behavior. -/// On an exact miss, aliases share a label only when stripping the manifest's -/// existing family-token prefix from the query and record keys yields exactly one -/// `databricks_v2` record; no or ambiguous stripped matches deliberately remain -/// uncurated. This accessor is discovery-only, so `resolve()` retains its exact- -/// record label contract. -pub fn databricks_registry_label(raw_model_id: &str) -> Option<&'static str> { +/// Display label for a Databricks endpoint id, or `None` to show the raw id. +/// Precedence: exact record → unique alias (family-token-stripped query and +/// record keys match exactly one `databricks_v2` record) → the generative label +/// grammar. An ambiguous alias match stays `None` rather than generating. This +/// accessor is display-only, so `resolve()` retains its exact-record label +/// contract. +pub fn databricks_registry_label(raw_model_id: &str) -> Option { let m = manifest(); - registry_label_for_databricks_records(raw_model_id, &m.exact_records, &m.label_family_tokens) + registry_label_for_databricks_records( + raw_model_id, + &m.exact_records, + &m.label_family_tokens, + true, + ) } -fn registry_label_for_databricks_records<'a>( +/// Exact-record and unique-alias tiers only, so tests can assert which tier +/// produced a label. +#[cfg(test)] +pub(crate) fn databricks_curated_label(raw_model_id: &str) -> Option { + let m = manifest(); + registry_label_for_databricks_records( + raw_model_id, + &m.exact_records, + &m.label_family_tokens, + false, + ) +} + +fn registry_label_for_databricks_records( raw_model_id: &str, - records: &'a [ExactRecord], + records: &[ExactRecord], family_tokens: &[String], -) -> Option<&'a str> { + generate: bool, +) -> Option { if raw_model_id.trim().is_empty() { return None; } @@ -483,13 +501,23 @@ fn registry_label_for_databricks_records<'a>( if let Some(rec) = records.iter().find(|rec| { rec.provider == "databricks_v2" && rec.raw_model_id.eq_ignore_ascii_case(raw_model_id) }) { - return Some(&rec.registry_label); + return Some(rec.registry_label.clone()); } + let generated = || { + generate + .then(|| { + crate::databricks_label_grammar::generate_databricks_label( + raw_model_id, + family_tokens, + ) + }) + .flatten() + }; let query_lower = raw_model_id.to_ascii_lowercase(); let stripped_query = strip_catalog_prefix(&query_lower, family_tokens); if stripped_query == query_lower { - return None; + return generated(); } let mut matching_record = None; for rec in records.iter().filter(|rec| rec.provider == "databricks_v2") { @@ -500,7 +528,7 @@ fn registry_label_for_databricks_records<'a>( return None; } } - matching_record.map(|rec| rec.registry_label.as_str()) + matching_record.map_or_else(generated, |rec| Some(rec.registry_label.clone())) } /// Semantic invariants that strict typed parsing cannot express. Structural @@ -1085,7 +1113,7 @@ mod tests { fn test_databricks_registry_label_lookup() { // Exact raw id remains case-insensitive and unchanged. assert_eq!( - databricks_registry_label("DATABRICKS-GPT-5-5"), + databricks_registry_label("DATABRICKS-GPT-5-5").as_deref(), Some("GPT-5.5") ); // Exact raw ids preserve their canonical labels. @@ -1095,18 +1123,18 @@ mod tests { ("databricks-kimi-k3", "Kimi K3"), ] { assert_eq!( - databricks_registry_label(model), + databricks_registry_label(model).as_deref(), Some(label), "model={model}" ); } // Aliases reuse the existing family-token stripper. assert_eq!( - databricks_registry_label("goose-gpt-5-6-sol"), + databricks_registry_label("goose-gpt-5-6-sol").as_deref(), Some("GPT-5.6 Sol") ); assert_eq!( - databricks_registry_label("goose-claude-fable-5"), + databricks_registry_label("goose-claude-fable-5").as_deref(), Some("Claude Fable 5") ); for (alias, label) in [ @@ -1119,7 +1147,7 @@ mod tests { ("goose-kimi-k3", "Kimi K3"), ] { assert_eq!( - databricks_registry_label(alias), + databricks_registry_label(alias).as_deref(), Some(label), "alias={alias}" ); @@ -1149,7 +1177,11 @@ mod tests { ("system.ai.glm-5-3-flash", "GLM-5.3 Flash"), ("system.ai.grok-4-6", "Grok 4.6"), ] { - assert_eq!(databricks_registry_label(fqn), Some(label), "fqn={fqn}"); + assert_eq!( + databricks_registry_label(fqn).as_deref(), + Some(label), + "fqn={fqn}" + ); } // Unknown ids, bare family ids, and blanks remain uncurated. assert_eq!(databricks_registry_label("custom-unlisted-endpoint"), None); @@ -1182,7 +1214,7 @@ mod tests { let family_tokens = vec!["gpt-".to_string()]; assert_eq!( - registry_label_for_databricks_records("goose-gpt-5-6", &records, &family_tokens), + registry_label_for_databricks_records("goose-gpt-5-6", &records, &family_tokens, true), None ); } diff --git a/desktop/src/features/agents/lib/agentCardModelLabel.test.mjs b/desktop/src/features/agents/lib/agentCardModelLabel.test.mjs index 4a1151c9ce..4ffdc88fc4 100644 --- a/desktop/src/features/agents/lib/agentCardModelLabel.test.mjs +++ b/desktop/src/features/agents/lib/agentCardModelLabel.test.mjs @@ -240,3 +240,14 @@ test("resolveAgentCardModelLabel — unspawned openai persona with Databricks-na }); assert.equal(label, "Default model (databricks-gpt-5-5)"); }); + +test("resolveModelLabel — uncurated Databricks ids humanize only under databricks_v2", () => { + const id = "data_workflow_tools.goose.goose-gpt-6-astra"; + assert.equal(resolveModelLabel(id, id, "databricks_v2"), "GPT-6 Astra"); + assert.equal(formatAgentModelLabel(id), id); + assert.equal(formatAgentModelLabel(id, "openai"), id); + assert.equal( + formatAgentModelLabel("builderbot-pr-reviews", "databricks_v2"), + "builderbot-pr-reviews", + ); +}); diff --git a/desktop/src/features/agents/lib/disambiguateModelLabels.test.mjs b/desktop/src/features/agents/lib/disambiguateModelLabels.test.mjs new file mode 100644 index 0000000000..be5276e576 --- /dev/null +++ b/desktop/src/features/agents/lib/disambiguateModelLabels.test.mjs @@ -0,0 +1,82 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { disambiguateModelLabels } from "./formatAgentModelLabel.ts"; + +const labels = (rows, provider = "databricks_v2") => + disambiguateModelLabels(rows, provider).map((row) => row.label); + +test("namespace collision gets the catalog.schema suffix", () => { + assert.deepEqual( + labels([ + { id: "system.ai.gpt-6-astra", label: "GPT-6 Astra" }, + { id: "sandbox.public.gpt-6-astra", label: "GPT-6 Astra" }, + ]), + ["GPT-6 Astra (system.ai)", "GPT-6 Astra (sandbox.public)"], + ); +}); + +test("workspace wrapper collision gets the wrapper suffix", () => { + assert.deepEqual( + labels([ + { id: "databricks-gpt-5-5", label: "GPT-5.5" }, + { id: "goose-gpt-5-5", label: "GPT-5.5" }, + { id: "kgoose-gpt-5-5", label: "GPT-5.5" }, + ]), + ["GPT-5.5 (databricks)", "GPT-5.5 (goose)", "GPT-5.5 (kgoose)"], + ); +}); + +test("rows still colliding after the source suffix get the full id", () => { + assert.deepEqual( + labels([ + { id: "system.ai.gemini-3-7-flash", label: "Gemini 3.7 Flash" }, + { id: "system.ai.gemini-3-7-flash-0925", label: "Gemini 3.7 Flash" }, + { id: "goose-gemini-3-7-flash", label: "Gemini 3.7 Flash" }, + ]), + [ + "Gemini 3.7 Flash (system.ai.gemini-3-7-flash)", + "Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925)", + "Gemini 3.7 Flash (goose)", + ], + ); +}); + +test("unique, raw-id, and default rows are unchanged", () => { + assert.deepEqual( + labels([ + { id: "", label: "Default model" }, + { id: "system.ai.gpt-6-sol", label: "GPT-6 Sol" }, + { id: "builderbot-pr-reviews", label: "builderbot-pr-reviews" }, + { id: "prod.ai.rag-agent-3", label: "prod.ai.rag-agent-3" }, + ]), + [ + "Default model", + "GPT-6 Sol", + "builderbot-pr-reviews", + "prod.ai.rag-agent-3", + ], + ); +}); + +test("non-Databricks providers are unchanged", () => { + const rows = [ + { id: "a", label: "Same" }, + { id: "b", label: "Same" }, + ]; + assert.deepEqual(labels(rows, "openai"), ["Same", "Same"]); + assert.deepEqual(labels(rows, null), ["Same", "Same"]); +}); + +test("grouping ignores case and the alias provider name", () => { + assert.deepEqual( + labels( + [ + { id: "goose-gpt-5-4-mini", label: "GPT-5.4 mini" }, + { id: "databricks-gpt-5-4-mini", label: "GPT-5.4 Mini" }, + ], + "DATABRICKS_V2", + ), + ["GPT-5.4 mini (goose)", "GPT-5.4 Mini (databricks)"], + ); +}); diff --git a/desktop/src/features/agents/lib/formatAgentModelLabel.ts b/desktop/src/features/agents/lib/formatAgentModelLabel.ts index 5bc3a0f299..a72e62419e 100644 --- a/desktop/src/features/agents/lib/formatAgentModelLabel.ts +++ b/desktop/src/features/agents/lib/formatAgentModelLabel.ts @@ -1,6 +1,7 @@ import { canonicalizeProvider, databricksRegistryLabel, + isDatabricksModelServiceFqn, resolveModelCapabilities, } from "../ui/modelCapabilities"; @@ -19,13 +20,15 @@ export { canonicalizeProvider }; * discovery contract (`{id, name: id}`) and any harness/version skew that * echoes the id as the name. * 2. Registry lookup by id: - * - `provider` supplied → Databricks v2 uses alias-aware exact records; - * every other provider uses provider-qualified exact records. On a miss + * - `provider` supplied → Databricks v2 uses alias-aware exact records, + * then the generative Databricks label grammar; every other provider + * uses provider-qualified exact records. On a miss * the raw id is returned; the providerless registry tier is NOT * consulted, so a Databricks endpoint id never leaks a curated label * through an anthropic/openai provider context (the P3-B contract). * - `provider` absent → alias-aware lookup over `databricks_v2` exact - * records, for legacy/inherited ids with no provider on hand. + * records only (no generated labels), for legacy/inherited ids with no + * provider on hand. * 3. Raw id unchanged. * * Returns the empty string when both id and discoveredName are blank; use @@ -56,8 +59,9 @@ export function resolveModelLabel( : resolveModelCapabilities(provider, trimmedId).registryLabel; return registryLabel ?? trimmedId; } - // Providerless path: alias-aware lookup for legacy/inherited ids. - return databricksRegistryLabel(trimmedId) ?? trimmedId; + // Providerless path: curated alias-aware lookup for legacy/inherited ids. + // Generated labels need an explicit databricks_v2 provider. + return databricksRegistryLabel(trimmedId, { generate: false }) ?? trimmedId; } /** @@ -65,8 +69,9 @@ export function resolveModelLabel( * "Auto" when no model is set (empty or whitespace-only). * * For known Databricks managed endpoints the registry-curated name is returned - * (e.g. "databricks-gpt-5-5" → "GPT-5.5"). Unknown or custom endpoint ids are - * returned unchanged — no heuristic string mangling. Pass `provider` when the + * (e.g. "databricks-gpt-5-5" → "GPT-5.5"); with a `databricks_v2` provider, an + * uncurated endpoint id gets a label parsed strictly from its own tokens. + * Anything else is returned unchanged. Pass `provider` when the * inference provider is known to get a provider-qualified registry label. */ export function formatAgentModelLabel( @@ -77,3 +82,54 @@ export function formatAgentModelLabel( if (!trimmed) return "Auto"; return resolveModelLabel(trimmed, null, provider); } + +const DATABRICKS_WRAPPERS = ["databricks-", "goose-", "kgoose-", "builderbot-"]; + +/** Where a Databricks id comes from: its UC `catalog.schema`, else its wrapper. */ +function databricksModelSource(id: string): string { + if (isDatabricksModelServiceFqn(id)) { + return id.split(".").slice(0, 2).join("."); + } + const lower = id.toLowerCase(); + const wrapper = DATABRICKS_WRAPPERS.find((w) => lower.startsWith(w)); + return wrapper ? wrapper.slice(0, -1) : id; +} + +function collidingLabels(rows: ReadonlyArray<{ id: string; label: string }>) { + const counts = new Map(); + for (const { label } of rows) { + const key = label.toLowerCase(); + counts.set(key, (counts.get(key) ?? 0) + 1); + } + return (label: string) => (counts.get(label.toLowerCase()) ?? 0) > 1; +} + +/** + * Tells apart Databricks rows whose humanized labels collide (the same model + * served from several catalogs or wrappers) by suffixing the source: + * `GPT-6 Astra (system.ai)`, `GPT-5.5 (goose)`. Rows still colliding after + * that get the full id. Raw-id rows, unique labels, the default `""` row, and + * other providers are unchanged. Presentation only; ids are never touched. + */ +export function disambiguateModelLabels< + T extends { id: string; label: string }, +>(rows: ReadonlyArray, provider: string | null | undefined): T[] { + if (canonicalizeProvider(provider ?? "") !== "databricks_v2") { + return [...rows]; + } + const humanized = (row: T) => row.id !== "" && row.label !== row.id; + const collides = collidingLabels(rows.filter(humanized)); + const suffixed = rows.map((row) => + humanized(row) && collides(row.label) + ? { ...row, label: `${row.label} (${databricksModelSource(row.id)})` } + : row, + ); + const stillCollides = collidingLabels( + suffixed.filter((row, i) => row !== rows[i]), + ); + return suffixed.map((row, i) => + row !== rows[i] && stillCollides(row.label) + ? { ...row, label: `${rows[i].label} (${row.id})` } + : row, + ); +} diff --git a/desktop/src/features/agents/ui/ModelPicker.tsx b/desktop/src/features/agents/ui/ModelPicker.tsx index 170adca0b8..7401119d12 100644 --- a/desktop/src/features/agents/ui/ModelPicker.tsx +++ b/desktop/src/features/agents/ui/ModelPicker.tsx @@ -26,7 +26,10 @@ import { DropdownMenuRadioItem, DropdownMenuTrigger, } from "@/shared/ui/dropdown-menu"; -import { resolveModelLabel } from "@/features/agents/lib/formatAgentModelLabel"; +import { + disambiguateModelLabels, + resolveModelLabel, +} from "@/features/agents/lib/formatAgentModelLabel"; export function ModelPicker({ agent, @@ -86,10 +89,24 @@ export function ModelPicker({ ); const currentValue = agent.model ?? modelsData?.agentDefaultModel ?? ""; + const modelRows = React.useMemo( + () => + disambiguateModelLabels( + (modelsData?.models ?? []).map((model) => ({ + id: model.id, + label: resolveModelLabel(model.id, model.name, agent.provider), + })), + agent.provider, + ), + [modelsData, agent.provider], + ); + const rowLabel = (id: string) => + modelRows.find((row) => row.id === id)?.label ?? + resolveModelLabel(id, null, agent.provider); const displayLabel = agent.model - ? resolveModelLabel(agent.model, null, agent.provider) + ? rowLabel(agent.model) : modelsData?.agentDefaultModel - ? `${resolveModelLabel(modelsData.agentDefaultModel, null, agent.provider)} (default)` + ? `${rowLabel(modelsData.agentDefaultModel)} (default)` : hasRequestedModels && loading ? "Loading..." : "Auto"; @@ -286,9 +303,9 @@ export function ModelPicker({ onValueChange={handleModelChange} value={currentValue} > - {modelsData.models.map((model) => ( - - {resolveModelLabel(model.id, model.name, agent.provider)} + {modelRows.map((row) => ( + + {row.label} ))} diff --git a/desktop/src/features/agents/ui/PersonaModelCombobox.tsx b/desktop/src/features/agents/ui/PersonaModelCombobox.tsx index 16efbecfea..762e292a20 100644 --- a/desktop/src/features/agents/ui/PersonaModelCombobox.tsx +++ b/desktop/src/features/agents/ui/PersonaModelCombobox.tsx @@ -35,8 +35,11 @@ export function PersonaModelCombobox({ const filteredOptions = React.useMemo(() => { if (query.trim() === "") return options; const lower = query.toLowerCase(); - return options.filter((option) => - option.label.toLowerCase().includes(lower), + // Search the raw id too, so `system.ai` finds rows whose label hides it. + return options.filter( + (option) => + option.label.toLowerCase().includes(lower) || + option.value.toLowerCase().includes(lower), ); }, [options, query]); diff --git a/desktop/src/features/agents/ui/agentConfigControls.tsx b/desktop/src/features/agents/ui/agentConfigControls.tsx index 677db669a3..7756a38a14 100644 --- a/desktop/src/features/agents/ui/agentConfigControls.tsx +++ b/desktop/src/features/agents/ui/agentConfigControls.tsx @@ -155,10 +155,12 @@ export function AgentDropdownSelect({ const filteredOptions = React.useMemo(() => { const trimmed = query.trim().toLowerCase(); if (!showSearch || trimmed === "") return options; - return options.filter((option) => - (typeof option.label === "string" ? option.label : option.value) - .toLowerCase() - .includes(trimmed), + // Search the raw id too, so `system.ai` finds rows whose label hides it. + return options.filter( + (option) => + (typeof option.label === "string" && + option.label.toLowerCase().includes(trimmed)) || + option.value.toLowerCase().includes(trimmed), ); }, [options, query, showSearch]); diff --git a/desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs b/desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs new file mode 100644 index 0000000000..1d2e84a7fe --- /dev/null +++ b/desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs @@ -0,0 +1,56 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { fileURLToPath } from "node:url"; +import test from "node:test"; + +import { + databricksRegistryLabel, + generateDatabricksLabel, +} from "./modelCapabilities.ts"; + +// TS half of the shared display-label contract; the Rust suite +// (databricks_label_grammar.rs) replays the same fixture file. +const readScript = (name) => + JSON.parse( + readFileSync( + fileURLToPath(new URL(`../../../../../scripts/${name}`, import.meta.url)), + "utf8", + ), + ); +const fixtures = readScript("databricks-label-fixtures.json"); +const records = readScript("model-capabilities.json").exact_records.filter( + (rec) => rec.provider === "databricks_v2", +); +const hasExactRecord = (id) => + records.some((rec) => rec.raw_model_id.toLowerCase() === id.toLowerCase()); + +test("shared fixtures replay through the label helper", () => { + for (const { id, tier, label } of fixtures.lookup) { + assert.equal( + databricksRegistryLabel(id), + label, + `id=${JSON.stringify(id)}`, + ); + assert.equal(hasExactRecord(id), tier === "exact", `tier=${tier} id=${id}`); + const curated = databricksRegistryLabel(id, { generate: false }); + if (tier === "exact" || tier === "alias") { + assert.equal(curated, label, `id=${id}`); + } else if (tier === "generated") { + assert.equal(curated, null, `generated fixture is masked: ${id}`); + assert.equal(generateDatabricksLabel(id), label, `id=${id}`); + } else { + assert.equal(tier, "raw", `unknown tier ${tier} for ${id}`); + assert.equal(label, null, `raw fixture has a label: ${id}`); + assert.equal(curated, null, `raw fixture is curated: ${id}`); + } + } +}); + +test("grammar reproduces every curated label except curator-only ones", () => { + const misses = records + .filter( + (rec) => generateDatabricksLabel(rec.raw_model_id) !== rec.registry_label, + ) + .map((rec) => rec.raw_model_id); + assert.deepEqual(misses, fixtures.curator_only_labels); +}); diff --git a/desktop/src/features/agents/ui/modelCapabilities.ts b/desktop/src/features/agents/ui/modelCapabilities.ts index 42cd450aa7..29e737a8cb 100644 --- a/desktop/src/features/agents/ui/modelCapabilities.ts +++ b/desktop/src/features/agents/ui/modelCapabilities.ts @@ -289,7 +289,7 @@ function toResult( }; } -function isDatabricksModelServiceFqn(model: string): boolean { +export function isDatabricksModelServiceFqn(model: string): boolean { const components = model.split("."); return ( components.length === 3 && @@ -428,11 +428,19 @@ export type RegistryLabelRecord = { readonly registry_label: string; }; +/** + * Display label for a Databricks endpoint id: exact record → unique alias → + * generative grammar (only when `generate` and no record matched) → `null` + * (show the raw id). An ambiguous alias match stays `null`. + */ export function databricksRegistryLabelForRecords( rawModelId: string, records: ReadonlyArray, familyTokens: ReadonlyArray, + generate = true, ): string | null { + const unmatched = () => + generate ? generateDatabricksLabel(rawModelId) : null; if (!rawModelId.trim()) return null; const idLower = rawModelId.toLowerCase(); @@ -444,7 +452,7 @@ export function databricksRegistryLabelForRecords( if (exact) return exact.registry_label; const strippedQuery = stripCatalogPrefix(idLower, familyTokens); - if (strippedQuery === idLower) return null; + if (strippedQuery === idLower) return unmatched(); let matchingRecord: RegistryLabelRecord | null = null; for (const rec of records) { if (rec.provider !== "databricks_v2") continue; @@ -457,13 +465,155 @@ export function databricksRegistryLabelForRecords( matchingRecord = rec; } } - return matchingRecord?.registry_label ?? null; + return matchingRecord?.registry_label ?? unmatched(); } -export function databricksRegistryLabel(rawModelId: string): string | null { +/** + * `generate: false` limits the lookup to curated records — for ids whose + * provider is unknown, which must not be humanized by Databricks grammar. + */ +export function databricksRegistryLabel( + rawModelId: string, + { generate = true }: { generate?: boolean } = {}, +): string | null { return databricksRegistryLabelForRecords( rawModelId, MANIFEST.exact_records, MANIFEST.label_family_tokens, + generate, + ); +} + +// --------------------------------------------------------------------------- +// Generative display-label grammar — mirrors crates/buzz-agent/src/ +// databricks_label_grammar.rs. Reached only after an exact-record and a +// unique-alias miss; returns a label built solely from the id's own tokens, or +// null so callers show the raw id. Only families in the manifest's +// label_family_tokens are named; adding a vendor means adding one entry there. +// Presentation-only: capabilities, routing, +// and the saved id never depend on it. Both interpreters replay +// scripts/databricks-label-fixtures.json. +// --------------------------------------------------------------------------- + +const CLAUDE_TIERS = new Set(["opus", "sonnet", "haiku"]); +const LABEL_WRAPPERS = ["databricks-", "goose-", "kgoose-", "builderbot-"]; + +type LabelPart = { readonly version: boolean; readonly text: string }; + +const isVersionToken = (tok: string) => /^(?:\d|[1-9]\d)$/.test(tok); +const isDateToken = (tok: string) => /^(?:\d{4}|\d{8})$/.test(tok); +const isLetterVersionToken = (tok: string) => /^[a-z]\d+$/.test(tok); +const isSizeToken = (tok: string) => /^[a-z]?\d+b$/.test(tok); +const capitalize = (word: string) => + word.charAt(0).toUpperCase() + word.slice(1); + +/** + * Parse a Databricks endpoint id into a display label, or `null` when any part + * of the id falls outside the grammar. + */ +export function generateDatabricksLabel(rawModelId: string): string | null { + // Refuse non-ASCII before trimming or folding, so no Unicode case or + // whitespace rule can turn an unsupported id into an ASCII-looking one. + if ([...rawModelId].some((ch) => ch.charCodeAt(0) > 0x7f)) return null; + const id = rawModelId.trim().toLowerCase(); + const isFqn = isDatabricksModelServiceFqn(id); + const service = isFqn ? id.slice(id.lastIndexOf(".") + 1) : id; + // Only a UC FQN or a wrapped endpoint name is attributable to Databricks; a + // bare id such as `gpt-5` stays raw. + const wrapper = LABEL_WRAPPERS.find((w) => service.startsWith(w)); + if (!wrapper && !isFqn) return null; + let body = wrapper ? service.slice(wrapper.length) : service; + if (body.startsWith("meta-llama-")) body = body.slice("meta-".length); + + const [head, ...rest] = body.split("-"); + const familyMatch = /^([a-z]+)(\d{0,2})$/.exec(head); + if (!familyMatch) return null; + const [, family, digits] = familyMatch; + if ( + !MANIFEST.label_family_tokens.some( + (token) => token.replace(/-$/, "") === family, + ) + ) + return null; + // Attached digits followed by a version (`qwen3-5`, `llama3-1`) cannot be + // read without guessing; checked before any reorder. + if (digits !== "" && isVersionToken(rest[0] ?? "")) return null; + // In-name digits stay whole (`nova12` → `Nova12`). The one exception is + // Qwen's own compact-decimal naming: `qwen35` → `Qwen3.5` when the second + // digit is nonzero. + const stem = + family === "qwen" && digits.length === 2 && digits[1] !== "0" + ? `${digits[0]}.${digits[1]}` + : digits; + if (family === "claude") { + // Goose's numeric-first Claude ids (`claude-4-7-opus`) name the tier after + // the version. Reorder only that exact shape; any other token after the + // version keeps its position. + let count = 0; + while (count < rest.length && isVersionToken(rest[count])) count += 1; + if (count >= 1 && count <= 2 && CLAUDE_TIERS.has(rest[count])) { + rest.splice(0, count + 1, rest[count], ...rest.slice(0, count)); + } + } + + const parts: LabelPart[] = []; + let afterMinor = false; + for (let i = 0; i < rest.length; i += 1) { + const tok = rest[i]; + const next = rest[i + 1]; + if (isDateToken(tok) && next === undefined) break; + const hasMinor = next !== undefined && isVersionToken(next); + // A third number (`3-7-1`) would read as a separate version. + if ( + hasMinor && + (isVersionToken(tok) || isLetterVersionToken(tok)) && + isVersionToken(rest[i + 2] ?? "") + ) + return null; + if (isVersionToken(tok)) { + afterMinor = hasMinor; + parts.push({ version: true, text: afterMinor ? `${tok}.${next}` : tok }); + if (afterMinor) i += 1; + } else if (isLetterVersionToken(tok)) { + const major = capitalize(tok); + parts.push({ + version: false, + text: hasMinor ? `${major}.${next}` : major, + }); + if (hasMinor) i += 1; + } else if (isSizeToken(tok)) { + parts.push({ version: false, text: tok.toUpperCase() }); + } else if (/^[a-z]+$/.test(tok)) { + const text = + tok === "oss" + ? "OSS" + : family === "gpt" && afterMinor && (tok === "mini" || tok === "nano") + ? tok + : capitalize(tok); + parts.push({ version: false, text }); + } else { + return null; + } + } + // A versionless id (`builderbot-pr-reviews`) is a named endpoint, not a model. + const hasNumber = + stem !== "" || parts.some((part) => part.version || /\d/.test(part.text)); + if (!hasNumber) return null; + + const brand = + family === "gpt" + ? "GPT" + : family === "glm" + ? "GLM" + : family === "deepseek" + ? "DeepSeek" + : capitalize(family); + const hyphenated = family === "gpt" || family === "glm"; + return parts.reduce( + (label, part, index) => + label + + (part.version && index === 0 && hyphenated ? "-" : " ") + + part.text, + brand + stem, ); } diff --git a/desktop/src/features/agents/ui/usePersonaModelDiscovery.ts b/desktop/src/features/agents/ui/usePersonaModelDiscovery.ts index 8f2ef49346..f72eec3bfe 100644 --- a/desktop/src/features/agents/ui/usePersonaModelDiscovery.ts +++ b/desktop/src/features/agents/ui/usePersonaModelDiscovery.ts @@ -12,7 +12,10 @@ import { } from "./personaModelDiscoveryStatus"; import type { PersonaModelOption } from "./agentConfigOptions"; import { providerRequiresExplicitModel } from "./agentConfigOptions"; -import { resolveModelLabel } from "@/features/agents/lib/formatAgentModelLabel"; +import { + disambiguateModelLabels, + resolveModelLabel, +} from "@/features/agents/lib/formatAgentModelLabel"; export const MODEL_DISCOVERY_LOADING_VALUE = "__model_discovery_loading__"; @@ -76,10 +79,13 @@ export function getDiscoveredPersonaModelOptions( return [ ...defaultModelOption, - ...explicitModels.map((model) => ({ - id: model.id, - label: resolveModelLabel(model.id, model.name, provider), - })), + ...disambiguateModelLabels( + explicitModels.map((model) => ({ + id: model.id, + label: resolveModelLabel(model.id, model.name, provider), + })), + provider, + ), ]; } diff --git a/desktop/tests/e2e/edit-agent.spec.ts b/desktop/tests/e2e/edit-agent.spec.ts index e7fbe3d7bd..25c8cc3d22 100644 --- a/desktop/tests/e2e/edit-agent.spec.ts +++ b/desktop/tests/e2e/edit-agent.spec.ts @@ -227,6 +227,82 @@ test.describe("edit agent dialog", () => { ); }); + test("tells colliding Databricks model names apart and persists the chosen id", async ({ + page, + }) => { + const goosePathId = "data_workflow_tools.goose.gpt-6-astra"; + const systemAiId = "system.ai.gpt-6-astra"; + await installMockBridge(page, { + managedAgents: [ + { + pubkey: AGENT_PUBKEY, + name: AGENT_NAME, + status: "stopped", + channelNames: ["agents"], + envVars: { DATABRICKS_HOST: "https://databricks.example.test" }, + }, + ], + // The agent inherits Databricks from the global defaults; the mock's + // update_managed_agent does not echo a per-agent provider. + globalAgentConfig: { + preferred_runtime: "goose", + provider: "databricks_v2", + model: null, + env_vars: {}, + }, + discoverAgentModels: { + models: [ + { id: goosePathId, name: goosePathId }, + { id: systemAiId, name: systemAiId }, + ], + supportsSwitching: true, + selectedModel: null, + }, + }); + + await openEditDialog(page); + + await page.locator("#edit-agent-model").click(); + await expect( + page.getByRole("menuitemradio", { + name: "GPT-6 Astra (data_workflow_tools.goose)", + }), + ).toBeVisible(); + await page + .getByRole("menuitemradio", { name: "GPT-6 Astra (system.ai)" }) + .click(); + await expect(page.locator("#edit-agent-model")).toHaveText( + "GPT-6 Astra (system.ai)", + ); + + const submit = page.getByTestId("edit-agent-dialog-submit"); + await expect(submit).toBeEnabled({ timeout: 10_000 }); + await submit.click(); + await expect(page.getByTestId("edit-agent-dialog")).not.toBeVisible(); + + const persistedModel = () => + page.evaluate(async (pubkey) => { + const agents = (await ( + window as typeof window & { + __BUZZ_E2E_INVOKE_MOCK_COMMAND__?: ( + command: string, + payload: unknown, + ) => Promise; + } + ).__BUZZ_E2E_INVOKE_MOCK_COMMAND__?.("list_managed_agents", null)) as + | Array<{ pubkey: string; model: string | null }> + | undefined; + return agents?.find((agent) => agent.pubkey === pubkey)?.model; + }, AGENT_PUBKEY); + await expect.poll(persistedModel).toBe(systemAiId); + + await page.getByTestId("user-profile-edit-agent").click(); + await expect(page.locator("#edit-agent-model")).toHaveText( + "GPT-6 Astra (system.ai)", + { timeout: 10_000 }, + ); + }); + test("keeps the custom command visible without opening Advanced", async ({ page, }) => { diff --git a/desktop/tests/e2e/global-agent-config-screenshots.spec.ts b/desktop/tests/e2e/global-agent-config-screenshots.spec.ts index a6ecf0ec56..d79495eaa8 100644 --- a/desktop/tests/e2e/global-agent-config-screenshots.spec.ts +++ b/desktop/tests/e2e/global-agent-config-screenshots.spec.ts @@ -335,6 +335,93 @@ test.describe("global agent config screenshots", () => { expect(persisted).toMatchObject({ model: modelId }); }); + test("defaults humanize uncurated Databricks ids without changing persisted ids", async ({ + page, + }) => { + // Neither id has a curated record; both labels come from the generative + // grammar, and two distinct ids sharing a label must both stay selectable. + const modelId = "data_workflow_tools.goose.goose-gpt-6-astra"; + const siblingId = "system.ai.gpt-6-astra"; + await installMockBridge(page, { + globalAgentConfig: { + preferred_runtime: "goose", + provider: "databricks_v2", + model: modelId, + env_vars: {}, + }, + discoverAgentModels: { + models: [ + { id: modelId, name: modelId }, + { id: siblingId, name: siblingId }, + { id: "builderbot-pr-reviews", name: "builderbot-pr-reviews" }, + ], + supportsSwitching: true, + selectedModel: modelId, + }, + runtimeFileConfigs: { + goose: { + provider: "databricks_v2", + model: modelId, + satisfiedEnvKeys: ["DATABRICKS_HOST"], + }, + }, + }); + + await openAiDefaultsSettings(page); + + const model = page.getByTestId("global-agent-model"); + await expect(model).toHaveText("GPT-6 Astra (data_workflow_tools.goose)"); + await expect(model).toHaveAttribute("data-value", modelId); + + await model.click(); + const option = (id: string) => + page.getByTestId(`global-agent-model-option-${id}`); + // Colliding labels are told apart by their catalog.schema. + await expect(option(modelId)).toHaveText( + "GPT-6 Astra (data_workflow_tools.goose)", + ); + await expect(option(siblingId)).toHaveText("GPT-6 Astra (system.ai)"); + await expect(option("builderbot-pr-reviews")).toHaveText( + "builderbot-pr-reviews", + ); + + // Search matches the raw id as well as the label. + await page + .getByTestId("global-agent-model-search") + .fill("system.ai.gpt-6-astra"); + await expect(option(siblingId)).toBeVisible(); + await expect(option(modelId)).toHaveCount(0); + await expect(option("builderbot-pr-reviews")).toHaveCount(0); + + await page.getByRole("option", { name: "GPT-6 Astra (system.ai)" }).click(); + await expect(model).toHaveText("GPT-6 Astra (system.ai)"); + await expect(model).toHaveAttribute("data-value", siblingId); + + await page + .getByRole("button", { name: "Save defaults" }) + .filter({ visible: true }) + .click(); + const persisted = () => + page.evaluate(async () => + ( + window as typeof window & { + __BUZZ_E2E_INVOKE_MOCK_COMMAND__?: ( + command: string, + payload: unknown, + ) => Promise; + } + ).__BUZZ_E2E_INVOKE_MOCK_COMMAND__?.("get_global_agent_config", null), + ); + await expect.poll(persisted).toMatchObject({ model: siblingId }); + + // Reopen without a reload: the closed trigger recomputes its label from + // the saved raw id. + await page.getByTestId("settings-nav-appearance").click(); + await page.getByTestId("settings-nav-agents").click(); + await expect(model).toHaveText("GPT-6 Astra (system.ai)"); + await expect(model).toHaveAttribute("data-value", siblingId); + }); + test("defaults honor credentials set in the harness config file", async ({ page, }) => { diff --git a/scripts/databricks-label-fixtures.json b/scripts/databricks-label-fixtures.json new file mode 100644 index 0000000000..fe69c472e7 --- /dev/null +++ b/scripts/databricks-label-fixtures.json @@ -0,0 +1,80 @@ +{ + "_comment": "Shared Databricks display-label fixtures, replayed by crates/buzz-agent/src/databricks_label_grammar.rs and desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs. `lookup` rows run through the full label helper (exact record -> unique alias -> generative grammar -> raw); a null label means the raw id is shown. Every `generated` row must stay absent from exact records so a curated addition cannot mask a broken grammar. `curator_only_labels` lists the exact records whose curated label intentionally differs from what the grammar derives from the id: curator suffixes like `(latest)` / `Preview`, a version alias, or a versionless name the grammar deliberately refuses.", + "lookup": [ + { "id": "databricks-gpt-5-5", "tier": "exact", "label": "GPT-5.5" }, + { "id": "DATABRICKS-GPT-5-5", "tier": "exact", "label": "GPT-5.5" }, + { "id": "databricks-claude-sonnet-4", "tier": "exact", "label": "Claude Sonnet 4.5" }, + { "id": "databricks-gemini-3-pro", "tier": "exact", "label": "Gemini 3 Pro Preview" }, + { "id": "goose-claude-4-7-opus", "tier": "exact", "label": "Claude Opus 4.7" }, + { "id": "data_workflow_tools.goose.goose-claude-opus-5-5", "tier": "exact", "label": "Claude Opus 5.5" }, + { "id": "system.ai.claude-opus-5-5", "tier": "alias", "label": "Claude Opus 5.5" }, + { "id": "goose-gpt-5-6-sol", "tier": "alias", "label": "GPT-5.6 Sol" }, + { "id": "goose-claude-haiku-4-5", "tier": "alias", "label": "Claude Haiku 4.5 (latest)" }, + { "id": "system.ai.deepseek-v4-pro-0813", "tier": "alias", "label": "DeepSeek V4 Pro" }, + { "id": "builderbot-glm-5-2", "tier": "alias", "label": "GLM-5.2" }, + { "id": "system.ai.gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" }, + { "id": "data_workflow_tools.goose.goose-gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" }, + { "id": "data_workflow_tools.goose.goose-gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" }, + { "id": "data_workflow_tools.goose.goose-gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" }, + { "id": "system.ai.gemini-3-7-flash", "tier": "generated", "label": "Gemini 3.7 Flash" }, + { "id": "system.ai.gemini-3-8-flash", "tier": "generated", "label": "Gemini 3.8 Flash" }, + { "id": "system.ai.deepseek-v4-1-flash", "tier": "generated", "label": "DeepSeek V4.1 Flash" }, + { "id": "databricks-gpt-5-3-codex", "tier": "generated", "label": "GPT-5.3 Codex" }, + { "id": "databricks-gpt-5-5-pro", "tier": "generated", "label": "GPT-5.5 Pro" }, + { "id": "system.ai.mistral-large-3", "tier": "raw", "label": null }, + { "id": "system.ai.grok-5-mini", "tier": "generated", "label": "Grok 5 Mini" }, + { "id": "system.ai.claude-opus-5-5-thinking", "tier": "generated", "label": "Claude Opus 5.5 Thinking" }, + { "id": "system.ai.gemini-3-7-flash-0925", "tier": "generated", "label": "Gemini 3.7 Flash" }, + { "id": "system.ai.phi-5-mini", "tier": "raw", "label": null }, + { "id": "system.ai.minimax-m3", "tier": "raw", "label": null }, + { "id": "system.ai.qwen36-35b-a3b", "tier": "generated", "label": "Qwen3.6 35B A3B" }, + { "id": "system.ai.gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" }, + { "id": "system.ai.gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" }, + { "id": "system.ai.nova12-pro", "tier": "raw", "label": null }, + { "id": "system.ai.qwen10-32b", "tier": "generated", "label": "Qwen10 32B" }, + { "id": "system.ai.gpt10", "tier": "generated", "label": "GPT10" }, + { "id": "system.ai.qwen35-122b-a10b-instruct", "tier": "generated", "label": "Qwen3.5 122B A10B Instruct" }, + { "id": "system.ai.claude-5-6-opus", "tier": "generated", "label": "Claude Opus 5.6" }, + { "id": "system.ai.claude-5-6-v2", "tier": "generated", "label": "Claude 5.6 V2" }, + { "id": "system.ai.claude-5-6-thinking", "tier": "generated", "label": "Claude 5.6 Thinking" }, + { "id": "system.ai.gemini-3-7-1-flash", "tier": "raw", "label": null }, + { "id": "system.ai.deepseek-v4-1-2-flash", "tier": "raw", "label": null }, + { "id": "system.ai.\u212aimi-k4", "tier": "raw", "label": null }, + { "id": "databricks-\u212aimi-k4", "tier": "raw", "label": null }, + { "id": "\ufeffsystem.ai.nova-3\ufeff", "tier": "raw", "label": null }, + { "id": "\u0085system.ai.nova-3\u0085", "tier": "raw", "label": null }, + { "id": "goose-qwen3-5-122b-a10b", "tier": "raw", "label": null }, + { "id": "system.ai.llama3-1-8b", "tier": "raw", "label": null }, + { "id": "system.ai.claude3-5-sonnet", "tier": "raw", "label": null }, + { "id": "system.ai.nova12-3-4", "tier": "raw", "label": null }, + { "id": "builderbot-pr-reviews-v2", "tier": "raw", "label": null }, + { "id": "prod.ai.rag-agent-3", "tier": "raw", "label": null }, + { "id": "dev.ai.rag-agent-3", "tier": "raw", "label": null }, + { "id": "builderbot-pr-reviews", "tier": "raw", "label": null }, + { "id": "gpt-5", "tier": "raw", "label": null }, + { "id": "gpt-5.5", "tier": "raw", "label": null }, + { "id": "qwen3:32b", "tier": "raw", "label": null }, + { "id": "claude-opus-4.7", "tier": "raw", "label": null }, + { "id": "command-r-plus", "tier": "raw", "label": null }, + { "id": "system.ai.qwen3:32b", "tier": "raw", "label": null }, + { "id": "system.ai.claude-opus-4.7", "tier": "raw", "label": null }, + { "id": "system.ai.command-r-plus", "tier": "raw", "label": null }, + { "id": "system.ai.gpt", "tier": "raw", "label": null }, + { "id": "system.ai.gpt-05", "tier": "raw", "label": null }, + { "id": "databricks-team-2025-01", "tier": "raw", "label": null }, + { "id": "custom-unlisted-endpoint", "tier": "raw", "label": null }, + { "id": " ", "tier": "raw", "label": null } + ], + "curator_only_labels": [ + "databricks-claude-haiku-4-5", + "databricks-claude-opus-4-1", + "databricks-claude-opus-4-5", + "databricks-claude-sonnet-4", + "databricks-claude-sonnet-4-5", + "databricks-gemini-3-1-flash-lite", + "databricks-gemini-3-1-pro", + "databricks-gemini-3-flash", + "databricks-gemini-3-pro", + "databricks-inkling" + ] +}