mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
feat(agents): humanize uncurated Databricks model ids with a label grammar (#7844)
New Databricks endpoints (Claude Opus 5.5, GPT-6 Astra/Sol, etc.) showed as raw FQNs in the model picker until someone added a curated corpus record for each. This adds a strict token-class grammar as a third label tier inside the Databricks label helpers (Rust `model_capabilities.rs` + TS `modelCapabilities.ts`): exact record → unique alias → **generated label** → raw id. In TS it fires only for an explicit `databricks_v2` provider; providerless ids stay raw. ## Grammar Takes the last component of a UC FQN (or strips one wrapper: `databricks-`, `goose-`, `kgoose-`, `builderbot-`; an id that is neither stays raw), splits on `-`, and classifies each token as version, letter+digit version, size, trailing date (dropped), or word. Labels are built only from the id's own text, and the grammar never reinterprets a version. When in doubt it returns raw: - **Known families only.** The family (leading letters of the first token) must be in the manifest's existing `label_family_tokens`. Custom endpoints such as `prod.ai.rag-agent-3` or `builderbot-pr-reviews-v2` stay raw. Supporting a new vendor means adding one entry to that list, not a record per model. - **Attached digits stay whole** (`gpt10` → `GPT10`). If the family already carries digits and the next token is a version (`goose-qwen3-5-122b-a10b`, `system.ai.llama3-1-8b`, `system.ai.claude3-5-sonnet`), the id stays raw; this is checked before any reorder. - A third consecutive number (`gemini-3-7-1-flash`) returns raw rather than splitting off a patch version. - Any non-ASCII input returns raw, checked before trimming or case folding. - Any unclassifiable token, or no version-bearing token (`builderbot-pr-reviews`), returns raw. `(latest)`/`Preview` never appear unless an exact record supplies them. Special cases, each justified by an unseen-id shape rather than the corpus score: - `meta-llama-` → `Llama`: a namespace convention. - Claude numeric-first reorder: Goose names Claude endpoints `claude-4-7-opus`. Only 1–2 version tokens followed by `opus`/`sonnet`/`haiku` reorder (`claude-5-6-opus` → `Claude Opus 5.6`); anything else keeps its order (`claude-5-6-thinking` → `Claude 5.6 Thinking`). Both current numeric-first ids also have exact records. - Qwen compact decimal: Qwen's own naming `qwen35` → `Qwen3.5` when the second digit is nonzero. ## Colliding labels in the picker The same model is often served from several sources, so humanized labels collide (four rows are all `GPT-5.5`). `disambiguateModelLabels` in `formatAgentModelLabel.ts` suffixes only colliding Databricks labels with their source: - UC FQN: its `catalog.schema`, e.g. `GPT-6 Astra (system.ai)` / `GPT-6 Astra (sandbox.public)`. - Workspace endpoint: its wrapper, e.g. `GPT-5.5 (databricks)` / `GPT-5.5 (goose)` / `GPT-5.5 (kgoose)`. - Still colliding after that (e.g. a date variant in the same schema): the full id, e.g. `Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925)`. Unique labels, raw-id rows, the default row, and other providers are unchanged. It runs in `getDiscoveredPersonaModelOptions` (covering `AgentDropdownSelect`, `PersonaModelCombobox`, `PersonaDropdownField`, whose closed triggers read the option label) and in `ModelPicker`. Model search in `AgentDropdownSelect` and `PersonaModelCombobox` also matches the raw id, so searching `system.ai` finds those rows. Rust `ModelEntry.name` is unchanged. ## What labels drive Picker values, persistence, routing, catalog dedup, `resolve()` and capabilities all stay keyed on the raw id; two ids sharing a label both remain selectable. Labels do drive picker search (alongside the raw id), the workspace catalog's equal-time sort, and `TeamIdentityCard`'s "Mixed models" summary, which compares lowercased base labels, so `goose-` and `system.ai` GPT-6 Astra read as one model there (correct, since they are the same model). ## Tests Shared fixtures in `scripts/databricks-label-fixtures.json` are replayed by both Rust and TS. Each row asserts its tier: `exact` rows match a record, `alias` rows resolve with generation off, `generated` rows return nothing with generation off (so a future alias or record cannot mask them), and `raw` rows return nothing either way. A curator guard requires that across all 57 exact records, the ids the grammar does not reproduce equal exactly the 10 curator-only ids. `disambiguateModelLabels` has unit tests for namespace, wrapper, and full-id-fallback collisions, untouched rows, other providers, and case-insensitive grouping. The defaults-picker e2e checks the suffixed rows and trigger, raw-id search, and that choosing the second uncurated FQN by its visible name persists its raw id and restores its label. An edit-agent e2e covers the existing-agent model dropdown (`#edit-agent-model`): both suffixed rows render, the `system.ai` row is chosen by name, and after save and reopen the agent stores the `system.ai` id and shows `GPT-6 Astra (system.ai)`. --------- Signed-off-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
This commit is contained in:
Will Pfleger
@@ -27,8 +27,9 @@ use crate::{
|
||||
/// A discovered model entry: `id` is the picker value (the raw endpoint id or
|
||||
/// Unity Catalog model-service FQN, and the wire/config value), `name` is the
|
||||
/// display label. Databricks catalog APIs do not provide a consistently useful
|
||||
/// picker label, so discovery curates names from the capability manifest when
|
||||
/// an exact known id exists and otherwise uses the raw id.
|
||||
/// picker label, so discovery derives `name` as exact record → unique alias →
|
||||
/// generated label grammar (known `label_family_tokens` families only) → raw
|
||||
/// id. Adding a vendor means adding one `label_family_tokens` entry.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ModelEntry {
|
||||
pub id: String,
|
||||
@@ -80,12 +81,10 @@ const UNITY_CATALOG_DESCRIPTOR: CatalogDescriptor<ModelEntry> = CatalogDescripto
|
||||
parse_page: parse_uc_model_services_page,
|
||||
};
|
||||
|
||||
/// Curated display label for a discovered Databricks endpoint or model-service
|
||||
/// id. Unknown ids deliberately pass through unchanged.
|
||||
/// Display label for a discovered Databricks endpoint or model-service id. Ids
|
||||
/// the label grammar cannot fully parse deliberately pass through unchanged.
|
||||
fn curated_model_name(id: &str) -> String {
|
||||
crate::model_capabilities::databricks_registry_label(id)
|
||||
.unwrap_or(id)
|
||||
.to_string()
|
||||
crate::model_capabilities::databricks_registry_label(id).unwrap_or_else(|| id.to_string())
|
||||
}
|
||||
|
||||
/// Fallback catalog used only when both authenticated Databricks v2 catalogs
|
||||
@@ -1903,7 +1902,7 @@ mod tests {
|
||||
// `name` is the curated label + provenance suffix, not the raw id.
|
||||
assert!(models.iter().all(|model| {
|
||||
let label = crate::model_capabilities::databricks_registry_label(&model.id)
|
||||
.unwrap_or(model.id.as_str());
|
||||
.unwrap_or_else(|| model.id.clone());
|
||||
model.name == format!("{label}{AUTHENTICATED_EMPTY_CATALOG_SUFFIX}")
|
||||
}));
|
||||
}
|
||||
|
||||
281
crates/buzz-agent/src/databricks_label_grammar.rs
Normal file
281
crates/buzz-agent/src/databricks_label_grammar.rs
Normal file
@@ -0,0 +1,281 @@
|
||||
//! Deterministic display-label grammar for Databricks endpoint ids.
|
||||
//!
|
||||
//! The last tier of [`crate::model_capabilities::databricks_registry_label`],
|
||||
//! reached only after an exact-record miss and a unique-alias miss. The result
|
||||
//! is either a label built solely from the id's own tokens or `None`, in which
|
||||
//! case callers show the raw id. It is presentation-only: capabilities, wire
|
||||
//! routing, and the saved model id never depend on it. Ids from families not
|
||||
//! in the manifest's `label_family_tokens` stay raw; supporting a new vendor
|
||||
//! means adding one entry to that list, not a record per model.
|
||||
//!
|
||||
//! Mirrored by `desktop/src/features/agents/ui/modelCapabilities.ts`; both
|
||||
//! replay `scripts/databricks-label-fixtures.json`.
|
||||
|
||||
use crate::model_capabilities::is_databricks_model_service_fqn;
|
||||
|
||||
/// Endpoint-name wrappers that precede the model name in Databricks ids.
|
||||
const WRAPPERS: [&str; 4] = ["databricks-", "goose-", "kgoose-", "builderbot-"];
|
||||
|
||||
enum Part {
|
||||
/// Numeric version (`5`, `5.5`) — the one part GPT/GLM join with a hyphen.
|
||||
Version(String),
|
||||
Word(String),
|
||||
}
|
||||
|
||||
/// Parse a Databricks endpoint id into a display label, or `None` when any part
|
||||
/// of the id falls outside the grammar. Only families listed in the manifest's
|
||||
/// `label_family_tokens` are named, so custom endpoints (`rag-agent-3`) stay raw.
|
||||
pub(crate) fn generate_databricks_label(
|
||||
raw_model_id: &str,
|
||||
family_tokens: &[String],
|
||||
) -> Option<String> {
|
||||
// Refuse non-ASCII before trimming or folding, so no Unicode case or
|
||||
// whitespace rule can turn an unsupported id into an ASCII-looking one.
|
||||
if !raw_model_id.is_ascii() {
|
||||
return None;
|
||||
}
|
||||
let id = raw_model_id.trim().to_ascii_lowercase();
|
||||
let is_fqn = is_databricks_model_service_fqn(&id);
|
||||
let service = if is_fqn {
|
||||
id.rsplit('.').next()?
|
||||
} else {
|
||||
id.as_str()
|
||||
};
|
||||
// Only a UC FQN or a wrapped endpoint name is attributable to Databricks; a
|
||||
// bare id such as `gpt-5` stays raw.
|
||||
let body = match WRAPPERS.iter().find_map(|w| service.strip_prefix(w)) {
|
||||
Some(body) => body,
|
||||
None if is_fqn => service,
|
||||
None => return None,
|
||||
};
|
||||
let body = body
|
||||
.strip_prefix("meta-")
|
||||
.filter(|rest| rest.starts_with("llama-"))
|
||||
.unwrap_or(body);
|
||||
|
||||
let mut tokens = body.split('-');
|
||||
let (family, stem) = split_family(tokens.next()?)?;
|
||||
if !family_tokens
|
||||
.iter()
|
||||
.any(|t| t.strip_suffix('-').unwrap_or(t) == family)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
let mut rest: Vec<&str> = tokens.collect();
|
||||
// Attached digits followed by a version (`qwen3-5`, `llama3-1`) cannot be
|
||||
// read without guessing; checked before any reorder.
|
||||
if stem.is_some() && rest.first().is_some_and(|t| is_version(t)) {
|
||||
return None;
|
||||
}
|
||||
if family == "claude" {
|
||||
// Goose's numeric-first Claude ids (`claude-4-7-opus`) name the tier
|
||||
// after the version. Reorder only that exact shape; any other token
|
||||
// after the version keeps its position.
|
||||
let digits = rest.iter().take_while(|t| is_version(t)).count();
|
||||
if (1..=2).contains(&digits)
|
||||
&& rest
|
||||
.get(digits)
|
||||
.is_some_and(|t| matches!(*t, "opus" | "sonnet" | "haiku"))
|
||||
{
|
||||
rest[..=digits].rotate_right(1);
|
||||
}
|
||||
}
|
||||
|
||||
let mut parts = Vec::new();
|
||||
let mut has_number = stem.is_some();
|
||||
let mut after_minor = false;
|
||||
let mut rest = rest.into_iter().peekable();
|
||||
while let Some(tok) = rest.next() {
|
||||
if is_date(tok) && rest.peek().is_none() {
|
||||
break;
|
||||
}
|
||||
let part = if is_version(tok) {
|
||||
let minor = rest.next_if(|t| is_version(t));
|
||||
after_minor = minor.is_some();
|
||||
if after_minor && rest.peek().is_some_and(|t| is_version(t)) {
|
||||
// A third number (`3-7-1`) would read as a separate version.
|
||||
return None;
|
||||
}
|
||||
Part::Version(minor.map_or_else(|| tok.to_string(), |m| format!("{tok}.{m}")))
|
||||
} else if is_letter_version(tok) {
|
||||
let minor = rest.next_if(|t| is_version(t));
|
||||
if minor.is_some() && rest.peek().is_some_and(|t| is_version(t)) {
|
||||
return None;
|
||||
}
|
||||
let major = capitalize(tok);
|
||||
Part::Word(minor.map_or_else(|| major.clone(), |m| format!("{major}.{m}")))
|
||||
} else if is_size(tok) {
|
||||
Part::Word(tok.to_ascii_uppercase())
|
||||
} else if !tok.is_empty() && tok.bytes().all(|b| b.is_ascii_lowercase()) {
|
||||
Part::Word(match tok {
|
||||
"oss" => "OSS".to_string(),
|
||||
"mini" | "nano" if family == "gpt" && after_minor => tok.to_string(),
|
||||
_ => capitalize(tok),
|
||||
})
|
||||
} else {
|
||||
return None;
|
||||
};
|
||||
has_number |= match &part {
|
||||
Part::Version(_) => true,
|
||||
Part::Word(word) => word.bytes().any(|b| b.is_ascii_digit()),
|
||||
};
|
||||
parts.push(part);
|
||||
}
|
||||
// A versionless id (`builderbot-pr-reviews`) is a named endpoint, not a model.
|
||||
if !has_number {
|
||||
return None;
|
||||
}
|
||||
|
||||
let (brand, hyphenated) = match family {
|
||||
"gpt" => ("GPT".to_string(), true),
|
||||
"glm" => ("GLM".to_string(), true),
|
||||
"deepseek" => ("DeepSeek".to_string(), false),
|
||||
_ => (capitalize(family), false),
|
||||
};
|
||||
let mut label = brand + &stem.unwrap_or_default();
|
||||
for (index, part) in parts.iter().enumerate() {
|
||||
let (sep, text) = match part {
|
||||
Part::Version(v) if index == 0 && hyphenated => ('-', v),
|
||||
Part::Version(text) | Part::Word(text) => (' ', text),
|
||||
};
|
||||
label.push(sep);
|
||||
label.push_str(text);
|
||||
}
|
||||
Some(label)
|
||||
}
|
||||
|
||||
/// Split the leading token into its family name and an optional in-name
|
||||
/// version, kept whole (`nova12` → `12`). The one exception is Qwen's own
|
||||
/// compact-decimal naming: `qwen35` → `3.5` when the second digit is nonzero.
|
||||
fn split_family(tok: &str) -> Option<(&str, Option<String>)> {
|
||||
let alpha = tok.bytes().take_while(u8::is_ascii_lowercase).count();
|
||||
let (family, digits) = tok.split_at(alpha);
|
||||
if family.is_empty() || digits.len() > 2 || !digits.bytes().all(|b| b.is_ascii_digit()) {
|
||||
return None;
|
||||
}
|
||||
let stem = match digits.as_bytes() {
|
||||
[] => None,
|
||||
[major, minor] if family == "qwen" && *minor != b'0' => {
|
||||
Some(format!("{}.{}", *major as char, *minor as char))
|
||||
}
|
||||
_ => Some(digits.to_string()),
|
||||
};
|
||||
Some((family, stem))
|
||||
}
|
||||
|
||||
/// `0` or a 1–2-digit number without a leading zero.
|
||||
fn is_version(tok: &str) -> bool {
|
||||
match tok.as_bytes() {
|
||||
[d] => d.is_ascii_digit(),
|
||||
[a, b] => (b'1'..=b'9').contains(a) && b.is_ascii_digit(),
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// A 4- or 8-digit date stamp (`0731`, `20260101`).
|
||||
fn is_date(tok: &str) -> bool {
|
||||
matches!(tok.len(), 4 | 8) && tok.bytes().all(|b| b.is_ascii_digit())
|
||||
}
|
||||
|
||||
/// A letter-prefixed version such as `v4` or `k2`.
|
||||
fn is_letter_version(tok: &str) -> bool {
|
||||
matches!(tok.as_bytes(), [l, digits @ ..] if l.is_ascii_lowercase()
|
||||
&& !digits.is_empty()
|
||||
&& digits.iter().all(u8::is_ascii_digit))
|
||||
}
|
||||
|
||||
/// A parameter-size token: `120b`, or letter-prefixed `a3b`.
|
||||
fn is_size(tok: &str) -> bool {
|
||||
let Some(body) = tok.strip_suffix('b') else {
|
||||
return false;
|
||||
};
|
||||
let digits = body
|
||||
.strip_prefix(|c: char| c.is_ascii_lowercase())
|
||||
.unwrap_or(body);
|
||||
!digits.is_empty() && digits.bytes().all(|b| b.is_ascii_digit())
|
||||
}
|
||||
|
||||
fn capitalize(word: &str) -> String {
|
||||
let mut chars = word.chars();
|
||||
chars.next().map_or_else(String::new, |first| {
|
||||
first.to_ascii_uppercase().to_string() + chars.as_str()
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::model_capabilities::{databricks_curated_label, databricks_registry_label};
|
||||
use serde_json::Value;
|
||||
|
||||
const FIXTURES_JSON: &str = include_str!("../../../scripts/databricks-label-fixtures.json");
|
||||
const MANIFEST_JSON: &str = include_str!("../../../scripts/model-capabilities.json");
|
||||
|
||||
fn generate(id: &str) -> Option<String> {
|
||||
let family_tokens: Vec<String> =
|
||||
serde_json::from_value(json(MANIFEST_JSON)["label_family_tokens"].clone())
|
||||
.expect("label_family_tokens");
|
||||
super::generate_databricks_label(id, &family_tokens)
|
||||
}
|
||||
|
||||
fn json(source: &str) -> Value {
|
||||
serde_json::from_str(source).expect("fixture JSON must parse")
|
||||
}
|
||||
|
||||
/// `(raw_model_id, registry_label)` for every `databricks_v2` exact record.
|
||||
fn exact_records() -> Vec<(String, String)> {
|
||||
json(MANIFEST_JSON)["exact_records"]
|
||||
.as_array()
|
||||
.expect("exact_records")
|
||||
.iter()
|
||||
.filter(|rec| rec["provider"] == "databricks_v2")
|
||||
.map(|rec| {
|
||||
let field = |key: &str| rec[key].as_str().expect(key).to_string();
|
||||
(field("raw_model_id"), field("registry_label"))
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn shared_fixtures_replay_through_the_label_helper() {
|
||||
let fixtures = json(FIXTURES_JSON);
|
||||
let records = exact_records();
|
||||
for row in fixtures["lookup"].as_array().expect("lookup") {
|
||||
let id = row["id"].as_str().expect("id");
|
||||
let label = row["label"].as_str();
|
||||
assert_eq!(databricks_registry_label(id).as_deref(), label, "id={id:?}");
|
||||
let has_exact = records.iter().any(|(raw, _)| raw.eq_ignore_ascii_case(id));
|
||||
let curated = databricks_curated_label(id);
|
||||
let tier = row["tier"].as_str().expect("tier");
|
||||
assert_eq!(has_exact, tier == "exact", "tier={tier} id={id:?}");
|
||||
match tier {
|
||||
"exact" | "alias" => assert_eq!(curated.as_deref(), label, "id={id:?}"),
|
||||
"generated" => {
|
||||
assert_eq!(curated, None, "generated fixture is masked: {id:?}");
|
||||
assert_eq!(generate(id).as_deref(), label, "id={id:?}");
|
||||
}
|
||||
"raw" => {
|
||||
assert_eq!(label, None, "raw fixture has a label: {id:?}");
|
||||
assert_eq!(curated, None, "raw fixture is curated: {id:?}");
|
||||
}
|
||||
tier => panic!("unknown tier {tier:?} for {id}"),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn grammar_reproduces_every_curated_label_except_curator_only_ones() {
|
||||
let fixtures = json(FIXTURES_JSON);
|
||||
let curator_only: Vec<&str> = fixtures["curator_only_labels"]
|
||||
.as_array()
|
||||
.expect("curator_only_labels")
|
||||
.iter()
|
||||
.map(|id| id.as_str().expect("id"))
|
||||
.collect();
|
||||
let misses: Vec<String> = exact_records()
|
||||
.into_iter()
|
||||
.filter(|(raw, label)| generate(raw).as_deref() != Some(label))
|
||||
.map(|(raw, _)| raw)
|
||||
.collect();
|
||||
assert_eq!(misses, curator_only);
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,7 @@ mod builtin;
|
||||
pub mod catalog;
|
||||
pub mod config;
|
||||
pub mod databricks;
|
||||
mod databricks_label_grammar;
|
||||
mod handoff;
|
||||
mod hints;
|
||||
mod llm;
|
||||
@@ -413,8 +414,7 @@ async fn resolve_models_catalog(
|
||||
fn configured_model_fallback(model: &str) -> Vec<ModelEntry> {
|
||||
let model = model.trim().to_string();
|
||||
let name = crate::model_capabilities::databricks_registry_label(&model)
|
||||
.unwrap_or(&model)
|
||||
.to_string();
|
||||
.unwrap_or_else(|| model.clone());
|
||||
vec![ModelEntry { id: model, name }]
|
||||
}
|
||||
|
||||
|
||||
@@ -459,23 +459,41 @@ pub fn databricks_v2_known_models() -> &'static [String] {
|
||||
&manifest().databricks_v2_known_models
|
||||
}
|
||||
|
||||
/// Curated display label for a Databricks endpoint id, or `None` when no exact
|
||||
/// record covers it. Exact raw-id hits preserve the resolver's current behavior.
|
||||
/// On an exact miss, aliases share a label only when stripping the manifest's
|
||||
/// existing family-token prefix from the query and record keys yields exactly one
|
||||
/// `databricks_v2` record; no or ambiguous stripped matches deliberately remain
|
||||
/// uncurated. This accessor is discovery-only, so `resolve()` retains its exact-
|
||||
/// record label contract.
|
||||
pub fn databricks_registry_label(raw_model_id: &str) -> Option<&'static str> {
|
||||
/// Display label for a Databricks endpoint id, or `None` to show the raw id.
|
||||
/// Precedence: exact record → unique alias (family-token-stripped query and
|
||||
/// record keys match exactly one `databricks_v2` record) → the generative label
|
||||
/// grammar. An ambiguous alias match stays `None` rather than generating. This
|
||||
/// accessor is display-only, so `resolve()` retains its exact-record label
|
||||
/// contract.
|
||||
pub fn databricks_registry_label(raw_model_id: &str) -> Option<String> {
|
||||
let m = manifest();
|
||||
registry_label_for_databricks_records(raw_model_id, &m.exact_records, &m.label_family_tokens)
|
||||
registry_label_for_databricks_records(
|
||||
raw_model_id,
|
||||
&m.exact_records,
|
||||
&m.label_family_tokens,
|
||||
true,
|
||||
)
|
||||
}
|
||||
|
||||
fn registry_label_for_databricks_records<'a>(
|
||||
/// Exact-record and unique-alias tiers only, so tests can assert which tier
|
||||
/// produced a label.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn databricks_curated_label(raw_model_id: &str) -> Option<String> {
|
||||
let m = manifest();
|
||||
registry_label_for_databricks_records(
|
||||
raw_model_id,
|
||||
&m.exact_records,
|
||||
&m.label_family_tokens,
|
||||
false,
|
||||
)
|
||||
}
|
||||
|
||||
fn registry_label_for_databricks_records(
|
||||
raw_model_id: &str,
|
||||
records: &'a [ExactRecord],
|
||||
records: &[ExactRecord],
|
||||
family_tokens: &[String],
|
||||
) -> Option<&'a str> {
|
||||
generate: bool,
|
||||
) -> Option<String> {
|
||||
if raw_model_id.trim().is_empty() {
|
||||
return None;
|
||||
}
|
||||
@@ -483,13 +501,23 @@ fn registry_label_for_databricks_records<'a>(
|
||||
if let Some(rec) = records.iter().find(|rec| {
|
||||
rec.provider == "databricks_v2" && rec.raw_model_id.eq_ignore_ascii_case(raw_model_id)
|
||||
}) {
|
||||
return Some(&rec.registry_label);
|
||||
return Some(rec.registry_label.clone());
|
||||
}
|
||||
|
||||
let generated = || {
|
||||
generate
|
||||
.then(|| {
|
||||
crate::databricks_label_grammar::generate_databricks_label(
|
||||
raw_model_id,
|
||||
family_tokens,
|
||||
)
|
||||
})
|
||||
.flatten()
|
||||
};
|
||||
let query_lower = raw_model_id.to_ascii_lowercase();
|
||||
let stripped_query = strip_catalog_prefix(&query_lower, family_tokens);
|
||||
if stripped_query == query_lower {
|
||||
return None;
|
||||
return generated();
|
||||
}
|
||||
let mut matching_record = None;
|
||||
for rec in records.iter().filter(|rec| rec.provider == "databricks_v2") {
|
||||
@@ -500,7 +528,7 @@ fn registry_label_for_databricks_records<'a>(
|
||||
return None;
|
||||
}
|
||||
}
|
||||
matching_record.map(|rec| rec.registry_label.as_str())
|
||||
matching_record.map_or_else(generated, |rec| Some(rec.registry_label.clone()))
|
||||
}
|
||||
|
||||
/// Semantic invariants that strict typed parsing cannot express. Structural
|
||||
@@ -1085,7 +1113,7 @@ mod tests {
|
||||
fn test_databricks_registry_label_lookup() {
|
||||
// Exact raw id remains case-insensitive and unchanged.
|
||||
assert_eq!(
|
||||
databricks_registry_label("DATABRICKS-GPT-5-5"),
|
||||
databricks_registry_label("DATABRICKS-GPT-5-5").as_deref(),
|
||||
Some("GPT-5.5")
|
||||
);
|
||||
// Exact raw ids preserve their canonical labels.
|
||||
@@ -1095,18 +1123,18 @@ mod tests {
|
||||
("databricks-kimi-k3", "Kimi K3"),
|
||||
] {
|
||||
assert_eq!(
|
||||
databricks_registry_label(model),
|
||||
databricks_registry_label(model).as_deref(),
|
||||
Some(label),
|
||||
"model={model}"
|
||||
);
|
||||
}
|
||||
// Aliases reuse the existing family-token stripper.
|
||||
assert_eq!(
|
||||
databricks_registry_label("goose-gpt-5-6-sol"),
|
||||
databricks_registry_label("goose-gpt-5-6-sol").as_deref(),
|
||||
Some("GPT-5.6 Sol")
|
||||
);
|
||||
assert_eq!(
|
||||
databricks_registry_label("goose-claude-fable-5"),
|
||||
databricks_registry_label("goose-claude-fable-5").as_deref(),
|
||||
Some("Claude Fable 5")
|
||||
);
|
||||
for (alias, label) in [
|
||||
@@ -1119,7 +1147,7 @@ mod tests {
|
||||
("goose-kimi-k3", "Kimi K3"),
|
||||
] {
|
||||
assert_eq!(
|
||||
databricks_registry_label(alias),
|
||||
databricks_registry_label(alias).as_deref(),
|
||||
Some(label),
|
||||
"alias={alias}"
|
||||
);
|
||||
@@ -1149,7 +1177,11 @@ mod tests {
|
||||
("system.ai.glm-5-3-flash", "GLM-5.3 Flash"),
|
||||
("system.ai.grok-4-6", "Grok 4.6"),
|
||||
] {
|
||||
assert_eq!(databricks_registry_label(fqn), Some(label), "fqn={fqn}");
|
||||
assert_eq!(
|
||||
databricks_registry_label(fqn).as_deref(),
|
||||
Some(label),
|
||||
"fqn={fqn}"
|
||||
);
|
||||
}
|
||||
// Unknown ids, bare family ids, and blanks remain uncurated.
|
||||
assert_eq!(databricks_registry_label("custom-unlisted-endpoint"), None);
|
||||
@@ -1182,7 +1214,7 @@ mod tests {
|
||||
let family_tokens = vec!["gpt-".to_string()];
|
||||
|
||||
assert_eq!(
|
||||
registry_label_for_databricks_records("goose-gpt-5-6", &records, &family_tokens),
|
||||
registry_label_for_databricks_records("goose-gpt-5-6", &records, &family_tokens, true),
|
||||
None
|
||||
);
|
||||
}
|
||||
|
||||
@@ -240,3 +240,14 @@ test("resolveAgentCardModelLabel — unspawned openai persona with Databricks-na
|
||||
});
|
||||
assert.equal(label, "Default model (databricks-gpt-5-5)");
|
||||
});
|
||||
|
||||
test("resolveModelLabel — uncurated Databricks ids humanize only under databricks_v2", () => {
|
||||
const id = "data_workflow_tools.goose.goose-gpt-6-astra";
|
||||
assert.equal(resolveModelLabel(id, id, "databricks_v2"), "GPT-6 Astra");
|
||||
assert.equal(formatAgentModelLabel(id), id);
|
||||
assert.equal(formatAgentModelLabel(id, "openai"), id);
|
||||
assert.equal(
|
||||
formatAgentModelLabel("builderbot-pr-reviews", "databricks_v2"),
|
||||
"builderbot-pr-reviews",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
|
||||
import { disambiguateModelLabels } from "./formatAgentModelLabel.ts";
|
||||
|
||||
const labels = (rows, provider = "databricks_v2") =>
|
||||
disambiguateModelLabels(rows, provider).map((row) => row.label);
|
||||
|
||||
test("namespace collision gets the catalog.schema suffix", () => {
|
||||
assert.deepEqual(
|
||||
labels([
|
||||
{ id: "system.ai.gpt-6-astra", label: "GPT-6 Astra" },
|
||||
{ id: "sandbox.public.gpt-6-astra", label: "GPT-6 Astra" },
|
||||
]),
|
||||
["GPT-6 Astra (system.ai)", "GPT-6 Astra (sandbox.public)"],
|
||||
);
|
||||
});
|
||||
|
||||
test("workspace wrapper collision gets the wrapper suffix", () => {
|
||||
assert.deepEqual(
|
||||
labels([
|
||||
{ id: "databricks-gpt-5-5", label: "GPT-5.5" },
|
||||
{ id: "goose-gpt-5-5", label: "GPT-5.5" },
|
||||
{ id: "kgoose-gpt-5-5", label: "GPT-5.5" },
|
||||
]),
|
||||
["GPT-5.5 (databricks)", "GPT-5.5 (goose)", "GPT-5.5 (kgoose)"],
|
||||
);
|
||||
});
|
||||
|
||||
test("rows still colliding after the source suffix get the full id", () => {
|
||||
assert.deepEqual(
|
||||
labels([
|
||||
{ id: "system.ai.gemini-3-7-flash", label: "Gemini 3.7 Flash" },
|
||||
{ id: "system.ai.gemini-3-7-flash-0925", label: "Gemini 3.7 Flash" },
|
||||
{ id: "goose-gemini-3-7-flash", label: "Gemini 3.7 Flash" },
|
||||
]),
|
||||
[
|
||||
"Gemini 3.7 Flash (system.ai.gemini-3-7-flash)",
|
||||
"Gemini 3.7 Flash (system.ai.gemini-3-7-flash-0925)",
|
||||
"Gemini 3.7 Flash (goose)",
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("unique, raw-id, and default rows are unchanged", () => {
|
||||
assert.deepEqual(
|
||||
labels([
|
||||
{ id: "", label: "Default model" },
|
||||
{ id: "system.ai.gpt-6-sol", label: "GPT-6 Sol" },
|
||||
{ id: "builderbot-pr-reviews", label: "builderbot-pr-reviews" },
|
||||
{ id: "prod.ai.rag-agent-3", label: "prod.ai.rag-agent-3" },
|
||||
]),
|
||||
[
|
||||
"Default model",
|
||||
"GPT-6 Sol",
|
||||
"builderbot-pr-reviews",
|
||||
"prod.ai.rag-agent-3",
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
test("non-Databricks providers are unchanged", () => {
|
||||
const rows = [
|
||||
{ id: "a", label: "Same" },
|
||||
{ id: "b", label: "Same" },
|
||||
];
|
||||
assert.deepEqual(labels(rows, "openai"), ["Same", "Same"]);
|
||||
assert.deepEqual(labels(rows, null), ["Same", "Same"]);
|
||||
});
|
||||
|
||||
test("grouping ignores case and the alias provider name", () => {
|
||||
assert.deepEqual(
|
||||
labels(
|
||||
[
|
||||
{ id: "goose-gpt-5-4-mini", label: "GPT-5.4 mini" },
|
||||
{ id: "databricks-gpt-5-4-mini", label: "GPT-5.4 Mini" },
|
||||
],
|
||||
"DATABRICKS_V2",
|
||||
),
|
||||
["GPT-5.4 mini (goose)", "GPT-5.4 Mini (databricks)"],
|
||||
);
|
||||
});
|
||||
@@ -1,6 +1,7 @@
|
||||
import {
|
||||
canonicalizeProvider,
|
||||
databricksRegistryLabel,
|
||||
isDatabricksModelServiceFqn,
|
||||
resolveModelCapabilities,
|
||||
} from "../ui/modelCapabilities";
|
||||
|
||||
@@ -19,13 +20,15 @@ export { canonicalizeProvider };
|
||||
* discovery contract (`{id, name: id}`) and any harness/version skew that
|
||||
* echoes the id as the name.
|
||||
* 2. Registry lookup by id:
|
||||
* - `provider` supplied → Databricks v2 uses alias-aware exact records;
|
||||
* every other provider uses provider-qualified exact records. On a miss
|
||||
* - `provider` supplied → Databricks v2 uses alias-aware exact records,
|
||||
* then the generative Databricks label grammar; every other provider
|
||||
* uses provider-qualified exact records. On a miss
|
||||
* the raw id is returned; the providerless registry tier is NOT
|
||||
* consulted, so a Databricks endpoint id never leaks a curated label
|
||||
* through an anthropic/openai provider context (the P3-B contract).
|
||||
* - `provider` absent → alias-aware lookup over `databricks_v2` exact
|
||||
* records, for legacy/inherited ids with no provider on hand.
|
||||
* records only (no generated labels), for legacy/inherited ids with no
|
||||
* provider on hand.
|
||||
* 3. Raw id unchanged.
|
||||
*
|
||||
* Returns the empty string when both id and discoveredName are blank; use
|
||||
@@ -56,8 +59,9 @@ export function resolveModelLabel(
|
||||
: resolveModelCapabilities(provider, trimmedId).registryLabel;
|
||||
return registryLabel ?? trimmedId;
|
||||
}
|
||||
// Providerless path: alias-aware lookup for legacy/inherited ids.
|
||||
return databricksRegistryLabel(trimmedId) ?? trimmedId;
|
||||
// Providerless path: curated alias-aware lookup for legacy/inherited ids.
|
||||
// Generated labels need an explicit databricks_v2 provider.
|
||||
return databricksRegistryLabel(trimmedId, { generate: false }) ?? trimmedId;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -65,8 +69,9 @@ export function resolveModelLabel(
|
||||
* "Auto" when no model is set (empty or whitespace-only).
|
||||
*
|
||||
* For known Databricks managed endpoints the registry-curated name is returned
|
||||
* (e.g. "databricks-gpt-5-5" → "GPT-5.5"). Unknown or custom endpoint ids are
|
||||
* returned unchanged — no heuristic string mangling. Pass `provider` when the
|
||||
* (e.g. "databricks-gpt-5-5" → "GPT-5.5"); with a `databricks_v2` provider, an
|
||||
* uncurated endpoint id gets a label parsed strictly from its own tokens.
|
||||
* Anything else is returned unchanged. Pass `provider` when the
|
||||
* inference provider is known to get a provider-qualified registry label.
|
||||
*/
|
||||
export function formatAgentModelLabel(
|
||||
@@ -77,3 +82,54 @@ export function formatAgentModelLabel(
|
||||
if (!trimmed) return "Auto";
|
||||
return resolveModelLabel(trimmed, null, provider);
|
||||
}
|
||||
|
||||
const DATABRICKS_WRAPPERS = ["databricks-", "goose-", "kgoose-", "builderbot-"];
|
||||
|
||||
/** Where a Databricks id comes from: its UC `catalog.schema`, else its wrapper. */
|
||||
function databricksModelSource(id: string): string {
|
||||
if (isDatabricksModelServiceFqn(id)) {
|
||||
return id.split(".").slice(0, 2).join(".");
|
||||
}
|
||||
const lower = id.toLowerCase();
|
||||
const wrapper = DATABRICKS_WRAPPERS.find((w) => lower.startsWith(w));
|
||||
return wrapper ? wrapper.slice(0, -1) : id;
|
||||
}
|
||||
|
||||
function collidingLabels(rows: ReadonlyArray<{ id: string; label: string }>) {
|
||||
const counts = new Map<string, number>();
|
||||
for (const { label } of rows) {
|
||||
const key = label.toLowerCase();
|
||||
counts.set(key, (counts.get(key) ?? 0) + 1);
|
||||
}
|
||||
return (label: string) => (counts.get(label.toLowerCase()) ?? 0) > 1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Tells apart Databricks rows whose humanized labels collide (the same model
|
||||
* served from several catalogs or wrappers) by suffixing the source:
|
||||
* `GPT-6 Astra (system.ai)`, `GPT-5.5 (goose)`. Rows still colliding after
|
||||
* that get the full id. Raw-id rows, unique labels, the default `""` row, and
|
||||
* other providers are unchanged. Presentation only; ids are never touched.
|
||||
*/
|
||||
export function disambiguateModelLabels<
|
||||
T extends { id: string; label: string },
|
||||
>(rows: ReadonlyArray<T>, provider: string | null | undefined): T[] {
|
||||
if (canonicalizeProvider(provider ?? "") !== "databricks_v2") {
|
||||
return [...rows];
|
||||
}
|
||||
const humanized = (row: T) => row.id !== "" && row.label !== row.id;
|
||||
const collides = collidingLabels(rows.filter(humanized));
|
||||
const suffixed = rows.map((row) =>
|
||||
humanized(row) && collides(row.label)
|
||||
? { ...row, label: `${row.label} (${databricksModelSource(row.id)})` }
|
||||
: row,
|
||||
);
|
||||
const stillCollides = collidingLabels(
|
||||
suffixed.filter((row, i) => row !== rows[i]),
|
||||
);
|
||||
return suffixed.map((row, i) =>
|
||||
row !== rows[i] && stillCollides(row.label)
|
||||
? { ...row, label: `${rows[i].label} (${row.id})` }
|
||||
: row,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -26,7 +26,10 @@ import {
|
||||
DropdownMenuRadioItem,
|
||||
DropdownMenuTrigger,
|
||||
} from "@/shared/ui/dropdown-menu";
|
||||
import { resolveModelLabel } from "@/features/agents/lib/formatAgentModelLabel";
|
||||
import {
|
||||
disambiguateModelLabels,
|
||||
resolveModelLabel,
|
||||
} from "@/features/agents/lib/formatAgentModelLabel";
|
||||
|
||||
export function ModelPicker({
|
||||
agent,
|
||||
@@ -86,10 +89,24 @@ export function ModelPicker({
|
||||
);
|
||||
|
||||
const currentValue = agent.model ?? modelsData?.agentDefaultModel ?? "";
|
||||
const modelRows = React.useMemo(
|
||||
() =>
|
||||
disambiguateModelLabels(
|
||||
(modelsData?.models ?? []).map((model) => ({
|
||||
id: model.id,
|
||||
label: resolveModelLabel(model.id, model.name, agent.provider),
|
||||
})),
|
||||
agent.provider,
|
||||
),
|
||||
[modelsData, agent.provider],
|
||||
);
|
||||
const rowLabel = (id: string) =>
|
||||
modelRows.find((row) => row.id === id)?.label ??
|
||||
resolveModelLabel(id, null, agent.provider);
|
||||
const displayLabel = agent.model
|
||||
? resolveModelLabel(agent.model, null, agent.provider)
|
||||
? rowLabel(agent.model)
|
||||
: modelsData?.agentDefaultModel
|
||||
? `${resolveModelLabel(modelsData.agentDefaultModel, null, agent.provider)} (default)`
|
||||
? `${rowLabel(modelsData.agentDefaultModel)} (default)`
|
||||
: hasRequestedModels && loading
|
||||
? "Loading..."
|
||||
: "Auto";
|
||||
@@ -286,9 +303,9 @@ export function ModelPicker({
|
||||
onValueChange={handleModelChange}
|
||||
value={currentValue}
|
||||
>
|
||||
{modelsData.models.map((model) => (
|
||||
<DropdownMenuRadioItem key={model.id} value={model.id}>
|
||||
{resolveModelLabel(model.id, model.name, agent.provider)}
|
||||
{modelRows.map((row) => (
|
||||
<DropdownMenuRadioItem key={row.id} value={row.id}>
|
||||
{row.label}
|
||||
</DropdownMenuRadioItem>
|
||||
))}
|
||||
</DropdownMenuRadioGroup>
|
||||
|
||||
@@ -35,8 +35,11 @@ export function PersonaModelCombobox({
|
||||
const filteredOptions = React.useMemo(() => {
|
||||
if (query.trim() === "") return options;
|
||||
const lower = query.toLowerCase();
|
||||
return options.filter((option) =>
|
||||
option.label.toLowerCase().includes(lower),
|
||||
// Search the raw id too, so `system.ai` finds rows whose label hides it.
|
||||
return options.filter(
|
||||
(option) =>
|
||||
option.label.toLowerCase().includes(lower) ||
|
||||
option.value.toLowerCase().includes(lower),
|
||||
);
|
||||
}, [options, query]);
|
||||
|
||||
|
||||
@@ -155,10 +155,12 @@ export function AgentDropdownSelect({
|
||||
const filteredOptions = React.useMemo(() => {
|
||||
const trimmed = query.trim().toLowerCase();
|
||||
if (!showSearch || trimmed === "") return options;
|
||||
return options.filter((option) =>
|
||||
(typeof option.label === "string" ? option.label : option.value)
|
||||
.toLowerCase()
|
||||
.includes(trimmed),
|
||||
// Search the raw id too, so `system.ai` finds rows whose label hides it.
|
||||
return options.filter(
|
||||
(option) =>
|
||||
(typeof option.label === "string" &&
|
||||
option.label.toLowerCase().includes(trimmed)) ||
|
||||
option.value.toLowerCase().includes(trimmed),
|
||||
);
|
||||
}, [options, query, showSearch]);
|
||||
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import test from "node:test";
|
||||
|
||||
import {
|
||||
databricksRegistryLabel,
|
||||
generateDatabricksLabel,
|
||||
} from "./modelCapabilities.ts";
|
||||
|
||||
// TS half of the shared display-label contract; the Rust suite
|
||||
// (databricks_label_grammar.rs) replays the same fixture file.
|
||||
const readScript = (name) =>
|
||||
JSON.parse(
|
||||
readFileSync(
|
||||
fileURLToPath(new URL(`../../../../../scripts/${name}`, import.meta.url)),
|
||||
"utf8",
|
||||
),
|
||||
);
|
||||
const fixtures = readScript("databricks-label-fixtures.json");
|
||||
const records = readScript("model-capabilities.json").exact_records.filter(
|
||||
(rec) => rec.provider === "databricks_v2",
|
||||
);
|
||||
const hasExactRecord = (id) =>
|
||||
records.some((rec) => rec.raw_model_id.toLowerCase() === id.toLowerCase());
|
||||
|
||||
test("shared fixtures replay through the label helper", () => {
|
||||
for (const { id, tier, label } of fixtures.lookup) {
|
||||
assert.equal(
|
||||
databricksRegistryLabel(id),
|
||||
label,
|
||||
`id=${JSON.stringify(id)}`,
|
||||
);
|
||||
assert.equal(hasExactRecord(id), tier === "exact", `tier=${tier} id=${id}`);
|
||||
const curated = databricksRegistryLabel(id, { generate: false });
|
||||
if (tier === "exact" || tier === "alias") {
|
||||
assert.equal(curated, label, `id=${id}`);
|
||||
} else if (tier === "generated") {
|
||||
assert.equal(curated, null, `generated fixture is masked: ${id}`);
|
||||
assert.equal(generateDatabricksLabel(id), label, `id=${id}`);
|
||||
} else {
|
||||
assert.equal(tier, "raw", `unknown tier ${tier} for ${id}`);
|
||||
assert.equal(label, null, `raw fixture has a label: ${id}`);
|
||||
assert.equal(curated, null, `raw fixture is curated: ${id}`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("grammar reproduces every curated label except curator-only ones", () => {
|
||||
const misses = records
|
||||
.filter(
|
||||
(rec) => generateDatabricksLabel(rec.raw_model_id) !== rec.registry_label,
|
||||
)
|
||||
.map((rec) => rec.raw_model_id);
|
||||
assert.deepEqual(misses, fixtures.curator_only_labels);
|
||||
});
|
||||
@@ -289,7 +289,7 @@ function toResult(
|
||||
};
|
||||
}
|
||||
|
||||
function isDatabricksModelServiceFqn(model: string): boolean {
|
||||
export function isDatabricksModelServiceFqn(model: string): boolean {
|
||||
const components = model.split(".");
|
||||
return (
|
||||
components.length === 3 &&
|
||||
@@ -428,11 +428,19 @@ export type RegistryLabelRecord = {
|
||||
readonly registry_label: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* Display label for a Databricks endpoint id: exact record → unique alias →
|
||||
* generative grammar (only when `generate` and no record matched) → `null`
|
||||
* (show the raw id). An ambiguous alias match stays `null`.
|
||||
*/
|
||||
export function databricksRegistryLabelForRecords(
|
||||
rawModelId: string,
|
||||
records: ReadonlyArray<RegistryLabelRecord>,
|
||||
familyTokens: ReadonlyArray<string>,
|
||||
generate = true,
|
||||
): string | null {
|
||||
const unmatched = () =>
|
||||
generate ? generateDatabricksLabel(rawModelId) : null;
|
||||
if (!rawModelId.trim()) return null;
|
||||
|
||||
const idLower = rawModelId.toLowerCase();
|
||||
@@ -444,7 +452,7 @@ export function databricksRegistryLabelForRecords(
|
||||
if (exact) return exact.registry_label;
|
||||
|
||||
const strippedQuery = stripCatalogPrefix(idLower, familyTokens);
|
||||
if (strippedQuery === idLower) return null;
|
||||
if (strippedQuery === idLower) return unmatched();
|
||||
let matchingRecord: RegistryLabelRecord | null = null;
|
||||
for (const rec of records) {
|
||||
if (rec.provider !== "databricks_v2") continue;
|
||||
@@ -457,13 +465,155 @@ export function databricksRegistryLabelForRecords(
|
||||
matchingRecord = rec;
|
||||
}
|
||||
}
|
||||
return matchingRecord?.registry_label ?? null;
|
||||
return matchingRecord?.registry_label ?? unmatched();
|
||||
}
|
||||
|
||||
export function databricksRegistryLabel(rawModelId: string): string | null {
|
||||
/**
|
||||
* `generate: false` limits the lookup to curated records — for ids whose
|
||||
* provider is unknown, which must not be humanized by Databricks grammar.
|
||||
*/
|
||||
export function databricksRegistryLabel(
|
||||
rawModelId: string,
|
||||
{ generate = true }: { generate?: boolean } = {},
|
||||
): string | null {
|
||||
return databricksRegistryLabelForRecords(
|
||||
rawModelId,
|
||||
MANIFEST.exact_records,
|
||||
MANIFEST.label_family_tokens,
|
||||
generate,
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Generative display-label grammar — mirrors crates/buzz-agent/src/
|
||||
// databricks_label_grammar.rs. Reached only after an exact-record and a
|
||||
// unique-alias miss; returns a label built solely from the id's own tokens, or
|
||||
// null so callers show the raw id. Only families in the manifest's
|
||||
// label_family_tokens are named; adding a vendor means adding one entry there.
|
||||
// Presentation-only: capabilities, routing,
|
||||
// and the saved id never depend on it. Both interpreters replay
|
||||
// scripts/databricks-label-fixtures.json.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CLAUDE_TIERS = new Set(["opus", "sonnet", "haiku"]);
|
||||
const LABEL_WRAPPERS = ["databricks-", "goose-", "kgoose-", "builderbot-"];
|
||||
|
||||
type LabelPart = { readonly version: boolean; readonly text: string };
|
||||
|
||||
const isVersionToken = (tok: string) => /^(?:\d|[1-9]\d)$/.test(tok);
|
||||
const isDateToken = (tok: string) => /^(?:\d{4}|\d{8})$/.test(tok);
|
||||
const isLetterVersionToken = (tok: string) => /^[a-z]\d+$/.test(tok);
|
||||
const isSizeToken = (tok: string) => /^[a-z]?\d+b$/.test(tok);
|
||||
const capitalize = (word: string) =>
|
||||
word.charAt(0).toUpperCase() + word.slice(1);
|
||||
|
||||
/**
|
||||
* Parse a Databricks endpoint id into a display label, or `null` when any part
|
||||
* of the id falls outside the grammar.
|
||||
*/
|
||||
export function generateDatabricksLabel(rawModelId: string): string | null {
|
||||
// Refuse non-ASCII before trimming or folding, so no Unicode case or
|
||||
// whitespace rule can turn an unsupported id into an ASCII-looking one.
|
||||
if ([...rawModelId].some((ch) => ch.charCodeAt(0) > 0x7f)) return null;
|
||||
const id = rawModelId.trim().toLowerCase();
|
||||
const isFqn = isDatabricksModelServiceFqn(id);
|
||||
const service = isFqn ? id.slice(id.lastIndexOf(".") + 1) : id;
|
||||
// Only a UC FQN or a wrapped endpoint name is attributable to Databricks; a
|
||||
// bare id such as `gpt-5` stays raw.
|
||||
const wrapper = LABEL_WRAPPERS.find((w) => service.startsWith(w));
|
||||
if (!wrapper && !isFqn) return null;
|
||||
let body = wrapper ? service.slice(wrapper.length) : service;
|
||||
if (body.startsWith("meta-llama-")) body = body.slice("meta-".length);
|
||||
|
||||
const [head, ...rest] = body.split("-");
|
||||
const familyMatch = /^([a-z]+)(\d{0,2})$/.exec(head);
|
||||
if (!familyMatch) return null;
|
||||
const [, family, digits] = familyMatch;
|
||||
if (
|
||||
!MANIFEST.label_family_tokens.some(
|
||||
(token) => token.replace(/-$/, "") === family,
|
||||
)
|
||||
)
|
||||
return null;
|
||||
// Attached digits followed by a version (`qwen3-5`, `llama3-1`) cannot be
|
||||
// read without guessing; checked before any reorder.
|
||||
if (digits !== "" && isVersionToken(rest[0] ?? "")) return null;
|
||||
// In-name digits stay whole (`nova12` → `Nova12`). The one exception is
|
||||
// Qwen's own compact-decimal naming: `qwen35` → `Qwen3.5` when the second
|
||||
// digit is nonzero.
|
||||
const stem =
|
||||
family === "qwen" && digits.length === 2 && digits[1] !== "0"
|
||||
? `${digits[0]}.${digits[1]}`
|
||||
: digits;
|
||||
if (family === "claude") {
|
||||
// Goose's numeric-first Claude ids (`claude-4-7-opus`) name the tier after
|
||||
// the version. Reorder only that exact shape; any other token after the
|
||||
// version keeps its position.
|
||||
let count = 0;
|
||||
while (count < rest.length && isVersionToken(rest[count])) count += 1;
|
||||
if (count >= 1 && count <= 2 && CLAUDE_TIERS.has(rest[count])) {
|
||||
rest.splice(0, count + 1, rest[count], ...rest.slice(0, count));
|
||||
}
|
||||
}
|
||||
|
||||
const parts: LabelPart[] = [];
|
||||
let afterMinor = false;
|
||||
for (let i = 0; i < rest.length; i += 1) {
|
||||
const tok = rest[i];
|
||||
const next = rest[i + 1];
|
||||
if (isDateToken(tok) && next === undefined) break;
|
||||
const hasMinor = next !== undefined && isVersionToken(next);
|
||||
// A third number (`3-7-1`) would read as a separate version.
|
||||
if (
|
||||
hasMinor &&
|
||||
(isVersionToken(tok) || isLetterVersionToken(tok)) &&
|
||||
isVersionToken(rest[i + 2] ?? "")
|
||||
)
|
||||
return null;
|
||||
if (isVersionToken(tok)) {
|
||||
afterMinor = hasMinor;
|
||||
parts.push({ version: true, text: afterMinor ? `${tok}.${next}` : tok });
|
||||
if (afterMinor) i += 1;
|
||||
} else if (isLetterVersionToken(tok)) {
|
||||
const major = capitalize(tok);
|
||||
parts.push({
|
||||
version: false,
|
||||
text: hasMinor ? `${major}.${next}` : major,
|
||||
});
|
||||
if (hasMinor) i += 1;
|
||||
} else if (isSizeToken(tok)) {
|
||||
parts.push({ version: false, text: tok.toUpperCase() });
|
||||
} else if (/^[a-z]+$/.test(tok)) {
|
||||
const text =
|
||||
tok === "oss"
|
||||
? "OSS"
|
||||
: family === "gpt" && afterMinor && (tok === "mini" || tok === "nano")
|
||||
? tok
|
||||
: capitalize(tok);
|
||||
parts.push({ version: false, text });
|
||||
} else {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
// A versionless id (`builderbot-pr-reviews`) is a named endpoint, not a model.
|
||||
const hasNumber =
|
||||
stem !== "" || parts.some((part) => part.version || /\d/.test(part.text));
|
||||
if (!hasNumber) return null;
|
||||
|
||||
const brand =
|
||||
family === "gpt"
|
||||
? "GPT"
|
||||
: family === "glm"
|
||||
? "GLM"
|
||||
: family === "deepseek"
|
||||
? "DeepSeek"
|
||||
: capitalize(family);
|
||||
const hyphenated = family === "gpt" || family === "glm";
|
||||
return parts.reduce(
|
||||
(label, part, index) =>
|
||||
label +
|
||||
(part.version && index === 0 && hyphenated ? "-" : " ") +
|
||||
part.text,
|
||||
brand + stem,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -12,7 +12,10 @@ import {
|
||||
} from "./personaModelDiscoveryStatus";
|
||||
import type { PersonaModelOption } from "./agentConfigOptions";
|
||||
import { providerRequiresExplicitModel } from "./agentConfigOptions";
|
||||
import { resolveModelLabel } from "@/features/agents/lib/formatAgentModelLabel";
|
||||
import {
|
||||
disambiguateModelLabels,
|
||||
resolveModelLabel,
|
||||
} from "@/features/agents/lib/formatAgentModelLabel";
|
||||
|
||||
export const MODEL_DISCOVERY_LOADING_VALUE = "__model_discovery_loading__";
|
||||
|
||||
@@ -76,10 +79,13 @@ export function getDiscoveredPersonaModelOptions(
|
||||
|
||||
return [
|
||||
...defaultModelOption,
|
||||
...explicitModels.map((model) => ({
|
||||
id: model.id,
|
||||
label: resolveModelLabel(model.id, model.name, provider),
|
||||
})),
|
||||
...disambiguateModelLabels(
|
||||
explicitModels.map((model) => ({
|
||||
id: model.id,
|
||||
label: resolveModelLabel(model.id, model.name, provider),
|
||||
})),
|
||||
provider,
|
||||
),
|
||||
];
|
||||
}
|
||||
|
||||
|
||||
@@ -227,6 +227,82 @@ test.describe("edit agent dialog", () => {
|
||||
);
|
||||
});
|
||||
|
||||
test("tells colliding Databricks model names apart and persists the chosen id", async ({
|
||||
page,
|
||||
}) => {
|
||||
const goosePathId = "data_workflow_tools.goose.gpt-6-astra";
|
||||
const systemAiId = "system.ai.gpt-6-astra";
|
||||
await installMockBridge(page, {
|
||||
managedAgents: [
|
||||
{
|
||||
pubkey: AGENT_PUBKEY,
|
||||
name: AGENT_NAME,
|
||||
status: "stopped",
|
||||
channelNames: ["agents"],
|
||||
envVars: { DATABRICKS_HOST: "https://databricks.example.test" },
|
||||
},
|
||||
],
|
||||
// The agent inherits Databricks from the global defaults; the mock's
|
||||
// update_managed_agent does not echo a per-agent provider.
|
||||
globalAgentConfig: {
|
||||
preferred_runtime: "goose",
|
||||
provider: "databricks_v2",
|
||||
model: null,
|
||||
env_vars: {},
|
||||
},
|
||||
discoverAgentModels: {
|
||||
models: [
|
||||
{ id: goosePathId, name: goosePathId },
|
||||
{ id: systemAiId, name: systemAiId },
|
||||
],
|
||||
supportsSwitching: true,
|
||||
selectedModel: null,
|
||||
},
|
||||
});
|
||||
|
||||
await openEditDialog(page);
|
||||
|
||||
await page.locator("#edit-agent-model").click();
|
||||
await expect(
|
||||
page.getByRole("menuitemradio", {
|
||||
name: "GPT-6 Astra (data_workflow_tools.goose)",
|
||||
}),
|
||||
).toBeVisible();
|
||||
await page
|
||||
.getByRole("menuitemradio", { name: "GPT-6 Astra (system.ai)" })
|
||||
.click();
|
||||
await expect(page.locator("#edit-agent-model")).toHaveText(
|
||||
"GPT-6 Astra (system.ai)",
|
||||
);
|
||||
|
||||
const submit = page.getByTestId("edit-agent-dialog-submit");
|
||||
await expect(submit).toBeEnabled({ timeout: 10_000 });
|
||||
await submit.click();
|
||||
await expect(page.getByTestId("edit-agent-dialog")).not.toBeVisible();
|
||||
|
||||
const persistedModel = () =>
|
||||
page.evaluate(async (pubkey) => {
|
||||
const agents = (await (
|
||||
window as typeof window & {
|
||||
__BUZZ_E2E_INVOKE_MOCK_COMMAND__?: (
|
||||
command: string,
|
||||
payload: unknown,
|
||||
) => Promise<unknown>;
|
||||
}
|
||||
).__BUZZ_E2E_INVOKE_MOCK_COMMAND__?.("list_managed_agents", null)) as
|
||||
| Array<{ pubkey: string; model: string | null }>
|
||||
| undefined;
|
||||
return agents?.find((agent) => agent.pubkey === pubkey)?.model;
|
||||
}, AGENT_PUBKEY);
|
||||
await expect.poll(persistedModel).toBe(systemAiId);
|
||||
|
||||
await page.getByTestId("user-profile-edit-agent").click();
|
||||
await expect(page.locator("#edit-agent-model")).toHaveText(
|
||||
"GPT-6 Astra (system.ai)",
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
});
|
||||
|
||||
test("keeps the custom command visible without opening Advanced", async ({
|
||||
page,
|
||||
}) => {
|
||||
|
||||
@@ -335,6 +335,93 @@ test.describe("global agent config screenshots", () => {
|
||||
expect(persisted).toMatchObject({ model: modelId });
|
||||
});
|
||||
|
||||
test("defaults humanize uncurated Databricks ids without changing persisted ids", async ({
|
||||
page,
|
||||
}) => {
|
||||
// Neither id has a curated record; both labels come from the generative
|
||||
// grammar, and two distinct ids sharing a label must both stay selectable.
|
||||
const modelId = "data_workflow_tools.goose.goose-gpt-6-astra";
|
||||
const siblingId = "system.ai.gpt-6-astra";
|
||||
await installMockBridge(page, {
|
||||
globalAgentConfig: {
|
||||
preferred_runtime: "goose",
|
||||
provider: "databricks_v2",
|
||||
model: modelId,
|
||||
env_vars: {},
|
||||
},
|
||||
discoverAgentModels: {
|
||||
models: [
|
||||
{ id: modelId, name: modelId },
|
||||
{ id: siblingId, name: siblingId },
|
||||
{ id: "builderbot-pr-reviews", name: "builderbot-pr-reviews" },
|
||||
],
|
||||
supportsSwitching: true,
|
||||
selectedModel: modelId,
|
||||
},
|
||||
runtimeFileConfigs: {
|
||||
goose: {
|
||||
provider: "databricks_v2",
|
||||
model: modelId,
|
||||
satisfiedEnvKeys: ["DATABRICKS_HOST"],
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
await openAiDefaultsSettings(page);
|
||||
|
||||
const model = page.getByTestId("global-agent-model");
|
||||
await expect(model).toHaveText("GPT-6 Astra (data_workflow_tools.goose)");
|
||||
await expect(model).toHaveAttribute("data-value", modelId);
|
||||
|
||||
await model.click();
|
||||
const option = (id: string) =>
|
||||
page.getByTestId(`global-agent-model-option-${id}`);
|
||||
// Colliding labels are told apart by their catalog.schema.
|
||||
await expect(option(modelId)).toHaveText(
|
||||
"GPT-6 Astra (data_workflow_tools.goose)",
|
||||
);
|
||||
await expect(option(siblingId)).toHaveText("GPT-6 Astra (system.ai)");
|
||||
await expect(option("builderbot-pr-reviews")).toHaveText(
|
||||
"builderbot-pr-reviews",
|
||||
);
|
||||
|
||||
// Search matches the raw id as well as the label.
|
||||
await page
|
||||
.getByTestId("global-agent-model-search")
|
||||
.fill("system.ai.gpt-6-astra");
|
||||
await expect(option(siblingId)).toBeVisible();
|
||||
await expect(option(modelId)).toHaveCount(0);
|
||||
await expect(option("builderbot-pr-reviews")).toHaveCount(0);
|
||||
|
||||
await page.getByRole("option", { name: "GPT-6 Astra (system.ai)" }).click();
|
||||
await expect(model).toHaveText("GPT-6 Astra (system.ai)");
|
||||
await expect(model).toHaveAttribute("data-value", siblingId);
|
||||
|
||||
await page
|
||||
.getByRole("button", { name: "Save defaults" })
|
||||
.filter({ visible: true })
|
||||
.click();
|
||||
const persisted = () =>
|
||||
page.evaluate(async () =>
|
||||
(
|
||||
window as typeof window & {
|
||||
__BUZZ_E2E_INVOKE_MOCK_COMMAND__?: (
|
||||
command: string,
|
||||
payload: unknown,
|
||||
) => Promise<unknown>;
|
||||
}
|
||||
).__BUZZ_E2E_INVOKE_MOCK_COMMAND__?.("get_global_agent_config", null),
|
||||
);
|
||||
await expect.poll(persisted).toMatchObject({ model: siblingId });
|
||||
|
||||
// Reopen without a reload: the closed trigger recomputes its label from
|
||||
// the saved raw id.
|
||||
await page.getByTestId("settings-nav-appearance").click();
|
||||
await page.getByTestId("settings-nav-agents").click();
|
||||
await expect(model).toHaveText("GPT-6 Astra (system.ai)");
|
||||
await expect(model).toHaveAttribute("data-value", siblingId);
|
||||
});
|
||||
|
||||
test("defaults honor credentials set in the harness config file", async ({
|
||||
page,
|
||||
}) => {
|
||||
|
||||
80
scripts/databricks-label-fixtures.json
Normal file
80
scripts/databricks-label-fixtures.json
Normal file
@@ -0,0 +1,80 @@
|
||||
{
|
||||
"_comment": "Shared Databricks display-label fixtures, replayed by crates/buzz-agent/src/databricks_label_grammar.rs and desktop/src/features/agents/ui/databricksLabelFixtures.test.mjs. `lookup` rows run through the full label helper (exact record -> unique alias -> generative grammar -> raw); a null label means the raw id is shown. Every `generated` row must stay absent from exact records so a curated addition cannot mask a broken grammar. `curator_only_labels` lists the exact records whose curated label intentionally differs from what the grammar derives from the id: curator suffixes like `(latest)` / `Preview`, a version alias, or a versionless name the grammar deliberately refuses.",
|
||||
"lookup": [
|
||||
{ "id": "databricks-gpt-5-5", "tier": "exact", "label": "GPT-5.5" },
|
||||
{ "id": "DATABRICKS-GPT-5-5", "tier": "exact", "label": "GPT-5.5" },
|
||||
{ "id": "databricks-claude-sonnet-4", "tier": "exact", "label": "Claude Sonnet 4.5" },
|
||||
{ "id": "databricks-gemini-3-pro", "tier": "exact", "label": "Gemini 3 Pro Preview" },
|
||||
{ "id": "goose-claude-4-7-opus", "tier": "exact", "label": "Claude Opus 4.7" },
|
||||
{ "id": "data_workflow_tools.goose.goose-claude-opus-5-5", "tier": "exact", "label": "Claude Opus 5.5" },
|
||||
{ "id": "system.ai.claude-opus-5-5", "tier": "alias", "label": "Claude Opus 5.5" },
|
||||
{ "id": "goose-gpt-5-6-sol", "tier": "alias", "label": "GPT-5.6 Sol" },
|
||||
{ "id": "goose-claude-haiku-4-5", "tier": "alias", "label": "Claude Haiku 4.5 (latest)" },
|
||||
{ "id": "system.ai.deepseek-v4-pro-0813", "tier": "alias", "label": "DeepSeek V4 Pro" },
|
||||
{ "id": "builderbot-glm-5-2", "tier": "alias", "label": "GLM-5.2" },
|
||||
{ "id": "system.ai.gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" },
|
||||
{ "id": "data_workflow_tools.goose.goose-gpt-6-astra", "tier": "generated", "label": "GPT-6 Astra" },
|
||||
{ "id": "data_workflow_tools.goose.goose-gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" },
|
||||
{ "id": "data_workflow_tools.goose.goose-gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" },
|
||||
{ "id": "system.ai.gemini-3-7-flash", "tier": "generated", "label": "Gemini 3.7 Flash" },
|
||||
{ "id": "system.ai.gemini-3-8-flash", "tier": "generated", "label": "Gemini 3.8 Flash" },
|
||||
{ "id": "system.ai.deepseek-v4-1-flash", "tier": "generated", "label": "DeepSeek V4.1 Flash" },
|
||||
{ "id": "databricks-gpt-5-3-codex", "tier": "generated", "label": "GPT-5.3 Codex" },
|
||||
{ "id": "databricks-gpt-5-5-pro", "tier": "generated", "label": "GPT-5.5 Pro" },
|
||||
{ "id": "system.ai.mistral-large-3", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.grok-5-mini", "tier": "generated", "label": "Grok 5 Mini" },
|
||||
{ "id": "system.ai.claude-opus-5-5-thinking", "tier": "generated", "label": "Claude Opus 5.5 Thinking" },
|
||||
{ "id": "system.ai.gemini-3-7-flash-0925", "tier": "generated", "label": "Gemini 3.7 Flash" },
|
||||
{ "id": "system.ai.phi-5-mini", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.minimax-m3", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.qwen36-35b-a3b", "tier": "generated", "label": "Qwen3.6 35B A3B" },
|
||||
{ "id": "system.ai.gpt-6-sol", "tier": "generated", "label": "GPT-6 Sol" },
|
||||
{ "id": "system.ai.gpt-6-luna", "tier": "generated", "label": "GPT-6 Luna" },
|
||||
{ "id": "system.ai.nova12-pro", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.qwen10-32b", "tier": "generated", "label": "Qwen10 32B" },
|
||||
{ "id": "system.ai.gpt10", "tier": "generated", "label": "GPT10" },
|
||||
{ "id": "system.ai.qwen35-122b-a10b-instruct", "tier": "generated", "label": "Qwen3.5 122B A10B Instruct" },
|
||||
{ "id": "system.ai.claude-5-6-opus", "tier": "generated", "label": "Claude Opus 5.6" },
|
||||
{ "id": "system.ai.claude-5-6-v2", "tier": "generated", "label": "Claude 5.6 V2" },
|
||||
{ "id": "system.ai.claude-5-6-thinking", "tier": "generated", "label": "Claude 5.6 Thinking" },
|
||||
{ "id": "system.ai.gemini-3-7-1-flash", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.deepseek-v4-1-2-flash", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.\u212aimi-k4", "tier": "raw", "label": null },
|
||||
{ "id": "databricks-\u212aimi-k4", "tier": "raw", "label": null },
|
||||
{ "id": "\ufeffsystem.ai.nova-3\ufeff", "tier": "raw", "label": null },
|
||||
{ "id": "\u0085system.ai.nova-3\u0085", "tier": "raw", "label": null },
|
||||
{ "id": "goose-qwen3-5-122b-a10b", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.llama3-1-8b", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.claude3-5-sonnet", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.nova12-3-4", "tier": "raw", "label": null },
|
||||
{ "id": "builderbot-pr-reviews-v2", "tier": "raw", "label": null },
|
||||
{ "id": "prod.ai.rag-agent-3", "tier": "raw", "label": null },
|
||||
{ "id": "dev.ai.rag-agent-3", "tier": "raw", "label": null },
|
||||
{ "id": "builderbot-pr-reviews", "tier": "raw", "label": null },
|
||||
{ "id": "gpt-5", "tier": "raw", "label": null },
|
||||
{ "id": "gpt-5.5", "tier": "raw", "label": null },
|
||||
{ "id": "qwen3:32b", "tier": "raw", "label": null },
|
||||
{ "id": "claude-opus-4.7", "tier": "raw", "label": null },
|
||||
{ "id": "command-r-plus", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.qwen3:32b", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.claude-opus-4.7", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.command-r-plus", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.gpt", "tier": "raw", "label": null },
|
||||
{ "id": "system.ai.gpt-05", "tier": "raw", "label": null },
|
||||
{ "id": "databricks-team-2025-01", "tier": "raw", "label": null },
|
||||
{ "id": "custom-unlisted-endpoint", "tier": "raw", "label": null },
|
||||
{ "id": " ", "tier": "raw", "label": null }
|
||||
],
|
||||
"curator_only_labels": [
|
||||
"databricks-claude-haiku-4-5",
|
||||
"databricks-claude-opus-4-1",
|
||||
"databricks-claude-opus-4-5",
|
||||
"databricks-claude-sonnet-4",
|
||||
"databricks-claude-sonnet-4-5",
|
||||
"databricks-gemini-3-1-flash-lite",
|
||||
"databricks-gemini-3-1-pro",
|
||||
"databricks-gemini-3-flash",
|
||||
"databricks-gemini-3-pro",
|
||||
"databricks-inkling"
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user