jerry/buzz
1
0
Fork 0
mirror of https://github.com/block/buzz.git synced 2026-10-27 04:30:52 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(model-capabilities): humanize Databricks UC model families (#6955)

Browse Source 
Chat-eligible Unity Catalog models from the Gemini, DeepSeek, GLM, Grok,
Llama, Qwen, Gemma, and Inkling families were rendering as raw FQNs such
as `system.ai.gemini-3-5-flash` in the agent model picker. #6918
surfaces these models, but the capability manifest carried no records
for most of them and the registry-label path could not strip their Unity
Catalog or `goose-` prefixes.

I added a display-only `label_family_tokens` list and `databricks_v2`
exact records for the sixteen previously unlabeled endpoint stems. All
26 target FQNs and their `goose-` aliases now resolve to curated labels
through `databricks_registry_label`, while capability resolution
continues to use only the existing `family_tokens` (`claude-`, `gpt-`,
and `kimi-`). This keeps label discovery from changing capability
profiles for unrelated model IDs.

Each new record includes reconciliation metadata against the pinned
[models.dev catalog](https://models.dev/api.json). Nine records adopt
first-party effort evidence that the Databricks MLflow Chat transport
can express: Gemini 3.5 Flash, Gemini 3.5 Flash Lite, Gemini 3.6 Flash,
Gemini 3 Pro Image, DeepSeek V4 Flash, DeepSeek V4 Pro, GLM-5.3 Flash,
Grok 4.6, and Inkling. The remaining seven deliberately retain the
`databricks_v2/concrete_unknown` fallback because upstream evidence is
absent, identifies a non-reasoning model, or provides only
toggle/token-budget controls that `reasoning_effort` cannot represent.

The normative corpus adds the sixteen exact records, four Unity Catalog
alias probes, and two capability-isolation probes. Existing base vectors
remain unchanged, and Rust and TypeScript consume the same strict
manifest and corpus.

Related: #6918

---------

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Thufir <7ebdb0b67dab08a570b9faf7bbada97535673b4ccaba2cbd546ad3ba84c87fa6@buzz.block.builderlab.xyz>
Co-authored-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Co-authored-by: Thufir <7ebdb0b67dab08a570b9faf7bbada97535673b4ccaba2cbd546ad3ba84c87fa6@buzz.block.builderlab.xyz>
This commit is contained in:
Will Pfleger
2026-08-27 17:25:19 -04:00
committed by GitHub
parent be8d7756d5
commit ae0ecd9a67
5 changed files with 880 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

437
scripts/normative-corpus.json
View File

@@ -2266,5 +2266,442 @@
"normalization_policy": "openai-standard",
"registry_label": null
}
},
{
"_group": "Databricks UC model-family humanization probes (#6918 follow-up)",
"_note": "Exact-record and UC-FQN strip probes for the Gemini/DeepSeek/GLM/Grok/Llama/Qwen/Gemma/Inkling families surfaced by UC discovery."
},
{
"id": "dbv2-gemini-3-1-flash-image-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemini-3-1-flash-image",
"_note": "Probes the Gemini 3.1 Flash Image endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Gemini 3.1 Flash Image"
}
},
{
"id": "dbv2-gemini-3-5-flash-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemini-3-5-flash",
"_note": "Probes the Gemini 3.5 Flash endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"minimal",
"low",
"medium",
"high"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Gemini 3.5 Flash"
}
},
{
"id": "dbv2-gemini-3-5-flash-lite-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemini-3-5-flash-lite",
"_note": "Probes the Gemini 3.5 Flash Lite endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"minimal",
"low",
"medium",
"high"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Gemini 3.5 Flash Lite"
}
},
{
"id": "dbv2-gemini-3-6-flash-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemini-3-6-flash",
"_note": "Probes the Gemini 3.6 Flash endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"minimal",
"low",
"medium",
"high"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Gemini 3.6 Flash"
}
},
{
"id": "dbv2-gemini-3-pro-image-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemini-3-pro-image",
"_note": "Probes the Gemini 3 Pro Image endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"high"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Gemini 3 Pro Image"
}
},
{
"id": "dbv2-deepseek-v4-flash-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-deepseek-v4-flash-0731",
"_note": "Probes the DeepSeek V4 Flash endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "DeepSeek V4 Flash"
}
},
{
"id": "dbv2-deepseek-v4-pro-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-deepseek-v4-pro-0813",
"_note": "Probes the DeepSeek V4 Pro endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "DeepSeek V4 Pro"
}
},
{
"id": "dbv2-glm-5-3-flash-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-glm-5-3-flash",
"_note": "Probes the GLM-5.3 Flash endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"high",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "GLM-5.3 Flash"
}
},
{
"id": "dbv2-grok-4-6-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-grok-4-6",
"_note": "Probes the Grok 4.6 endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Grok 4.6"
}
},
{
"id": "dbv2-llama-4-maverick-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-llama-4-maverick",
"_note": "Probes the Llama 4 Maverick endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Llama 4 Maverick"
}
},
{
"id": "dbv2-meta-llama-3-1-8b-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-meta-llama-3-1-8b-instruct",
"_note": "Probes the meta-llama record; the llama- token strips the meta- prefix identically for record and query.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Llama 3.1 8B Instruct"
}
},
{
"id": "dbv2-meta-llama-3-3-70b-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-meta-llama-3-3-70b-instruct",
"_note": "Probes the meta-llama 3.3 70B record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Llama 3.3 70B Instruct"
}
},
{
"id": "dbv2-qwen3-next-80b-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-qwen3-next-80b-a3b-instruct",
"_note": "Probes the Qwen3 Next 80B record; the bare qwen token strips on a hyphen boundary.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Qwen3 Next 80B A3B Instruct"
}
},
{
"id": "dbv2-qwen35-122b-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-qwen35-122b-a10b",
"_note": "Probes the Qwen3.5 122B record; the bare qwen token strips a qwen35 stem with no separator.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Qwen3.5 122B A10B"
}
},
{
"id": "dbv2-gemma-3-12b-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-gemma-3-12b",
"_note": "Probes the Gemma 3 12B endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": "Gemma 3 12B"
}
},
{
"id": "dbv2-inkling-exact-record-probe",
"provider": "databricks_v2",
"raw_model_id": "databricks-inkling",
"_note": "Probes the Inkling endpoint record and label.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"low",
"medium",
"high",
"xhigh",
"max"
],
"default_effort": null,
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-standard",
"registry_label": "Inkling"
}
},
{
"id": "dbv2-uc-fqn-gemini-3-5-flash-strip-probe",
"provider": "databricks_v2",
"raw_model_id": "system.ai.gemini-3-5-flash",
"_note": "Probes strip parity on a system.ai. UC FQN carrying the gemini- token (resolve carries no label; the alias label path is unit-tested).",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": null
}
},
{
"id": "dbv2-uc-fqn-meta-llama-strip-probe",
"provider": "databricks_v2",
"raw_model_id": "system.ai.meta-llama-3-3-70b-instruct",
"_note": "Probes strip parity on a UC FQN where the llama- token strips through meta-.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": null
}
},
{
"id": "dbv2-uc-goose-deepseek-strip-probe",
"provider": "databricks_v2",
"raw_model_id": "data_workflow_tools.goose.goose-deepseek-v4-pro-0813",
"_note": "Probes strip parity on a goose- prefixed UC FQN carrying the deepseek- token.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": null
}
},
{
"id": "dbv2-uc-fqn-inkling-strip-probe",
"provider": "databricks_v2",
"raw_model_id": "system.ai.inkling",
"_note": "Probes strip parity on a UC FQN carrying the bare inkling token.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"none",
"minimal",
"low",
"medium",
"high",
"xhigh"
],
"default_effort": "medium",
"databricks_v2_wire_route": "mlflow-chat",
"normalization_policy": "openai-clamp-max-to-xhigh",
"registry_label": null
}
},
{
"_group": "Label/capability token isolation probes (#6955 review pass 1)",
"_note": "Pins that label_family_tokens (the UC-humanization superset) never leaks into capability resolve(): capability stripping still uses only claude-/gpt-/kimi-, so a label token appearing before a gpt- marker must NOT displace the gpt-5-pro exact profile."
},
{
"id": "isolation-openai-gemini-gpt-5-pro-probe",
"provider": "openai",
"raw_model_id": "tenant-gemini-gpt-5-pro",
"_note": "The gemini- label token must not strip here; capability resolve keeps the gpt-5-pro high-only profile.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"high"
],
"default_effort": "high",
"databricks_v2_wire_route": "not-applicable",
"normalization_policy": "openai-standard",
"registry_label": null
}
},
{
"id": "isolation-openai-qwenchanted-gpt-5-pro-probe",
"provider": "openai",
"raw_model_id": "tenant-qwenchanted-gpt-5-pro",
"_note": "The bare qwen label token must not fire mid-segment; capability resolve keeps the gpt-5-pro high-only profile.",
"expect": {
"thinking_mode": "none",
"supported_efforts": [
"high"
],
"default_effort": "high",
"databricks_v2_wire_route": "not-applicable",
"normalization_policy": "openai-standard",
"registry_label": null
}
}
]