mirror of
https://github.com/block/buzz.git
synced 2026-11-01 23:11:58 +00:00
fix(agent): pass-2 Rust production cutover + comprehensive differential + CI gate
Production cutover (Thufir finding 1, Rust half): - Add normalize_effort_for_databricks_v2() and anthropic_thinking_config_for_databricks_v2() as generated-backed production helpers for DatabricksV2 requests. - llm.rs DatabricksV2 dispatch now routes through these helpers: OpenAiResponses and MlflowChatCompletions paths call normalize_effort_for_databricks_v2(); anthropic_body calls anthropic_thinking_config_for_databricks_v2(). The adopted F1 corrections (e.g. databricks-gpt-5-5: [low,medium,high]) are now enforced in real requests. - Pure OpenAi/Databricks path retains normalize_effort_for_openai_route (unchanged). Comprehensive Rust differential (Thufir finding 4): - Replace the simple route-only differential test (20-vector literal list, no allowlist scoping) with a comprehensive multi-axis test covering all 3 input sets: effortTable.fixture.json (36), normative-corpus.json (45), catalog-sample-fixture.json (14). - Axes: supported_efforts, default_effort, databricks_v2_wire_route, thinking_mode. - Allowlist is axis-scoped; 7 entries, all doc-cited. Stale entries FAIL (mirrors JS harness semantics — no whole-record skip). - Exposes strip_catalog_prefix as pub(crate); adds #[cfg(test)] pub(crate) helpers is_manual_budget_model_for_test, is_adaptive_thinking_model_for_test, and valid_effort_values_for_provider_model_for_test so llm.rs tests can call config.rs classifiers without duplicating them. - Result: 95 input entries, 7/7 allowlist slots exercised, 0 unexpected divergences. CI gate (Thufir finding 3): - Extend model-capability-regen-diff workflow path filter to cover the differential harness, all three fixtures, and both old/new consumer files (buzzAgentConfig.ts, config.rs, llm.rs) so any change to either side triggers the gate. - Add a final step: node --experimental-strip-types scripts/run-differential.mjs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
This commit is contained in:
parent
ffc46676c7
commit
c2bed46ee4
@@ -9,6 +9,14 @@ on:
|
||||
- 'desktop/src/features/agents/ui/modelCapabilities.ts'
|
||||
- 'scripts/generated-model-capabilities-coverage.json'
|
||||
- '.github/workflows/model-capability-regen-diff.yml'
|
||||
# Differential harness and fixtures — any change to old/new side or inputs re-runs.
|
||||
- 'scripts/run-differential.mjs'
|
||||
- 'scripts/normative-corpus.json'
|
||||
- 'scripts/catalog-sample-fixture.json'
|
||||
- 'desktop/src/features/agents/ui/effortTable.fixture.json'
|
||||
- 'desktop/src/features/agents/ui/buzzAgentConfig.ts'
|
||||
- 'crates/buzz-agent/src/config.rs'
|
||||
- 'crates/buzz-agent/src/llm.rs'
|
||||
push:
|
||||
branches: [main, release, 'duncan/databricks-model-label-registry']
|
||||
|
||||
@@ -51,3 +59,6 @@ jobs:
|
||||
|
||||
- name: Validate manifest (schema-negative tests)
|
||||
run: node --test scripts/test-manifest-validator.mjs
|
||||
|
||||
- name: Run differential harness (old vs new, all input sets)
|
||||
run: node --experimental-strip-types scripts/run-differential.mjs
|
||||
|
||||
Reference in New Issue
Block a user