mirror of
https://github.com/block/buzz.git
synced 2026-11-05 08:51:10 +00:00
a82d6916fb
Three cells, all at reasoning effort `high`, all on pinned OpenRouter
upstreams:
tb-solo-kimi-k3-high TB 2.1 solo, moonshotai/kimi-k3
tb-solo-deepseek-v4-flash-high TB 2.1 solo, deepseek-v4-flash-0731
lhtb-or-kimi-lead-2deepseek-high LHTB-46 team: kimi lead, deepseek
scout, deepseek worker
The first cells in the study on open-weight models and on a route other
than direct-OpenAI or Databricks. The solo pair spans a 22x input price
ratio ($0.133 vs $3.00/Mtok), making it the open-weight analogue of the
luna -> sol ladder; the team cell is the open-weight restaging of
tb-gt-sol-luna-terra-high, which is currently the study's only clean
long-horizon team win.
NO MEDIUM-EFFORT CELLS. The original plan had a high/medium pair per
model. Neither model supports it: both advertise supported_efforts
["max","high","low"], and while OpenRouter accepts `medium` without
error, measured reasoning length on deepseek did not separate low from
medium from high (medians 1097 / 1102 / 1067, within-level spread wider
than between-level). A cell labelled `medium` would name a condition the
provider does not implement, so the effort axis is dropped rather than
faked.
Effort is still pinned on all three cells even though `high` matches
deepseek's provider default -- an unpinned cell records no effort in its
manifest, so a later provider-side default change would silently redefine
the condition without moving the hash. kimi defaults to `max`, so there
`high` is a deliberate step down.
Prices are the PINNED endpoints' own rates, not the model ids' headline
rates -- gmicloud/fp8 bills $0.133/$0.266 where the model id advertises
$0.14/$0.28, and two of kimi's ten upstreams bill 1.5x the row here. The
pin is what makes these rows the correct ones, so dropping it would
require restating cost as well as re-establishing the condition.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Atish Patel <atish@squareup.com>