mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh (#5289)
Shared compute now has exactly two model choices: MeshLLM's virtual `mesh` model, or a model you name. Buzz picks between them in one place, and buzz-agent no longer knows meshes exist. ## What changed - **MeshLLM v0.74.0 → v0.75.1.** v0.75.0 added `degrade_to_single_model`, so a `model=mesh` request is answered by one served model when there is no committee to form, instead of failing. v0.75.1 adds Mesh-LLM#1196, which skips stale pre-0.75 runtime cache entries rather than aborting startup on them — without it, anyone who had run mesh on 0.73/0.74 could not start. - **Deleted the client-side mesh catalog probe.** buzz-agent used to poll `/v1/models` (5s TTL, 30s cooldown, two-observation debounce) to decide whether `mesh` was safe to send. MeshLLM now decides per request, so the polling, its hysteresis, and its 503 fallback are gone. - **One mapping point.** `relay_mesh_wire_model()` turns the stored value into a wire name: `auto` becomes `mesh`, a named model passes through. The spawn env, the ACP harness, and the readiness probe all use it, so they cannot disagree — previously `BUZZ_ACP_MODEL` and the probe both said `auto`, a name the mesh does not advertise. - **Removed the `nostr-relay-pool` advisory exception.** #5404 allowed RUSTSEC-2026-0243 "after mesh-llm migrates to nostr-sdk >= 0.45". v0.75.1 does, so the retired crate is gone from both lockfiles and the exception would only mask a future advisory for it. - **Deleted `scripts/ensure-mesh-native-runtime.sh`** and its six justfile call sites. It built llama.cpp from source into the runtime cache; the app already downloads the signed release runtime itself, and CI never called it. ## Why it is better **−639 lines of Rust.** Availability is decided by the node that knows the answer, per request, instead of by a client cache that could be stale for up to 30 seconds. A second worker joining now takes effect on the next request rather than after two confirming probes. ## Behaviour change A 503 on an explicit `mesh` request takes the ordinary transport retry under the same model instead of failing over to a second one — there is no second model to fail over to now. MoA repairs partial committee results internally before it reaches that point. ## Validation `crates/buzz-relay/examples/mesh_agent_e2e.rs` now sends `mesh` where it previously sent `auto` or the physical model id, so no leg was covering what Buzz actually puts on the wire. 4/4 on gemma-4-E4B, gemma-4-26B-A4B, and Qwen3-8B — including a real ACP tool call through `mesh` into buzz-dev-mcp, asserted by reading the written file back off disk. Hand-tested in the desktop app on both gemma-4 sizes: picked Auto, agent logged `model_id=mesh`, replied in channel. ## Not covered A committee that forms and then loses a worker returns 502, and that needs two workers to reproduce — not testable on one machine. --------- Signed-off-by: Michael Neale <michael.neale@gmail.com> Co-authored-by: Michael Neale <michael.neale@gmail.com>
This commit is contained in:
@@ -1,48 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
BACKEND="${1:-}"
|
||||
if [[ -z "$BACKEND" ]]; then
|
||||
case "$(uname -s)" in
|
||||
Darwin) BACKEND="metal" ;;
|
||||
*) BACKEND="cpu" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
CACHE_DIR="${MESH_LLM_NATIVE_RUNTIME_CACHE_DIR:-$ROOT/.cache/mesh-llm-native-runtime}"
|
||||
OUT_DIR="${MESH_LLM_NATIVE_RUNTIME_OUT_DIR:-$ROOT/.cache/mesh-llm-native-runtime-artifacts}"
|
||||
|
||||
metadata="$($ROOT/bin/cargo metadata --manifest-path "$ROOT/desktop/src-tauri/Cargo.toml" --features mesh-llm --format-version 1)"
|
||||
SDK_MANIFEST="$(python3 -c 'import json,sys; data=json.load(sys.stdin); print(next(p["manifest_path"] for p in data["packages"] if p["name"]=="mesh-llm-sdk"))' <<<"$metadata")"
|
||||
MESH_ROOT="$(cd "$(dirname "$SDK_MANIFEST")/../.." && pwd)"
|
||||
MESH_VERSION="$(python3 -c 'import json,sys; data=json.load(sys.stdin); print(next(p["version"] for p in data["packages"] if p["name"]=="mesh-llm-sdk"))' <<<"$metadata")"
|
||||
|
||||
case "$(uname -s)/$(uname -m)/$BACKEND" in
|
||||
Darwin/arm64/metal) RUNTIME_ID="meshllm-native-runtime-darwin-aarch64-metal" ;;
|
||||
Darwin/x86_64/metal) RUNTIME_ID="meshllm-native-runtime-darwin-x86_64-metal" ;;
|
||||
*/*/cpu)
|
||||
case "$(uname -s)/$(uname -m)" in
|
||||
Darwin/arm64) RUNTIME_ID="meshllm-native-runtime-darwin-aarch64-cpu" ;;
|
||||
Darwin/x86_64) RUNTIME_ID="meshllm-native-runtime-darwin-x86_64-cpu" ;;
|
||||
Linux/x86_64) RUNTIME_ID="meshllm-native-runtime-linux-x86_64-cpu" ;;
|
||||
Linux/aarch64) RUNTIME_ID="meshllm-native-runtime-linux-aarch64-cpu" ;;
|
||||
*) RUNTIME_ID="" ;;
|
||||
esac
|
||||
;;
|
||||
*) RUNTIME_ID="" ;;
|
||||
esac
|
||||
|
||||
if [[ -n "$RUNTIME_ID" && -f "$CACHE_DIR/$MESH_VERSION/$RUNTIME_ID/manifest.json" ]]; then
|
||||
printf '%s\n' "$CACHE_DIR"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "Preparing MeshLLM native runtime ($BACKEND) for MeshLLM $MESH_VERSION..." >&2
|
||||
runtime_dir="$(cd "$MESH_ROOT" && scripts/ci-prepare-native-runtime.sh "$OUT_DIR" "$BACKEND")"
|
||||
version="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["runtime"].get("mesh_version") or "unknown")' "$runtime_dir/manifest.json")"
|
||||
id="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1]))["runtime"]["id"])' "$runtime_dir/manifest.json")"
|
||||
mkdir -p "$CACHE_DIR/$version"
|
||||
rm -rf "$CACHE_DIR/$version/$id"
|
||||
cp -a "$runtime_dir" "$CACHE_DIR/$version/$id"
|
||||
printf '%s\n' "$CACHE_DIR"
|
||||
Reference in New Issue
Block a user