chore: mesh upgrade, clean up legacy special case code, simplify model selection for mesh (#5289)

Shared compute now has exactly two model choices: MeshLLM's virtual
`mesh`
model, or a model you name. Buzz picks between them in one place, and
buzz-agent no longer knows meshes exist.

## What changed

- **MeshLLM v0.74.0 → v0.75.1.** v0.75.0 added
`degrade_to_single_model`, so a
  `model=mesh` request is answered by one served model when there is no
committee to form, instead of failing. v0.75.1 adds Mesh-LLM#1196, which
skips stale pre-0.75 runtime cache entries rather than aborting startup
on
them — without it, anyone who had run mesh on 0.73/0.74 could not start.
- **Deleted the client-side mesh catalog probe.** buzz-agent used to
poll
`/v1/models` (5s TTL, 30s cooldown, two-observation debounce) to decide
whether `mesh` was safe to send. MeshLLM now decides per request, so the
  polling, its hysteresis, and its 503 fallback are gone.
- **One mapping point.** `relay_mesh_wire_model()` turns the stored
value into
a wire name: `auto` becomes `mesh`, a named model passes through. The
spawn
env, the ACP harness, and the readiness probe all use it, so they cannot
disagree — previously `BUZZ_ACP_MODEL` and the probe both said `auto`, a
name
  the mesh does not advertise.
- **Removed the `nostr-relay-pool` advisory exception.** #5404 allowed
RUSTSEC-2026-0243 "after mesh-llm migrates to nostr-sdk >= 0.45".
v0.75.1
does, so the retired crate is gone from both lockfiles and the exception
  would only mask a future advisory for it.
- **Deleted `scripts/ensure-mesh-native-runtime.sh`** and its six
justfile call
sites. It built llama.cpp from source into the runtime cache; the app
already
  downloads the signed release runtime itself, and CI never called it.

## Why it is better

**−639 lines of Rust.** Availability is decided by the node that knows
the
answer, per request, instead of by a client cache that could be stale
for up to
30 seconds. A second worker joining now takes effect on the next request
rather
than after two confirming probes.

## Behaviour change

A 503 on an explicit `mesh` request takes the ordinary transport retry
under
the same model instead of failing over to a second one — there is no
second
model to fail over to now. MoA repairs partial committee results
internally
before it reaches that point.

## Validation

`crates/buzz-relay/examples/mesh_agent_e2e.rs` now sends `mesh` where it
previously sent `auto` or the physical model id, so no leg was covering
what
Buzz actually puts on the wire. 4/4 on gemma-4-E4B, gemma-4-26B-A4B, and
Qwen3-8B — including a real ACP tool call through `mesh` into
buzz-dev-mcp,
asserted by reading the written file back off disk.

Hand-tested in the desktop app on both gemma-4 sizes: picked Auto, agent
logged
`model_id=mesh`, replied in channel.

## Not covered

A committee that forms and then loses a worker returns 502, and that
needs two
workers to reproduce — not testable on one machine.

---------

Signed-off-by: Michael Neale <michael.neale@gmail.com>
Co-authored-by: Michael Neale <michael.neale@gmail.com>
This commit is contained in:
Mic Neale
2026-08-12 01:40:07 +10:00
committed by GitHub
parent 83ca595ada
commit 240cdd3ea1
19 changed files with 1195 additions and 1505 deletions

View File

@@ -833,12 +833,6 @@ pub struct Config {
pub anthropic_api_version: String,
/// OpenAI endpoint selection. See [`OpenAiApi`].
pub openai_api: OpenAiApi,
/// Prefer mesh-llm's virtual `mesh` model when the configured/effective
/// OpenAI model is `auto` and the live model catalog advertises it.
/// Set by Buzz's relay-mesh provider via
/// `BUZZ_AGENT_PREFER_MESH_FOR_AUTO=1`; other providers keep their
/// existing `auto` semantics.
pub prefer_mesh_for_auto: bool,
pub hints_enabled: bool,
/// Thinking/reasoning effort level. `None` = use provider default (no
/// thinking config sent). Set via `BUZZ_AGENT_THINKING_EFFORT`.
@@ -936,7 +930,6 @@ impl Config {
base_url,
anthropic_api_version: env_or("ANTHROPIC_API_VERSION", "2023-06-01"),
openai_api,
prefer_mesh_for_auto: parse_env("BUZZ_AGENT_PREFER_MESH_FOR_AUTO", 0u8)? != 0,
max_rounds: parse_env("BUZZ_AGENT_MAX_ROUNDS", 0)?,
max_output_tokens: parse_env("BUZZ_AGENT_MAX_OUTPUT_TOKENS", 32_768)?,
llm_timeout: Duration::from_secs(parse_env("BUZZ_AGENT_LLM_TIMEOUT_SECS", 240)?),
@@ -988,7 +981,6 @@ impl Config {
system_prompt: String::new(),
anthropic_api_version: "2023-06-01".into(),
openai_api: OpenAiApi::Chat,
prefer_mesh_for_auto: false,
max_rounds: 0,
max_output_tokens: 1,
llm_timeout: Duration::from_secs(30),

File diff suppressed because it is too large Load Diff

View File

@@ -84,8 +84,8 @@ async-compression = { version = "0.4.42", features = ["tokio", "gzip"] }
dev = ["buzz-auth/dev"]
[dev-dependencies]
mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"] }
mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"] }
mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.75.1", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"] }
mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.75.1", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"] }
# Relay-driven mesh lifecycle smoke (examples/mesh_relay_lifecycle_smoke.rs):
# the relay client for discovery notes and the exact ed25519 the mesh owner
# keys use for binding verification.

View File

@@ -54,16 +54,11 @@ fn env(name: &str) -> anyhow::Result<String> {
std::env::var(name).map_err(|_| anyhow::anyhow!("{name} is required for this role"))
}
/// Installs the signed release runtime when none is cached — the same path the
/// desktop takes. Deliberately no "is it installed?" precondition: this runs in
/// three separate processes, and a guard here would refuse before reaching the
/// call that does the installing.
async fn init_native_runtime() -> anyhow::Result<()> {
let cache = mesh_llm_sdk::native_runtime::native_runtime_cache(None)?;
let current = mesh_llm_sdk::native_runtime::CURRENT_MESH_VERSION;
if !cache
.installed()?
.iter()
.any(|runtime| runtime.mesh_version == current)
{
anyhow::bail!("MeshLLM native runtime for MeshLLM {current} is not installed; run `just mesh-e2e-hardware` once to prepare it");
}
mesh_llm_host_runtime::initialize_host_runtime()
.await
.map_err(|error| anyhow::anyhow!("MeshLLM host runtime init failed: {error}"))

View File

@@ -112,10 +112,12 @@ async fn run() -> anyhow::Result<()> {
Err(e) => record("P1 explicit-model chat", false, e.to_string()),
}
// P2: auto — router picks the model.
// P2: the virtual `mesh` model — exactly what apply_relay_mesh_env now puts
// on the wire for shared compute. With one served model there is no
// committee, so this proves MeshLLM's degrade_to_single_model path.
let r = agent_chat(
&base,
"auto",
"mesh",
None,
"Reply with exactly one word: PONG",
&[],
@@ -123,11 +125,15 @@ async fn run() -> anyhow::Result<()> {
.await;
match r {
Ok(text) => record(
"P2 auto-model chat",
"P2 virtual-mesh chat (degrades to single model)",
text.to_uppercase().contains("PONG"),
text,
),
Err(e) => record("P2 auto-model chat", false, e.to_string()),
Err(e) => record(
"P2 virtual-mesh chat (degrades to single model)",
false,
e.to_string(),
),
}
// P3: regression — an output budget no served model's context can hold
@@ -139,7 +145,7 @@ async fn run() -> anyhow::Result<()> {
// GLM-4.7-Flash.
let r = agent_chat(
&base,
&served_id,
"mesh",
Some("150000"),
"Reply with exactly one word: PONG",
&[],
@@ -169,7 +175,7 @@ async fn run() -> anyhow::Result<()> {
);
let mcp = vec![("dev".to_string(), repo_bin("buzz-dev-mcp")?)];
let (r, marker) =
agent_chat_with_marker(&base, &served_id, None, &prompt, &mcp, &marker_name).await;
agent_chat_with_marker(&base, "mesh", None, &prompt, &mcp, &marker_name).await;
let file_ok = std::fs::read_to_string(&marker)
.map(|c| c.contains("BUZZ_OK"))
.unwrap_or(false);
@@ -270,7 +276,9 @@ async fn agent_chat_in_isolated_home(
.env_clear()
.env("PATH", std::env::var("PATH").unwrap_or_default())
.env("HOME", &home)
// Exactly the environment apply_relay_mesh_env() supplies.
// The transport subset of apply_relay_mesh_env(): provider, base URL,
// model, key, and chat API. Not BUZZ_AGENT_REQUIRE_REPLY, which needs
// Buzz's publish tools to mean anything.
.env("BUZZ_AGENT_PROVIDER", "openai")
.env("BUZZ_AGENT_MODEL", model)
.env("OPENAI_COMPAT_BASE_URL", base)

View File

@@ -41,15 +41,11 @@ const CLIENT_CONSOLE_PORT: u16 = 13132;
async fn main() -> anyhow::Result<()> {
let model = std::env::var("MESH_SMOKE_MODEL").unwrap_or_else(|_| DEFAULT_MODEL.to_string());
eprintln!("[smoke] model: {model}");
let cache = mesh_llm_sdk::native_runtime::native_runtime_cache(None)?;
let current = mesh_llm_sdk::native_runtime::CURRENT_MESH_VERSION;
if !cache
.installed()?
.iter()
.any(|runtime| runtime.mesh_version == current)
{
anyhow::bail!("MeshLLM native runtime for MeshLLM {current} is not installed; run `just mesh-e2e-hardware` to prepare it");
}
// No cache precondition: `initialize_host_runtime` installs the signed
// release runtime itself when none is present, which is how the desktop
// gets one too. The guard that used to stand here refused before reaching
// this line and told the reader to run the very recipe that runs this
// example.
mesh_llm_host_runtime::initialize_host_runtime()
.await
.map_err(|error| anyhow::anyhow!("MeshLLM host runtime init failed: {error}"))?;

View File

@@ -11,7 +11,9 @@
//! # Running (manual / runbook)
//!
//! ```text
//! # 1. prepare the matching native runtime with `scripts/ensure-mesh-native-runtime.sh`
//! # 1. the mesh-enabled desktop installs the signed native runtime itself on
//! # first init; stale pre-0.75 cache entries are skipped rather than
//! # fatal (MeshLLM >= 0.75.1), so no manual cleanup is needed
//! # 2. start the normal membership-gated relay and a mesh-enabled desktop
//! # 3. have that desktop publish status, then run the trust assertions:
//! RELAY_URL=ws://localhost:3000 \