mirror of
https://github.com/block/buzz.git
synced 2026-10-27 12:40:30 +00:00
mesh: upgrade runtime, enforce membership, add shared compute provider (#1656)
This commit is contained in:
Michael Neale
167
docs/buzz-shared-compute-dev.md
Normal file
167
docs/buzz-shared-compute-dev.md
Normal file
@@ -0,0 +1,167 @@
|
||||
# Buzz shared compute: local GUI verification
|
||||
|
||||
This runbook verifies the actual desktop path used by the built-in **Fizz** agent:
|
||||
|
||||
`Buzz Desktop → buzz-acp → buzz-agent → MeshLLM SDK → local/remote compute`
|
||||
|
||||
It does not use a substitute agent harness.
|
||||
|
||||
## Before starting
|
||||
|
||||
Run from the `block/buzz` repository root on the mesh-enabled branch.
|
||||
|
||||
For a completely fresh, deterministic local state, use:
|
||||
|
||||
```bash
|
||||
. ./bin/activate-hermit
|
||||
just mesh-dev-fresh
|
||||
```
|
||||
|
||||
This removes development app data, the development keyring entry,
|
||||
`~/.buzz-dev`, and local Docker volumes; it preserves the installed Buzz app's
|
||||
data, production keyring, and `~/.buzz`. The first dev page load also clears
|
||||
only that dev server origin's WebKit storage, so saved fields from an earlier
|
||||
run cannot leak into the fresh state. It then seeds local channels and starts
|
||||
the mesh-enabled desktop with the repository's public Tyler test identity.
|
||||
That identity is a fixture and must never be pointed at staging or production.
|
||||
|
||||
If using `mesh-dev-fresh`, the clean window opens at **Welcome to Buzz**. Join
|
||||
the seeded local community before continuing:
|
||||
|
||||
1. Click **Join a community**.
|
||||
2. Use any local name, such as **Local Buzz**.
|
||||
3. Set **Community URL** to `ws://localhost:3000` and join.
|
||||
4. Complete the short profile setup if it appears.
|
||||
|
||||
The recipe already supplied the repository's public test identity and seeded
|
||||
the local channels. Do not import or generate another key. Continue at **Share
|
||||
this machine** below.
|
||||
|
||||
Free the development ports if a previous run was interrupted:
|
||||
|
||||
```bash
|
||||
lsof -nP -iTCP:3000 -iTCP:8080 -iTCP:9102 -iTCP:9337 -iTCP:3131
|
||||
```
|
||||
|
||||
Stop only stale Buzz/MeshLLM processes shown by that command. Do not leave a
|
||||
standalone `mesh-llm` process using `9337` or `3131`; the desktop owns those
|
||||
ports during this test.
|
||||
|
||||
## 1. Launch the mesh-enabled desktop
|
||||
|
||||
```bash
|
||||
. ./bin/activate-hermit
|
||||
just mesh=1 dev
|
||||
```
|
||||
|
||||
Keep that terminal open. The first run may build/install the native runtime and
|
||||
take several minutes. Wait for the Buzz window to open and for the terminal to
|
||||
stop printing build progress.
|
||||
|
||||
Using plain `just dev` is not sufficient: the Compute UI and embedded MeshLLM
|
||||
runtime are behind the `mesh-llm` feature.
|
||||
|
||||
## 2. Share this machine
|
||||
|
||||
1. Open **Settings**.
|
||||
2. Select **Compute**.
|
||||
3. Under **Share compute**, choose a suggested model.
|
||||
- On a 16 GB Apple Silicon machine, use a suggested Qwen3.5 4B quantized
|
||||
model when available.
|
||||
- `unsloth/Qwen3.5-4B-GGUF:Q4_K_M` is the model used by the hardware proof.
|
||||
- Do not use a sub-1B model for the channel-reply proof. It can prove that
|
||||
inference is reachable while still failing the agent's long prompt and
|
||||
required message-send tool call.
|
||||
4. Turn on **Share this machine**.
|
||||
5. Wait until the card says it is sharing/running. Do not start Fizz while the
|
||||
card says downloading, preparing, or starting.
|
||||
|
||||
Buzz may download the model on first use. The model picker ranks models for the
|
||||
current hardware; avoid entering a model the card marks too large.
|
||||
|
||||
## 3. Make shared compute the agent default
|
||||
|
||||
1. Open **Agents** from the left sidebar.
|
||||
2. In **Agent defaults**, set **Default LLM provider** to
|
||||
**Buzz shared compute**.
|
||||
3. Set **Default model** to **Default (auto)**.
|
||||
4. Click **Save defaults** and wait for **Saved**.
|
||||
|
||||
Fizz has no pinned runtime/provider/model, so it inherits these defaults and
|
||||
resolves to the bundled `buzz-agent`. No API key is required.
|
||||
|
||||
## 4. Start the real Fizz path
|
||||
|
||||
1. Find the **Fizz** card on the Agents screen.
|
||||
2. If Fizz is stopped, click the small play badge over its avatar. If it is
|
||||
running, the badge is a green status dot instead of a stop control.
|
||||
3. Wait for its runtime indicator to become active.
|
||||
4. Add Fizz to a channel if it is not already a channel member.
|
||||
5. In that channel, send:
|
||||
|
||||
```text
|
||||
@Fizz Reply exactly: FIZZ_MESH_OK
|
||||
```
|
||||
|
||||
6. Confirm that Fizz replies `FIZZ_MESH_OK` in the channel.
|
||||
|
||||
That channel response is the end-to-end proof. A green Compute card alone proves
|
||||
only model serving; it does not prove the Fizz harness and provider inheritance.
|
||||
|
||||
To stop a running agent, click the body/name of its card to open its profile,
|
||||
then click **Stop** near the top. The green avatar badge is status-only while the
|
||||
agent is running. Once stopped, the profile action becomes **Respawn** and the
|
||||
avatar badge becomes a play button.
|
||||
|
||||
To create a separate test agent, choose **New agent → New agent**, use
|
||||
**buzz-agent** as the runtime, **Buzz shared compute** as the LLM provider,
|
||||
**Default (auto)** as the model, and **This computer** under **Run on**. Shared
|
||||
compute is an LLM provider; do not select a remote compute backend as the run
|
||||
location merely because its name mentions mesh.
|
||||
|
||||
## 5. Optional diagnostics
|
||||
|
||||
While Buzz is running:
|
||||
|
||||
```bash
|
||||
# The desktop should own both ports.
|
||||
lsof -nP -iTCP:9337 -iTCP:3131
|
||||
|
||||
# The embedded OpenAI-compatible ingress should advertise the model.
|
||||
curl -sS http://127.0.0.1:9337/v1/models | jq '.data[].id'
|
||||
|
||||
# Fizz should resolve through the real managed-agent subprocesses.
|
||||
ps -eo pid,ppid,command | grep -E '[b]uzz-(desktop|acp|agent)'
|
||||
```
|
||||
|
||||
If Fizz fails, open its runtime details from the Agents screen first. Common
|
||||
causes are:
|
||||
|
||||
- launched with `just dev` instead of `just mesh=1 dev`;
|
||||
- a stale process owns `9337`/`3131`;
|
||||
- the model is still downloading or preparing;
|
||||
- Fizz is not a member of the channel;
|
||||
- defaults were changed but not saved;
|
||||
- no current Buzz membership snapshot is available (admission fails closed).
|
||||
|
||||
## Security boundary
|
||||
|
||||
Buzz publishes member-signed discovery notes through an ordinary relay-supported
|
||||
NIP-51 event. The note includes a MeshLLM-key signature binding the member to the
|
||||
advertised MeshLLM node identity, plus a second signature over the exact endpoint
|
||||
tokens in the note. Current Buzz membership controls which node identities are
|
||||
admitted. A serving target is selectable only when its endpoint signature is
|
||||
valid, its invite token decodes as a bounded Iroh endpoint, and every advertised
|
||||
relay URL matches this machine's locally configured Iroh relay policy.
|
||||
|
||||
`BUZZ_MESH_IROH_RELAYS` defaults to Iroh's production relay set. Set it to `0`
|
||||
for direct QUIC only, or to a comma-separated HTTPS allowlist for custom relays.
|
||||
Plain HTTP is accepted only for loopback development relays. Remote status notes
|
||||
cannot expand this local allowlist.
|
||||
|
||||
MeshLLM—not the Buzz relay—carries inference over direct QUIC or its encrypted
|
||||
iroh relays and enforces the owner allowlist. The dependency is pinned to the
|
||||
post-v0.72.2 admission fix that prevents a non-member with a leaked invite token
|
||||
from using passive inference streams. MeshLLM v0.73.1 still performs its owner
|
||||
check during gossip after transport connection; authenticating before any gossip
|
||||
is an upstream protocol change and is not claimed by the Buzz-side checks above.
|
||||
@@ -1,124 +0,0 @@
|
||||
# Mesh LLM local build prerequisites
|
||||
|
||||
Buzz embeds mesh-llm through the Rust SDK pinned in Cargo. mesh-llm's native
|
||||
skippy/llama layer is linked into the relay and desktop binaries.
|
||||
|
||||
## Local Mac demo path
|
||||
|
||||
For the first local milestone, use mesh-llm's default native build path. On macOS
|
||||
this compiles patched llama.cpp/ggml with Metal support the first time a Buzz
|
||||
binary that depends on mesh is built. The result is cached under Cargo's git
|
||||
checkout of mesh-llm, so subsequent builds are much faster.
|
||||
|
||||
Prerequisites:
|
||||
|
||||
```bash
|
||||
xcode-select --install # if Command Line Tools are not installed yet
|
||||
brew install cmake # if cmake is not already available
|
||||
```
|
||||
|
||||
Then build normally:
|
||||
|
||||
```bash
|
||||
cargo build -p buzz-relay --bin buzz-relay
|
||||
cargo check --manifest-path desktop/src-tauri/Cargo.toml
|
||||
```
|
||||
|
||||
To run the desktop app with mesh enabled, use the opt-in recipes — plain
|
||||
`just dev` / `just staging` are mesh-off for fast iteration:
|
||||
|
||||
```bash
|
||||
just mesh=1 dev # local relay, mesh-llm on
|
||||
just mesh=1 staging # staging relay, mesh-llm on + native runtime prep
|
||||
```
|
||||
|
||||
Expect the first build to take several minutes while mesh-llm prepares and builds
|
||||
patched llama.cpp. This is intentional for the local demo: there is no external
|
||||
binary artifact to fetch and no separate dylib path to configure.
|
||||
|
||||
## CI / release path
|
||||
|
||||
CI should not rebuild llama.cpp from scratch on every job. CI and release now
|
||||
prebuild the llama native libraries in a dedicated step (`prepare-llama.sh` +
|
||||
`build-llama.sh -DCMAKE_OSX_DEPLOYMENT_TARGET=10.15`) and the Tauri build reuses
|
||||
them via `SKIPPY_LLAMA_AUTO_BUILD=0` + `LLAMA_STAGE_BUILD_DIR`. That build is
|
||||
cached with `actions/cache` keyed on the mesh-llm rev (resolved from
|
||||
`Cargo.lock`), so a cache hit skips the rebuild and a dependency bump
|
||||
invalidates the cache automatically — no workflow edit required.
|
||||
|
||||
A dynamic-link artifact path remains a possible future optimization. The
|
||||
mesh-llm build script supports dynamic linking with:
|
||||
|
||||
```bash
|
||||
export LLAMA_STAGE_LINK_MODE=dynamic
|
||||
export LLAMA_STAGE_LIB_DIR=/path/to/prebuilt/llama/libs
|
||||
```
|
||||
|
||||
Do not use dynamic-link locally unless you already have compatible `llama`,
|
||||
`llama-common`, and `mtmd` dynamic libraries. The default static build is the
|
||||
supported local path for M1.
|
||||
|
||||
## Connectivity model: public iroh relays off, raw STUN on (WAN)
|
||||
|
||||
Buzz Desktop starts the embedded mesh node with `disable_iroh_relays(true)`
|
||||
(mesh-llm fork rev `bc2f1106`, `RelayPolicy::ExplicitlyDisabled`): **no public
|
||||
iroh relays** (no `*.iroh.link` traffic, no relay transport), so there is no
|
||||
empty-relay fallback to public infrastructure.
|
||||
|
||||
Raw STUN **remains on** under this policy: the node discovers its public address
|
||||
(via `stun.l.google.com` / `stun.cloudflare.com`) and injects it into the invite
|
||||
token / `EndpointAddr`. That address rides the relay-signed `24621`/`24622`
|
||||
call-me-now exchange, and peers hole-punch directly over UDP — so this works over
|
||||
**WAN**, not just LAN. STUN is a "what's my public IP" lookup, not a relay or a
|
||||
data path; the Buzz relay performs the address-exchange coordination.
|
||||
|
||||
Residual limit (intentional v1): with iroh relays off there is **no relay
|
||||
transport fallback**, so two peers both behind **symmetric NATs** may fail to
|
||||
hole-punch. Works for the common cases (≥1 side cone-NAT / port-forwarded /
|
||||
server). A hosted private-iroh-relay fallback (relay advertises an
|
||||
`iroh_relay_url` + NIP-98 auth) is the planned follow-up once the relay side is
|
||||
wired end-to-end.
|
||||
|
||||
## Mesh-compute e2e acceptance matrix
|
||||
|
||||
"Full e2e" for mesh-compute is necessarily layered: the Playwright harness
|
||||
drives the **web build** of the desktop UI through a bridge, so it cannot
|
||||
execute real Tauri mesh commands or real GGUF inference. We therefore split
|
||||
coverage into three layers and are explicit about what is real vs mocked.
|
||||
|
||||
| # | What it proves | Where | Real / Mocked | Runs in CI? | How to run |
|
||||
|---|----------------|-------|---------------|-------------|------------|
|
||||
| 1 | serve node + client node + mesh routing + **real inference** | `crates/buzz-relay/examples/mesh_serve_client_smoke.rs` | **REAL** (loads a model, runs inference, joins a real mesh) | No — hardware-gated | `just mesh-e2e-hardware` (or `cargo run -p buzz-relay --example mesh_serve_client_smoke`) |
|
||||
| 2 | admission **invariant**: relay membership is the only factor | `crates/buzz-relay/src/handlers/mesh_signaling.rs` (`*_admitted` / `denied_is_not_admitted` tests) | REAL policy logic, no I/O | **Yes** | `cargo test -p buzz-relay mesh_signaling` |
|
||||
| 2b | live db-membership admission + member/non-member status reads | `crates/buzz-test-client/tests/e2e_mesh_llm.rs` (`trust_*`) | REAL relay over ws | No — env-gated (`MEMBER_NSEC`/`STRANGER_NSEC`, live relay) | see that file's module docs |
|
||||
| 3 | desktop UI contract: Share-compute start/stop, Run-on-relay-mesh preset, **ensure-before-spawn** order, membership-gated toggle | `desktop/tests/e2e/mesh-compute.spec.ts` | UI REAL, Tauri mesh commands MOCKED via the e2e bridge | **Yes** | `cd desktop && pnpm test:e2e:integration -- mesh-compute.spec.ts` |
|
||||
|
||||
`just mesh-e2e` runs the two CI-safe layers (2 + 3). Layer 1 is run on hardware.
|
||||
`just mesh-e2e-hardware` prepares a matching MeshLLM native runtime in a Buzz-controlled local cache before starting the real serve/client smoke, so it does not depend on a manually preseeded `MESH_LLM_NATIVE_RUNTIME_CACHE_DIR`.
|
||||
|
||||
### What "real" means per layer
|
||||
|
||||
- **Layer 1 is the only layer that proves inference.** It starts a real serve
|
||||
node on the GPU, a real client node that joins via the serve node's invite
|
||||
token, and asserts a chat completion *routed through the client* returns
|
||||
`finish_reason=stop` with non-empty content. Verified locally with
|
||||
SmolLM2-135M; point `MESH_SMOKE_MODEL` at a larger `.gguf` for scale.
|
||||
The just target first runs `scripts/ensure-mesh-native-runtime.sh`, which
|
||||
builds/packages/installs the MeshLLM native runtime matching the pinned SDK
|
||||
into `.cache/mesh-llm-native-runtime` and exports `MESH_LLM_NATIVE_RUNTIME_CACHE_DIR`
|
||||
for the smoke. Note: with `disable_iroh_relays(true)` the join bootstraps via
|
||||
STUN-discovered public addresses + relay-signed call-me-now (no public iroh
|
||||
relay) — see the connectivity model above.
|
||||
- **Layer 2 proves the auth invariant without faking it.** The policy mapping
|
||||
(`MembershipDecision` → admit/deny) is exercised directly: member → allow,
|
||||
open relay → allow, non-member → deny, owner-delegation → deny (v1),
|
||||
error → deny. A valid NIP-98 identity or possession of dial metadata is, by
|
||||
itself, never sufficient. The membership-only gate is what matters **when
|
||||
membership enforcement is enabled** (`require_relay_membership = true`); an
|
||||
open relay (`OpenRelay`) intentionally admits any valid NIP-98 signer, so
|
||||
`open relay → allow` is the deliberate disabled-enforcement case, not a
|
||||
contradiction of the membership gate.
|
||||
- **Layer 3 proves the UI contract, not inference.** The mesh Tauri commands
|
||||
are mocked, but the assertions are on real UI behavior and real command
|
||||
*ordering* (`mesh_ensure_client_node` is recorded before `create_managed_agent`),
|
||||
and on the membership gate (a non-member cannot enable relay-mesh at all).
|
||||
@@ -1043,13 +1043,9 @@ registers a subscription only after:
|
||||
`Err(_)` emit `CLOSED "restricted: …"` (`:127-132`).
|
||||
5. Global path (`channel_id = None`): per-filter p/engram/author gates must
|
||||
hold against *p* (`:144-167`); otherwise `CLOSED`.
|
||||
6. Only then is `sub_registry.register(conn_id, sub_id, filters, channel_id)`
|
||||
called (`:202-204`). `rg -U -n "sub_registry\s*\n?\s*\.register\(" crates/buzz-relay/src`
|
||||
(`-U` is required — the prod call splits `.sub_registry` and `.register(`
|
||||
across `:203-204`, so the single-line pattern would miss it) returns
|
||||
exactly three sites: `req.rs:204` is the **sole** production caller; the
|
||||
two others (`mesh_signaling.rs:550`, `event.rs:1058`) are inside
|
||||
`#[cfg(test)]` modules.
|
||||
6. Only then is `sub_registry.register_scoped(...)` called. Direct `register`
|
||||
calls are confined to test setup; production subscription registration goes
|
||||
through the community-scoped API in `req.rs`.
|
||||
|
||||
#### G2 — delivery (`crates/buzz-relay/src/handlers/event.rs:59-113`)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user