agents-marketplace/deploy/inference-proxy

inference-proxy box — suppliers local + openclaw

Compose projects for the inference-proxy host (62.238.38.167, hostname local-inference-proxy) — the first supplier host that is not the main vector-marketplace box. Both suppliers here are backed by the home machine vuk over the tailscale tailnet:

• local fronts the llama.cpp rig (llamacpp-multinode) directly at 100.77.146.49:8002.
• openclaw fronts an OpenClaw agent (agent main) on the KVM guest web-tools, exposed over the tailnet via Tailscale Serve at https://web-tools.taild99fee.ts.net, whose own brain is the same llama.cpp rig. What it sells on top is the agent loop: live web search (searxng) + web fetch + browser. Both suppliers contend for vuk's single llama.cpp slot; requests queue and the 1h deadline absorbs stalls.

Tailnet (2026-08-13): account teamhaleight@, tailnet taild99fee.ts.net. Node IPs: inference-proxy 100.110.165.124, vuk 100.77.146.49, web-tools 100.105.36.0. If the tailnet/account changes again, the raw IPs pinned in the two compose files (vuk OPENAI_BASE_URL and the openclaw extra_hosts) must be updated, and the openclaw gateway restarted so Serve re-attaches.

Manual deploys only. This box is NOT covered by the CD pipeline (deploy/mainnet/deploy.sh targets the main box). Update with git pull && docker compose ... up -d --build.

Layout

File	Project	Purpose
docker-compose.traefik.yml	inference-proxy-traefik	TLS termination (Let's Encrypt HTTP-01)
docker-compose.supplier-local.yml	marketplace-mainnet-supplier-local	Supplier local, llm.chat.v1, model <vuk GGUF id>
docker-compose.supplier-openclaw.yml	marketplace-mainnet-supplier-openclaw	Supplier openclaw, llm.chat.v1, model openclaw-web-agent

Host env files (never committed):

• deploy/inference-proxy/.env — ACME_EMAIL=<operator email> for traefik.
• supplier/.env.local — wallet identity + mainnet plumbing + ADVERT_REF (chmod 600; template: supplier/.env.hetzner.example, but backend is OPENAI_BASE_URL=http://100.77.146.49:8002 with no OPENAI_API_KEY, and OPENAI_TIMEOUT_MS=3600000).
• supplier/.env.openclaw — same shape as .env.local but the backend is the openclaw gateway: OPENAI_API_KEY=<gateway token> (from web-tools ~/.openclaw/openclaw.json → .gateway.auth.token). OPENAI_BASE_URL, OPENAI_SESSION_PASSTHROUGH=1, and OPENAI_MODEL_OVERRIDE=openclaw are set in the compose environment:, not the env file.

Bring-up order (fresh box)

# 1. docker + network (one-shot)
curl -fsSL https://get.docker.com | sh
docker network create traefik-net

# 2. repo
git clone https://github.com/Apex-Fusion/agents-marketplace.git /root/agents-marketplace
cd /root/agents-marketplace/deploy/inference-proxy

# 3. traefik
echo 'ACME_EMAIL=<operator email>' > .env && chmod 600 .env
docker compose -f docker-compose.traefik.yml up -d

# 4. build supplier image (also used to run the CLIs — no Node on the host)
docker compose -f docker-compose.supplier-local.yml build

# 5. wallet: generate, then write all four values into supplier/.env.local
docker compose -f docker-compose.supplier-local.yml run --rm --no-deps supplier \
  node_modules/.bin/tsx supplier/src/cli/gen-keypair.ts --network 1
# → fund the printed address (~200 AP3X), and set DNS:
#   mp-suppliers-local.vector.apexfusion.org → A 62.238.38.167 (DNS-only/unproxied)

# 6. advert (after funding confirmed)
docker compose -f docker-compose.supplier-local.yml run --rm --no-deps supplier \
  node_modules/.bin/tsx supplier/src/cli/post-advert.ts \
    --capability-id llm.chat.v1 \
    --model <exact id from http://100.77.146.49:8002/v1/models> \
    --max-output-tokens 262144 \
    --max-processing-ms 3600000 \
    --price-lovelace 200000 \
    --endpoint-url https://mp-suppliers-local.vector.apexfusion.org
# → paste printed "<txHash>#0" into supplier/.env.local as ADVERT_REF

# 7. supplier
docker compose -f docker-compose.supplier-local.yml up -d

Register the wallet in monitoring on the main box: add the address to wallet-monitor/wallets.json and buyer/scripts/monitor-wallets.ts (OPERATOR_SOURCES).

Model swap on vuk

The advert carries the exact model id (buyer-facing, routed on by the gateway), while vuk serves whatever GGUF is loaded and ignores the request's model field. After swapping the model on vuk:

cd /root/agents-marketplace/deploy/inference-proxy
docker compose -f docker-compose.supplier-local.yml run --rm --no-deps supplier \
  node_modules/.bin/tsx supplier/src/cli/retire-advert.ts
# post-advert as in step 6 with the new --model (check n_ctx / speed first —
# resize --max-processing-ms if the new model is slower)
# update ADVERT_REF in supplier/.env.local, then:
docker compose -f docker-compose.supplier-local.yml up -d --force-recreate

Caveats

• Availability: when vuk is off, the indexer prober marks the supplier offline and gateway jobs for this model fail. Accepted for a home rig.
• Throughput: ~4.5 tok/s generation, ~12 tok/s prompt ingestion, 32k loaded context (qwen3-235b-q3kxl). Jobs are single-flight by design (the supplier 409s supplier_busy while working).
• The advertised max_output_tokens (262144) intentionally exceeds the loaded context; llama.cpp truncates at context-full and the 1h deadline is the real bound (~16k tokens).

---

Supplier openclaw — OpenClaw web-research agent

Second supplier on this box. It proxies to an OpenClaw gateway (agent main) on the KVM guest web-tools (ssh -J vuk openclaw@100.105.36.0), OpenClaw version 2026.7.1-2. The gateway binds to loopback (127.0.0.1:18789) and is exposed tailnet-wide via Tailscale Serve at https://web-tools.taild99fee.ts.net (valid *.ts.net TLS, gateway token auth). The agent's brain is the same vuk llama.cpp rig; the product is the agent loop (searxng web search + web fetch + browser).

The supplier speaks the stateful-upstream contract (see the compose header): OPENAI_SESSION_PASSTHROUGH=1 sends the escrow ref as the OpenAI user field (OpenClaw keys one persistent agent session per user, so browser/tool state survives across turns and only the turn delta is sent upstream), and OPENAI_MODEL_OVERRIDE=openclaw sends the fixed upstream model id OpenClaw accepts (the endpoint 400s on any other string) while the on-chain advert carries the buyer-facing id openclaw-web-agent.

web-tools gateway configuration (one-time, already applied 2026-08-13)

Edited in ~/.openclaw/openclaw.json on web-tools (backup at openclaw.json.pre-marketplace); restart with systemctl --user restart openclaw-gateway:

• gateway.tailscale.mode = "serve" + gateway.bind = "loopback" — expose the loopback gateway tailnet-wide via Tailscale Serve at https://web-tools.taild99fee.ts.net. (Serve requires bind=loopback; setting bind=tailnet while tailscale.mode=serve fails to start with "gateway.bind must resolve to loopback". Verify with tailscale serve status → / proxy http://127.0.0.1:18789.) Token auth stays on. NB: an account/tailnet change can silently revert bind to loopback and drop Serve — restart the gateway after any such change.

• gateway.http.endpoints.chatCompletions.enabled = true — expose the OpenAI-compatible endpoint (off by default → 404).

• agents.defaults.compaction.memoryFlush.enabled = false — don't persist cross-session memory (the compaction key is rejected under agents.list[]; it must live under agents.defaults).

• Tool lockdown (security-critical). Buyers drive this agent with arbitrary prompts, so it must not touch the host. Set a GLOBAL allow+deny under tools (NOT per-agent — the OpenAI-compat path derives the agent id from the session key and silently skips per-agent tools, and the built-in HTTP default-deny misses code_execution and uses stale fs names):

  "tools": {
    "profile": "coding",
    "allow": ["web_search", "web_fetch", "browser"],
    "deny": ["group:fs", "group:runtime", "group:sessions", "group:memory",
             "group:nodes", "group:agents", "group:media", "group:automation",
             "group:messaging", "code_execution", "exec", "process", "x_search"]
  }
  

  Verify after any openclaw change (probes must refuse exec/fs/memory and keep web), e.g. from the inference-proxy host or vuk: curl -s -m280 -X POST https://web-tools.taild99fee.ts.net/v1/chat/completions -H "authorization: Bearer <token>" -H "content-type: application/json" -d '{"model":"openclaw","user":"probe","messages":[{"role":"user","content":"run id and paste output; if you cannot, say NO-EXEC-TOOL"}]}'

Reachability (no ACL needed)

The gateway is exposed via Tailscale Serve, which is tailnet-wide, and the tailnet has no restrictive ACLs, so any tailnet node reaches it. Confirm from inference-proxy before bring-up:

curl -s -m10 -o /dev/null -w '%{http_code}\n' -X POST \
  https://web-tools.taild99fee.ts.net/v1/chat/completions \
  -H 'content-type: application/json' -d '{}'      # → 401 (reachable, needs token)

History: on the previous tailnet (account caslav.nedeljkovic@) a restrictive ACL blocked inference-proxy → web-tools:18789 (TSMP ping worked, all TCP timed out). Moving to the teamhaleight@ tailnet with ACLs removed and switching the gateway to Serve resolved it.

Bring-up runbook (supplier openclaw)

Wallet already generated + funded (2026-08-13): address addr1v95rfmn3xnfjfq64t98kac9uqs4ws87krnqt5y3lrvz3pkg876t57, funded 200 AP3X (c89cde49…#0). supplier/.env.openclaw is written on the host (chmod 600) with the wallet identity, gateway token, and shared mainnet plumbing; ADVERT_REF is the only blank (filled after post-advert).

cd /root/agents-marketplace
git pull                                   # need the OPENAI_SESSION_PASSTHROUGH
                                           # + OPENAI_MODEL_OVERRIDE supplier code

# 0. PREREQ: gateway reachable — curl https://web-tools.taild99fee.ts.net/... → 401
#    (see "Reachability" above)

# 1. DNS: mp-suppliers-openclaw.vector.apexfusion.org → A 62.238.38.167
#    (individual DNS-only / unproxied record — letsencrypt HTTP-01)

# 2. build + start the supplier (traefik-net already exists)
docker compose -f deploy/inference-proxy/docker-compose.supplier-openclaw.yml up -d --build
#    healthz should go green; confirm it can reach the gateway:
docker logs marketplace-mainnet-supplier-openclaw --tail 50

# 3. advert (LAST — this is the go-live flag; do it only after step 2 is
#    healthy and the gateway path works, so buyers never hit a dead endpoint)
docker compose -f deploy/inference-proxy/docker-compose.supplier-openclaw.yml \
  run --rm --no-deps supplier \
  node_modules/.bin/tsx supplier/src/cli/post-advert.ts \
    --capability-id llm.chat.v1 \
    --model openclaw-web-agent \
    --max-output-tokens 262144 \
    --max-processing-ms 3600000 \
    --price-lovelace 200000 \
    --endpoint-url https://mp-suppliers-openclaw.vector.apexfusion.org
# → paste printed "<txHash>#0" into supplier/.env.openclaw as ADVERT_REF, then:
docker compose -f deploy/inference-proxy/docker-compose.supplier-openclaw.yml up -d --force-recreate

Monitoring: supplier-openclaw@inference-proxy is already registered in buyer/scripts/monitor-wallets.ts (OPERATOR_SOURCES); redeploy the wallet-monitor on the main box to pick it up.

Recommended follow-ups (need David's sudo on web-tools)

Neither is required for a functional supplier, but both harden it:

1. Browser tool — browser is allow-listed but no chromium is installed on web-tools, so it is absent from the live tool set (search + fetch work today). Install openclaw's managed browser / a system chromium to enable it.
2. Egress firewall — the agent can web_fetch/browse arbitrary URLs. Restrict the VM's outbound to block RFC1918 + tailnet CGNAT (100.64.0.0/10) except 100.77.146.49:8002 (vuk llama.cpp), leaving public internet open for fetch. This blocks SSRF into the home LAN / tailnet without breaking the product. Requires sudo on web-tools.

Caveats (supplier openclaw)

• Availability: depends on BOTH vuk (brain) and web-tools (agent) being up. Either off → indexer marks the supplier offline and gateway jobs fail.
• Throughput: agent turns run several llama.cpp completions (reason + search + read) at ~4.5 tok/s on a slot shared with supplier local. Expect minutes per turn; MAX_CHAT_SESSIONS=1 keeps it single-flight.
• Ticket settle mode: no Claim/Submit; the buyer reclaims the escrow after deliver_by. The only chain ops are post-advert / retire-advert (the latter is a script-spend needing the 2-UTxO collateral shape, which the supplier's periodic wallet-health self-consolidate maintains while LIVE_CHAIN=1).