agents-marketplace/docs/HETZNER_SETUP.md

Hetzner Inference Supplier Fleet — Setup Runbook

Status: shipped; fleet trimmed 2026-08-21. Hetzner retired Kimi-K2.7-Code, GLM-5.2-NVFP4, and DeepSeek-V4-Flash-0731 from Inference (gone from /v1/models), so the 6 suppliers serving them were deregistered (adverts retired on-chain, containers removed). Qwen3.8-27B was added in their place, reusing the retired kimi-code wallets. Configures mainnet supplier nodes that use Hetzner Inference (https://inference.hetzner.com) as their compute backend for bonded inference work commissioned on Vector. Config only — no code changes.

---

1. What this is

Hetzner Inference is an OpenAI-compatible hosted API. Because the supplier's upstream client (supplier/src/openai.ts) already speaks plain POST /v1/chat/completions with a Bearer token, pointing it at Hetzner is pure configuration: LLM_BACKEND=openai + OPENAI_BASE_URL + the API key. This is the same pattern as the DeepSeek-direct and OpenRouter suppliers, and the direct sibling of docs/HUGGINGFACE_ROUTER_SETUP.md.

One supplier per (model × capability) — both the one-off llm.text.generate.v1 and the multi-turn llm.chat.v1 capability:

Supplier name	Model (on-chain + API param, verbatim)	Capability	Advert max_output_tokens
qwen35b	Qwen/Qwen3.6-35B-A3B-FP8	llm.text.generate.v1	262144
qwen35b-chat	Qwen/Qwen3.6-35B-A3B-FP8	llm.chat.v1	262144
qwen38	Qwen3.8-27B	llm.text.generate.v1	262144
qwen38-chat	Qwen3.8-27B	llm.chat.v1	262144

Deregistered 2026-08-21 (models dropped by Hetzner): kimi-code, kimi-code-chat (Kimi-K2.7-Code), glm, glm-chat (GLM-5.2-NVFP4), ds-flash-htz, ds-flash-htz-chat (DeepSeek-V4-Flash-0731). The qwen38 pair reuses the kimi-code / kimi-code-chat wallets; the other four wallets still hold funds and their env files remain on the host.

Shared advert parameters: price 200000 lovelace (0.2 AP3X) flat per job, bonds 1000000 lovelace (1 AP3X) both sides, max_processing_ms 300000 for one-off / 1800000 for chat (the session spans the whole conversation). max_output_tokens is set to the model's context length: no max_tokens is forwarded upstream (OPENAI_MAX_TOKENS unset), so the advert value only gates what buyers may request.

Names follow the brand-by-model convention; the retired ds-flash-htz carried a -htz suffix only because deepseek-flash-* was already taken by the OpenRouter fleet serving deepseek/deepseek-v4-flash.

2. Prerequisites

1. A Hetzner account with Inference access and an API key (usage-based billing to you as the node operator — the flat 0.2 AP3X job price is a business choice, not cost-derived). One key is shared by all suppliers; Hetzner rate limits apply to the aggregate.
2. One funded supplier wallet per supplier (50 AP3X each recommended; bonds are 1 AP3X per in-flight job and returned on completion).
3. The usual mainnet supplier deploy prerequisites (deploy/README.md).
4. DNS: one A record per supplier — mp-suppliers-<name>.vector.apexfusion.org → 91.98.147.172, DNS-only/unproxied. There is NO wildcard for *.vector.apexfusion.org; and because traefik issues certs on-box via Let's Encrypt HTTP-01 (port 80), a proxied/CDN record would break issuance. Certs are issued automatically within ~a minute of the record propagating — no restarts needed.

3. Verify the backend before spending on-chain

The model ids come from GET https://inference.hetzner.com/api/v1/models (this list changes — Hetzner retires models; re-check before any advert) and are sent verbatim as the upstream model param (the runtime uses advert.model). Before posting adverts, smoke-test each model:

curl -sS -X POST https://inference.hetzner.com/api/v1/chat/completions \
  -H "Authorization: Bearer $HETZNER_KEY" -H "Content-Type: application/json" \
  -d '{"model":"Qwen3.8-27B","messages":[{"role":"user","content":"ping"}]}' \
  | jq '.choices[0].message.content'

choices[0].message.content must be a non-empty string with no max_tokens sent (matches runtime config) — the supplier throws openai_malformed otherwise and forfeits its 1 AP3X bond per failed job. For chat suppliers also verify a tools/tool_choice round-trip. The original four models passed both checks on 2026-08-11; Qwen3.8-27B passed both on 2026-08-21.

4. Per-supplier bring-up (repeat per supplier)

# 1. wallet — prints privateKeyHex, publicKeyHex, pubKeyHash, address
pnpm --filter @marketplace/supplier tx:gen-keypair --network 1
# fund the printed address (50 AP3X recommended), then:

# 2. env file on the mainnet host
cp supplier/.env.hetzner.example /root/agents-marketplace/supplier/.env.<name>
chmod 600 /root/agents-marketplace/supplier/.env.<name>
# fill in: ALL FOUR wallet vars (PRIV_KEY_HEX, ADDRESS, PKH, PUB_KEY_HEX —
# they are NOT derived at boot; missing derived vars ⇒ 403 wrong_supplier),
# OPENAI_API_KEY, and the ADVERT_* values from the table above.

# 3. advert (one-off shown; chat: --capability-id llm.chat.v1 --max-processing-ms 1800000)
pnpm --filter @marketplace/supplier tx:post-advert \
  --capability-id llm.text.generate.v1 \
  --model 'GLM-5.2-NVFP4' \
  --max-output-tokens 512000 \
  --max-processing-ms 300000 \
  --price-lovelace 200000 \
  --endpoint-url https://mp-suppliers-<name>.vector.apexfusion.org
# paste the printed "<txHash>#0" into ADVERT_REF in the env file

# 4. first start is always manual (CD skips compose projects with no running
#    containers); subsequent rollouts ride the mainnet CD pipeline
docker compose -f deploy/mainnet/docker-compose.supplier-<name>.yml up -d

# 5. register the wallet for balance alerts
#    add {name, address} to wallet-monitor/wallets.json on the host

5. Verification

• curl https://mp-suppliers-<name>.vector.apexfusion.org/healthz → {"ok":true}; /capability shows the right model + pkh.
• Indexer: buyer GET /v1/indexer/suppliers lists the new entries with advert_status: Active, status: free.
• Gateway GET /openai/v1/models includes the four Hetzner model ids.
• Watch docker logs -f marketplace-mainnet-supplier-<name> through the first jobs for openai_malformed (reasoning-model content-shape risk).

6. Footguns

• OPENAI_BASE_URL=https://inference.hetzner.com/api — no /v1; the client appends /v1/chat/completions.
• Do not set OPENAI_REASONING — reasoning:{enabled:false} is an OpenRouter-only param.
• The advert --model must match the Hetzner id exactly, including the Qwen/ prefix on Qwen/Qwen3.6-35B-A3B-FP8 (slash is fine on-chain; precedent moonshotai/kimi-k2.6).
• Kimi-K2.7-Code and DeepSeek-V4-Flash-0731 auto-join the buyer's PDF pool (case-insensitive substring allowlist ["kimi","deepseek","gpt"] in buyer/src/pdf/caps.ts); GLM and Qwen stay out. Deliberate — adjust PDF_MODEL_ALLOWLIST/PDF_MODEL_DENYLIST in the buyer env to change it.
• OPENAI_TIMEOUT_MS must stay ≤ the advert's max_processing_ms.