e8261d9500
Hetzner Inference dropped Kimi-K2.7-Code, GLM-5.2-NVFP4 and DeepSeek-V4-Flash-0731 from /v1/models; the 6 suppliers serving them are deregistered (adverts retired on-chain 2026-08-21, containers removed): kimi-code 3f37082166faf2b2220e6476235bae8723af9a3434a6aaf60277366ff1f62bfb kimi-code-chat 3d1b43645c00bdb7b429f29ced798bc4af206c9dc56074e955d84b603b64f50c glm bf3ee48addb32e2b0726066f1806b9d84ad64b86c7d41d379abd4b439c522696 glm-chat b910327023edb6a3f31696b7feaa9448a151c671c2ff303c4c3dd5a07d03c6b5 ds-flash-htz 61dd30deb1cd3d6c3f73792bda46ecea6890735237d8329ade2b8449d298f24e ds-flash-htz-chat 348a278c81a2bf664ea65705a6245df31e81f64c0ffa6e04a59b248fcdae3d9f New qwen38 + qwen38-chat suppliers serve Hetzner's new Qwen3.8-27B (ctx 262144; ping + tools round-trip verified 2026-08-21), reusing the retired kimi-code / kimi-code-chat wallets. Adverts: qwen38 bb8a7078098dc5e60b3f97b040e38f05e6cca662497d4bea471e1dff19c1e1ae#0 qwen38-chat 31e3a6e258ceaa0d1eaabe61fa3fbe3abcbbb400a7b885e05c8dbcdcf4924eda#0
142 lines
7.0 KiB
Markdown
142 lines
7.0 KiB
Markdown
# Hetzner Inference Supplier Fleet — Setup Runbook
|
||
|
||
> **Status:** shipped; fleet trimmed 2026-08-21. Hetzner retired
|
||
> `Kimi-K2.7-Code`, `GLM-5.2-NVFP4`, and `DeepSeek-V4-Flash-0731` from
|
||
> Inference (gone from `/v1/models`), so the 6 suppliers serving them were
|
||
> deregistered (adverts retired on-chain, containers removed). `Qwen3.8-27B`
|
||
> was added in their place, reusing the retired kimi-code wallets. Configures
|
||
> mainnet supplier nodes that use **Hetzner Inference**
|
||
> (`https://inference.hetzner.com`) as their **compute backend** for bonded
|
||
> inference work commissioned on Vector. Config only — no code changes.
|
||
|
||
---
|
||
|
||
## 1. What this is
|
||
|
||
Hetzner Inference is an OpenAI-compatible hosted API. Because the supplier's
|
||
upstream client (`supplier/src/openai.ts`) already speaks plain
|
||
`POST /v1/chat/completions` with a Bearer token, pointing it at Hetzner is
|
||
**pure configuration**: `LLM_BACKEND=openai` + `OPENAI_BASE_URL` + the API key.
|
||
This is the same pattern as the DeepSeek-direct and OpenRouter suppliers, and
|
||
the direct sibling of `docs/HUGGINGFACE_ROUTER_SETUP.md`.
|
||
|
||
One supplier per (model × capability) — both the one-off
|
||
`llm.text.generate.v1` and the multi-turn `llm.chat.v1` capability:
|
||
|
||
| Supplier name | Model (on-chain + API param, verbatim) | Capability | Advert max_output_tokens |
|
||
|---|---|---|---|
|
||
| `qwen35b` | `Qwen/Qwen3.6-35B-A3B-FP8` | `llm.text.generate.v1` | 262144 |
|
||
| `qwen35b-chat` | `Qwen/Qwen3.6-35B-A3B-FP8` | `llm.chat.v1` | 262144 |
|
||
| `qwen38` | `Qwen3.8-27B` | `llm.text.generate.v1` | 262144 |
|
||
| `qwen38-chat` | `Qwen3.8-27B` | `llm.chat.v1` | 262144 |
|
||
|
||
Deregistered 2026-08-21 (models dropped by Hetzner): `kimi-code`,
|
||
`kimi-code-chat` (`Kimi-K2.7-Code`), `glm`, `glm-chat` (`GLM-5.2-NVFP4`),
|
||
`ds-flash-htz`, `ds-flash-htz-chat` (`DeepSeek-V4-Flash-0731`). The qwen38
|
||
pair reuses the kimi-code / kimi-code-chat wallets; the other four wallets
|
||
still hold funds and their env files remain on the host.
|
||
|
||
Shared advert parameters: price **200000 lovelace (0.2 AP3X)** flat per job,
|
||
bonds **1000000 lovelace (1 AP3X)** both sides, `max_processing_ms` **300000**
|
||
for one-off / **1800000** for chat (the session spans the whole conversation).
|
||
`max_output_tokens` is set to the model's context length: no `max_tokens` is
|
||
forwarded upstream (`OPENAI_MAX_TOKENS` unset), so the advert value only gates
|
||
what buyers may request.
|
||
|
||
Names follow the brand-by-model convention; the retired `ds-flash-htz`
|
||
carried a `-htz` suffix only because `deepseek-flash-*` was already taken by
|
||
the OpenRouter fleet serving `deepseek/deepseek-v4-flash`.
|
||
|
||
## 2. Prerequisites
|
||
|
||
1. A Hetzner account with Inference access and an API key (usage-based billing
|
||
to you as the node operator — the flat 0.2 AP3X job price is a business
|
||
choice, not cost-derived). One key is shared by all suppliers; Hetzner
|
||
rate limits apply to the aggregate.
|
||
2. One funded supplier wallet per supplier (50 AP3X each recommended; bonds
|
||
are 1 AP3X per in-flight job and returned on completion).
|
||
3. The usual mainnet supplier deploy prerequisites (`deploy/README.md`).
|
||
4. **DNS**: one A record per supplier — `mp-suppliers-<name>.vector.apexfusion.org`
|
||
→ `91.98.147.172`, **DNS-only/unproxied**. There is NO wildcard for
|
||
`*.vector.apexfusion.org`; and because traefik issues certs on-box via
|
||
Let's Encrypt HTTP-01 (port 80), a proxied/CDN record would break issuance.
|
||
Certs are issued automatically within ~a minute of the record propagating —
|
||
no restarts needed.
|
||
|
||
## 3. Verify the backend before spending on-chain
|
||
|
||
The model ids come from `GET https://inference.hetzner.com/api/v1/models`
|
||
(this list changes — Hetzner retires models; re-check before any advert)
|
||
and are sent **verbatim** as the upstream `model` param (the runtime uses
|
||
`advert.model`). Before posting adverts, smoke-test each model:
|
||
|
||
```bash
|
||
curl -sS -X POST https://inference.hetzner.com/api/v1/chat/completions \
|
||
-H "Authorization: Bearer $HETZNER_KEY" -H "Content-Type: application/json" \
|
||
-d '{"model":"Qwen3.8-27B","messages":[{"role":"user","content":"ping"}]}' \
|
||
| jq '.choices[0].message.content'
|
||
```
|
||
|
||
`choices[0].message.content` must be a non-empty string with **no `max_tokens`
|
||
sent** (matches runtime config) — the supplier throws `openai_malformed`
|
||
otherwise and forfeits its 1 AP3X bond per failed job. For chat suppliers also
|
||
verify a `tools`/`tool_choice` round-trip. The original four models passed
|
||
both checks on 2026-08-11; `Qwen3.8-27B` passed both on 2026-08-21.
|
||
|
||
## 4. Per-supplier bring-up (repeat per supplier)
|
||
|
||
```bash
|
||
# 1. wallet — prints privateKeyHex, publicKeyHex, pubKeyHash, address
|
||
pnpm --filter @marketplace/supplier tx:gen-keypair --network 1
|
||
# fund the printed address (50 AP3X recommended), then:
|
||
|
||
# 2. env file on the mainnet host
|
||
cp supplier/.env.hetzner.example /root/agents-marketplace/supplier/.env.<name>
|
||
chmod 600 /root/agents-marketplace/supplier/.env.<name>
|
||
# fill in: ALL FOUR wallet vars (PRIV_KEY_HEX, ADDRESS, PKH, PUB_KEY_HEX —
|
||
# they are NOT derived at boot; missing derived vars ⇒ 403 wrong_supplier),
|
||
# OPENAI_API_KEY, and the ADVERT_* values from the table above.
|
||
|
||
# 3. advert (one-off shown; chat: --capability-id llm.chat.v1 --max-processing-ms 1800000)
|
||
pnpm --filter @marketplace/supplier tx:post-advert \
|
||
--capability-id llm.text.generate.v1 \
|
||
--model 'GLM-5.2-NVFP4' \
|
||
--max-output-tokens 512000 \
|
||
--max-processing-ms 300000 \
|
||
--price-lovelace 200000 \
|
||
--endpoint-url https://mp-suppliers-<name>.vector.apexfusion.org
|
||
# paste the printed "<txHash>#0" into ADVERT_REF in the env file
|
||
|
||
# 4. first start is always manual (CD skips compose projects with no running
|
||
# containers); subsequent rollouts ride the mainnet CD pipeline
|
||
docker compose -f deploy/mainnet/docker-compose.supplier-<name>.yml up -d
|
||
|
||
# 5. register the wallet for balance alerts
|
||
# add {name, address} to wallet-monitor/wallets.json on the host
|
||
```
|
||
|
||
## 5. Verification
|
||
|
||
- `curl https://mp-suppliers-<name>.vector.apexfusion.org/healthz` → `{"ok":true}`;
|
||
`/capability` shows the right model + pkh.
|
||
- Indexer: buyer `GET /v1/indexer/suppliers` lists the new entries with
|
||
`advert_status: Active`, `status: free`.
|
||
- Gateway `GET /openai/v1/models` includes the four Hetzner model ids.
|
||
- Watch `docker logs -f marketplace-mainnet-supplier-<name>` through the first
|
||
jobs for `openai_malformed` (reasoning-model content-shape risk).
|
||
|
||
## 6. Footguns
|
||
|
||
- `OPENAI_BASE_URL=https://inference.hetzner.com/api` — **no `/v1`**; the
|
||
client appends `/v1/chat/completions`.
|
||
- Do **not** set `OPENAI_REASONING` — `reasoning:{enabled:false}` is an
|
||
OpenRouter-only param.
|
||
- The advert `--model` must match the Hetzner id exactly, including the
|
||
`Qwen/` prefix on `Qwen/Qwen3.6-35B-A3B-FP8` (slash is fine on-chain;
|
||
precedent `moonshotai/kimi-k2.6`).
|
||
- `Kimi-K2.7-Code` and `DeepSeek-V4-Flash-0731` auto-join the buyer's PDF pool
|
||
(case-insensitive substring allowlist `["kimi","deepseek","gpt"]` in
|
||
`buyer/src/pdf/caps.ts`); GLM and Qwen stay out. Deliberate — adjust
|
||
`PDF_MODEL_ALLOWLIST`/`PDF_MODEL_DENYLIST` in the buyer env to change it.
|
||
- `OPENAI_TIMEOUT_MS` must stay ≤ the advert's `max_processing_ms`.
|