feat(supplier): HuggingFace Inference Providers router backend preset
Some checks failed
CI / typecheck + vitest (Node 20) (push) Has been cancelled

OpenAI-compatible compute-backend preset (config only, reuses LLM_BACKEND=openai) for pointing a supplier node at the HuggingFace Inference Providers router. Adds a boot guard that rejects the OpenRouter-only reasoning param against the HF router, unit tests, and an operator runbook.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
HalukG
2026-08-10 12:49:34 +03:00
parent 2f7b0194a5
commit 5377d379b7
4 changed files with 353 additions and 1 deletions

View File

@@ -0,0 +1,168 @@
# HuggingFace Router Supplier — Setup Runbook (V1 / W1)
> **Status:** shipped. Configures a Vector supplier node to use the HuggingFace
> **Inference Providers router** as its **compute backend** for bonded inference
> work commissioned on Vector. Config only — no code changes.
---
## 1. What this is
The HuggingFace **Inference Providers router** (`https://router.huggingface.co`)
is an OpenAI-compatible endpoint that fronts hundreds of hosted models across
partner providers (Together, Fireworks, DeepInfra, Groq, …). It is the direct
analogue of the OpenRouter backend the supplier already supports.
Because the supplier's upstream client (`supplier/src/openai.ts`) already speaks
plain `POST /v1/chat/completions` with a Bearer token, pointing it at HuggingFace
is **pure configuration**: set `LLM_BACKEND=openai` and point `OPENAI_BASE_URL` at
the router. It is the same as adding any OpenAI-compatible compute backend, with
the HuggingFace endpoint and token substituted in.
```
buyer commissions bonded work ──▶ supplier node ──▶ HuggingFace router
(AP3X escrow on Vector) │ (compute backend, hf_ token)
▼
supplier fulfils the work; the escrow settles on-chain
```
The supplier node runs the inference to **fulfil work a buyer has commissioned
under stake**; HuggingFace is simply the compute it uses to produce the result.
**What this does NOT do:** it does not add creator/model-author work-sharing,
change protocol economics, or change how an advert's committed amount is sized.
Those are separate, larger pieces of work tracked in the team's internal planning
notes. This preset is the fast, low-risk on-ramp that proves the path.
---
## 2. Prerequisites
1. A HuggingFace account with **Inference Providers access** (a free monthly
credit allowance, then usage-based per token — this is HuggingFace's billing
to you as the node operator).
2. An access token from <https://huggingface.co/settings/tokens> with the
**"Make calls to Inference Providers"** permission. It starts with `hf_`.
Treat it like a password.
3. A funded supplier wallet (~5 AP3X for tx fees + bonds). Generate a seed with
`openssl rand -hex 32`.
4. The usual supplier deploy prerequisites (Ogmios URL, a publicly reachable
HTTPS endpoint for buyers). See `deploy/README.md`.
---
## 3. Steps
### 3.1 Create the env file
```bash
cp supplier/.env.huggingface-chat.example supplier/.env
chmod 600 supplier/.env
```
Fill in `SUPPLIER_PRIV_KEY_HEX`, `OPENAI_API_KEY` (your `hf_…` token),
`OGMIOS_URL`, and the `ADVERT_*` values. Key settings for this preset:
| Var | Value | Why |
|-----|-------|-----|
| `LLM_BACKEND` | `openai` | routes chat through the OpenAI-compatible client |
| `OPENAI_BASE_URL` | `https://router.huggingface.co` | HF router; the `/v1/chat/completions` suffix is appended by the client — do **not** add it |
| `OPENAI_API_KEY` | `hf_…` | your HF token (Bearer) |
| `OPENAI_REASONING` | **unset** | HF 400s on the OpenRouter-only `reasoning` param (see §5) |
| `ADVERT_MODEL` | provider-qualified id, e.g. `deepseek-ai/DeepSeek-V3:fastest` | see §4 |
### 3.2 Choose a model
HF uses **provider-qualified** model names. The value goes on-chain in
`AdvertDatum.model` and is forwarded verbatim to the router:
- `deepseek-ai/DeepSeek-V3:fastest` — bare id + `:fastest` picks the fastest provider
- `deepseek-ai/DeepSeek-V3:together` — pin a specific provider
- `openai/gpt-oss-120b` — open-weight, HF-hosted
Browse models and their providers at
<https://huggingface.co/models?inference_provider=all>.
Because the model is on-chain data, you can advertise any router-supported chat
model without changing the supplier code — just post a new advert.
### 3.3 Smoke-test the token before going on-chain
Confirm your token + model + base URL work end to end, independent of the chain:
```bash
curl https://router.huggingface.co/v1/chat/completions \
-H "Authorization: Bearer $HF_TOKEN" \
-H "Content-Type: application/json" \
-d '{"model":"deepseek-ai/DeepSeek-V3:fastest",
"messages":[{"role":"user","content":"say hi in 3 words"}],
"stream":false}'
```
A `choices[0].message.content` in the response means the compute backend is
reachable. (Note: **no** `reasoning` field — see §5.)
### 3.4 Post the advert
```bash
pnpm --filter @marketplace/supplier tx:post-advert \
--capability-id llm.text.generate.v1 \
--model deepseek-ai/DeepSeek-V3:fastest \
--max-output-tokens 512 \
--max-processing-ms 60000 \
--price-lovelace 2000000 \
--endpoint-url https://your-supplier.example.org
```
Copy the printed `<txHash>#0` into `ADVERT_REF` in `supplier/.env`. Use
`--dry-run` first to inspect the tx without submitting.
### 3.5 Boot
Start read-only first (`LIVE_CHAIN` unset) to confirm the node boots and can
reach both Ogmios and the HF router, then set `LIVE_CHAIN=1` and restart to
enable real Claim/Submit. Deploy via the supplier compose file as in
`deploy/README.md`.
---
## 4. Sizing the advert (read before you post one)
`ADVERT_PRICE_LOVELACE` is the **flat AP3X amount committed in escrow per job**,
but the HuggingFace backend meters **per token**. The committed amount is fixed
per job while upstream token usage varies, so:
- Size it for your **worst-case** prompt length, not the average.
- Cap output tightly with `--max-output-tokens` — it is your main lever on
upstream token usage.
- Prefer **small** models here; large models have wide token-usage variance and
are harder to size a fixed per-job amount around.
- Buyer **input** tokens are not capped on-chain today. Until an input-token
ceiling lands, stick to small models or add your own request-size guard.
This is a pre-existing protocol design point (fixed per-job advert amount vs.
variable upstream token usage), not specific to HuggingFace. It is the main thing
to account for when sizing an advert.
---
## 5. Footgun: `OPENAI_REASONING` must stay unset
`reasoning:{enabled:false}` is an **OpenRouter-specific** parameter. The HF
router **rejects it with HTTP 400 on every request**. When copying an existing
OpenRouter env, it is easy to carry `OPENAI_REASONING=off` over by accident.
The supplier guards against this: `loadConfig` **refuses to boot** if
`OPENAI_REASONING` is disabled while `OPENAI_BASE_URL` resolves to
`router.huggingface.co`, with a clear error naming the fix
(`supplier/src/config.ts`). Just leave `OPENAI_REASONING` unset for HF.
---
## 6. Where this sits
This preset is the multi-model on-ramp that uses the HuggingFace router as a
compute backend. Further work items (additional task adapters, advert-sizing
refinements, and protocol-economics changes) are tracked separately in the team's
internal planning notes, along with the upstream-terms check every operator must
complete before pointing a node at a third-party compute backend.

View File

@@ -0,0 +1,114 @@
# supplier/.env.huggingface-chat.example
#
# Preset for a chat supplier that uses the HuggingFace Inference Providers router
# (https://router.huggingface.co) as its COMPUTE BACKEND to fulfil bonded
# inference work commissioned on Vector. It is the OpenAI-compatible on-ramp:
# the router is OpenAI-compatible, so this reuses the existing LLM_BACKEND=openai
# path with NO code changes — it is config only.
#
# Copy to supplier/.env on the host (chmod 600) and fill in real values.
# .env is gitignored — NEVER commit a populated copy.
#
# Runbook: docs/HUGGINGFACE_ROUTER_SETUP.md
#
# To generate a fresh Ed25519 wallet seed: openssl rand -hex 32
# Lines marked SECRET must never be shared, logged, or committed.
# ─── Required ─────────────────────────────────────────────────────────────
# 64-char hex Ed25519 private key seed for this supplier's wallet. SECRET.
SUPPLIER_PRIV_KEY_HEX=PASTE_64_HEX_CHARS_HERE
# Ogmios WebSocket URL. Testnet shown; use the mainnet URL for NETWORK_ID=1:
# testnet: wss://ogmios.vector.testnet.apexfusion.org
# mainnet: wss://ogmios.vector.mainnet.apexfusion.org
OGMIOS_URL=wss://ogmios.vector.testnet.apexfusion.org
# OutputReference of this supplier's on-chain advertisement UTxO.
# Format "<64-hex-tx-hash>#<int>". Post it first (see tx:post-advert below),
# then paste the printed "<txHash>#0" here.
ADVERT_REF=PASTE_TXHASH_HERE#0
# Cardano network ID. "0" = testnet, "1" = mainnet. Keep "0" until you have
# validated the setup end-to-end on testnet.
NETWORK_ID=0
# ─── Backend: HuggingFace router (OpenAI-compatible) ──────────────────────
# Route the chat job runner at the HF Inference Providers router instead of a
# local Ollama. The supplier POSTs to ${OPENAI_BASE_URL}/v1/chat/completions.
LLM_BACKEND=openai
# HF router base URL. The supplier appends /v1/chat/completions itself, so do
# NOT include the /v1 suffix here.
OPENAI_BASE_URL=https://router.huggingface.co
# HuggingFace access token (starts with "hf_"). Create one at
# https://huggingface.co/settings/tokens with "Make calls to Inference
# Providers" permission. Forwarded as `Authorization: Bearer <token>`. SECRET.
OPENAI_API_KEY=PASTE_HF_TOKEN_HERE
# First-token latency across hosted providers can exceed local Ollama — keep a
# generous request timeout.
OPENAI_TIMEOUT_MS=180000
# ─── IMPORTANT: do NOT set OPENAI_REASONING here ──────────────────────────
# `reasoning:{enabled:false}` is an OpenRouter-only param. The HF router
# rejects it with HTTP 400 on every request. The supplier refuses to boot if
# OPENAI_REASONING is disabled while OPENAI_BASE_URL points at the HF router.
# (This is the #1 footgun when copying an OpenRouter env — leave it unset.)
# OPENAI_REASONING=
# Optional hard ceiling forwarded as `max_tokens`. Unset = provider grants the
# max available (context − prompt). Keep below the model's context window.
# OPENAI_MAX_TOKENS=
# ─── Enable real chain submissions ────────────────────────────────────────
# LIVE_CHAIN: only the literal "1" opts in to real Claim/Submit txs. Leave
# unset for a dry run (read-only, no chain writes) while testing.
LIVE_CHAIN=
# HTTP listen port inside the container. Compose maps this to the host.
PORT=8080
# ─── tx:post-advert — run once before first deploy ────────────────────────
# Execute (from repo root):
# pnpm --filter @marketplace/supplier tx:post-advert \
# --capability-id llm.text.generate.v1 \
# --model deepseek-ai/DeepSeek-V3:fastest \
# --max-output-tokens 512 \
# --max-processing-ms 60000 \
# --price-lovelace 2000000 \
# --endpoint-url https://mp-supplier.vector.testnet.apexfusion.org
# Copy the printed "<txHash>#0" into ADVERT_REF above.
#
# NOTE on --model: HF uses PROVIDER-QUALIFIED model names. The bare id picks a
# default provider; a ":<hint>" suffix pins routing:
# deepseek-ai/DeepSeek-V3:fastest # cheapest/fastest provider for the model
# deepseek-ai/DeepSeek-V3:together # pin a specific provider
# openai/gpt-oss-120b # open-weight, HF-hosted
# Browse models + providers at https://huggingface.co/models?inference_provider=all
# The value lives on-chain in AdvertDatum.model — it is data, not code, so you
# can advertise any router-supported model without touching the supplier.
ADVERT_CAPABILITY_ID=llm.text.generate.v1
ADVERT_MODEL=deepseek-ai/DeepSeek-V3:fastest
ADVERT_MAX_OUTPUT_TOKENS=512
ADVERT_MAX_PROCESSING_MS=60000
# Flat AP3X amount committed in escrow per job (1 AP3X = 1_000_000 lovelace).
# The HF backend meters PER TOKEN while this committed amount is fixed per job,
# so size it for your worst-case prompt length and cap output tightly via
# ADVERT_MAX_OUTPUT_TOKENS. See docs/HUGGINGFACE_ROUTER_SETUP.md §4.
ADVERT_PRICE_LOVELACE=2000000
ADVERT_SUPPLIER_BOND_LOVELACE=1000000
ADVERT_BUYER_BOND_LOVELACE=1000000
# HTTPS endpoint buyers will call (must be publicly reachable).
ADVERT_ENDPOINT_URL=https://mp-supplier.vector.testnet.apexfusion.org
# Optional off-chain detail pointer + its sha256 (leave empty for v1).
ADVERT_DETAIL_URI=
ADVERT_DETAIL_HASH=
# ─── Optional: CIP-33 reference scripts (recommended on mainnet) ──────────
# ESCROW_REF_UTXO=<txhash>#0
# ADVERT_REF_UTXO=<txhash>#1

View File

@@ -15,7 +15,10 @@
* *
* Required for CAPABILITY_KIND="chat" + LLM_BACKEND="openai": * Required for CAPABILITY_KIND="chat" + LLM_BACKEND="openai":
* OPENAI_BASE_URL — http(s):// URL for an OpenAI-compatible /v1/chat/completions * OPENAI_BASE_URL — http(s):// URL for an OpenAI-compatible /v1/chat/completions
* host (e.g. ChatMock localhost proxy backed by Codex OAuth). * host. Examples: ChatMock localhost proxy (Codex OAuth),
* OpenRouter (https://openrouter.ai/api), or the HuggingFace
* Inference Providers router (https://router.huggingface.co).
* See docs/HUGGINGFACE_ROUTER_SETUP.md for the HF preset.
* *
* Required for CAPABILITY_KIND="tts": * Required for CAPABILITY_KIND="tts":
* PIPER_URL — http(s):// URL for the openedai-speech-min PiperTTS host * PIPER_URL — http(s):// URL for the openedai-speech-min PiperTTS host
@@ -51,6 +54,15 @@ const TX_HASH_RE = /^[0-9a-fA-F]{64}$/;
const NON_NEG_INT_RE = /^(?:0|[1-9]\d*)$/; const NON_NEG_INT_RE = /^(?:0|[1-9]\d*)$/;
const POS_INT_RE = /^[1-9]\d*$/; const POS_INT_RE = /^[1-9]\d*$/;
/** Lowercased hostname of a URL, or "" if it doesn't parse. */
function hostnameOf(url: string): string {
try {
return new URL(url).hostname.toLowerCase();
} catch {
return "";
}
}
export interface AdvertRef { export interface AdvertRef {
txHash: string; txHash: string;
index: number; index: number;
@@ -269,6 +281,24 @@ export function loadConfig(env: Record<string, string | undefined>): SupplierCon
reasoningRaw === "0" || reasoningRaw === "0" ||
reasoningRaw === "no"; reasoningRaw === "no";
// HuggingFace router guard. The `reasoning:{enabled:false}` param is
// OpenRouter-specific; the HF Inference Providers router
// (router.huggingface.co) rejects it with HTTP 400 on EVERY request. This is
// the easy footgun when copying an OpenRouter env to the HF preset, so fail
// fast at boot with a clear message instead of 400-ing every job.
// See docs/HUGGINGFACE_ROUTER_SETUP.md.
if (
llmBackend === "openai" &&
openaiReasoningDisabled &&
hostnameOf(openaiBaseUrl) === "router.huggingface.co"
) {
throw new Error(
"loadConfig: OPENAI_REASONING must be unset for the HuggingFace router " +
"(OPENAI_BASE_URL=https://router.huggingface.co) — it rejects " +
"reasoning:{enabled:false} with HTTP 400. Remove OPENAI_REASONING.",
);
}
// OPENAI_MAX_TOKENS — optional hard ceiling forwarded as `max_tokens`. Unset // OPENAI_MAX_TOKENS — optional hard ceiling forwarded as `max_tokens`. Unset
// or non-positive omits it (provider grants max available = context − prompt). // or non-positive omits it (provider grants max available = context − prompt).
// Keep it below the model context (kimi 262144) or the provider 400s when // Keep it below the model context (kimi 262144) or the provider 400s when

View File

@@ -184,6 +184,46 @@ describe("loadConfig() — LIVE_CHAIN parsing", () => {
}); });
}); });
// ─── loadConfig — HuggingFace router preset (W1) ────────────────────────────
// The HF Inference Providers router (router.huggingface.co) is OpenAI-compatible
// and reuses LLM_BACKEND=openai, but rejects the OpenRouter-specific
// reasoning:{enabled:false} param with HTTP 400 — so OPENAI_REASONING must stay
// unset for it. loadConfig fails fast at boot to catch the copy-from-OpenRouter
// footgun. See docs/HUGGINGFACE_ROUTER_SETUP.md.
describe("loadConfig() — HuggingFace router guard", () => {
const hfEnv = (over: Record<string, string> = {}) => ({
...buildSampleEnv(),
LLM_BACKEND: "openai",
OPENAI_BASE_URL: "https://router.huggingface.co",
OPENAI_API_KEY: "hf_sampletoken",
...over,
});
it("accepts the HF router with OPENAI_REASONING unset", () => {
const cfg = loadConfig(hfEnv());
expect(cfg.openaiBaseUrl).toBe("https://router.huggingface.co");
expect(cfg.llmBackend).toBe("openai");
expect(cfg.openaiReasoningDisabled).toBe(false);
});
it("throws when OPENAI_REASONING=off is set against the HF router", () => {
expect(() => loadConfig(hfEnv({ OPENAI_REASONING: "off" }))).toThrow(
/OPENAI_REASONING must be unset for the HuggingFace router/i,
);
});
it("still allows OPENAI_REASONING=off for non-HF openai backends (e.g. OpenRouter)", () => {
const cfg = loadConfig({
...buildSampleEnv(),
LLM_BACKEND: "openai",
OPENAI_BASE_URL: "https://openrouter.ai/api",
OPENAI_REASONING: "off",
});
expect(cfg.openaiReasoningDisabled).toBe(true);
});
});
// ─── parseAdvertRef ────────────────────────────────────────────────────────── // ─── parseAdvertRef ──────────────────────────────────────────────────────────
describe("parseAdvertRef()", () => { describe("parseAdvertRef()", () => {