diff --git a/docs/HUGGINGFACE_ROUTER_SETUP.md b/docs/HUGGINGFACE_ROUTER_SETUP.md new file mode 100644 index 0000000..28a50dd --- /dev/null +++ b/docs/HUGGINGFACE_ROUTER_SETUP.md @@ -0,0 +1,168 @@ +# HuggingFace Router Supplier — Setup Runbook (V1 / W1) + +> **Status:** shipped. Configures a Vector supplier node to use the HuggingFace +> **Inference Providers router** as its **compute backend** for bonded inference +> work commissioned on Vector. Config only — no code changes. + +--- + +## 1. What this is + +The HuggingFace **Inference Providers router** (`https://router.huggingface.co`) +is an OpenAI-compatible endpoint that fronts hundreds of hosted models across +partner providers (Together, Fireworks, DeepInfra, Groq, …). It is the direct +analogue of the OpenRouter backend the supplier already supports. + +Because the supplier's upstream client (`supplier/src/openai.ts`) already speaks +plain `POST /v1/chat/completions` with a Bearer token, pointing it at HuggingFace +is **pure configuration**: set `LLM_BACKEND=openai` and point `OPENAI_BASE_URL` at +the router. It is the same as adding any OpenAI-compatible compute backend, with +the HuggingFace endpoint and token substituted in. + +``` +buyer commissions bonded work ──▶ supplier node ──▶ HuggingFace router + (AP3X escrow on Vector) │ (compute backend, hf_ token) + ▼ + supplier fulfils the work; the escrow settles on-chain +``` + +The supplier node runs the inference to **fulfil work a buyer has commissioned +under stake**; HuggingFace is simply the compute it uses to produce the result. + +**What this does NOT do:** it does not add creator/model-author work-sharing, +change protocol economics, or change how an advert's committed amount is sized. +Those are separate, larger pieces of work tracked in the team's internal planning +notes. This preset is the fast, low-risk on-ramp that proves the path. + +--- + +## 2. Prerequisites + +1. A HuggingFace account with **Inference Providers access** (a free monthly + credit allowance, then usage-based per token — this is HuggingFace's billing + to you as the node operator). +2. An access token from with the + **"Make calls to Inference Providers"** permission. It starts with `hf_`. + Treat it like a password. +3. A funded supplier wallet (~5 AP3X for tx fees + bonds). Generate a seed with + `openssl rand -hex 32`. +4. The usual supplier deploy prerequisites (Ogmios URL, a publicly reachable + HTTPS endpoint for buyers). See `deploy/README.md`. + +--- + +## 3. Steps + +### 3.1 Create the env file + +```bash +cp supplier/.env.huggingface-chat.example supplier/.env +chmod 600 supplier/.env +``` + +Fill in `SUPPLIER_PRIV_KEY_HEX`, `OPENAI_API_KEY` (your `hf_…` token), +`OGMIOS_URL`, and the `ADVERT_*` values. Key settings for this preset: + +| Var | Value | Why | +|-----|-------|-----| +| `LLM_BACKEND` | `openai` | routes chat through the OpenAI-compatible client | +| `OPENAI_BASE_URL` | `https://router.huggingface.co` | HF router; the `/v1/chat/completions` suffix is appended by the client — do **not** add it | +| `OPENAI_API_KEY` | `hf_…` | your HF token (Bearer) | +| `OPENAI_REASONING` | **unset** | HF 400s on the OpenRouter-only `reasoning` param (see §5) | +| `ADVERT_MODEL` | provider-qualified id, e.g. `deepseek-ai/DeepSeek-V3:fastest` | see §4 | + +### 3.2 Choose a model + +HF uses **provider-qualified** model names. The value goes on-chain in +`AdvertDatum.model` and is forwarded verbatim to the router: + +- `deepseek-ai/DeepSeek-V3:fastest` — bare id + `:fastest` picks the fastest provider +- `deepseek-ai/DeepSeek-V3:together` — pin a specific provider +- `openai/gpt-oss-120b` — open-weight, HF-hosted + +Browse models and their providers at +. + +Because the model is on-chain data, you can advertise any router-supported chat +model without changing the supplier code — just post a new advert. + +### 3.3 Smoke-test the token before going on-chain + +Confirm your token + model + base URL work end to end, independent of the chain: + +```bash +curl https://router.huggingface.co/v1/chat/completions \ + -H "Authorization: Bearer $HF_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{"model":"deepseek-ai/DeepSeek-V3:fastest", + "messages":[{"role":"user","content":"say hi in 3 words"}], + "stream":false}' +``` + +A `choices[0].message.content` in the response means the compute backend is +reachable. (Note: **no** `reasoning` field — see §5.) + +### 3.4 Post the advert + +```bash +pnpm --filter @marketplace/supplier tx:post-advert \ + --capability-id llm.text.generate.v1 \ + --model deepseek-ai/DeepSeek-V3:fastest \ + --max-output-tokens 512 \ + --max-processing-ms 60000 \ + --price-lovelace 2000000 \ + --endpoint-url https://your-supplier.example.org +``` + +Copy the printed `#0` into `ADVERT_REF` in `supplier/.env`. Use +`--dry-run` first to inspect the tx without submitting. + +### 3.5 Boot + +Start read-only first (`LIVE_CHAIN` unset) to confirm the node boots and can +reach both Ogmios and the HF router, then set `LIVE_CHAIN=1` and restart to +enable real Claim/Submit. Deploy via the supplier compose file as in +`deploy/README.md`. + +--- + +## 4. Sizing the advert (read before you post one) + +`ADVERT_PRICE_LOVELACE` is the **flat AP3X amount committed in escrow per job**, +but the HuggingFace backend meters **per token**. The committed amount is fixed +per job while upstream token usage varies, so: + +- Size it for your **worst-case** prompt length, not the average. +- Cap output tightly with `--max-output-tokens` — it is your main lever on + upstream token usage. +- Prefer **small** models here; large models have wide token-usage variance and + are harder to size a fixed per-job amount around. +- Buyer **input** tokens are not capped on-chain today. Until an input-token + ceiling lands, stick to small models or add your own request-size guard. + +This is a pre-existing protocol design point (fixed per-job advert amount vs. +variable upstream token usage), not specific to HuggingFace. It is the main thing +to account for when sizing an advert. + +--- + +## 5. Footgun: `OPENAI_REASONING` must stay unset + +`reasoning:{enabled:false}` is an **OpenRouter-specific** parameter. The HF +router **rejects it with HTTP 400 on every request**. When copying an existing +OpenRouter env, it is easy to carry `OPENAI_REASONING=off` over by accident. + +The supplier guards against this: `loadConfig` **refuses to boot** if +`OPENAI_REASONING` is disabled while `OPENAI_BASE_URL` resolves to +`router.huggingface.co`, with a clear error naming the fix +(`supplier/src/config.ts`). Just leave `OPENAI_REASONING` unset for HF. + +--- + +## 6. Where this sits + +This preset is the multi-model on-ramp that uses the HuggingFace router as a +compute backend. Further work items (additional task adapters, advert-sizing +refinements, and protocol-economics changes) are tracked separately in the team's +internal planning notes, along with the upstream-terms check every operator must +complete before pointing a node at a third-party compute backend. diff --git a/supplier/.env.huggingface-chat.example b/supplier/.env.huggingface-chat.example new file mode 100644 index 0000000..a8dc271 --- /dev/null +++ b/supplier/.env.huggingface-chat.example @@ -0,0 +1,114 @@ +# supplier/.env.huggingface-chat.example +# +# Preset for a chat supplier that uses the HuggingFace Inference Providers router +# (https://router.huggingface.co) as its COMPUTE BACKEND to fulfil bonded +# inference work commissioned on Vector. It is the OpenAI-compatible on-ramp: +# the router is OpenAI-compatible, so this reuses the existing LLM_BACKEND=openai +# path with NO code changes — it is config only. +# +# Copy to supplier/.env on the host (chmod 600) and fill in real values. +# .env is gitignored — NEVER commit a populated copy. +# +# Runbook: docs/HUGGINGFACE_ROUTER_SETUP.md +# +# To generate a fresh Ed25519 wallet seed: openssl rand -hex 32 +# Lines marked SECRET must never be shared, logged, or committed. + +# ─── Required ───────────────────────────────────────────────────────────── + +# 64-char hex Ed25519 private key seed for this supplier's wallet. SECRET. +SUPPLIER_PRIV_KEY_HEX=PASTE_64_HEX_CHARS_HERE + +# Ogmios WebSocket URL. Testnet shown; use the mainnet URL for NETWORK_ID=1: +# testnet: wss://ogmios.vector.testnet.apexfusion.org +# mainnet: wss://ogmios.vector.mainnet.apexfusion.org +OGMIOS_URL=wss://ogmios.vector.testnet.apexfusion.org + +# OutputReference of this supplier's on-chain advertisement UTxO. +# Format "<64-hex-tx-hash>#". Post it first (see tx:post-advert below), +# then paste the printed "#0" here. +ADVERT_REF=PASTE_TXHASH_HERE#0 + +# Cardano network ID. "0" = testnet, "1" = mainnet. Keep "0" until you have +# validated the setup end-to-end on testnet. +NETWORK_ID=0 + +# ─── Backend: HuggingFace router (OpenAI-compatible) ────────────────────── +# Route the chat job runner at the HF Inference Providers router instead of a +# local Ollama. The supplier POSTs to ${OPENAI_BASE_URL}/v1/chat/completions. +LLM_BACKEND=openai + +# HF router base URL. The supplier appends /v1/chat/completions itself, so do +# NOT include the /v1 suffix here. +OPENAI_BASE_URL=https://router.huggingface.co + +# HuggingFace access token (starts with "hf_"). Create one at +# https://huggingface.co/settings/tokens with "Make calls to Inference +# Providers" permission. Forwarded as `Authorization: Bearer `. SECRET. +OPENAI_API_KEY=PASTE_HF_TOKEN_HERE + +# First-token latency across hosted providers can exceed local Ollama — keep a +# generous request timeout. +OPENAI_TIMEOUT_MS=180000 + +# ─── IMPORTANT: do NOT set OPENAI_REASONING here ────────────────────────── +# `reasoning:{enabled:false}` is an OpenRouter-only param. The HF router +# rejects it with HTTP 400 on every request. The supplier refuses to boot if +# OPENAI_REASONING is disabled while OPENAI_BASE_URL points at the HF router. +# (This is the #1 footgun when copying an OpenRouter env — leave it unset.) +# OPENAI_REASONING= + +# Optional hard ceiling forwarded as `max_tokens`. Unset = provider grants the +# max available (context − prompt). Keep below the model's context window. +# OPENAI_MAX_TOKENS= + +# ─── Enable real chain submissions ──────────────────────────────────────── +# LIVE_CHAIN: only the literal "1" opts in to real Claim/Submit txs. Leave +# unset for a dry run (read-only, no chain writes) while testing. +LIVE_CHAIN= + +# HTTP listen port inside the container. Compose maps this to the host. +PORT=8080 + +# ─── tx:post-advert — run once before first deploy ──────────────────────── +# Execute (from repo root): +# pnpm --filter @marketplace/supplier tx:post-advert \ +# --capability-id llm.text.generate.v1 \ +# --model deepseek-ai/DeepSeek-V3:fastest \ +# --max-output-tokens 512 \ +# --max-processing-ms 60000 \ +# --price-lovelace 2000000 \ +# --endpoint-url https://mp-supplier.vector.testnet.apexfusion.org +# Copy the printed "#0" into ADVERT_REF above. +# +# NOTE on --model: HF uses PROVIDER-QUALIFIED model names. The bare id picks a +# default provider; a ":" suffix pins routing: +# deepseek-ai/DeepSeek-V3:fastest # cheapest/fastest provider for the model +# deepseek-ai/DeepSeek-V3:together # pin a specific provider +# openai/gpt-oss-120b # open-weight, HF-hosted +# Browse models + providers at https://huggingface.co/models?inference_provider=all +# The value lives on-chain in AdvertDatum.model — it is data, not code, so you +# can advertise any router-supported model without touching the supplier. +ADVERT_CAPABILITY_ID=llm.text.generate.v1 +ADVERT_MODEL=deepseek-ai/DeepSeek-V3:fastest +ADVERT_MAX_OUTPUT_TOKENS=512 +ADVERT_MAX_PROCESSING_MS=60000 + +# Flat AP3X amount committed in escrow per job (1 AP3X = 1_000_000 lovelace). +# The HF backend meters PER TOKEN while this committed amount is fixed per job, +# so size it for your worst-case prompt length and cap output tightly via +# ADVERT_MAX_OUTPUT_TOKENS. See docs/HUGGINGFACE_ROUTER_SETUP.md §4. +ADVERT_PRICE_LOVELACE=2000000 +ADVERT_SUPPLIER_BOND_LOVELACE=1000000 +ADVERT_BUYER_BOND_LOVELACE=1000000 + +# HTTPS endpoint buyers will call (must be publicly reachable). +ADVERT_ENDPOINT_URL=https://mp-supplier.vector.testnet.apexfusion.org + +# Optional off-chain detail pointer + its sha256 (leave empty for v1). +ADVERT_DETAIL_URI= +ADVERT_DETAIL_HASH= + +# ─── Optional: CIP-33 reference scripts (recommended on mainnet) ────────── +# ESCROW_REF_UTXO=#0 +# ADVERT_REF_UTXO=#1 diff --git a/supplier/src/config.ts b/supplier/src/config.ts index df17b26..51b2cc1 100644 --- a/supplier/src/config.ts +++ b/supplier/src/config.ts @@ -15,7 +15,10 @@ * * Required for CAPABILITY_KIND="chat" + LLM_BACKEND="openai": * OPENAI_BASE_URL — http(s):// URL for an OpenAI-compatible /v1/chat/completions - * host (e.g. ChatMock localhost proxy backed by Codex OAuth). + * host. Examples: ChatMock localhost proxy (Codex OAuth), + * OpenRouter (https://openrouter.ai/api), or the HuggingFace + * Inference Providers router (https://router.huggingface.co). + * See docs/HUGGINGFACE_ROUTER_SETUP.md for the HF preset. * * Required for CAPABILITY_KIND="tts": * PIPER_URL — http(s):// URL for the openedai-speech-min PiperTTS host @@ -51,6 +54,15 @@ const TX_HASH_RE = /^[0-9a-fA-F]{64}$/; const NON_NEG_INT_RE = /^(?:0|[1-9]\d*)$/; const POS_INT_RE = /^[1-9]\d*$/; +/** Lowercased hostname of a URL, or "" if it doesn't parse. */ +function hostnameOf(url: string): string { + try { + return new URL(url).hostname.toLowerCase(); + } catch { + return ""; + } +} + export interface AdvertRef { txHash: string; index: number; @@ -336,6 +348,24 @@ export function loadConfig(env: Record): SupplierCon reasoningRaw === "0" || reasoningRaw === "no"; + // HuggingFace router guard. The `reasoning:{enabled:false}` param is + // OpenRouter-specific; the HF Inference Providers router + // (router.huggingface.co) rejects it with HTTP 400 on EVERY request. This is + // the easy footgun when copying an OpenRouter env to the HF preset, so fail + // fast at boot with a clear message instead of 400-ing every job. + // See docs/HUGGINGFACE_ROUTER_SETUP.md. + if ( + llmBackend === "openai" && + openaiReasoningDisabled && + hostnameOf(openaiBaseUrl) === "router.huggingface.co" + ) { + throw new Error( + "loadConfig: OPENAI_REASONING must be unset for the HuggingFace router " + + "(OPENAI_BASE_URL=https://router.huggingface.co) — it rejects " + + "reasoning:{enabled:false} with HTTP 400. Remove OPENAI_REASONING.", + ); + } + // OPENAI_MAX_TOKENS — optional hard ceiling forwarded as `max_tokens`. Unset // or non-positive omits it (provider grants max available = context − prompt). // Keep it below the model context (kimi 262144) or the provider 400s when diff --git a/tests/unit/supplier-config.test.ts b/tests/unit/supplier-config.test.ts index 990307e..10500a7 100644 --- a/tests/unit/supplier-config.test.ts +++ b/tests/unit/supplier-config.test.ts @@ -184,6 +184,46 @@ describe("loadConfig() — LIVE_CHAIN parsing", () => { }); }); +// ─── loadConfig — HuggingFace router preset (W1) ──────────────────────────── +// The HF Inference Providers router (router.huggingface.co) is OpenAI-compatible +// and reuses LLM_BACKEND=openai, but rejects the OpenRouter-specific +// reasoning:{enabled:false} param with HTTP 400 — so OPENAI_REASONING must stay +// unset for it. loadConfig fails fast at boot to catch the copy-from-OpenRouter +// footgun. See docs/HUGGINGFACE_ROUTER_SETUP.md. + +describe("loadConfig() — HuggingFace router guard", () => { + const hfEnv = (over: Record = {}) => ({ + ...buildSampleEnv(), + LLM_BACKEND: "openai", + OPENAI_BASE_URL: "https://router.huggingface.co", + OPENAI_API_KEY: "hf_sampletoken", + ...over, + }); + + it("accepts the HF router with OPENAI_REASONING unset", () => { + const cfg = loadConfig(hfEnv()); + expect(cfg.openaiBaseUrl).toBe("https://router.huggingface.co"); + expect(cfg.llmBackend).toBe("openai"); + expect(cfg.openaiReasoningDisabled).toBe(false); + }); + + it("throws when OPENAI_REASONING=off is set against the HF router", () => { + expect(() => loadConfig(hfEnv({ OPENAI_REASONING: "off" }))).toThrow( + /OPENAI_REASONING must be unset for the HuggingFace router/i, + ); + }); + + it("still allows OPENAI_REASONING=off for non-HF openai backends (e.g. OpenRouter)", () => { + const cfg = loadConfig({ + ...buildSampleEnv(), + LLM_BACKEND: "openai", + OPENAI_BASE_URL: "https://openrouter.ai/api", + OPENAI_REASONING: "off", + }); + expect(cfg.openaiReasoningDisabled).toBe(true); + }); +}); + // ─── parseAdvertRef ────────────────────────────────────────────────────────── describe("parseAdvertRef()", () => {