agents-marketplace/docs/gateway.md

# OpenAI-Compatible API Gateway for the Vector Marketplace

## Context

Today the only way to send a prompt to the marketplace is the single-operator **buyer** web app (cookie auth, one shared wallet). We want external developers to use **any OpenAI SDK** against the marketplace. Design (locked through interview + a 5-lens adversarial audit of an earlier draft):

- **Public, multi-tenant, custodial gateway.** Each API key is its **own buyer wallet** (own Ed25519 key, held AES-GCM-encrypted by the gateway). The user funds their own deposit address with AP3X; each request posts an on-chain escrow **from that user's wallet**. Not the deployed operator buyer.
- **Two surfaces, two escrow models:**
  - **One-shot completions** — `POST /openai/v1/chat/completions` (stateless, OpenAI-standard). Routes to an `llm.text.generate.v1` supplier. **One escrow per call.** `stream:true` → buffered pseudo-stream.
  - **Chat sessions** — a Vector REST extension. Routes to an `llm.chat.v1` supplier. **One escrow per session**: fixed price locked at open, **real token streaming** per turn (no per-turn charge), settled at session close.
- **Routing matches capability + model, not cheapest.** Per-prompt cost = the matched supplier's advertised price.
- **Gateway auto-maintains each user's wallet** (collateral + consolidation) so requests past #1 keep working.
- Onboarding is **self-serve via a small web UI**; the response is a **spec-clean OpenAI body + `x_vector` receipt extension**; custody is protected with **settle-on-success + a state-aware reclaim sweeper + a withdrawal path**.

The audit confirmed 39 holes in the first draft (10 distinct themes). This document incorporates every fix; the "Audit-driven corrections" section maps each theme to its resolution.

## Locked decisions

| Area | Decision |
|---|---|
| Scope | Public multi-tenant OpenAI-compatible gateway; new `gateway/` pnpm package reusing the buyer `Marketplace` SDK |
| Identity | Each API key = its own custodial buyer wallet |
| Custody | AES-256-GCM, `GATEWAY_MASTER_KEY` (env), per-row nonce; decrypt in-memory at request time; **rotation CLI**; **withdrawal endpoint** is the user exit |
| One-shot | `/openai/v1/chat/completions` → `llm.text.generate.v1`, 1 escrow/call, buffered (+ pseudo-stream) |
| Chat session | `/openai/v1/chat/sessions*` → `llm.chat.v1`, 1 escrow/**session**, real SSE per turn, settle at close |
| Routing | match `model` + required capability; pick an available supplier (status `free`\|`unknown`); **not cheapest**; fallback to next match on busy/offline |
| Pricing | cost = matched supplier `price_lovelace`; **required balance = price + buyer_bond + supplier_bond + 5 ADA collateral + fee reserve** |
| Wallet health | gateway **auto-consolidates** each wallet to `{5-ADA collateral, working}` after settle / on a tick |
| Settlement | resolve the **Submitted** ref via indexer, `acceptResult`, **await confirm** before returning success; sweeper is backstop |
| Sweeper | state-aware: Open/Claimed past `deliver_by` → reclaim; Submitted → retry Accept in-window; never blind-reclaim Submitted |
| Response | OpenAI object `{id, object, created, model, choices, usage}` + `x_vector:{receipt, receipt_signature, escrow_ref}`; full streaming chunk transform + final usage chunk |
| Hardening | `/signup` IP rate-limit, per-key request rate-limit, `SdkRegistry` LRU cap, structured logging + audit trail |

## Architecture

New standalone pnpm package `gateway/` (add to `pnpm-workspace.yaml`). Boots **one** shared `ChainProvider` (Ogmios, `LIVE_CHAIN=1`) + indexer URL, and an `SdkRegistry` that lazily builds **one `Marketplace` per API key** bound to that user's `WalletKey` (LRU-capped, evicted by idle). All on-chain work for a key is serialized by a per-key promise-chain mutex.

### Verified reuse points (do not re-implement)
- `Marketplace` ctor `MarketplaceOpts{chain, indexerUrl, walletKey, networkParams, historyStore?}` — `buyer/src/sdk/Marketplace.ts:117`. Per-user field is only `walletKey`.
- `deriveWalletKey` + ed25519 hook — `buyer/src/index.ts:24-64`.
- SDK methods: `discoverSuppliers`, `submitPrompt` (`Marketplace.ts:242`, verifies receipt but **does not settle**), `startChat`/`endChat` (chat session; `endChat` returns `acceptedRef` and awaits Accept), `acceptResult`, `reclaim`.
- Escrow value/bonds: buyer funds `totalLocked = price + buyer_bond + supplier_bond` at PostEscrow — `packages/shared/src/tx/escrow/postEscrow.ts:132-200`. On Accept supplier gets `payment+supplier_bond`, buyer gets `buyer_bond` back.
- Submitted-ref resolution: the buyer `/v1/accept` queries the indexer and matches the Submitted row by `posted_at` — `buyer/src/server.ts` `/v1/accept`. Gateway must mirror this.
- Wallet health: `planConsolidate` / `consolidateWallet` (`{collateral, working}` shape, `DEFAULT_COLLATERAL_LOVELACE = 5 ADA`) — `packages/shared/src/tx/wallet/`. Collateral assertion: `assertCollateralCandidate` — `packages/shared/src/tx/internal/liveCbor.ts`.
- Supplier chat-session SSE `{type:"token"|"done"|"error"}` + headers — `supplier/src/server.ts` `makeChatSessionHandlers`; buyer relay in `buyer/src/server.ts` `/v1/chat/message` (note: **verbatim** passthrough — the gateway must transform to OpenAI chunks).
- SQLite pattern (`better-sqlite3`, WAL) — `buyer/src/db/archive.ts`. Auth helpers `timingSafeCompareStrings`, `createRateLimiter` — `buyer/src/auth.ts`.
- Indexer `SupplierView` (`model`, `capability_id`, `price_lovelace`, `buyer_bond_lovelace`, `supplier_bond_lovelace`, `status` free/working/offline/unknown, `advert_status`) — `indexer/src/routes/suppliers.ts`.

## Package layout (`gateway/`)
```
src/
  index.ts          # boot: config → shared chain/indexer → GatewayStore → SdkRegistry → app → listen + sweeper + wallet-health tick
  config.ts         # GATEWAY_MASTER_KEY(64hex), INDEXER_URL, OGMIOS_URL, LIVE_CHAIN, NETWORK_ID, GATEWAY_PORT, GATEWAY_DB_DIR, rate-limit knobs
  wallet.ts         # deriveWalletKey + ed25519 hook + genPrivKey
  crypto/seal.ts    # aes-256-gcm seal()/open(), per-row nonce
  db/{schema.ts,store.ts}        # api_keys, usage, sessions
  sdk/registry.ts   # per-key Marketplace cache (LRU) + per-key Mutex
  routing/selectSupplier.ts      # indexer query → capability+model match → available → fallback
  openai/{shapes.ts,errors.ts,chatCompletions.ts,sessions.ts,models.ts}
  account/routes.ts # POST /signup, GET /account, POST /account/withdraw
  middleware/apiKeyAuth.ts, middleware/rateLimit.ts
  walletHealth.ts   # post-settle + ticker consolidation (reuse planConsolidate)
  sweeper.ts        # state-aware reclaim/accept-retry
  ui/index.html
  cli/rotate-master-key.ts       # re-encrypt all rows old→new master key
```
Build wiring: add `"exports": { "./sdk": "./dist/sdk/index.js" }` to `buyer/package.json`, depend `@marketplace/buyer: workspace:*` + `@marketplace/shared: workspace:*`.

## SQLite schema
```sql
CREATE TABLE api_keys ( id TEXT PRIMARY KEY, key_hash TEXT UNIQUE NOT NULL, key_prefix TEXT NOT NULL, label TEXT,
  wallet_pkh TEXT NOT NULL, deposit_address TEXT NOT NULL,
  enc_priv_nonce TEXT NOT NULL, enc_priv_ct TEXT NOT NULL, enc_priv_tag TEXT NOT NULL,
  master_key_version INTEGER NOT NULL DEFAULT 1, created_at INTEGER NOT NULL, disabled INTEGER NOT NULL DEFAULT 0 );
CREATE TABLE usage ( id TEXT PRIMARY KEY, key_id TEXT NOT NULL, created_at INTEGER NOT NULL, kind TEXT NOT NULL,
  model TEXT, capability_id TEXT, supplier_pkh TEXT, escrow_ref TEXT, cost_lovelace TEXT,
  prompt_tokens INTEGER DEFAULT 0, completion_tokens INTEGER DEFAULT 0, status TEXT NOT NULL, failure_reason TEXT );
CREATE TABLE sessions ( id TEXT PRIMARY KEY, key_id TEXT NOT NULL, escrow_ref TEXT NOT NULL, session_nonce TEXT NOT NULL,
  supplier_base_url TEXT NOT NULL, model TEXT NOT NULL, state TEXT NOT NULL, opened_at INTEGER NOT NULL, closed_at INTEGER );
```
Raw API key never persisted (only `sha256(key)`); `key_prefix` for display. `master_key_version` enables rotation.

## Endpoints

**One-shot (OpenAI standard):**
- `POST /openai/v1/chat/completions` — Bearer. Parse `{model, messages, max_tokens?, stream?}`; **reject** `tools`/`tool_choice`/`functions` with 400; **ignore** `temperature/n/stop/top_p/...` (documented). Lifecycle below.
- `GET /openai/v1/models` — distinct `model` over Active suppliers → OpenAI list shape.

**Chat sessions (Vector extension — documented as non-standard):**
- `POST /openai/v1/chat/sessions {model}` — open: route to `llm.chat.v1`, `startChat` (lock fixed price), persist session row → `{session_id, model}`.
- `POST /openai/v1/chat/sessions/{id}/messages {messages|content, stream}` — real SSE turn; transform supplier `{type:token}` → `chat.completion.chunk`; final usage chunk + `[DONE]`. No new escrow.
- `POST /openai/v1/chat/sessions/{id}/close` — settle: `endChat` (Submit+Accept, awaits), record usage, then run wallet-health.

**Account / UI:**
- `POST /signup {label?}` — IP rate-limited. Generate priv → derive → seal → store → `{api_key (once), deposit_address, key_prefix}`.
- `GET /account` — Bearer. `{deposit_address, balance:{available_lovelace, locked_in_escrow_lovelace, ap3x}, collateral_ok:bool, spend:{total_cost_lovelace, request_count}, recent_usage[]}`.
- `POST /account/withdraw {to_address, amount_lovelace?}` — Bearer. Build a wallet→address transfer (new `buildWithdrawTx` in packages/shared; repo has no send tx today) sending available funds out. **The custodial exit path.**
- `GET /` — self-serve UI (create key, copy deposit address, live balances, usage, withdraw).

## Request lifecycle — one-shot `/chat/completions`
All on-chain steps run inside the per-key `mutex.run(...)`.
1. **Route** (`selectSupplier`): indexer rows where `model` matches, `capability_id==="llm.text.generate.v1"`, `advert_status==="Active"`, live `status ∈ {free, unknown}`. Pick an available one (not cheapest). No match → 404 `model_not_found` (`type: invalid_request_error`).
2. **Pre-flight**: required = `price + buyer_bond + supplier_bond + 5_000_000 (collateral) + ~2 AP3X fee`. Verify total balance **and** a pure-ADA UTxO ≥5 ADA exists (`assertCollateralCandidate`-style). Else 402 `insufficient_funds` + `x_vector:{required_lovelace, available_lovelace, collateral_ok, deposit_address}`.
3. **Execute**: `submitPrompt({advertRef, messages, payment_lovelace:price, max_output_tokens})` (posts escrow → supplier → verifies receipt).
4. **Settle**: resolve the **Submitted** escrow ref via the indexer (mirror buyer `/v1/accept`: match by buyer pkh + `posted_at`), `acceptResult(submittedRef)`, **await confirmation**. On 409/offline mid-route, retry next matching supplier (bounded). On any post-escrow failure, record + leave for the sweeper (don't block the response on reclaim).
5. **Respond**: `{id:"chatcmpl-…", object:"chat.completion", created:<sec>, model, choices:[{index:0,message:{role:"assistant",content},finish_reason:"stop"}], usage:{prompt_tokens,completion_tokens,total_tokens}, x_vector:{receipt,receipt_signature,escrow_ref}}`. Record `usage`.
6. **Wallet health**: after settle, consolidate the wallet back to `{collateral, working}` (inside the mutex or queued).
7. `stream:true`: same flow; emit one (or chunked) `chat.completion.chunk` delta + final `finish_reason:"stop"` chunk **with `usage`** + `data: [DONE]`. Emit `: keepalive` SSE comments during the escrow-confirm gap.

## Chat-session lifecycle
- **open** → `startChat` (escrow at fixed price), store `{escrow_ref, session_nonce, supplier_base_url}`.
- **messages** → POST the turn to `${supplier_base_url}/v1/chat/message` with `X-Escrow-Ref`; transform each `{type:token,value}` → `chat.completion.chunk`; `{type:done}` → final chunk + usage + `[DONE]`. (System messages: fold into the first turn's content; documented limitation since chat.v1 has no system channel.)
- **close** → `endChat` (Submit + Accept, awaits) → settle + bond refund; record usage; wallet-health.
- Sessions are reclaimable by the sweeper if abandoned before close.

### Settle modes (`CHAT_SETTLE_MODE`, gateway + supplier, default `full`)
- **full** — the lifecycle above: supplier Claims at start, Submits a transcript receipt at end, gateway Accepts (buyer charged).
- **ticket** — the buyer's escrow post at open is the ONLY chain op. The supplier verifies the Open escrow as an entry ticket (`/v1/chat/start` → `{status:"ticket"}`) but never Claims/Submits; `/v1/chat/end` → `{status:"ended", settle_mode:"ticket"}` with no receipt; the sweeper Reclaims the Open escrow after `deliver_by`, returning all locked funds (net cost ≈ tx fees). Usage rows record `cost_lovelace: 0`. Sessions may outlive `deliver_by` (no hard-cap) — a live session whose ticket got reclaimed keeps running; only the idle janitor/close ends it. Close behavior is driven per-session by the supplier's `/end` response, so mixed-mode rollout windows settle correctly.
- Ticket caveats (accepted, demo-grade): a supplier restart forgets used tickets, so an unexpired un-reclaimed escrow ref could reopen a session (bounded by `deliver_by` + the sweep); `/account.locked_in_escrow_lovelace` over-reports because the indexer never updates spent rows.
- Suppliers additionally take `MAX_CHAT_SESSIONS` (default 1) — concurrent session slots; `/status` reports `active_sessions`/`max_sessions` and stays `free` until all slots are taken.

## Sweeper & wallet-health
- **Sweeper** (interval ~60s): per key, classify escrows. Open/Claimed past `deliver_by` → `reclaim`. Submitted **with a verified receipt** still in accept-window → retry `acceptResult`. Submitted past accept-window → leave for supplier Release (log). Never blind-`reclaim` Submitted.
- **Wallet-health tick**: periodically + after each settle, `planConsolidate` → `consolidateWallet` so every wallet keeps a ≥5 ADA collateral UTxO. Skip if already healthy.

## Errors (OpenAI shape `{error:{message, type, code, param:null}}`)
no supplier → 404 `invalid_request_error`/`model_not_found`; underfunded/no-collateral → 402 `invalid_request_error`/`insufficient_funds`; bad/missing key → 401 `authentication_error`/`invalid_api_key`; rate limit → 429 `rate_limit_error` + `Retry-After`; supplier timeout → 504 `server_error`; supplier busy race (after retries) → 503 `server_error`/`overloaded`; receipt verify → 502 `server_error`/`receipt_verification_failed`; tx build → 409/502 `server_error`/`escrow_failed`; unsupported `tools`/`functions` → 400 `invalid_request_error`/`unsupported_parameter`.

## Audit-driven corrections (themes → resolution)
1. **Settlement/Submitted-ref + fire-and-forget Accept** → lifecycle step 4 (indexer resolution + awaited Accept); sweeper backstop.
2. **No withdrawal** → `POST /account/withdraw` + new `buildWithdrawTx`.
3. **Master-key SPOF / offline-stranding** → documented backup, `rotate-master-key` CLI (`master_key_version`), withdrawal as exit; trustless recovery (pre-signed reclaim / multisig) explicitly **out of scope**, documented as residual risk.
4. **Stateless↔stateful chat** → sessions are explicit (open/messages/close), one escrow/session; system-message folding documented.
5. **Collateral/fragmentation** → corrected required-balance formula + pre-flight collateral check + auto wallet-health consolidation.
6. **~0.5 req/min throughput** (awaitTx in mutex) → documented per-key SLA; "move awaitTx out of the lock" noted as fast-follow.
7. **Sweeper can't reclaim Submitted** → state-aware sweeper.
8. **OpenAI wire format** → full response object + chunk transform + final usage chunk + keepalive + error taxonomy + param policy.
9. **Hardening** → `/signup` IP limit, per-key request limit, `SdkRegistry` LRU cap, `status ∈ {free,unknown}` + freshness + fallback retry.
10. **Economics/observability** → v1 = **no on-chain margin** (operator subsidizes infra; a fee needs a separate transfer or off-chain credit — folded into the deferred "billing" phase); add structured per-request logging + audit (request_id, key_id, model, supplier_pkh, escrow_ref, status, latency) + basic metrics/alerts (indexer stale, Ogmios slow, sweeper idle).
- **Doc fix**: clarify the **off-chain tx builder** enforces `payment==advert.price`; the validator only enforces the state machine, not pricing.

## Out of scope for v1 (documented)
Real billing/markup & fee hook; trustless/self-custody recovery; multi-funding-UTxO optimization beyond the consolidation loop; moving `awaitTx` out of the per-key lock; real chat.v1 system-role support (needs a supplier API change).

## Verification (testnet, end-to-end)
1. Indexer + Ogmios + ≥1 `llm.text.generate.v1` and ≥1 `llm.chat.v1` supplier for the same `model`, Active.
2. Boot gateway: `LIVE_CHAIN=1`, `GATEWAY_MASTER_KEY=$(openssl rand -hex 32)`, `OGMIOS_URL`, `INDEXER_URL`, `NETWORK_ID`, `GATEWAY_DB_DIR`, `GATEWAY_PORT`.
3. `POST /signup` → key + deposit address. Fund ≥ price + both bonds + 5 ADA collateral + ~2 ADA fees; confirm via `GET /account` (`collateral_ok:true`).
4. One-shot non-stream via curl and via the OpenAI Python/JS SDK (`base_url=…/openai/v1`) → valid `chat.completion` + `x_vector`; confirm the escrow **settled (Accepted)** and a `usage` row exists.
5. **Run a 2nd and 3rd request** → confirm wallet-health kept `collateral_ok` and they succeed (the audit's request-#2 failure must not reproduce).
6. One-shot `stream:true` → chunks + final usage + `[DONE]`.
7. Chat session: open → stream 2–3 turns → close; confirm **one** escrow for the whole session, settled at close.
8. `POST /account/withdraw` → unspent AP3X returns to an external address.
9. Underfunded / no-collateral key → 402 with guidance. Unknown model → 404. `tools:[…]` → 400.
10. Kill gateway mid-request → restart → sweeper reclaims Open/Claimed and retries Accept on Submitted; balances recover.