Two structural fixes for demo concurrency (3 single-slot suppliers thrashed
under Open WebUI's ~3 conversations per message):
Multi-slot suppliers (MAX_CHAT_SESSIONS, default 1):
- SupplierState is now capacity-aware: a Set of active escrow refs with
free-until-full status semantics, ref-scoped release(ref) (also fixes the
latent bug where handler catch-alls released another request's slot), and
a walletMutex serializing every wallet-spending chain op — Claim, Submit,
and wallet-health consolidation queue instead of racing the single
working/collateral UTxO. /status gains active_sessions/max_sessions.
Entry-ticket settle mode (CHAT_SETTLE_MODE=ticket, default full, both sides):
- The buyer's escrow post at session open is the ONLY chain op. The supplier
verifies the Open escrow as an entry ticket but never Claims/Submits
(/v1/chat/start → {status:"ticket"}; /v1/chat/end → receipt-less
{status:"ended"}); no hard-cap timer, so sessions may outlive deliver_by.
Used tickets can't reopen sessions (records retained; 409 session_exists).
- Buyer SDK: startChat/endChat are settle-mode aware, driven by the
supplier's response — mixed-mode rollout windows settle correctly.
- Gateway: ticket closes bill cost 0; the sweeper Reclaims Open tickets
after deliver_by WITHOUT closing live sessions (the conversation and its
affinity mirror survive their reclaimed ticket), gates reclaims on
deliver_by, and keeps an in-process settled-refs memory so terminal
indexer rows aren't retried every sweep forever.
Deployment (kimi-chat trio + gateway): ticket mode, 8 slots per supplier,
idle policy gateway 10 min / supplier 15 min (fixes the inversion where the
supplier idle-closed before the gateway janitor).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The demo endpoint lost its session on nearly every second message: affinity
required clients to re-echo the assistant reply byte-identically, so clients
that strip <think> blocks or reshape whitespace (Open WebUI, litellm) missed
every turn, opened a fresh escrow session on the next free single-slot
supplier, and left the old one pinned. Open WebUI's background title/tag-gen
requests each pinned another supplier — one message could exhaust all three
kimi replicas ("no available chat supplier").
- affinity: lenient assistant-content matching (think-strip + whitespace
collapse); user/system/tool messages and tool_calls stay byte-strict;
strict prefixes outrank lenient ones. The mirror is never rewritten with
client text, so receipt hashes stay valid.
- demoChat: trim trailing assistant/system messages (regenerate semantics)
instead of 400 "the last message must have role user or tool".
- supplier: roll the transcript back when the LLM call fails so retries
don't duplicate the delta (ChatSessionStore.truncateTranscript).
- demoChat: LRU-evict a demo session when no supplier is available (supplier
/v1/chat/end frees the slot, sweeper Accepts the escrow) and retry the
open with selectCandidates ignoreStatusFor to bypass the ~20s-stale
indexer status poll. Max 2 evictions per request.
- sweeper: idle janitor closes demo sessions unused for DEMO_SESSION_IDLE_MS
(default 3 min, below the supplier's 5 min CHAT_IDLE_TIMEOUT_MS) so
abandoned one-off sessions release their suppliers quickly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New multi-tenant gateway package: external developers point any OpenAI SDK at
the marketplace. Each API key is its own custodial buyer wallet (AES-256-GCM
sealed under GATEWAY_MASTER_KEY); every request posts an on-chain escrow from
that user's wallet.
- POST /openai/v1/chat/completions (one escrow/call, llm.text.generate.v1,
buffered + pseudo-stream), chat sessions (one escrow/session, llm.chat.v1,
real SSE), GET /openai/v1/models, /signup, /account, /account/withdraw.
- Settles by resolving the Submitted ref via the indexer + awaited Accept;
state-aware reclaim sweeper + per-wallet auto-consolidation for collateral.
- Reuses the buyer Marketplace SDK via new buyer exports ./sdk; new shared
buildWithdrawTx (custodial exit). Runs via tsx; Dockerfile + mainnet compose.
Design in docs/gateway.md. Typecheck passes for all packages; 20 vitest tests
pass. Not yet exercised against a live chain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
