Two structural fixes for demo concurrency (3 single-slot suppliers thrashed
under Open WebUI's ~3 conversations per message):
Multi-slot suppliers (MAX_CHAT_SESSIONS, default 1):
- SupplierState is now capacity-aware: a Set of active escrow refs with
free-until-full status semantics, ref-scoped release(ref) (also fixes the
latent bug where handler catch-alls released another request's slot), and
a walletMutex serializing every wallet-spending chain op — Claim, Submit,
and wallet-health consolidation queue instead of racing the single
working/collateral UTxO. /status gains active_sessions/max_sessions.
Entry-ticket settle mode (CHAT_SETTLE_MODE=ticket, default full, both sides):
- The buyer's escrow post at session open is the ONLY chain op. The supplier
verifies the Open escrow as an entry ticket but never Claims/Submits
(/v1/chat/start → {status:"ticket"}; /v1/chat/end → receipt-less
{status:"ended"}); no hard-cap timer, so sessions may outlive deliver_by.
Used tickets can't reopen sessions (records retained; 409 session_exists).
- Buyer SDK: startChat/endChat are settle-mode aware, driven by the
supplier's response — mixed-mode rollout windows settle correctly.
- Gateway: ticket closes bill cost 0; the sweeper Reclaims Open tickets
after deliver_by WITHOUT closing live sessions (the conversation and its
affinity mirror survive their reclaimed ticket), gates reclaims on
deliver_by, and keeps an in-process settled-refs memory so terminal
indexer rows aren't retried every sweep forever.
Deployment (kimi-chat trio + gateway): ticket mode, 8 slots per supplier,
idle policy gateway 10 min / supplier 15 min (fixes the inversion where the
supplier idle-closed before the gateway janitor).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The demo endpoint lost its session on nearly every second message: affinity
required clients to re-echo the assistant reply byte-identically, so clients
that strip <think> blocks or reshape whitespace (Open WebUI, litellm) missed
every turn, opened a fresh escrow session on the next free single-slot
supplier, and left the old one pinned. Open WebUI's background title/tag-gen
requests each pinned another supplier — one message could exhaust all three
kimi replicas ("no available chat supplier").
- affinity: lenient assistant-content matching (think-strip + whitespace
collapse); user/system/tool messages and tool_calls stay byte-strict;
strict prefixes outrank lenient ones. The mirror is never rewritten with
client text, so receipt hashes stay valid.
- demoChat: trim trailing assistant/system messages (regenerate semantics)
instead of 400 "the last message must have role user or tool".
- supplier: roll the transcript back when the LLM call fails so retries
don't duplicate the delta (ChatSessionStore.truncateTranscript).
- demoChat: LRU-evict a demo session when no supplier is available (supplier
/v1/chat/end frees the slot, sweeper Accepts the escrow) and retry the
open with selectCandidates ignoreStatusFor to bypass the ~20s-stale
indexer status poll. Max 2 evictions per request.
- sweeper: idle janitor closes demo sessions unused for DEMO_SESSION_IDLE_MS
(default 3 min, below the supplier's 5 min CHAT_IDLE_TIMEOUT_MS) so
abandoned one-off sessions release their suppliers quickly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MVP demo: one operator-funded API key anyone can plug into an IDE/harness,
running real escrow-per-session marketplace conversations (llm.chat.v1)
behind the standard OpenAI /chat/completions wire shape.
- shared: ChatMessage gains tool role + tool_calls; normalizeChatMessage/
chatMessagesEquivalent keep supplier transcript and gateway mirror
hash-identical for receipts (field presence is part of the identity)
- supplier: /v1/chat/message accepts {messages,tools,tool_choice} alongside
legacy {content}; callOpenAiStream forwards tools, accumulates fragmented
delta.tool_calls, allows content-less tool turns; done frame carries the
assistant message verbatim + finish_reason + usage
- gateway: demo flag on api_keys (additive migration), create-demo-key CLI,
withdraw 403 + hidden deposit address for demo keys, per-IP demo rate
limit (demo keys skip the shared per-key limit), models filtered to chat
capability
- gateway executor (demoChat.ts): transcript-prefix affinity maps stateless
calls onto open sessions, forwards only unseen deltas, transparently
reopens on supplier idle-close; never falls back to one-shot (its settle
path assumes <=1 in-flight escrow per wallet)
- mutex scope: key mutex held only for chain work (open/close); off-chain
turns serialize on per-session locks so replicas serve users concurrently
- sweeper: chat-aware reclaim guard (35min vs 11min) and abandoned-session
cleanup; terminal state confirmed via on-chain queryUtxo (indexer keeps
lineage rows at their old state forever)
- deploy: kimi-chat replicas 2+3 compose files; chat idle timeout 2.5min
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The supplier sent a bare {model,messages,stream} to the OpenAI/OpenRouter
endpoint — no max_tokens and no reasoning control — so reasoning ("thinking")
models like kimi-k2.6 emitted reasoning tokens that could consume the
completion budget and finish with reason=length (truncated/empty answer =
failed job). deepseek-v4-flash has no thinking; kimi-k2.6 does by default.
Add two optional, env-gated request params (defaults preserve current behavior,
so ChatMock/DeepSeek-direct suppliers are unaffected):
- OPENAI_REASONING=off -> send reasoning:{enabled:false} (disable thinking:
faster, cheaper, straight answers; no reasoning-starved length truncation)
- OPENAI_MAX_TOKENS=N -> forward max_tokens:N. Must stay below the model
context (kimi context = 262144), so the advert's 262144 cap cannot be sent
verbatim (provider 400s when prompt+max_tokens > context).
openai.ts builds the body conditionally; config.ts parses the env; wired
through jobRunner.ts (text) and server.ts (chat-session stream).
Deployed on the 4 kimi + 3 deepseek-flash suppliers with OPENAI_REASONING=off
and OPENAI_MAX_TOKENS=131072 (env files are host-only/gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add 3 single-slot suppliers serving deepseek/deepseek-v4-flash via OpenRouter,
increasing concurrent capacity for the independent buyer machines. Same shape
as the kimi replicas: capability llm.text.generate.v1, price 200000 lovelace,
max_output_tokens=262144, max_processing_ms=3600000, OPENAI_TIMEOUT_MS=3600000,
sharing the one OpenRouter key (accepted concurrency ceiling; no 429/backoff).
- deploy/mainnet/docker-compose.supplier-deepseek-flash-{1,2,3}.yml (new)
- buyer/scripts/monitor-wallets.ts: register the 3 replica wallets
Per-instance wallets, .env files, and on-chain adverts are provisioned on the
mainnet host (gitignored); the 3 containers are live and Active on-chain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add 3 parallel-capacity replicas of supplier-kimi (kimi-2/3/4) so the
marketplace can serve more concurrent jobs from the independent buyer
machines. Each is a single-slot worker on the same capability
(llm.text.generate.v1) and model (moonshotai/kimi-k2.6) at the same 0.2 AP3X
flat price, sharing the one OpenRouter key (accepted concurrency ceiling).
Also bring the original supplier-kimi up to the same config: raise its advert
caps (re-posted on-chain) and OPENAI_TIMEOUT_MS to match. All four kimi
suppliers now advertise max_output_tokens=262144, max_processing_ms=3600000
(OpenRouter's ceiling for kimi-k2.6), with OPENAI_TIMEOUT_MS=3600000 so the
upstream call isn't cut short before the advertised deadline.
- deploy/mainnet/docker-compose.supplier-kimi-{2,3,4}.yml (new)
- deploy/mainnet/docker-compose.supplier-kimi.yml: OPENAI_TIMEOUT_MS 300000 -> 3600000
- buyer/scripts/monitor-wallets.ts: register the 3 replica wallets
Per-instance wallets, .env files, and on-chain adverts are provisioned on the
mainnet host (gitignored); all four containers are live and Active on-chain.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an "API Keys" page to the buyer SPA (marketplace.vector.apexfusion.org)
that mints a custodial gateway API key by POSTing directly to the gateway's
public /signup, then shows the key once + deposit address + a base_url/curl
usage snippet. Page lives behind the SPA's existing operator login; per-user
registration is a later feature.
Gateway:
- GATEWAY_CORS_ORIGINS env -> corsOrigins allowlist; corsMiddleware echoes
Access-Control-* only for allowlisted Origins, answers OPTIONS preflight
204, no credentials. Empty allowlist = CORS off (default safe).
Buyer:
- GATEWAY_PUBLIC_URL env -> injected into __BUYER_BOOT__.gatewayUrl (only in
the authenticated boot block). SPA resolveGatewayUrl() falls back to
deriving "api." + location.host so the post-login-no-reload case (boot
block absent) still works.
- New ApiKeys page + /api-keys route + nav link.
Deploy (mainnet compose): gateway GATEWAY_CORS_ORIGINS and buyer
GATEWAY_PUBLIC_URL set to the marketplace/api hosts. NOTE: both images must be
REBUILT (gateway runs tsx from copied source; buyer SPA is vite-built at image
build) — an env-only recreate would silently ship the old code.
Typecheck + vite build clean; 205 tests pass. Adversarial multi-agent review
(4 dimensions, per-finding verification) surfaced only the rebuild-required
deploy note above; no code/security defects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
buildLiveTxForEscrow called .complete() with no presetWalletInputs, so lucid was
free to spend the dedicated 5-ADA collateral UTxO as an ordinary input. On
back-to-back requests this left the wallet with no collateral candidate and the
subsequent Accept failed with collateral_required, stranding the escrow.
Reserve the smallest pure-ADA UTxO >= 5 ADA and fund PostEscrow from the rest
(falls back to default selection for single/large-UTxO cases, which keep an
ample pure-ADA change UTxO anyway). Benefits the buyer too (same shared path).
Adds reserveCollateralInputs unit tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Without VECTOR_ZERO_TIME_MS the live tx builder defaults to the testnet genesis,
placing PostEscrow validity bounds ~51 days in the future (Ogmios 3118). Escrow
spends (Accept/Reclaim) also need ESCROW_REF_UTXO. Mirror buyer/supplier mainnet env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New multi-tenant gateway package: external developers point any OpenAI SDK at
the marketplace. Each API key is its own custodial buyer wallet (AES-256-GCM
sealed under GATEWAY_MASTER_KEY); every request posts an on-chain escrow from
that user's wallet.
- POST /openai/v1/chat/completions (one escrow/call, llm.text.generate.v1,
buffered + pseudo-stream), chat sessions (one escrow/session, llm.chat.v1,
real SSE), GET /openai/v1/models, /signup, /account, /account/withdraw.
- Settles by resolving the Submitted ref via the indexer + awaited Accept;
state-aware reclaim sweeper + per-wallet auto-consolidation for collateral.
- Reuses the buyer Marketplace SDK via new buyer exports ./sdk; new shared
buildWithdrawTx (custodial exit). Runs via tsx; Dockerfile + mainnet compose.
Design in docs/gateway.md. Typecheck passes for all packages; 20 vitest tests
pass. Not yet exercised against a live chain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The no-post branch logged "all wallets healthy" even when wallets were below
threshold but simply already alerted within the reminder window — confusing in
the cron log. Now distinguish: report the low-but-deduped count vs genuinely
all-healthy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When an alert fires, the Slack message now includes the full roster — low (🔴)
and healthy (🟢) wallets alike — each with its complete, copy-pasteable address
(backticked), instead of only the low wallets with a truncated address. The
posting trigger and dedup/reminder logic are unchanged: still silent when all
wallets are healthy; the buyer (normally healthy) now appears for context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Keeps secret wallet dumps (*-wallet.json, e.g. supplier/.deepseek-wallet.json
which contains privateKeyHex) and hand-made .bak hotfix backups out of git so
operator boxes stay clean. None of these were ever tracked.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A one-shot monitor for the vector-marketplace box: checks each wallet in
wallets.json against a threshold (default 10 APEX) via Ogmios
(ReadOnlyOgmiosProvider) and posts a combined Slack message when any wallet is
below it. Re-reminds every 6h while a wallet stays low and posts a recovery
message when it climbs back above, deduped through a persisted state file.
Runs hourly under host cron via `docker compose run --rm` — no long-running
server, no ports/healthcheck. Reuses the buyer/supplier tsx Docker pattern and
the existing balance-query client; addresses come from a static, gitignored
wallets.json so the monitor never reads a private key.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mainnet buyer compose never set TTS_PIPER_BASE_URL (only testnet did),
so the buyer booted with an empty value and POST /v1/synth-speech — the
no-escrow demo proxy behind the "PiperTTS demo" tile — returned 503
"booted without TTS_PIPER_BASE_URL". Point it at the private piper backend
over internal docker DNS (buyer is already on marketplace-mainnet-net).
Verified: authenticated POST /v1/synth-speech returns 200 audio/mpeg.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the static "…" in the streaming bubble with a "Thinking" label plus
three staggered bouncing dots, shown until the model's first token streams in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The supplier is single-slot, but the buyer posted the escrow BEFORE learning
the supplier was busy, stranding a second user's funds in an Open escrow. Now:
- startChat does a pre-flight GET /status and refuses (409, no escrow posted)
when the supplier is "working".
- ChatForm polls the supplier status (indexer) every 8s and disables the
Start chat button (\"Supplier busy\") until it returns to free.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds GET /v1/wallet/balance (sums the buyer wallet UTxOs via
queryUtxosByAddress from Ogmios) and renders it on the Wallet page as AP3X
with a UTxO count and a refresh button.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
End chat 502'd with "escrow ref not on chain": endChat queried the original
Open escrow ref, but the supplier's Claim already spent it (Open->Claimed),
so queryUtxo returned null before the supplier was ever called. Now endChat
uses the supplier endpoint cached by the buyer-app at startChat, resolves
supplier identity from /capability for verification, and Accepts the Submitted
UTxO the supplier returns. Adds supplierBaseUrl to EndChatOptions; the
/v1/chat/end route passes it from chatRouteState (404 if the session is gone).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The static import of verifyReceipt pulled receipt/sign.ts (top-level
`import { createHash } from "crypto"`) into the Vite SPA bundle, breaking
`build:ui`. endChat now does structural-only receipt verification, matching
the existing submitPrompt/submitTts discipline (M1-E/M1-F note).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a conversational supplier capability and a free demo, both streaming.
New paid `llm.chat.v1` type: one escrow per conversation (flat fee). "Start
chat" posts the escrow with a session-init placeholder prompt_hash and the
supplier Claims (reserving its single slot); turns then stream off-chain via
SSE with zero chain interaction; "End chat" has the supplier Submit a receipt
over the full transcript and the buyer Accept — the moment the user is charged.
The on-chain validator is content-agnostic (pins but never verifies the
hashes), so this reuses the existing contract unchanged. A supplier-side idle
watchdog auto-ends abandoned sessions before deliver_by.
Free Kimi K2.6 chat demo: a synthetic "demo:kimi-chat" tile that uses the same
chat UI but proxies straight to OpenRouter (no escrow, no payment), mirroring
the existing Piper demo.
Shared:
- postChatEscrow.ts + chatSessionPromptHash() (session-init prompt_hash helper
shared by the buyer builder and the supplier start gate).
Supplier:
- CAPABILITY_KIND="chat-session" (+ CHAT_IDLE_TIMEOUT_MS); callOpenAiStream()
streaming client; ChatSessionStore + endChatSession(); /v1/chat/{start,
message,end} handlers with idle + hard-cap watchdog timers.
Buyer:
- Marketplace.startChat()/endChat() with relaxed (session-init) receipt
verification + crypto verifyReceipt; /v1/chat/{start,message,end} routes
(SSE passthrough) and a free /v1/chat-demo/message OpenRouter proxy;
OPENROUTER_API_KEY/OPENROUTER_BASE_URL config; ChatForm.tsx + Dashboard
KIMI_DEMO_SUPPLIER tile and llm.chat.v1 dispatch.
Deploy:
- docker-compose.supplier-kimi-chat.yml + supplier/.env.kimi-chat.example.
Typecheck passes across all workspaces; full vitest suite green except 15
pre-existing time-bomb/RED failures (tx-claim/accept/post-advert live-CBOR
fixtures with hardcoded past deadlines + the M0-C golden-datum test), confirmed
identical on a clean tree.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mainnet buyer mounts a named volume at /repo/data/archive, but the
container runs as a non-root user, so a fresh root-owned volume left the
SQLite response archive + pdf-jobs.db unopenable ("unable to open
database file"). Create the dir with the runtime user's ownership before
the chown so Docker seeds the empty volume with the right ownership.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The summarization runs server-side, but the SPA held the job only in
component state and jobs lived only in memory — so navigating away from
the Book Summarizer reset the UI (losing the handle to a still-running
job) and past summaries vanished on restart.
- PdfJobDb: SQLite store (its own file under ARCHIVE_DIR) persisting
every job on each progress update.
- JobStore: createdAt, persist-on-update, viewOf()/list() that serve
live jobs first then fall back to the durable record; non-live jobs
whose worker died surface as a new "interrupted" status.
- routes: GET /v1/pdf-jobs (newest-first list); GET /:id + summary.md
serve past jobs via viewOf; /:id/events sends a single terminal frame
for a finished/past job (no hang) and live frames otherwise.
- SPA: RecentJobs list (open any job → reconnect to live progress or
show result), localStorage resume of the active job on return, and a
"back to list (keeps running)" affordance.
- mainnet buyer compose: persistent named volume + ARCHIVE_DIR so the
archive + pdf-jobs.db survive container recreation/redeploys.
Tests: persistence-across-restart, interrupted detection, list, and
past-job serving (incl. terminal SSE). Full buyer suite green (185).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds an authenticated upload flow to the buyer web app: an operator
uploads a PDF, the server extracts + paragraph-aligned-chunks it
(~2.2k tokens), then runs a hierarchical map-reduce summarization by
paying marketplace suppliers per chunk — one on-chain escrow per map
call and per reduce node. One book upload fans out into dozens of real
mainnet escrows (the intended marketplace traffic) while producing a
genuine full-book summary.
Server module (buyer/src/pdf/): PDF extraction (text-only, rejects
scanned), token-aware chunker, supplier discovery/round-robin filtered
to capable models (kimi/deepseek/gpt; excludes qwen2.5:0.5b), pre-spend
cost estimate + wallet-floor guard, and a JobStore orchestrator that
reuses submitPrompt → findSubmittedEscrowRef → runAccept per call.
v1 serializes PostEscrow under a mutex (the shared tx builder can't pin
wallet inputs, so parallel escrows would double-spend); retries failed
calls on a different supplier then continues with gaps. 6 /v1/pdf-*
routes (multipart upload, estimate, confirm-gated start, SSE progress,
result, .md download) behind the existing session auth + page/chunk/
balance caps. SPA adds a 4-step BookSummarizer wizard.
Tests: chunk/estimate/supplier-pool/extract unit tests + orchestrator
(happy/gap/failover) + routes integration. Full buyer suite green
(181 tests); SPA build + typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Periodic setInterval ticker + on-failure backstop from the buildClaimTx
catch in server.ts. Both reuse the existing single-slot lock and call
runConsolidateWallet under the hood — planConsolidate short-circuits to
already-healthy in the steady state so no tx, no fee on a clean wallet.
Driven by deepseek-lane outage 2026-05-28 (supplier wallet drifted to 14
UTxOs with 9 dust 2-AP3X outputs; Claim tx coin-selector picked dust as
working input → chain_submit_failed; lane stalled until manual
tx:consolidate-wallet).
buyer/scripts/monitor-wallets: surface deepseek + kimi as operator wallets.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caught the indexer stuck for 3.5h at slot 23463870 today while the
chain tip had advanced to 23477134. Ogmios TCP socket was alive but
the server had stopped pushing nextBlock responses; /healthz reported
ogmios_status: "connected" because that flag only tracked tip_slot > 0,
not actual block flow.
OgmiosSource now arms a 90s response watchdog on every nextBlock /
findIntersection send and clears it on any well-formed protocol reply.
On timeout it ws.close()s, re-entering the existing onclose →
scheduleReconnect → findIntersection path that resumes from the saved
cursor. Errors and malformed messages intentionally leave the watchdog
armed so a stuck Ogmios error path also recycles instead of wedging.
OGMIOS_RESPONSE_TIMEOUT_MS env var tunes the window (0 disables).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fourth mainnet supplier, branded by model rather than provider so future
OpenRouter-backed suppliers can claim their own compose files without
naming collisions. LLM_BACKEND=openai pointed at https://openrouter.ai/api
with a Bearer-token API key in supplier/.env.kimi. Traefik host
mp-suppliers-kimi.vector.apexfusion.org. No code changes needed — the
OPENAI_API_KEY plumbing landed in 1466442.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional OPENAI_API_KEY env var that supplier/src/openai.ts forwards as
`Authorization: Bearer <key>`. Empty/unset preserves the ChatMock behaviour
(no auth header). Unlocks pointing the openai backend at DeepSeek directly
(https://api.deepseek.com) — no local proxy needed.
New compose: deploy/mainnet/docker-compose.supplier-deepseek.yml (third mainnet
supplier, model deepseek-v4-pro, mp-suppliers-deepseek.vector.apexfusion.org).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two mainnet buyer-bot wallets on vector-modules-simulation
(178.105.3.133, SSH user `user`, env files at ~/marketplace-agents/
buyer-{a,b}.env) were missing from OPERATOR_SOURCES so monitor:wallets
didn't report on them.
State at time of discovery (2026-05-20): both BROKEN with the same
fragmentation pattern as the open-webui supplier — 11 and 12 UTxOs
respectively, max 3.00 AP3X, no UTxO ≥ 5 AP3X for collateral.
Cron-driven Accept/Reclaim cycles were failing silently.
Both consolidated to {5 AP3X collateral, ~22 AP3X working, 1.5 AP3X
collateral-return} via the tx:consolidate-wallet CLI (commit 972c0f9)
run inside the vector-marketplace buyer container with --priv-key:
- buyer-a@vector-modules-simulation
addr1vytah0zpcn3gn00lvu83g4qur4th0cg02v9hslaqlq9k88g5squ09
tx b789c2719bcee221f935578b4ac277d96295fa6e240ec2b63e0f4c83efc18cad
- buyer-b@vector-modules-simulation
addr1vy2wlyyat59fx572jgecwdmkxze7v65dl9jvg3pulgxudngvlpeau
tx ee1269be8aa01113db827cf02b6d9ffa0309524fb5cd2d3a9744cab9ef89f923
Total operator wallets monitored: 6 → 8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a one-shot operational tx that re-shapes any operator wallet to
{5 AP3X collateral candidate, working remainder} so subsequent Plutus
script-spends (Claim/Submit/Accept/Reclaim/RetireAdvert) find a UTxO
that satisfies lucid's setCollateral=5_000_000n default coin selector.
Symptom this addresses: open-webui mainnet supplier wallet held 8 UTxOs
max 2.40 AP3X, every Accept tx failed at lucid .complete() with the
"wallet must hold a pure-ADA UTxO ≥ 5 ADA" error path. Same fragmentation
risk on the other 4 mainnet operator wallets (one big UTxO surrounded by
dust — first script-spend consumes the big one and leaves nothing usable
as collateral).
Phase 1 — new tx:consolidate-wallet CLI (supplier + buyer):
- planConsolidate.ts — pure planner; supports consolidate (N>1 → 2),
split (1 → 2), and already-healthy skip.
- consolidateWallet.ts — live builder; presetWalletInputs pins all
UTxOs, two pay.ToAddress outputs (collateral + working).
- runConsolidateWallet.ts — orchestration (build + submit + awaitTx).
- supplier/src/cli/consolidate-wallet.ts + buyer mirror.
- monitor-wallets.ts — add supplier-tts@vector-marketplace (was
misclassified as 3rd-party).
Phase 2 — defensive pre-check in the 5 script-spend builders:
- assertCollateralCandidate(utxos) fires at the top of
buildSpendOpenOrClaimedTx (Claim/Submit), buildLiveTxForAccept,
buildLiveTxForReclaim, buildLiveTxForRetireAdvert. Throws a
TxConstructionError("collateral_required") pointing at
tx:consolidate-wallet instead of waiting for lucid's deep failure.
Tests: 12 new passing (planConsolidate edge cases + assert helper).
Pre-existing lucid 0.4.30 WASM "unreachable" failures in tx-*-live.test.ts
fixtures are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- jobRunner.ts: console.warn at each jobs.fail site (chat + tts paths, 6 sites). Three mainnet GPT-supplier 502s on 2026-05-19 left no trace beyond Traefik — adding a one-line log restores grep-ability.
- docker-compose.supplier-gpt.yml: OPENAI_TIMEOUT_MS 180000→300000. The 3 observed 502s were all exactly 90 polls × 2s, i.e. the supplier's openai_timeout firing.
- docker-compose.chatmock.yml: serve --reasoning-effort low --reasoning-summary none. Default 'medium'/'auto' pushed GPT-5.4 think-time over the prior 180s budget.
Note: on-chain advert ADVERT_MAX_PROCESSING_MS still pins 180000 until a re-advert tx is posted — buyers compare against the on-chain value, not the runtime env.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reads BUYER_/SUPPLIER_PRIV_KEY_HEX from the 5 mainnet wallets across
vector-marketplace + open-webui via batch-mode ssh (hex never touches
local disk), derives addresses, and queries lovelace via
ReadOnlyOgmiosProvider. Cross-references mp-indexer /suppliers and
labels any on-chain supplier whose pkh isn't an operator wallet as
third-party. Per-role thresholds (--buyer-min, --supplier-min,
defaults 10/3 AP3X) flag LOW operator wallets; exit 1 on any LOW or
ssh/query error.
Run from the buyer workspace: `monitor:wallets [--json] [--buyer-min N] [--supplier-min N]`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors deploy/testnet/docker-compose.supplier-tts.yml and the simulation
PiperTTS stack, co-located on vector-marketplace as a private internal
service on marketplace-mainnet-net (no Traefik labels, no public DNS for
Piper itself). Supplier reaches Piper at http://marketplace-mainnet-tts-piper:8000.
Voice: en_US-lessac-high. New supplier wallet + on-chain advert are
published out-of-band by the operator per the runbook header.
Files:
- deploy/mainnet/docker-compose.tts-piper.yml openedai-speech-min, private
- deploy/mainnet/docker-compose.supplier-tts.yml CAPABILITY_KIND=tts supplier
- deploy/mainnet/install-voice-extras.sh voice download + yaml mapping
- deploy/mainnet/voice-mappings.py idempotent yaml editor
- supplier/.env.tts.example operator env template
- .gitignore whitelist .env.*.example
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The comment at every setMinFee call site said "500K is a comfortable
floor" but the actual value was 200K — the floor wasn't high enough to
cover lucid 0.4.30's Conway-era fee underestimation. Smoke-tested
mainnet Accept tx hit Ogmios code 3122: lucid declared 343,132 lovelace
fee but Ogmios required 347,726 (off by ~5K). With the floor at 200K
the floor was inert (343K > 200K), so the underestimated fee stood.
Bumping all six builder paths (PostEscrow, Claim/Submit, Accept,
Reclaim, PostAdvert, UpdateAdvert/RetireAdvert) to 500K — comfortably
above the empirical minimum across operations, ~0.15 AP3X surcharge
per tx, excess becomes change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore the canonical mainnet docker-compose.supplier.yml to mirror
testnet's ollama-backed supplier (LLM_BACKEND=ollama, OLLAMA_URL points
at marketplace-mainnet-ollama, Traefik routes mp-suppliers.vector.apex-
fusion.org). The ChatGPT/Codex variant introduced in 965fabd moves to
docker-compose.supplier-gpt.yml with project name + container name
suffixed -gpt so both can coexist on the same host.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The mainnet marketplace-indexer was stuck in an event-replay loop since
2026-05-14. Diagnosis: OgmiosSource.intersectAt is captured ONCE in
start() from the cursor at process startup. On every WebSocket reconnect
(Cloudflare-fronted WS times out at ~100s idle, plus periodic Ogmios
server restarts), scheduleReconnect calls sendFindIntersection with the
stale start-time intersect, never the latest persisted cursor. Ogmios
seeks to that old point, sends nextBlock forward from there, the worker
re-processes the same blocks and writes saveCursor over the advanced
cursor with the old slot — DB cursor effectively never moves past the
startup point, and events get duplicated each iteration.
Fix:
- Add ChainSyncSource.updateIntersect({ slot, id }) to the interface.
- OgmiosSource.updateIntersect mutates this.intersectAt, so the next
reconnect's findIntersection picks up from the latest saved cursor.
- worker.handleBlock calls source.updateIntersect after every saveCursor.
- worker.handleRollback also calls it so a WS drop right after a rollback
doesn't resume past the rollback point.
- MockChainSyncSource implements as no-op (no reconnect path in tests).
Verified: same shape as the testnet projection-gap bug (memory 95fe6946),
manifesting differently. After this fix the cursor will only ever advance,
and reconnects resume from the freshest persisted point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The storagetime/chatmock image has neither wget nor curl, so the
previous wget-based healthcheck always failed with "executable file
not found" and the container sat in "starting" forever. The image
does carry a Python interpreter (chatmock is a Flask app), so swap
to a 1-liner urllib probe of /health. /health returns 200 even
before OAuth login, so a red healthcheck now means the Flask server
itself is down.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
storagetime/chatmock on Docker Hub publishes only {latest, 1.36, v1.36}
(all the same digest sha256:757c65c8b28941...). The :v1.35 pin from
the initial commit resolves to "not found". The compose's own comment
forbids :latest, so bump to :v1.36.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a Codex/ChatGPT backend path for the supplier alongside the
existing Ollama one. The mainnet supplier is repointed at a localhost
ChatMock proxy (RayBytes/ChatMock, pinned :v1.35) that authenticates
against a ChatGPT Pro subscription via OAuth.
- supplier/src/openai.ts: HTTP client for OpenAI-compatible
/v1/chat/completions. Same return shape as ollama.callOllama so
jobRunner can branch cleanly.
- supplier/src/jobRunner.ts: branch on config.llmBackend.
- supplier/src/config.ts: LLM_BACKEND ("ollama" default | "openai"),
OPENAI_BASE_URL, OPENAI_TIMEOUT_MS (default 180_000 — GPT first-token
latency).
- supplier/src/cli/gen-keypair.ts: standalone Ed25519 keypair generator
+ bech32 address derivation. Used to bootstrap a fresh supplier
wallet without round-tripping through tx:post-advert --dry-run.
- deploy/mainnet/docker-compose.chatmock.yml: long-running chatmock
serve + one-shot --profile login flow.
- deploy/mainnet/docker-compose.supplier.yml: rewritten for openai
backend; Traefik routes at mp-suppliers-gpt.vector.apexfusion.org so
the unsuffixed slot stays free for a future supplier with a
different backend.
- tests/unit/supplier-openai.test.ts: unit coverage for callOpenAi.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The mainnet host (vector-marketplace, 91.98.147.172) has no local
apex-dashboard-mainnet stack — the buyer + indexer already reach the
chain via the public HTTPS endpoint
https://ogmios.vector.mainnet.apexfusion.org. Pin the supplier to the
same pattern: drop the ws://ogmios:1337 env override (let supplier/.env
supply OGMIOS_URL) and drop the apex-net binding so compose-up doesn't
fail on a missing external network.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
lucidContext.ts captured VECTOR_ZERO_TIME at module-load time, so CLIs
that read a .env file at runtime (rather than letting Docker env_file
populate process.env before the runtime starts) got the testnet default
and built txs with a wrong slot origin -- Ogmios rejected with code
3118 "outside validity interval" on mainnet.
Two fixes:
- lucidContext.ts: read process.env.VECTOR_ZERO_TIME_MS inside
applyVectorSlotConfig() so the defensive re-apply in createLucidContext
picks up env vars set after module-load.
- run-cycle.ts / reclaim-orphans.ts: inject VECTOR_ZERO_TIME_MS,
OGMIOS_URL, INDEXER_URL, NETWORK_ID into process.env right after
reading the --env file, before any tx work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the "DEFERRED — Do NOT compose up" banner from
docker-compose.ollama.yml and docker-compose.supplier.yml (mirrors
cd0edd7 for buyer.yml + indexer.yml) and adds the missing LIVE_CHAIN=1
inline override that the testnet supplier compose has at line 45 —
without it the mainnet supplier silently falls back to
ReadOnlyOgmiosProvider and Claim/Submit never broadcast.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
run-cycle.ts: one full PostEscrow → Submit-wait → Accept cycle per
invocation, then exit. Lifts the stale-Open-ref workaround from the
run-agent-a.ts demo (Marketplace.submitPrompt returns a spent ref;
findSubmittedEscrowRef polls the indexer for the current Submitted
state). Flags: --env, --prompt-text, --payment-lovelace, --log-file,
--tag. Exit codes 0/2/4.
reclaim-orphans.ts: scans indexer for buyer's Open|Claimed escrows and
reclaims any past deliver_by. Skips not-yet-expired via the
"reclaim before deliver_by" TxConstructionError. Generalizes the
hardcoded-ref demo script.
Both for the two-mainnet-bot deploy on vector-modules-simulation
(see ~/.claude/plans/i-have-an-instance-ticklish-newt.md). No Claude
Code in the per-cycle path; cron + hand-curated prompts.txt files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Symptom: first login on a fresh session lands on Dashboard but Tasks shows
"buyer pkh missing — boot script not injected" until the operator manually
refreshes.
Cause: __BUYER_BOOT__ is injected by the Express server at SPA-shell-serve
time, gated on a valid session cookie. The pre-login HTML payload has no
boot block, so the in-flight Marketplace SDK is constructed with an empty
walletKey.pubKeyHash. After the 204 from /v1/auth/login the cookie is set
but the SDK instance in this page keeps its empty pkh, breaking any
component that reads it (TaskHistory.tsx:202).
Fix: replace auth.refresh() with window.location.reload() on success. The
browser re-fetches / with the new cookie, the server injects the boot
block, and the SPA mounts cleanly with the right walletKey.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
buyer and indexer mainnet stacks have been running on vector-marketplace
(marketplace-mainnet-buyer + marketplace-mainnet-indexer healthy, 4+ days
uptime for indexer, buyer redeployed today with the auth gate). The
"Wired but not yet exercised" header contradicts reality and would spook
the next operator who reads it.
Note in the new headers points at marketplace.vector.apexfusion.org and
mp-indexer.vector.apexfusion.org so the connection between the compose
file and the live URL is explicit. The genuinely-still-open mainnet-
safety follow-up (ARCHITECTURE.md §9 #5: env-gate / wallet allowlist /
MAINNET=1 flag) is kept as a one-line reference in the buyer header.
supplier and ollama mainnet composes are intentionally left alone — I
don't have evidence of those being deployed via these compose files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
