Files
agents-marketplace/deploy
jelisaveta-talbot 1f1a7d5a59 buyer: response archive (SQLite + filesystem) for off-chain audit trail
The on-chain Submit tx anchors sha256(canonicalize(receipt)) so the
receipt's signature is provably tied to a specific Cardano block — but
without persisting the receipt content + the actual response bytes
nowhere can buyer or supplier produce the artefacts a dispute would
need to argue about. This commit closes that gap on the buyer side.

What's new

- buyer/src/db/schema.ts (new):
  `responses` table keyed by escrow_ref, indexes on completed_at /
  capability_id / supplier_pkh. Stores metadata + receipt JSON +
  Ed25519 signature; large blobs (request envelope, response bytes)
  live on disk per-escrow.

- buyer/src/db/archive.ts (new):
  ResponseArchive class — persistChat / persistTts / list / get /
  readRequest / readResponse. Filesystem layout under ARCHIVE_DIR:
    <txhash>_<index>/request.json   (always JSON)
    <txhash>_<index>/response.{ext} (txt/json for chat, mp3/wav/… for TTS)
  Synchronous (better-sqlite3 + fs.writeFileSync) — volume is low and
  the simplicity is worth more than async overhead.

- buyer/src/server.ts:
  - AppDeps.archive plumbed through createApp.
  - /v1/submit-prompt + /v1/submit-tts persist BEFORE returning to
    the SPA, using canonicalize() for chat (same bytes the supplier
    hashed; JSON.stringify produced different bytes → wrong sha256).
    Persist failures are logged but never abort the response — the
    on-chain receipt is still authoritative.
  - New read-only endpoints:
      GET /v1/responses?limit=…           list metadata + parsed receipts
      GET /v1/responses/:ref               single-row detail
      GET /v1/responses/:ref/request       request artefact (application/json)
      GET /v1/responses/:ref/response      response artefact (verbatim bytes)
  - URL path uses "<txhash>_<index>" (the dir-safe form) since "#"
    can't appear unencoded in URLs (fragment delimiter); server
    normalises back to canonical "<txhash>#<index>" for SQLite lookup.
  - 503 service_unavailable on /v1/responses* when boot has no
    archive (ARCHIVE_DIR unset or DB unopenable) — the rest of
    the buyer-app still works.

- buyer/src/ui/pages/TaskHistory.tsx:
  Joins indexer escrows (chain state) + /v1/responses (archived
  artefacts) by escrow_ref. Each completed lifecycle gets an
  archive panel showing:
  - Request panel (above, chronological order): human-readable
    summary — first user message for chat, `text + voice + format +
    speed` for TTS, raw JSON fallback otherwise.
  - Response panel: assistant text inline for chat, <audio controls>
    for TTS; Download response / Download request anchors.
  - "Verify hash" button — fetches the response bytes, computes
    sha256 via crypto.subtle, compares to receipt.response_hash;
    green ✓ on match, red ✗ with truncated computed/expected on
    mismatch.
  - Collapsible "Receipt + signature" with the full canonical
    receipt JSON + the supplier's Ed25519 hex sig.

- buyer/src/config.ts: + ARCHIVE_DIR env (default "./data/archive").

- buyer/Dockerfile: builder gains python3/make/g++ + the
  better-sqlite3 native-binding fetch step (mirrors indexer/Dockerfile);
  runtime image stays toolchain-free.

- deploy/testnet/docker-compose.buyer.yml: bind-mounts
  ../../data/buyer/archive (repo-root-relative, not file-relative)
  into /data/archive. ARCHIVE_DIR env points at that path.

Tests

- tests/unit/buyer-archive.test.ts (new, 14 tests): persistChat /
  persistTts unit coverage (filesystem layout, INSERT-OR-REPLACE
  semantics, content-type → extension mapping for mp3/wav/opus/aac/
  flac, list ordering + limit, single-row get/read miss-vs-hit) +
  HTTP integration (503 when disabled, happy paths returning
  metadata / request / response artefact bytes with correct
  content-types, 404 on unknown ref).
- tests/unit/buyer-ui-smoke.test.tsx: TaskHistory fetch mock now
  also handles /v1/responses (returns empty list — same code path as
  "archive disabled" production case).

45 buyer tests passing; chat + TTS lifecycle verified end-to-end on
Vector testnet; "Verify hash" button confirms sha256(response bytes)
matches receipt.response_hash for both capabilities.
2026-05-11 08:49:07 +02:00
..

Deploy runbook — Local Agents Marketplace (M1-F)

Operator-facing instructions for bringing up the four-service stack on a Hetzner VM (or any Docker host) against an external Ogmios endpoint.

This is a REVIEW DRAFT. The artifacts (Dockerfiles, per-service compose files under deploy/testnet/ and deploy/mainnet/, .env.examples) have not been deployed end-to-end. Run the stack only after a careful review per-section below.


1. Concepts

Per-service compose projects

The four services (ollama, supplier, indexer, buyer) live in independent compose projects under deploy/testnet/. The operator can start, stop, rebuild, or pull images for each service in isolation:

deploy/testnet/
  docker-compose.ollama.yml      project name: marketplace-ollama
  docker-compose.supplier.yml    project name: marketplace-supplier
  docker-compose.indexer.yml     project name: marketplace-indexer
  docker-compose.buyer.yml       project name: marketplace-buyer

A parallel deploy/mainnet/ directory mirrors the testnet layout with marketplace-mainnet- prefixes and the mainnet ogmios network. The mainnet stubs are wired but deferred — see §7.

Three-axis network model

Each service joins one or more external Docker networks. The networks must already exist on the host before any of these compose files come up.

                          ┌─────────────────────────────────┐
                          │       dashboard_default         │  ← Traefik project
                          │  (Traefik ingress, port 80/443) │
                          └────┬───────────┬───────────┬────┘
                               │           │           │
                               ▼           ▼           ▼
   ┌──────────────────────────────────────────────────────────────────┐
   │                       marketplace-net                            │  ← service-to-service
   │   ┌──────────┐    ┌──────────┐    ┌──────────┐    ┌──────────┐   │
   │   │  ollama  │◀───│ supplier │    │ indexer  │◀───│  buyer   │   │
   │   └──────────┘    └────┬─────┘    └────┬─────┘    └──────────┘   │
   └──────────────────────────────────────────────────────────────────┘
                            │               │
                            ▼               ▼
              ┌─────────────────────────────────────────┐
              │     apex-dashboard_apex-net             │  ← apex-dashboard testnet stack
              │   ws://ogmios:1337   (Vector testnet)   │
              └─────────────────────────────────────────┘
Network Owner Purpose
marketplace-net this stack (docker network create marketplace-net) service-to-service hostnames (marketplace-ollama, marketplace-indexer)
apex-dashboard_apex-net apex-dashboard testnet compose project gives supplier+indexer access to ws://ogmios:1337
dashboard_default the Traefik compose project gives supplier+indexer+buyer a public web entrypoint

The buyer does not join apex-dashboard_apex-net — only the indexer talks to Ogmios; the buyer reads chain state via the indexer's HTTP API.

Container names + Traefik hosts (testnet)

Service container_name Traefik host Internal port
ollama marketplace-ollama (none, internal only) 11434
supplier marketplace-supplier mp-suppliers.vector.testnet.apexfusion.org 8080
indexer marketplace-indexer mp-indexer.vector.testnet.apexfusion.org 8090
buyer marketplace-buyer mp.vector.testnet.apexfusion.org 8070

Mainnet domain mirror: mp-suppliers.vector.apexfusion.org, mp-indexer.vector.apexfusion.org, mp.vector.apexfusion.org. Router names suffixed -mainnet to stay globally unique on the shared Traefik (mp-supplier-mainnet, mp-indexer-mainnet, mp-buyer-mainnet).

2. Prerequisites

  1. Docker host with Docker Engine 24+ and the Compose v2 plugin (docker compose ..., not legacy docker-compose).
  2. apex-dashboard testnet stack running on the same host, providing ws://ogmios:1337 on the apex-dashboard_apex-net network. Verify:
    docker network ls | grep apex-dashboard_apex-net
    
    Fallback: if you can't run apex-dashboard locally, point each service's OGMIOS_URL at a hosted endpoint (e.g. wss://ogmios.vector.testnet.apexfusion.org) via .env — but then drop apex-net from the supplier+indexer compose files, since they won't be reaching ogmios via the docker network. (This is a review-time decision; the wired default uses the docker network.)
  3. Traefik running on the host and managing dashboard_default. Verify:
    docker network ls | grep dashboard_default
    
    Fallback: bring up Traefik via the apex-dashboard repo's docker-compose.traefik.local.yml — it creates dashboard_default identically.
  4. One-shot setup (run once on the host):
    docker network create marketplace-net
    
    The compose files declare marketplace-net as external: true and will refuse to come up if it doesn't exist.

3. .env setup per service

Each service reads its env file at <service>/.env (paths resolved relative to the repo root, since the compose files use ../.. as the build context). Copy the examples:

cp supplier/.env.example supplier/.env
cp indexer/.env.example  indexer/.env
cp buyer/.env.example    buyer/.env
$EDITOR supplier/.env indexer/.env buyer/.env

Generate Ed25519 seeds:

openssl rand -hex 32   # → SUPPLIER_PRIV_KEY_HEX (64 hex chars)
openssl rand -hex 32   # → BUYER_PRIV_KEY_HEX

Fund the supplier address with ~5 AP3X via the Vector testnet faucet before starting the supplier.

ADVERT_REF is left empty until the post-advert CLI ships (see §8 known gaps). Until then, supplier /capability returns 503 and the buyer chat endpoint will fail at the Claim stage.

The env_file: directive uses required: false, so the compose files parse and come up even with an empty/missing .env — the service will crash at boot if mandatory env vars are absent, which is the desired loud-fail behavior.

4. Per-service operations (testnet)

All commands assume you're at the repo root. Start order matters (supplier depends on ollama, buyer depends on indexer) — Compose's depends_on doesn't cross project boundaries, so the operator sequences these manually.

Ollama

docker compose -f deploy/testnet/docker-compose.ollama.yml up -d
docker compose -f deploy/testnet/docker-compose.ollama.yml logs -f
docker compose -f deploy/testnet/docker-compose.ollama.yml down
docker compose -f deploy/testnet/docker-compose.ollama.yml pull

After the container is healthy, pull the model (one-shot, ~400 MB):

docker exec marketplace-ollama ollama pull qwen2.5:0.5b

Supplier

docker compose -f deploy/testnet/docker-compose.supplier.yml build
docker compose -f deploy/testnet/docker-compose.supplier.yml up -d
docker compose -f deploy/testnet/docker-compose.supplier.yml logs -f
docker compose -f deploy/testnet/docker-compose.supplier.yml down

Indexer

docker compose -f deploy/testnet/docker-compose.indexer.yml build
docker compose -f deploy/testnet/docker-compose.indexer.yml up -d
docker compose -f deploy/testnet/docker-compose.indexer.yml logs -f
docker compose -f deploy/testnet/docker-compose.indexer.yml down

Buyer

docker compose -f deploy/testnet/docker-compose.buyer.yml build
docker compose -f deploy/testnet/docker-compose.buyer.yml up -d
docker compose -f deploy/testnet/docker-compose.buyer.yml logs -f
docker compose -f deploy/testnet/docker-compose.buyer.yml down

5. Bringing up the whole testnet stack

End-to-end first-time bring-up:

# One-shot (skip if marketplace-net already exists)
docker network create marketplace-net

# Ollama, then pull the model
docker compose -f deploy/testnet/docker-compose.ollama.yml up -d
docker exec marketplace-ollama ollama pull qwen2.5:0.5b

# Supplier (needs ollama healthy)
docker compose -f deploy/testnet/docker-compose.supplier.yml up -d

# Indexer (independent of supplier)
docker compose -f deploy/testnet/docker-compose.indexer.yml up -d

# Buyer (needs indexer healthy)
docker compose -f deploy/testnet/docker-compose.buyer.yml up -d

Health summary:

docker ps --format 'table {{.Names}}\t{{.Status}}' | grep marketplace-

If a service stays unhealthy for more than ~60s, inspect logs via the per-service logs -f commands in §4.

6. Stopping

Per-service:

docker compose -f deploy/testnet/docker-compose.<svc>.yml down

Wipe the named volumes (Ollama models, indexer DB):

docker compose -f deploy/testnet/docker-compose.ollama.yml  down -v
docker compose -f deploy/testnet/docker-compose.indexer.yml down -v

The supplier and buyer don't own volumes; their down is sufficient.

7. Mainnet

See deploy/mainnet/. Deferred per ARCHITECTURE.md §9 #5 — do not docker compose up until the mainnet safety env-gate, wallet allowlist, and MAINNET=1 flag are wired in. The compose files parse cleanly so the operator can flip them on later by removing the warning headers.

8. Known gaps (post-M1-F-1)

These ARCHITECTURE.md §9 follow-ups affect deployment:

  • #5 mainnet safety — hard requirement before any mainnet attempt.
  • #6 Ollama-failure leaves Claimed — if Ollama crashes mid-request, on-chain escrow stays in Claimed until deliver_by. M1 has no automated recovery; the operator waits for the buyer to Reclaim.
  • #8 supplier boots ReadOnlyOgmiosProvider — submitTx will throw. Real-chain Claim/Submit is broken until M1-F-2 swaps in LiveOgmiosProvider. The supplier image will boot and serve /status correctly; chat completions will fail at the Claim stage with chain_submit_failed. The supplier compose now points at real Ogmios via apex-dashboard_apex-net — no further compose changes needed when M1-F-2 lands.
  • #9 structural-vs-real-Ed25519 — buyer SDK currently does a structural receipt-signature check; tier-3 lifecycle exercises real Ed25519 by virtue of using real keys.
  • ADVERT_REF posting flow (M1-F-3) — no CLI yet; supplier /capability returns 503 until a real advert UTxO exists.
  • Vector relay + Ogmios in compose — Ogmios remains external (lives with apex-dashboard). Bringing it into the marketplace compose project is M1-F-vector-stack and is not planned.
  • INDEXER_TIMEOUT_MS — referenced in buyer/.env.example but not yet honoured by buyer/src/config.ts. Buyer SDK uses a hardcoded timeout.
  • Buyer Vite UI build — Node-only imports leak into browser bundle — packages/shared/src/tx/blueprint.ts imports Node-only modules into the buyer's browser bundle. This is separate from the M1-F-1 alias-ordering fix and will need to be addressed by splitting @marketplace/shared into browser-safe and Node-only entry points. The Express server side of the buyer (which is what Docker actually runs) is unaffected — only pnpm --filter @marketplace/buyer build:ui is currently blocked.

9. Quick reference

Service Internal URL Health endpoint Compose project name
ollama http://marketplace-ollama:11434 (CMD ollama list) marketplace-ollama
supplier http://marketplace-supplier:8080 /healthz marketplace-supplier
indexer http://marketplace-indexer:8090 /healthz marketplace-indexer
buyer http://marketplace-buyer:8070 /healthz marketplace-buyer

For the M1-F-2 (LiveOgmiosProvider) handoff: the supplier compose already joins apex-dashboard_apex-net and overrides OGMIOS_URL=ws://ogmios:1337, so once the supplier code stops booting ReadOnlyOgmiosProvider and starts talking to a real Ogmios, no compose changes are needed.