deploy: add apex-simulation host stack (Traefik + PiperTTS + voice extras)

Stand up a second deploy target on apex-simulation dedicated to
"simulation"-style supplier capabilities, starting with CPU-only PiperTTS
served via openedai-speech-min behind Traefik on :80.

- deploy/simulation/docker-compose.traefik.yml: Traefik latest with
  introspection API, settles on Docker daemon 29.x API negotiation
- deploy/simulation/docker-compose.tts-piper.yml: openedai-speech-min
  with curl-based healthcheck (image lacks wget, which Docker would have
  marked unhealthy and Traefik would filter from the docker provider)
- deploy/simulation/deploy.sh: rsync-based deploy mirroring the
  apex-dashboard pattern; selective --traefik-only / --tts-only flags
- deploy/simulation/install-voice-extras.sh + voice-mappings.py:
  idempotent post-deploy step that downloads en_US-lessac-high voice
  files into the bind-mount and registers a "lessac" alias in
  voice_to_speaker.yaml; survives a host wipe without manual steps
- deploy/simulation/README.md: full host runbook
- .gitignore: exclude data/ recursively, *.log, .vitest-cache, .DS_Store,
  __pycache__/, *.pyc, docs/*.zip
This commit is contained in:
jelisaveta-talbot
2026-05-06 18:59:20 +02:00
parent cbab81efbd
commit 3f3bc72680
7 changed files with 531 additions and 1 deletions

8
.gitignore vendored
View File

@@ -9,7 +9,8 @@ coverage
.env.* .env.*
!.env.example !.env.example
# Local data / caches # Local data / caches (matches at any depth — covers
# deploy/simulation/data/, indexer/data/, etc.)
data/ data/
*.log *.log
.vitest-cache .vitest-cache
@@ -26,3 +27,8 @@ docs/img/*.mmd
# secrets inside their entries that .gitignore can't strip from a zip. # secrets inside their entries that .gitignore can't strip from a zip.
# Don't republish them. # Don't republish them.
docs/*.zip docs/*.zip
# Python bytecode (only present under docs/ reference material).
# .pyc files can embed absolute dev-machine paths via co_filename.
__pycache__/
*.pyc

152
deploy/simulation/README.md Normal file
View File

@@ -0,0 +1,152 @@
# `apex-simulation` host runbook
A second deploy target dedicated to "simulation"-style supplier capabilities
(starting with CPU-only PiperTTS as `audio.synthesize.v1`). Independent of
`apex-dashboard` so heavy local models can come and go without affecting
the marketplace's chain-follower / buyer-app.
## Layout
```
deploy/simulation/
├── docker-compose.traefik.yml # Standalone Traefik on :80
├── docker-compose.tts-piper.yml # PiperTTS behind OpenAI-compat shim
├── deploy.sh # rsync from laptop → apex-simulation, compose up
└── data/ # bind-mount target (created on host; gitignored)
└── tts-piper/{voices,config}/ # downloaded Piper voices persist here
```
## Prereqs
**On the laptop**: SSH alias `apex-simulation` resolving to the new host
(see `~/.ssh/config`). `rsync` available locally.
**On `apex-simulation` (one-shot)**:
```sh
# As root (or via sudo) — create the deploy user:
adduser --disabled-password --gecos "" deploy
usermod -aG docker deploy
sudo -u deploy mkdir -m 700 /home/deploy/.ssh
# Paste your laptop's public key:
sudo -u deploy tee /home/deploy/.ssh/authorized_keys >/dev/null <<'EOF'
ssh-ed25519 AAAA… your-laptop-key
EOF
sudo -u deploy chmod 600 /home/deploy/.ssh/authorized_keys
# Optional but recommended — disable password SSH:
sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
systemctl reload sshd
```
## Cloudflare DNS
Add an A record `tts-piper.vector.testnet.apexfusion.org` → apex-simulation
public IP. Orange cloud (proxied) is fine — CF terminates TLS, forwards
HTTP to Traefik on :80.
If CF later flags the path with a 502, mirror the existing
`Skip mp-buyers on testnet` rule for `tts-piper.…` (see
`deploy/CLOUDFLARE_WAF_BYPASS.md`). TTS bodies are short text so this is
unlikely, but the runbook is there if needed.
## Deploy
From the laptop, repo root:
```sh
# First time (and every subsequent change):
bash deploy/simulation/deploy.sh
```
`deploy.sh` does:
1. `mkdir -p` the bind-mount target dirs on the remote.
2. `rsync` only `deploy/simulation/*` to `/home/deploy/marketplace-simulation/deploy/simulation/` on apex-simulation. The voices dir (`data/`) is excluded — Piper downloads voices on first run.
3. `docker network create traefik-net` (idempotent).
4. `docker compose ... up -d` for Traefik then TTS.
5. Print container status + last 20 lines of TTS logs.
Selective re-deploys:
```sh
bash deploy/simulation/deploy.sh --traefik-only
bash deploy/simulation/deploy.sh --tts-only
```
## Voice quality
The compose file defaults to `PRELOAD_MODEL=en_US-lessac-high` — Piper's
highest-quality American-English single-speaker voice (~60 MB). Other
top-tier options:
| Model | Description | Size |
|---|---|---|
| `en_US-lessac-high` | Female, very natural, default | 60 MB |
| `en_GB-alan-high` | British male | 65 MB |
| `en_US-ryan-high` | American male | 60 MB |
| `en_US-libritts_r-medium` | Multi-speaker (904 voices) | 75 MB |
Browse the rest at <https://huggingface.co/rhasspy/piper-voices/tree/main>.
To switch, edit `PRELOAD_MODEL` in `docker-compose.tts-piper.yml`, then:
```sh
bash deploy/simulation/deploy.sh --tts-only
```
## Verify TTS end-to-end
From any laptop that resolves the CF hostname:
```sh
curl -s -X POST https://tts-piper.vector.testnet.apexfusion.org/v1/audio/speech \
-H "Content-Type: application/json" \
-d '{
"model": "tts-1-hd",
"input": "Hello from the local agents marketplace. PiperTTS, served via openedai-speech, signed via the future bonded escrow.",
"voice": "alloy",
"response_format": "wav"
}' \
-o /tmp/out.wav
aplay /tmp/out.wav # or `mpv /tmp/out.wav` / `paplay /tmp/out.wav`
```
The OpenAI shim maps the six standard voice names (`alloy`, `echo`, `fable`,
`onyx`, `nova`, `shimmer`) onto the bundled Piper voices. The `model`
field is informational — Piper has no model picker beyond voice.
## Health probes
```sh
# Traefik (returns 404 with no host = alive)
curl -i http://<server-ip>:80
# TTS service via Traefik
curl -s https://tts-piper.vector.testnet.apexfusion.org/v1/models | jq .
```
## Wiring as a marketplace supplier (later)
Once the audio is sound, this becomes a second supplier deployment:
1. Add a `marketplace-supplier-tts` compose on apex-simulation (clone of
`deploy/testnet/docker-compose.supplier.yml`, swap `OLLAMA_URL` →
`http://marketplace-tts-piper:8000`, advertise capability
`audio.synthesize.v1`).
2. New wallet (`SUPPLIER_PRIV_KEY_HEX`) so the bond is independent of
the chat supplier.
3. `pnpm tx:post-advert` against this host's wallet to publish the
capability. The buyer Dashboard then sees two suppliers — one for
`llm.text.generate.v1`, one for `audio.synthesize.v1`.
Out of scope here — file under a future `UX-supplier-tts` task.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| `404` on the CF hostname | Traefik up but no router matches | Container labels missed; `bash deploy.sh --tts-only` |
| `502` on first call | Cold start downloading the Piper voice (30–90 s) | Wait & retry, or pre-warm with a short curl |
| Audio file is 0 bytes | Voice model failed to download (no outbound network from host?) | `ssh apex-simulation 'docker logs marketplace-tts-piper'` — look for huggingface fetch errors |
| Want a different voice | Change `PRELOAD_MODEL` in `docker-compose.tts-piper.yml` | `bash deploy/simulation/deploy.sh --tts-only` |

91
deploy/simulation/deploy.sh Executable file
View File

@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# deploy/simulation/deploy.sh — push the simulation-host stack to apex-simulation.
#
# Mirrors the rsync-only pattern used for apex-dashboard (no git clone on
# the remote — only the files we need actually leave the laptop). Idempotent:
# safe to re-run after editing either compose file.
#
# Usage:
# bash deploy/simulation/deploy.sh # rsync + compose up
# bash deploy/simulation/deploy.sh --traefik-only # rsync + (re)start Traefik
# bash deploy/simulation/deploy.sh --tts-only # rsync + (re)start TTS
#
# Prereqs on local laptop:
# - SSH alias `apex-simulation` resolves (see ~/.ssh/config)
#
# Prereqs on remote:
# - Docker engine + compose plugin
# - User `deploy` exists, in `docker` group
# - Run once: `ssh apex-simulation 'docker network create traefik-net'`
set -euo pipefail
REMOTE_HOST="${REMOTE_HOST:-apex-simulation}"
REMOTE_DIR="${REMOTE_DIR:-/home/deploy/marketplace-simulation}"
cd "$(dirname "$0")/../.." # repo root
WHAT="${1:-all}"
case "$WHAT" in
--traefik-only) WHAT=traefik ;;
--tts-only) WHAT=tts ;;
""|all|--all) WHAT=all ;;
*) echo "unknown arg: $WHAT" >&2; exit 1 ;;
esac
echo "→ rsync deploy/simulation/ → $REMOTE_HOST:$REMOTE_DIR/deploy/simulation/"
ssh "$REMOTE_HOST" "mkdir -p $REMOTE_DIR/deploy/simulation/data/tts-piper/voices $REMOTE_DIR/deploy/simulation/data/tts-piper/config"
# Only the files we need on the host. Excludes data/ so we don't ship the
# voice models from the laptop (they're downloaded on the host on first run).
rsync -avz --delete \
--exclude='data/' \
--exclude='*.bak' \
deploy/simulation/ \
"$REMOTE_HOST:$REMOTE_DIR/deploy/simulation/"
echo
echo "→ ensure traefik-net exists on remote"
ssh "$REMOTE_HOST" "docker network inspect traefik-net >/dev/null 2>&1 || docker network create traefik-net"
if [ "$WHAT" = "all" ] || [ "$WHAT" = "traefik" ]; then
echo
echo "→ traefik up"
ssh "$REMOTE_HOST" "cd $REMOTE_DIR && docker compose -f deploy/simulation/docker-compose.traefik.yml up -d"
fi
if [ "$WHAT" = "all" ] || [ "$WHAT" = "tts" ]; then
echo
echo "→ tts-piper up (force-recreate so env / image changes take effect)"
ssh "$REMOTE_HOST" "cd $REMOTE_DIR && docker compose -f deploy/simulation/docker-compose.tts-piper.yml up -d --force-recreate"
# Wait for the container to become healthy before applying voice extras.
# On a fresh host the bind-mount config dir is empty until openedai-speech-min
# populates it, which only happens after the server starts answering /v1/models.
# If we touch /app/config/voice_to_speaker.yaml before then, the file isn't
# there yet. The healthcheck (`curl /v1/models`) gates that for us.
echo
echo "→ waiting for tts-piper to become healthy (start_period up to 90s)"
ssh "$REMOTE_HOST" "until [ \"\$(docker inspect marketplace-tts-piper --format '{{.State.Health.Status}}')\" = healthy ]; do sleep 5; done"
# Install extra voices (e.g. en_US-lessac-high) and the yaml aliases for
# them. Idempotent — does nothing on subsequent runs once everything is
# in place. Restarts the container only if it actually changed something.
echo
echo "→ apply voice extras (idempotent; restarts container if it changed anything)"
ssh "$REMOTE_HOST" "cd $REMOTE_DIR && bash deploy/simulation/install-voice-extras.sh"
fi
echo
echo "→ container status"
ssh "$REMOTE_HOST" "docker ps --filter name=traefik --filter name=marketplace-tts-piper --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}'"
echo
echo "→ tail tts-piper logs (first 20 lines)"
ssh "$REMOTE_HOST" "docker logs --tail 20 marketplace-tts-piper 2>&1 || true"
echo
echo "Done. Test from your laptop once CF DNS is set:"
echo " curl -s -X POST https://tts-piper.vector.testnet.apexfusion.org/v1/audio/speech \\"
echo " -H 'Content-Type: application/json' \\"
echo " -d '{\"model\":\"tts-1\",\"input\":\"Hello.\",\"voice\":\"alloy\",\"response_format\":\"wav\"}' \\"
echo " -o /tmp/out.wav && aplay /tmp/out.wav"

View File

@@ -0,0 +1,69 @@
# deploy/simulation/docker-compose.traefik.yml — apex-simulation Traefik
#
# Runs as its own compose project so individual services (TTS today,
# more later) can be brought up/down independently without touching the
# reverse proxy. Cloudflare terminates TLS in front, so we expose a
# single :80 entrypoint and rely on CF Full / Flexible SSL for the
# browser-side connection (same pattern as the apex-dashboard host).
#
# Bring up:
# docker compose -f deploy/simulation/docker-compose.traefik.yml up -d
#
# Verify:
# curl -i http://<server-ip>:80 # 404 from Traefik = healthy / no router matched
name: traefik
services:
traefik:
# `:latest` (currently v3.5+) carries an updated Docker SDK that
# negotiates correctly against Docker daemon 29.x. v3.1 — and even
# v3.4 with `DOCKER_API_VERSION=1.45` — still requested API 1.24,
# which Docker 29 rejects ("Minimum supported API version is 1.40").
# Result: provider error loop, no routers ever published.
image: traefik:latest
container_name: traefik
restart: unless-stopped
command:
# Docker provider — services join `traefik-net` and add labels.
- --providers.docker=true
- --providers.docker.exposedbydefault=false
- --providers.docker.network=traefik-net
# Single HTTP entrypoint (Cloudflare does TLS in front).
- --entrypoints.web.address=:80
# Health probe + access logs.
- --ping=true
# API (insecure) so we can introspect routers/services on :8080
# via `docker exec traefik wget -qO- http://localhost:8080/api/...`.
# Not published on the host, so only reachable from inside the
# traefik-net or via docker exec.
- --api=true
- --api.insecure=true
- --log.level=DEBUG
- --log.format=json
- --accesslog=true
- --accesslog.format=json
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
- traefik-net
healthcheck:
test: ["CMD", "traefik", "healthcheck", "--ping"]
interval: 30s
timeout: 3s
retries: 3
logging:
driver: json-file
options:
max-size: "20m"
max-file: "3"
networks:
# traefik-net is created once out-of-band (`deploy.sh` does
# `docker network create traefik-net`) so the TTS compose project
# can also attach to it. Both compose projects mark it `external`.
traefik-net:
external: true
name: traefik-net

View File

@@ -0,0 +1,80 @@
# deploy/simulation/docker-compose.tts-piper.yml — PiperTTS as OpenAI-compatible /v1/audio/speech
#
# Wraps rhasspy/piper (CPU neural TTS) behind matatonic/openedai-speech,
# which exposes the OpenAI Audio API (`POST /v1/audio/speech`). Once
# online behind Traefik on `apex-simulation`, this becomes a viable
# capability for a future `audio.synthesize.v1` supplier in the
# marketplace — same OpenAI-shaped HTTP surface as Ollama for chat.
#
# `*-min` image is the CPU/Piper-only variant (the full image pulls
# parler-tts + xtts which want a GPU). Default model is `en_US-amy-low`
# — small (~10 MB) and fast on CPU. Add `PRELOAD_MODEL=...` env to
# pre-warm a different voice on boot.
#
# Bring up:
# docker compose -f deploy/simulation/docker-compose.tts-piper.yml up -d
#
# Quick test (from any host that resolves the Traefik route):
# curl -s -X POST https://tts-piper.vector.testnet.apexfusion.org/v1/audio/speech \
# -H "Content-Type: application/json" \
# -d '{"model":"tts-1","input":"Hello from the marketplace.","voice":"alloy","response_format":"wav"}' \
# -o /tmp/out.wav && aplay /tmp/out.wav # or `mpv` / `ffplay`
name: marketplace-tts-piper
services:
tts-piper:
image: ghcr.io/matatonic/openedai-speech-min:latest
container_name: marketplace-tts-piper
restart: unless-stopped
environment:
# Pre-load Piper's highest-quality American-English voice so first
# /v1/audio/speech doesn't pay the model-download tax. `-high`
# variants are 60–70 MB and noticeably more natural than `-low`/
# `-medium` on CPU. Browse alternatives at
# https://huggingface.co/rhasspy/piper-voices/tree/main
# en_US-lessac-high — clean female, very natural (default)
# en_GB-alan-high — British male
# en_US-ryan-high — American male
# en_US-libritts_r-medium — multi-speaker (904 voices)
PRELOAD_MODEL: en_US-lessac-high
# Expose port — matches the loadbalancer label below.
PORT: "8000"
volumes:
# Host bind mounts (not Docker named volumes) so the deploy user can
# `ls`/`du`/back-up the downloaded Piper voices directly without
# `docker volume inspect`. Paths are relative to THIS compose file,
# i.e. `deploy/simulation/data/tts-piper/{voices,config}` from the
# repo root. Create the dirs once before `compose up` (see README).
- ./data/tts-piper/voices:/app/voices
- ./data/tts-piper/config:/app/config
healthcheck:
# /v1/models is the OpenAI-shape model list; a 200 here proves the
# server is up and at least one voice is loaded. Image ships with
# curl (and python3) but NOT wget — using wget here marks the
# container unhealthy on every probe, and Traefik then filters it
# out of the docker provider ("Filtering unhealthy or starting
# container"), which means no router ever registers.
test: ["CMD-SHELL", "curl -fsS http://localhost:8000/v1/models >/dev/null || exit 1"]
interval: 30s
timeout: 5s
retries: 3
start_period: 90s
networks:
- traefik-net
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik-net"
- "traefik.http.routers.tts-piper.entrypoints=web"
- "traefik.http.routers.tts-piper.rule=Host(`tts-piper.vector.testnet.apexfusion.org`)"
- "traefik.http.services.tts-piper.loadbalancer.server.port=8000"
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
networks:
traefik-net:
external: true
name: traefik-net

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
# install-voice-extras.sh — runs ON THE HOST after `docker compose up`
# for marketplace-tts-piper to install Piper voices and yaml mappings
# beyond what the openedai-speech-min image ships with.
#
# Idempotent: every step is a no-op if it already finished previously.
# Safe to re-run after a host wipe — `deploy.sh` invokes it at the end
# of every tts deploy so a fresh `data/` dir gets the same voices back.
#
# What it does (in order):
# 1. Ensure the bind-mount voice files are present (curl -sSL with `[ -s ]` guard).
# Voice files are large (~100 MB) and gitignored under `data/`, so they
# have to be (re)downloaded onto each fresh host.
# 2. Copy voice-mappings.py into the container and execute it as root.
# The script edits /app/config/voice_to_speaker.yaml (which is a host
# bind-mount) to add custom voice aliases. It prints "modified" or
# "unchanged" — we restart only on the former.
# 3. Restart the container if either step (1) or (2) changed something,
# so Piper picks up the new files / yaml.
#
# Usage:
# bash deploy/simulation/install-voice-extras.sh
set -euo pipefail
SIM_DIR="$(cd "$(dirname "$0")" && pwd)"
VOICES_DIR="$SIM_DIR/data/tts-piper/voices"
CONTAINER="marketplace-tts-piper"
mkdir -p "$VOICES_DIR"
changes=0
# --- 1. Voice files to install (filename → URL) ---
# Add entries here when you want another voice baked into the deploy.
# Each .onnx pairs with a .onnx.json — both must be downloaded.
voices=(
"en_US-lessac-high.onnx|https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/high/en_US-lessac-high.onnx"
"en_US-lessac-high.onnx.json|https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/lessac/high/en_US-lessac-high.onnx.json"
)
for entry in "${voices[@]}"; do
fn="${entry%%|*}"
url="${entry##*|}"
if [ -s "$VOICES_DIR/$fn" ]; then
echo "voice: already installed — $fn"
continue
fi
echo "voice: downloading $fn"
# `--fail` so HTTP 4xx/5xx aborts (HF returns html on 404 by default).
# Download to a tmp file then mv, so a partial download from a network
# blip doesn't leave a half-file that the next run treats as installed.
tmp="$VOICES_DIR/.${fn}.partial"
curl -sSL --fail --output "$tmp" "$url"
mv "$tmp" "$VOICES_DIR/$fn"
changes=1
done
# --- 2. Yaml mapping additions inside the container ---
docker cp "$SIM_DIR/voice-mappings.py" "$CONTAINER:/tmp/voice-mappings.py" >/dev/null
mapping_output="$(docker exec "$CONTAINER" python3 /tmp/voice-mappings.py)"
echo "mapping: $mapping_output"
if [[ "$mapping_output" == modified* ]]; then
changes=1
fi
# --- 3. Restart only if anything changed ---
if [ "$changes" -eq 1 ]; then
echo "→ restarting $CONTAINER to pick up new voices/mappings"
docker restart "$CONTAINER" >/dev/null
else
echo "✓ voice extras already up to date — no restart needed"
fi

View File

@@ -0,0 +1,60 @@
#!/usr/bin/env python3
# voice-mappings.py — idempotently apply project-specific voice aliases to
# /app/config/voice_to_speaker.yaml inside the marketplace-tts-piper container.
#
# Why this exists:
# The openedai-speech-min image populates voice_to_speaker.yaml on first
# start with its OWN defaults (alloy/echo/fable/onyx/nova/shimmer mapped
# onto the bundled LibriTTS + northern-english-male voices). Voices we
# install ourselves (e.g. en_US-lessac-high) need a name in that yaml so
# the OpenAI-shape `voice` field can resolve to them. This script adds
# those entries — and only those — without disturbing the defaults.
#
# Idempotency contract:
# Reads the yaml, compares each desired entry against what's there, only
# rewrites if at least one entry is missing or stale. Prints "modified"
# on write, "unchanged" otherwise — install-voice-extras.sh greps for
# "modified" to decide whether to restart the container.
#
# Run via:
# docker cp voice-mappings.py marketplace-tts-piper:/tmp/voice-mappings.py
# docker exec marketplace-tts-piper python3 /tmp/voice-mappings.py
import sys
import yaml
CONFIG_PATH = "/app/config/voice_to_speaker.yaml"
# Mappings to ensure exist under `tts-1`. Adding more voices later is a
# matter of dropping the .onnx + .onnx.json into voices/ (handled by
# install-voice-extras.sh) and adding an entry here.
DESIRED_TTS1 = {
# en_US-lessac-high — high-quality American female single-speaker voice.
# Single-speaker model, so no `speaker:` key.
"lessac": {"model": "voices/en_US-lessac-high.onnx"},
}
def main() -> int:
with open(CONFIG_PATH) as f:
cfg = yaml.safe_load(f) or {}
cfg.setdefault("tts-1", {})
modified_keys = []
for name, mapping in DESIRED_TTS1.items():
if cfg["tts-1"].get(name) != mapping:
cfg["tts-1"][name] = mapping
modified_keys.append(name)
if modified_keys:
with open(CONFIG_PATH, "w") as f:
yaml.safe_dump(cfg, f, sort_keys=False)
print(f"modified — applied: {', '.join(modified_keys)}")
else:
print("unchanged — all mappings already present")
return 0
if __name__ == "__main__":
sys.exit(main())