buyer: PDF book summarizer (/book) — chunk → map-reduce over suppliers
Adds an authenticated upload flow to the buyer web app: an operator uploads a PDF, the server extracts + paragraph-aligned-chunks it (~2.2k tokens), then runs a hierarchical map-reduce summarization by paying marketplace suppliers per chunk — one on-chain escrow per map call and per reduce node. One book upload fans out into dozens of real mainnet escrows (the intended marketplace traffic) while producing a genuine full-book summary. Server module (buyer/src/pdf/): PDF extraction (text-only, rejects scanned), token-aware chunker, supplier discovery/round-robin filtered to capable models (kimi/deepseek/gpt; excludes qwen2.5:0.5b), pre-spend cost estimate + wallet-floor guard, and a JobStore orchestrator that reuses submitPrompt → findSubmittedEscrowRef → runAccept per call. v1 serializes PostEscrow under a mutex (the shared tx builder can't pin wallet inputs, so parallel escrows would double-spend); retries failed calls on a different supplier then continues with gaps. 6 /v1/pdf-* routes (multipart upload, estimate, confirm-gated start, SSE progress, result, .md download) behind the existing session auth + page/chunk/ balance caps. SPA adds a 4-step BookSummarizer wizard. Tests: chunk/estimate/supplier-pool/extract unit tests + orchestrator (happy/gap/failover) + routes integration. Full buyer suite green (181 tests); SPA build + typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -38,6 +38,29 @@ NETWORK_ID=0
|
||||
# along over loopback. Production behind TLS should leave this unset / "1".
|
||||
# COOKIE_SECURE=1
|
||||
|
||||
# ─── Optional: PDF Book Summarizer (/book page, /v1/pdf-* routes) ─────────
|
||||
# Upload a PDF → chunk it → pay suppliers to summarize each chunk (one escrow
|
||||
# per map + reduce call) → full-book summary. All knobs are optional; defaults
|
||||
# shown. Tune the caps to bound spend/abuse on mainnet.
|
||||
#
|
||||
# Hard caps (a job is refused if it would exceed these):
|
||||
# PDF_MAX_PAGES=400 # reject books larger than this
|
||||
# PDF_MAX_CHUNKS=200 # reject books that chunk into more calls
|
||||
# PDF_MAX_PDF_BYTES=25000000 # multipart upload size limit (bytes)
|
||||
# Refuse to START a job if projected spend would drop the wallet below this:
|
||||
# PDF_WALLET_FLOOR_LOVELACE=50000000 # 50 AP3X
|
||||
# Chunking + reduce shape:
|
||||
# PDF_CHUNK_TARGET_TOKENS=2200 # target tokens per chunk
|
||||
# PDF_CHUNK_MAX_TOKENS=3000 # hard ceiling per chunk
|
||||
# PDF_REDUCE_FANIN=8 # summaries collapsed per reduce node
|
||||
# PDF_RETRY_K=2 # per-call retries on a different supplier
|
||||
# Supplier selection (comma-separated model-name substrings):
|
||||
# PDF_MODEL_ALLOWLIST=kimi,deepseek,gpt
|
||||
# PDF_MODEL_DENYLIST=qwen2.5:0.5b,qwen2.5-0.5b
|
||||
# Concurrency. v1 must stay 1 (serial PostEscrow — the shared tx builder
|
||||
# can't pin wallet inputs yet, so parallel escrows would double-spend):
|
||||
# PDF_LANE_COUNT=1
|
||||
|
||||
# ─── Optional: CIP-33 reference scripts ───────────────────────────────────
|
||||
# When set, buyer-signed script-spending txs (Accept, Reclaim) read the
|
||||
# escrow validator from an on-chain reference UTxO instead of inlining
|
||||
|
||||
Reference in New Issue
Block a user