## Overview
**Category:** feat (env-gated experiment + one exact always-on
optimization)
**Problem:** Speech-end -> first TTS audio through the desktop huddle
pipeline measures **924–1087 ms** on an M4 Max with a 0 ms LLM leg.
Voice turns feel sluggish no matter how fast the agent replies. Baseline
breakdown: ~300 ms hardcoded VAD silence flush + ~150–250 ms Parakeet
decode + ~380–550 ms TTS synthesis before the first player append.
**Outcome:** With all levers enabled, e2e time-to-first-audio measures
**347–384 ms** (307–357 ms on a longer utterance) on the same hardware,
harness, and production pipelines. Defaults preserve production behavior
everywhere except one deterministic, bit-exact cache win.
## What's in here
### Levers (all default-off, env-gated)
| Lever | Env | Effect (measured medians, short utterance) |
|---|---|---|
| Speculative Parakeet decode | `BUZZ_STT_SPECULATIVE=1` | STT leg ->
~max(flush, decode) |
| Streaming TTS synthesis | `BUZZ_TTS_STREAMING=1`,
`BUZZ_TTS_EMIT_FRAMES` | first audio 380–550 -> 211–320 ms (emit=12,
bit-exact) |
| ONNX intra-op threads | `BUZZ_STT_THREADS`, `BUZZ_TTS_THREADS` | TTS
first audio 211–320 -> 129–180 ms (4 threads) |
- **Speculative decode** starts the Parakeet decode at the *first*
silent VAD frame, overlapping it with the flush window. Resumed speech
invalidates the result (voiced-frame-count check); held silence emits it
instantly at the flush boundary.
- **Streaming TTS**: new `synth_chunk_streaming` (buzz-voice)
interleaves the Flow LM frame loop with incremental *stateful* Mimi
decoding, emitting PCM deltas to the player via the existing
`PlaybackChunkAudio` decoration. At `emit_frames=12` (the decoder's
native chunk) streamed audio is **bit-identical** to the batch path —
verified by the ignored test
`incremental_stateful_decode_matches_batch_decode` (max|diff|=0).
Smaller deltas are faster but diverge (~23 dB SNR; decoder intra-chunk
lookahead), hence the default of 12.
> **Removed after live testing:** the `BUZZ_STT_FLUSH_MS` flush-window
override. Lowering the silence window below natural mid-sentence pauses
(the fast-path recipe said 150 ms) split single spoken sentences into
multiple messages and confused the listening agents. The window is a
turn-taking quality knob, not a latency lever — it is now fixed at the
production 300 ms value.
### Push-to-talk grouping fix (always-on)
A held push-to-talk shortcut is an explicit "I am not done talking"
signal, so silence never ends the utterance while it is held — even when
the microphone is also manually open. The utterance flushes on shortcut
release (existing transmit-edge flush); a manually open mic with the
shortcut up keeps normal VAD pause flushing. Gate is the pure
`vad_flush_allowed` function with a unit-test truth table.
### Always-on (exact): voice-conditioning cache
Phase profiling (`BUZZ_TTS_PHASE_LOG=1`) showed a fixed ~160 ms
`condition_voice` Flow-LM pass on *every* chunk, re-deriving the same
post-conditioning state for the same reference voice. The state is now
snapshotted after first computation and restored per chunk (dtype-tagged
tensor copies, keyed identically to the existing `cached_voice`).
Deterministic — same tensors in, same tensors out. The default path's
TTS leg drops from 380–550 ms to 225–355 ms with no configuration.
### Bench harness
`huddle::latency_bench` (`#[cfg(test)]` + `#[ignore]`) drives the real
`SttPipeline` and `TtsPipeline`, feeding a 48 kHz WAV in real-time 100
ms batches (AudioWorklet cadence) with a configurable fake LLM in place
of the relay leg, timing speech-end -> transcript -> speak() -> first
accepted player append.
```
BUZZ_STT_SPECULATIVE=1 BUZZ_TTS_STREAMING=1 \
BUZZ_TTS_THREADS=4 BUZZ_STT_THREADS=2 \
BUZZ_BENCH_WAV=<48k f32 mono wav> \
cargo test --release -p buzz-desktop --lib huddle::latency_bench -- --ignored --nocapture
```
## Tradeoffs to weigh before promoting any lever to a default
- **Speculative decode**: the speculative buffer has ~1 silent tail
frame vs ~19; observed one CTC wobble ("fail" vs "failed") in 24 turns.
Mitigation if productionized: zero-pad the speculative buffer to match
the flush-path shape.
- **Threads**: defaults stay 1 pending the min-spec (4-core Intel) A/B
flagged in the existing `STT_NUM_THREADS` comment.
- **Streaming at emit<12** is NOT the same waveform — don't ship below
12 without an ear pass.
## Validation
- Full desktop lib suite: **2408 passed / 0 failed** at this head
(`18fab2e1c`).
- buzz-voice suite green; bit-exactness test passes against the
production batch decode.
- Defaults-only bench rerun stays in the baseline family everywhere
except the exact conditioning-cache win (stt 525–532, tts 225–355).
- `cargo clippy --workspace --all-targets -- -D warnings` + fmt clean
(pre-push hook battery green).
Measurement notes with per-lever logs: Eva's workspace,
`RESEARCH/HUDDLE_E2E_LATENCY_OPTIMIZATION_2026_08_12.md` +
`RESEARCH/HUDDLE_E2E_STT_FAKELLM_TTS_BASELINE_2026_08_12.md`.
## Suggested promotion order
1. Conditioning cache (in this PR, always-on, exact).
2. Streaming TTS at emit=12: bit-exact audio, biggest UX win — needs the
env-gate removed + barge-in soak + an ear pass on a real huddle.
3. Speculative decode with silence padding: near-free ~100–150 ms.
4. Threads: after min-spec A/B.
---------
Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Signed-off-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Signed-off-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
Signed-off-by: tlongwell-block <tlongwell@block.xyz>
Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Max <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Co-authored-by: Mari <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
Co-authored-by: tlongwell-block <tlongwell@block.xyz>
## Overview
**Category:** improvement
**User impact:** Link previews appear in the composer and travel as
privacy-safe sender-authored snapshots, so recipients never contact the
linked site merely by opening a conversation.
**Problem:** Cold-cache link paste could freeze the composer before the
URL painted; recipient-side unfurling leaked visits; invalid or
unresolved preview work could interfere with sending or leave dead cards
behind.
**Solution:** Paint pasted links before starting cold resolver work,
resolve only in the sender's composer, attach only complete validated
snapshots at Send, and render authored snapshots without recipient
fallback fetching.
## Behavior
- **Cold paste stays responsive:** bare and angle-bracket URL paste
paths commit the visible link before resolver work begins.
- **Sender-only fetching:** metadata is resolved while composing;
recipients render only the sender-authored snapshot.
- **Send never waits:** pending, failed, invalid, and unsendable
previews are omitted. They do not block or cancel the message.
- **Terminal misses disappear:** failed, timed-out, or 404 resolver
results remove the composer card while preserving visible link text.
- **Display-text links work:** Markdown links such as `[review the pull
request](…)` produce and send the same snapshots as bare URLs.
- **Compact and Rich presentation:** Compact remains the default; Rich
preserves source description line breaks and paragraphs.
- **Immediate draft-wide dismissal:** clicking × immediately hides all
previews for the draft, suppresses links pasted later, and emits only
`["link-preview", "none"]`. No confirmation detour. Suppression resets
after send or clearing the draft.
- **Zero recipient fallback:** missing, stale, malformed, off-relay,
unsupported, or suppressed snapshots remain ordinary visible links;
recipients never regenerate them.
## Implementation
- Resolve previews from deferred composer URL state so paste can paint
first.
- Upload finished preview media to the active community relay and
snapshot only valid, sendable media references.
- Atomically capture ready snapshots at submit time; never append a late
preview after send.
- Validate snapshot and suppression tags in desktop/native and relay
ingestion, rejecting duplicate or mixed forms.
- Render composer previews as stable 55px attachment cards at desktop
and narrow widths.
- Add deterministic E2E coverage for cold paste,
ready/pending/failed/invalid previews, display-text links, multiline
Rich descriptions, immediate dismissal, later-pasted links, and
suppression reset.
## Validation
Validated head: `9807ba8952f190e76153834abf8ab61dd40be5e2`
- Push hooks passed: `check-push-org`, branch skew, desktop check,
mobile tests, desktop tests, Rust tests, and desktop Tauri checks.
- Focused screenshot E2E at the validated head: 5/5 passed across
Compact/Rich composer and recipient states, 800px/420px geometry,
display-text links, multiline descriptions, and immediate dismissal.
- PR CI was triggered for this exact head and is currently running;
completed checks are green at the time of this update.
- Worktree is clean and both PR head and validated branch resolve to
`9807ba895…`.
## Screenshots
### Compact composer
| Loading | Ready |
|---|---|
| 
| 
|
### Rich composer
| Loading | Ready |
|---|---|
| 
| 
|
### Responsive composer
| 800px loading | 800px ready |
|---|---|
| 
| 
|
| 420px loading | 420px ready |
|---|---|
| 
| 
|
### Recipient presentation
| Compact | Rich |
|---|---|
| 
| 
|
### Display-text Markdown link
| Composer | Recipient |
|---|---|
| 
| 
|
### Rich multiline description
### Immediate dismissal
| Before × | Immediately after × |
|---|---|
| 
| 
|
---------
Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
## Summary
- add a visible Stop control for interrupting agent speech
- make push-to-talk available by default while preserving manual mute
controls
- refine agent management, muted audio states, drawer layering, and
return navigation
- suppress duplicate notification sounds for Huddle messages
## Why
Huddles could trap users behind long agent speech, hide useful agent
controls, and leave temporary Huddle state visible after the call. The
drawer also regressed when the terminal substrate began painting behind
the rounded app surface.
## Validation
- `just desktop-ci`
- focused Huddle Playwright coverage for the drawer, speech
interruption, agent picker, and leave navigation
---------
Signed-off-by: kenny lopez <klopez4212@gmail.com>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
## Summary
- open Huddles in a focused companion window with a clean handoff back
to the in-app drawer and backing channel
- redesign the participant film strip, sidebar control, transcript
surface, and themed shell treatment
- preserve microphone and device control across windows, start agent
voice on the first reply, and show agent speaking activity in the film
strip
- give each agent a distinct session voice, beginning with the
configured default, plus compact per-agent text-to-speech and voice
controls
- enroll only agents explicitly mentioned or deliberately added through
an agent panel into the live Huddle roster
- keep temporary Huddle channels out of the sidebar unless the user
explicitly brings one into the main app
- remove Huddle-only avatar policy badges and filter short silence or
noise segments before speech-to-text posts
## Why
The previous flow exposed the temporary channel as product UI, obscured
who was present or speaking, and split transcript and audio state
between the main and companion windows. This keeps backing channels as
implementation details unless a user explicitly brings a Huddle into the
app, while sharing the live conversation and audio lifecycle across both
surfaces. Agent participants now join only after an explicit invitation,
distinct voices make multi-agent Huddles easier to follow, and short
microphone noise no longer becomes stray transcript messages.
## Validation
- `pnpm check`
- `pnpm build:e2e`
- `pnpm exec playwright test tests/e2e/huddle-transcription.spec.ts
--project=smoke` (13 passed)
- Huddle sidebar visibility unit coverage (4 passed)
- focused managed-agent and persona-mention E2E coverage (2 passed)
- `pnpm test` (3,910 passed)
- `cargo clippy --manifest-path desktop/src-tauri/Cargo.toml
--all-targets -- -D warnings`
- `cargo test --manifest-path desktop/src-tauri/Cargo.toml` (2,093
passed, 14 ignored; 3 diagnostics passed)
---------
Signed-off-by: kenny lopez <klopez4212@gmail.com>
Signed-off-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Wes <wesbillman@users.noreply.github.com>
Co-authored-by: Carl <c7ebe626f000404285d3686e1dc74cc07cc60a9754a150041ba132e14bd3e2ec@buzz.block.builderlab.xyz>
## Context
Pocket TTS currently offers bundled reference voices. People also need a
local, private way to add a voice without sending audio to a cloud
service.
## Summary
Add a Pocket voice import flow to Voice settings. Buzz opens the native
file picker, decodes common audio formats in the reusable `buzz-voice`
crate, canonicalizes the selected audio, stores it under a
content-derived identity in app data, selects it, and lets the user
delete it later.
## Changes
- Accept WAV, M4A, MP3, FLAC, OGG, and AIFF files between 2 and 30
seconds, including multichannel sources.
- Decode and downmix accepted audio to canonical mono 32 kHz PCM16 WAV
before hashing and storage.
- Store imported voices behind stable `pocket:imported:<sha256>`
identities and content-addressed files.
- Keep absolute file paths inside the native process and expose only
voice metadata to React.
- Include imported voices in Pocket preview and live huddle playback.
- Add Add voice and delete controls while preserving the bundled Pocket
voice catalog.
- Fall back to Mary when the selected imported voice is deleted.
- Keep durable import, selection, and deletion successful when a live
TTS worker acknowledgement is delayed.
- Preserve bundled voices when optional import metadata is unreadable
and keep failed deletion retryable.
## Related issue
None found.
## Testing
Production decoding was exercised with WAV, M4A with AAC, MP3, FLAC, OGG
Vorbis, and AIFF fixtures. Each format canonicalized to mono 32 kHz
PCM16 WAV. Manual validation in the combined daily-driver build covered
native-picker import, Preview, live-huddle playback, deletion, and Mary
fallback.
## Screenshots
The Voice settings card preserves the bundled Pocket catalog and adds
the local Add voice action.
## Reviewer-reproducible examples
Create common-format fixtures and run them through the production
importer:
```bash
. ./bin/activate-hermit
fixtures="$(mktemp -d)"
ffmpeg -hide_banner -loglevel error -f lavfi -i "sine=frequency=220:duration=3" -ac 2 -ar 44100 "$fixtures/voice.wav"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a aac "$fixtures/voice.m4a"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" "$fixtures/voice.mp3"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" "$fixtures/voice.flac"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a libvorbis "$fixtures/voice.ogg"
ffmpeg -hide_banner -loglevel error -i "$fixtures/voice.wav" -c:a pcm_s16be "$fixtures/voice.aiff"
BUZZ_VOICE_IMPORT_TEST_DIR="$fixtures" \
cargo test -p buzz-voice imports_common_audio_format_fixtures -- --ignored --nocapture
```
Exercise import persistence, synthesis, deletion, and bundled-voice
fallback with an installed Pocket model:
```bash
BUZZ_POCKET_MODEL_DIR=/path/to/pocket-model-bundle \
cargo test -p buzz-voice --test pocket_import_audio \
objective_import_synthesis_delete_and_mary_fallback \
-- --ignored --nocapture
```
Exercise the native-picker boundary, selection, preview dispatch,
deletion, cancellation, and invalid-file states:
```bash
cd desktop
pnpm build:e2e
pnpm exec playwright test tests/e2e/voice-settings.spec.ts --project=smoke
```
---------
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: John Tennant <johnmatthewtennant@gmail.com>
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Signed-off-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: John Tennant <jtennant@block.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub12gtutshhh76rx0jx697f32f9tffd4hhp3hx58fp4x6u4uemkm7sqf8f757 <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Relands **#2467** (extract `buzz-voice` crate) and **#3208** (Pocket
voice settings) onto main, after #3266 and #3180 merged.
## Why a fresh PR
The repo is squash-only with delete-branch-on-merge. Squashing #3266
deleted `jtennant/pocket-tts-2026-04`, which was #2467's base — GitHub
auto-closed #2467 and it cannot be reopened. Squash merges also sever
ancestry, so GitHub's natural merge-base reports phantom conflicts for
the whole remaining stack.
## Content provenance
- Byte-identical to the blessed `jt/buzz-voice-refactor` branch
(`93029c577`, tree `6729e0eff` — reviewed by Dawn (#2467) and Max
(#3208) at exact heads) **except** the three files where #3180 and #3208
genuinely interact.
- Three-file resolution (union of both sides):
- `huddle/mod.rs` — #3180's pipeline re-exports + #3208's
`agent_tts_routing` imports.
- `huddle/state.rs` — `reset_preserving_generation` preserves both
`huddle_generation` (#3180) and `tts_enabled` (#3208); test sets merged
into one `tests` module.
- `desktop/src/testing/e2eBridge.ts` — both switch arms kept; no
duplicate case labels.
## Verification at cf32dacd7
- `cargo test` (desktop/src-tauri, pinned 1.95.0): **2047 + 3 pass / 0
fail** (14 ignored: 8 keychain, 4 real_relay, 2 flag-gated)
- `cargo clippy --all-targets -- -D warnings`: clean; `cargo fmt
--check`: clean
- `cargo check --workspace` (root, includes new `buzz-voice` member):
clean; `cargo test -p buzz-voice`: 5/0
- `pnpm test`: **3885 / 0**; `tsc --noEmit`: clean; lint: clean
The 3180×3208 interaction resolution is getting an independent team
re-review before merge.
Buzz channel: buzz-desktop-voice `fd5fb402-b651-4238-89b1-bb3e2fa4dc96`,
thread `b4798ecc`.
Signed-off-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
Co-authored-by: npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
## Context
Before this change, every huddle initialized with transcription off.
Joining or adding an agent did not enable it, so the agent could not
receive spoken conversation until a person clicked the transcript
control. Starting a huddle from an agent DM could also omit that agent,
and adding an agent who already belonged to the parent channel could
attempt an unnecessary role change and show a warning.
Agent detection uses authoritative huddle membership. A participant
counts as an agent when the ephemeral membership identifies it with the
`bot` role, or when the existing agent identity model identifies the
participant in an agent DM.
## Summary
Buzz now enables transcription once when the first authoritative agent
is present. After that initial automatic action, explicit user control
is authoritative: manual ON or OFF survives membership refreshes,
reconnects, and UI remounts. Removing the last agent does not change the
current transcription state.
Agent-DM huddles enroll the agent automatically. Adding an agent who
already belongs to the parent channel preserves the existing parent role
and completes without a role-mutation warning.
| Scenario | Before | With this change |
| --- | --- | --- |
| First authoritative agent joins or is hydrated | Transcription stays
off | Transcription turns on once |
| User explicitly turns transcription on or off | Manual control exists
without an agent policy | The explicit choice suppresses later automatic
changes |
| Last agent leaves | No defined agent-presence behavior | The current
transcription state remains unchanged |
| Huddle starts from an agent DM | The agent can be omitted | The known
agent is enrolled automatically |
| Added agent already belongs to the parent channel | Buzz can attempt a
role rewrite and warn | Existing parent membership and role are
preserved |
| Transcription is active | The control is not visually distinct | The
control is highlighted and exposes `aria-pressed=true` |
## Changes
- Derive agent presence from authoritative bot-role huddle membership
and known agent-DM identity.
- Apply the one-time auto-enable rule during create, join, membership
hydration, reconnect, pipeline startup, and local agent addition.
- Preserve explicit user state and use huddle-generation guards so stale
asynchronous work cannot alter a replacement huddle.
- Keep backend and React transcription state synchronized, with a
visible and accessible active control.
- Enroll known agent-DM participants and make parent-channel membership
updates idempotent.
- Cover hydration ordering, reconnects, remounts, explicit OFF,
last-agent removal, DM enrollment, existing membership, and active
styling.
## Related issue
None found.
## Testing
Manual validation in `pending-seed` confirmed the product contract:
1. Started a huddle from the owned, running Fizz agent DM.
2. Confirmed the authoritative roster contained the human and Fizz as an
agent.
3. Confirmed transcription enabled without clicking the control: `Stop
transcript`, `aria-pressed=true`, with the highlighted active
background.
4. Turned transcription off and confirmed `Start transcript`,
`aria-pressed=false` remained stable.
5. Removed Fizz while transcription was off and confirmed the state
stayed off.
6. Left the huddle cleanly.
## Screenshots
The same control has distinct active and inactive states.
## Reviewer-reproducible examples
From a fresh checkout:
```bash
pnpm --dir desktop build:e2e
pnpm --dir desktop exec playwright test tests/e2e/huddle-transcription.spec.ts --project=smoke
pnpm --dir desktop exec playwright test tests/e2e/mentions.spec.ts --project=smoke --grep "system agent profile exposes owned agent actions|system agent avatar exposes owned agent actions|owned bot profile exposes message and huddle actions|owned agent mention profile exposes message and huddle actions"
```
The huddle scenario exercises initial authoritative hydration, exactly
one automatic enable, explicit OFF persistence, unchanged state after
last-agent removal, newer events winning over delayed hydration,
agent-DM enrollment, and idempotent parent membership. It also asserts
`aria-pressed` and distinct computed active styling.
---------
Signed-off-by: John Tennant <jtennant@squareup.com>
Signed-off-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
Co-authored-by: npub1jh9wn95s0472h86ahapupaf7m6kx4v9sx2n0atj2hltcfer8k06s5n3pyf <95cae996907d7cab9f5dbf43c0f53edeac6ab0b032a6feae4abfd784e467b3f5@buzz.block.builderlab.xyz>
## Context
Buzz Desktop currently installs an older Pocket TTS model bundle. The
current
bundle changes the tokenizer, learned BOS input, recurrent-state
contract, and
prompt behavior, so updating download URLs alone is not compatible.
## Summary
This PR upgrades Buzz Desktop to the current pinned Pocket TTS model. It
preserves existing product behavior and the hard 50-token model-input
limit
while adding the required runtime support, verified acquisition, and
crash-safe
cache migration.
## Changes
- Pins an immutable Pocket TTS revision, artifact names, exact byte
sizes,
SHA-256 checksums, Mary reference voice, and license.
- Loads the bundle-matched SentencePiece tokenizer, learned BOS
embedding, and
bundle-declared recurrent states.
- Uses one pinned Pocket TTS configuration; no precision or
model-version
selector is added.
- Preserves the resident engine's exact `<= 50` token contract without
changing
Desktop segmentation policy.
- Bumps the Pocket cache manifest to v4, verifies size and checksum
before
adoption, atomically swaps the cache, and recovers the last verified
cache
after interrupted installs, including an incomplete final directory.
- Keeps acquisition, cache migration, worker adoption, and tests within
the
existing Desktop implementation.
- Removes the obsolete model-quality harness, which was coupled to the
superseded production prompt and model layout.
## Related issue
None.
## Testing
Manual listening completed on the exact Desktop build. The updated model
improved speech quality and resolved the phrase-start and sample-onset
artifacts. Reproducible integrity and model checks are below.
## Screenshots
N/A. This changes model installation and speech synthesis, not a visual
surface.
## Reviewer-reproducible examples
### Before and after model identity
```sh
git show 35305bfc8fd456ca9a17caa1ddbfaabd87d46981:desktop/src-tauri/src/huddle/models.rs \
| grep -E 'sherpa-onnx-pocket-tts|TTS_MODEL_VERSION'
git show 211d17c58567448fe7ac95c4fa0ad2b88378849a:desktop/src-tauri/src/huddle/pocket_models.rs \
| grep -E 'MODEL_REPOSITORY|MODEL_REVISION|MODEL_PRECISION|MAX_TOKENS'
```
The target branch identifies the January bundle. The PR branch
identifies the
immutable April revision, INT8 precision, and 50-token maximum.
### Deterministic runtime validation
Use the pinned artifacts listed in `pocket_models.rs` and run the
model-dependent Pocket tests with the model directory supplied by the
test environment. The checked-in long-sentence fixture must preserve its
expected 48 and 44 token split and produce non-silent PCM.
### Manual listening validation
John listened to an untrimmed Pocket TTS onset-stress clip generated
from the exact user-provided passage, with every sentence synthesized
separately and identical 100 ms digital-silence boundaries. The clip
used no leading period, onset trimming, gain adjustment, or loudness
normalization.
The updated model produced better-quality speech and resolved the
start-of-sample artifacts.
---------
Signed-off-by: John Tennant <jtennant@block.xyz>
Signed-off-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
Co-authored-by: John Tennant <jtennant@block.xyz>
Co-authored-by: npub1mprnacetjua2xx3p5eddmhxyk6wv929ymm5py8kd2xfxurxahspqqlgyta <d8473ee32b973aa31a21a65adddcc4b69cc2a8a4dee8121ecd51926e0cddbc02@buzz.block.builderlab.xyz>
**Category:** new-feature
**User Impact:** Users can create, download, and verify a
password-protected backup of their private identity from desktop
Settings.
**Problem:** Buzz does not currently give signed-in users a
Settings-based path to protect or validate their private identity
independently of onboarding. **Solution:** Add a focused backup menu to
the private-key row, keep encryption and verification local in Rust, and
preserve completed encrypted backups briefly so native saves can be
retried without repeating encryption.
<details>
<summary>File changes</summary>
**desktop/src/features/settings/**
Adds the background backup lifecycle, create and test dialogs,
private-key menu integration, password handling, and focused unit
coverage.
**desktop/src/features/onboarding/ui/NsecMaskedDisplay.tsx**
Extends the masked private-key display with reusable overflow-menu
actions used by Settings.
**desktop/src/app/App.tsx**
Mounts the backup provider at app scope so encryption and save work
survive closing Settings or the modal.
**desktop/src/shared/api/tauriIdentity.ts**
Adds typed desktop bindings for local backup creation, save, selection,
and verification.
**desktop/src-tauri/src/key_backup.rs and
desktop/src-tauri/src/commands/identity.rs**
Implements local NIP-49 encryption, password generation, file handling,
and public-identity-only verification results.
**desktop/src-tauri/src/egress_guard.rs and guarded call sites**
Blocks encrypted secret material from relay, websocket, snapshot,
sharing, and huddle egress paths.
**desktop/src-tauri tests and fixtures**
Covers encryption, verification, file behavior, and fail-closed
no-egress protections.
**desktop/src/testing/e2eBridge.ts, desktop/tests/, and
desktop/playwright.config.ts**
Expands the mock native bridge and browser coverage across create,
retry, expiry, and current/different-identity verification states.
**desktop/src-tauri/Cargo.toml, Cargo.lock, and assets**
Adds the local cryptography/password-generation dependencies and
embedded short-word list.
</details>
## Reproduction steps
1. Run the desktop app and open **Settings → Profile → Identity**.
2. Open the private-key overflow menu and choose **Create backup**.
3. Enter or generate a valid password, submit, and confirm progress
continues if the dialog or Settings is closed.
4. Save the resulting `.ncryptsec` file; cancel and retry to confirm the
temporary download remains available.
5. Choose **Test backup**, select the file, enter a wrong password, then
retry with the correct password.
6. Confirm success identifies whether the backup matches the current
identity and displays only the public `npub`.
## Screenshots
| Settings identity | Private-key menu | Create backup |
|---|---|---|
| <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/981e391b-6829-4081-95ca-ca75a369de71"
/> | <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/7972c68e-7635-47d8-b0ad-9639390d3e6c"
/> | <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/4709c8f7-cf02-46f1-bec9-b3f98fe56fb2"
/> |
| Encrypting | Download available | Test success |
|---|---|---|
| <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/1ac3e934-2b4b-4135-bae6-126c715c8c59"
/> | <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/cb6f07ee-a16f-44a5-b9a0-6b9fe0e4d40d"
/> | <img width="1280" height="720" alt="image"
src="https://github.com/user-attachments/assets/ea58b1b1-966c-46aa-8d59-92c9f06a25bd"
/> |
Visual review and additional states: [Buzz
thread](buzz://message?channel=50ca7ef1-201e-4159-9499-40de3964b7c3&id=87eceb5f0f82fd50c32e560de3d35be48e293760f6620718aafdcef289d475fe)
---------
Signed-off-by: Taylor Ho <taylorkmho@gmail.com>
Co-authored-by: npub1223z34hd7vtwc6qj4s7flsxkj644nlre2nthu7lrrmkumhu3xddsrx9r6w <52a228d6edf316ec6812ac3c9fc0d696ab59fc7954d77e7be31eedcddf91335b@buzz.block.builderlab.xyz>
## Summary
- Preserve plain-string errors returned by Tauri and show an actionable
message when huddle audio is unavailable in the relay deployment.
- Use one error formatter across channel, timeline, wave, and profile
huddle actions while preserving other relay and device errors.
- Complete the huddle lifecycle when audio setup fails after publishing
a start event, preventing peers and reloaded clients from reconstructing
a phantom active huddle.
- Cover unavailable-audio formatting and START → rollback-END
reconstruction with regression tests.
## Behavior
The Tauri huddle commands reject with a plain string. Several desktop
toast call sites only preserved JavaScript `Error` objects, so the relay
message was discarded and replaced with “Failed to join huddle.”
The desktop now recognizes `huddle_audio_unavailable` and the current
relay message, then shows:
> Huddle audio isn’t available on this server. Ask an administrator to
turn it on.
Other relay and device messages remain intact, including microphone
errors.
`start_huddle` also publishes `KIND_HUDDLE_STARTED` before audio setup.
If setup fails, rollback now publishes `KIND_HUDDLE_ENDED` through the
normal end-and-archive path before resetting local state. This makes the
failed start observable to lifecycle reconstruction and prevents stale
join affordances.
## Checks
- `cd desktop && pnpm test` — 3,405 passed
- `cd desktop/src-tauri && cargo test` — 1,560 passed, 13 ignored; 3
diagnostic tests passed
- `cd desktop && pnpm exec playwright test tests/e2e/channels.spec.ts
--project=smoke --grep 'huddle rollback end event'` — passed
- `just desktop-tauri-check`
- `cd desktop && pnpm typecheck`
- `cd desktop && pnpm check`
- Pre-commit and pre-push hooks
