mirror of
https://github.com/block/buzz.git
synced 2026-10-29 13:31:44 +00:00
7ebe3ea699
## Problem and intent Buzz huddles have been clipping speech into fragments: one live turn arrived as the single letter “M,” and another ended mid-sentence. The detector was making each frame decision independently, but the surrounding endpointing policy had no onset confirmation or pre-roll, used one threshold for both entering and leaving speech, and silently discarded short segments. This PR keeps Earshot 1.1.0 and fixes that policy around it. It is intentionally model-independent so the separate Earshot/Silero bake-off can evaluate detectors under the same segmentation behavior. ## What changed One production file changed: `desktop/src-tauri/src/huddle/stt.rs` (**+367/−79** versus `main`). - Add a pure `VadEndpoint` state machine around Earshot probabilities. - Preserve 256 ms (16 frames) of pre-roll before confirmed onset. - Require three consecutive frames above 0.50 to enter speech. - Use 0.35 to leave speech, preserving hysteresis-band audio. - Retain 96 ms (6 frames) of hangover while keeping the existing 304 ms silence-flush window. - Make short-segment drops visible in logs instead of silent. - Add boundary, onset, hysteresis, hangover, drop-path, and PTT policy tests. - Bind the thresholds to Earshot 1.1.0 in source. Earshot 1.2.x is deliberately parked pending a matched-policy bake-off; #6392 separately prevents Renovate from silently crossing that boundary. ## Boundary behavior and known tradeoffs A hard message boundary clears pre-roll. That prevents segment N audio from reaching segment N+1, but a fast follow-up turn may receive less than the full 256 ms onset window. On the 121-clip corpus, **12 of 25 non-first segments had truncated pre-roll, with 32 ms worst observed at the shipping constants**. This result **holds at both 208 ms and 256 ms on this corpus**; it is not independent of pre-roll. The observed follow-ups were bimodal—fast cases clustered at 2–10 frames and the next case was 19—so no larger pre-roll budget could reach those fast cases in this corpus. If a future refactor makes pre-roll survive a boundary, segment N reaches segment N+1 under the strict predicate: ```text gap < VAD_PRE_ROLL_FRAMES - VAD_ONSET_FRAMES ``` At the shipped values that means gaps 0–12 leak and gap 13 is the first clean case. Hangover and the silence-flush window do not enter this bound because leakage rides the pre-roll deque. Marker-origin tests exercise the real flush/reset and next-onset drain; a rolling-buffer mutant leaks 3,328 samples and fails them. Other known behavior: - `voiced_frames` now counts frames above the 0.35 exit threshold once speech begins, not only frames above 0.50. Corpus drop counts measured under `> 0.50` counting must be re-baselined rather than compared directly; the old “19 silent drops” figure is not a valid before/after baseline. - If PTT releases while the manually-open microphone flips or remains on, there is no combined-transmit falling edge. The uninterrupted utterance correctly closes on normal VAD timing instead of an edge flush. - The GUI-to-`push_audio_pcm` wiring is unchanged. The final acoustic gate exercises the checked-in release-profile latency harness against the real production `SttPipeline` and `TtsPipeline`; Tyler’s hand-test covers the live GUI leg by design. ## Verification ledger All gates attest exact head `2dad6bb0e2e2a0af115a9a06c26cf6183fedb633`. - **Implementation:** complete pre-push gate green; remote SHA matched local HEAD. - **Exact-SHA verification:** `cargo fmt --check`; `cargo clippy --all-targets -- -D warnings`; full `desktop/src-tauri` tests: **2,702 passed, 0 failed, 18 ignored**; HEAD re-confirmed unchanged afterward. - **Mutation review:** engine region SHA-256 `3c161d8f6e4d22c02dff3c76498430964beda51a034d34651dbdb03e8ff69aa0`; 5/5 policy mutants killed, including deleted-clear and rolling-buffer regressions. Direct gap sweep leaked at 0–12 and was clean from 13. - **Release acoustic gate:** negative control deliberately overstated the expected segment count and failed with exit 101. Soft/short onset produced 1/1 segment (`“I'm happy.”`). Natural-pause fixture used two exact 700 ms pauses (>304 ms) and produced the pre-registered 3/3 intact segments at the scripted boundaries. - **Release CPU beside Pocket TTS:** soft fixture median 32.1%, p95/max 56.8% (5 sparse samples); natural fixture median 14.55%, p95/max 17.2% (8 sparse samples). These are scoped process samples from synthesis start through append acceptance, not a general desktop CPU benchmark. Local review receipts (not committed to the repository): - `.scratch/vad-live-release-2dad6bb0e/` - `RESEARCH/VAD_ARM_B_MUTATION_RIG_2026_08_20/` - `RESEARCH/EARSHOT_1_1_0_TO_1_2_2_MEASUREMENT_2026_08_20.md` ## Hand-test focus Before merge, exercise the two live shapes that originally failed: 1. Soft short openers such as “M” and “yes.” 2. Natural mid-sentence pauses. Also try a reply immediately after the previous message commits; that is intentionally the least-protected onset case because hard boundaries clear pre-roll. --- Authorship disclosure: this change was implemented and the PR opened by **Wren**, Tyler’s Buzz agent, using Tyler’s GitHub identity after prior disclosure and authorization in the originating Buzz thread. Tyler remains the accountable human reviewer/operator. --------- Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>