jerry/buzz

mirror of https://github.com/block/buzz.git synced 2026-11-05 08:51:10 +00:00

Files

Tyler 6039fed565 fix(desktop): restore human barge-in over agent TTS in huddles (#6431 )

## Problem

Humans cannot interrupt agents in huddles: agent TTS keeps playing over
a talking human (reported by @tlongwell-block, 2026-08-20). The full
requirement: **any human talking — local or remote, any input mode, any
audio rig — must interrupt any agent on the huddle.**

## This is a restoration, not a new feature

- **`b29c8cdaa` (#4281, 2026-08-04) deleted working local VAD
barge-in.** The pre-image shipped `BARGE_IN_DEBOUNCE_FRAMES = 20` (320
ms sustained speech cancels TTS), live at the production call site
(`pipeline.rs` passed `Some(tts_cancel)` unconditionally). The same day,
`ce3cf3cd2` (#4694) flipped the default input mode from VAD to
push-to-talk, which masked the loss.
- **`068a83b09` (#5671, 2026-08-13) removed the remaining TTS-awareness
plumbing from STT** (deliberately, to keep transcribing over agent audio
— a good change). Consequence: this PR is a re-plumb through the
`PlaybackCoordinator` from #6341, not a revert.
- A gap that **never** worked is also closed: PTT-mode users who open
the mic via the mute button (`manual_mic_unmuted` postdates the
deletion) transcribed fine but could not barge in.

## Design

All floor state lives under the single `PlaybackCoordinator` lock —
onset acceptance, epoch bump, synthesis invalidation, player
replacement, and output lease are one committed transition. No cancel
flag can be observed out of order with the playback state it describes.

- **Human floor**: local + per-peer remote ownership with epoch
invalidation. Onset cancels playback by queue replacement; late
synthesis for a stale epoch cannot append or restart.
- **Local onset (VAD)**: on an **isolated output route** (all CoreAudio
output-stream terminals report headphones), a confirmed short onset
interrupts immediately — no echo path exists. On a **coupled route**
(speakers, unknown, virtual, mixed), the restored **20-frame / 320 ms
sustained-speech debounce** discriminates a real human from speaker
bleed; the deleted code's comment records that 80 ms was tried and
false-triggered on laptop speakers. Route classification is queried
fresh at each onset (never cached — default-device re-routing mid-huddle
would strand a stale verdict).
- **Mic-open gate is per-frame**: barge-in observes on any frame where
the mic is actually open — pure VAD mode, or PTT with the mic manually
unmuted (key-held frames defer to the shortcut's own cancel).
- **Remote onset**: sustained non-DTX frames from a peer enter the same
persistent floor (independent of whether playback is live — a human
speaking while TTS is idle blocks late-arriving synthesis from starting
over them). Release on sustained DTX/absence, peer departure, and
recv-loop exit, with guards so a vanished peer cannot wedge the floor.
- **Output lease**: accepted appends renew an `Active` lease;
drain/cancel/onset start a 100 ms tail hangover (conservative against
measured ~12 ms/~1 ms CoreAudio tails), so speaker-tail bleed in the
just-drained window cannot self-trigger the coupled path.

## Known limitations (phase 2 pointers)

- Coupled-route mid-output barge-in pays the 320 ms debounce; a
playback-reference echo discriminator would shorten it.
- A speakers-rig participant's bleed can enter their mic and hold the
floor for other machines (bounded by release debounce).
- Non-macOS routes classify as coupled (fail-safe).

## Verification

- Full `buzz-desktop --lib` suite at head `b0459ae4a`: **2707 passed, 0
failed, 18 ignored** (pinned cargo 1.95.0).
- Exact CI recipe `just desktop-tauri-clippy`: PASS at head; base arm at
merge-base `b728a2af3` confirms the two
`#[allow(clippy::too_many_arguments)]`s cover branch-caused threshold
crossings (human_floor threading), not inherited noise.
- Regression tests pin: 20-frame threshold + reset-on-gap, short-coupled
rejection, sustained-coupled acceptance, coupled-idle acceptance,
remote-idle delayed-TTS rejection, output-tail hangover boundary (during
= rejected, after = accepted), isolated onset, per-frame mic-open gate
truth table, PTT+manual-unmute sustained coupled acquisition.
- `LocalBargeIn::observe` is covered as two joined halves (gate truth
table in `local_barge_in.rs`, floor transition in `tts_playback.rs`);
its body is a straight-line wrapper around a live CoreAudio query, left
uninjected deliberately.
- Coverage precision (mutation-verified):
`manual_open_ptt_sustained_speech_acquires_coupled_floor` pins the gate
→ 20-frame debounce → acquire → floor-blocked chain on the
coupled-**idle** cell. The live-output override leg is carried by
`sustained_coupled_speech_overrides_live_output_suppression`
(tts_playback.rs); the joiner's `sustained_coupled` argument is not
load-bearing there (flipping it to `false` leaves the test green, while
shortening the debounce by one frame turns it red).
- Live arms in progress: pre-regression build `b29c8cdaa^` staged to
confirm the deleted mechanism worked; two-endpoint remote-leg test
pending a second human.

## Commits

1. `fb681a5a9` — restore human barge-in (coordinator floor, lease, route
isolation, remote floor, 320 ms coupled debounce)
2. `b4265418e` — enable barge-in for manually opened mics (per-frame
gate; closes the PTT-open-mic gap)
3. `886489f2b` — extract local barge-in policy module (file-size
ratchet; also hoists the CoreAudio route query from per-frame to
per-onset, named in the commit message)
4. `b0459ae4a` — two targeted clippy allows for the widened worker
signatures

## Credits

Built by **Wren**. Regression archaeology and the PTT-open-mic gap by
**Dawn** (who also killed her own first fix as vacuous and caught a
clippy blocker before it hit CI). Review blockers (coordinator
serialization, idle-onset floors, output lease) by **Mari**. Live rig
verification by **Max**. Coordination and verification by **Eva**.
Opened by Eva with Tyler's explicit direction; commits carry agent
trailers.

---------

Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>

2026-08-21 12:51:55 -04:00