Replace LiveKit with WebSocket Opus audio relay (#326)

This commit is contained in:
tlongwell-block
2026-04-15 15:02:27 -04:00
committed by GitHub
parent 3d48be8777
commit 811c8f3f07
34 changed files with 1644 additions and 1696 deletions

View File

@@ -1,19 +1,24 @@
// AudioWorklet processor — runs in the AudioWorklet thread.
// Accumulates PCM Float32 samples and sends 100ms batches to the main thread.
// Accumulates PCM Float32 samples and sends 20ms batches to the main thread.
//
// ASSUMPTION: AudioContext runs at 48 kHz (Tauri WebView default on desktop).
// The 960-sample buffer = exactly one 20ms Opus frame at 48 kHz. If the
// sample rate differs (e.g. 44.1 kHz), frames will be slightly misaligned
// with the Opus encoder. getUserMedia should request sampleRate: 48000.
//
// Supports push-to-talk (PTT) gating: when `this.transmitting` is false,
// incoming audio frames are discarded and the buffer is reset. The main thread
// sends `{ type: 'ptt', active: boolean }` messages to toggle transmission.
// Default: transmitting=true (open mic for VAD mode compatibility).
//
// Note: when the worklet is disconnected, any partial buffer (< 4800 samples)
// is silently dropped. The last ~100ms of speech may be lost on huddle leave.
// Note: when the worklet is disconnected, any partial buffer (< 960 samples)
// is silently dropped. The last ~20ms of speech may be lost on huddle leave.
// This is acceptable for voice — losing a partial syllable at disconnect is
// imperceptible compared to the natural end-of-conversation flow.
class SttTapProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.buffer = new Float32Array(4800); // ~100ms at 48kHz
this.buffer = new Float32Array(960); // ~20ms at 48kHz
this.offset = 0;
this.transmitting = true; // default: open (VAD mode). PTT mode sets false on init.
@@ -48,7 +53,7 @@ class SttTapProcessor extends AudioWorkletProcessor {
if (this.offset >= this.buffer.length) {
// Transfer ownership for zero-copy
this.port.postMessage(this.buffer, [this.buffer.buffer]);
this.buffer = new Float32Array(4800);
this.buffer = new Float32Array(960);
this.offset = 0;
// Handle leftover samples