fix: audio pipeline — 16 kHz AudioContext, 4096-sample buffering, SERVER_READY handshake

Root causes of disconnection and slow transcription:
- AudioWorklet was firing every 128 native samples (~48 kHz), sending
  ~375 tiny WebSocket messages/sec. Server flooded with tiny frames
  during silence → keepalive ping timed out → connection dropped.
- JS resampling 48 kHz → 16 kHz added CPU overhead on every chunk.
- Audio started on ws.onopen before server sent SERVER_READY, so early
  frames were dropped.

Fixes:
- audioWorklet.js: accumulate 4096 samples before posting (256 ms/chunk
  at 16 kHz, ~4 messages/sec), transfer ArrayBuffer zero-copy.
- transcriptionService: AudioContext({ sampleRate: 16000 }) — browser
  handles native resampling, no JS resampling needed. Remove
  resampleTo16kHZ entirely.
- Wait for SERVER_READY message before calling setupAudioProcessing().
- Send 'END_OF_AUDIO' string on stop so server can finalise last segment.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
This commit is contained in:
2026-05-23 07:26:20 +00:00
co-authored by Claude Sonnet 4.6
parent 308889937c
commit 4139eb8fd3
2 changed files with 44 additions and 36 deletions
+17 -3
View File
@@ -1,12 +1,26 @@
class AudioProcessor extends AudioWorkletProcessor {
constructor() {
super();
this._buffer = new Float32Array(4096);
this._bufferIndex = 0;
}
process(inputs) {
const input = inputs[0];
if (input.length > 0) {
const audioData = input[0];
this.port.postMessage(audioData);
const samples = input[0];
for (let i = 0; i < samples.length; i++) {
this._buffer[this._bufferIndex++] = samples[i];
if (this._bufferIndex >= 4096) {
// Transfer ownership (zero-copy) to main thread
this.port.postMessage(this._buffer.buffer, [this._buffer.buffer]);
this._buffer = new Float32Array(4096);
this._bufferIndex = 0;
}
}
}
return true;
}
}
registerProcessor('audio-processor', AudioProcessor);
registerProcessor('audio-processor', AudioProcessor);