fix: audio pipeline — 16 kHz AudioContext, 4096-sample buffering, SERVER_READY handshake
Root causes of disconnection and slow transcription:
- AudioWorklet was firing every 128 native samples (~48 kHz), sending
~375 tiny WebSocket messages/sec. Server flooded with tiny frames
during silence → keepalive ping timed out → connection dropped.
- JS resampling 48 kHz → 16 kHz added CPU overhead on every chunk.
- Audio started on ws.onopen before server sent SERVER_READY, so early
frames were dropped.
Fixes:
- audioWorklet.js: accumulate 4096 samples before posting (256 ms/chunk
at 16 kHz, ~4 messages/sec), transfer ArrayBuffer zero-copy.
- transcriptionService: AudioContext({ sampleRate: 16000 }) — browser
handles native resampling, no JS resampling needed. Remove
resampleTo16kHZ entirely.
- Wait for SERVER_READY message before calling setupAudioProcessing().
- Send 'END_OF_AUDIO' string on stop so server can finalise last segment.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
This commit is contained in:
+17
-3
@@ -1,12 +1,26 @@
|
||||
class AudioProcessor extends AudioWorkletProcessor {
|
||||
constructor() {
|
||||
super();
|
||||
this._buffer = new Float32Array(4096);
|
||||
this._bufferIndex = 0;
|
||||
}
|
||||
|
||||
process(inputs) {
|
||||
const input = inputs[0];
|
||||
if (input.length > 0) {
|
||||
const audioData = input[0];
|
||||
this.port.postMessage(audioData);
|
||||
const samples = input[0];
|
||||
for (let i = 0; i < samples.length; i++) {
|
||||
this._buffer[this._bufferIndex++] = samples[i];
|
||||
if (this._bufferIndex >= 4096) {
|
||||
// Transfer ownership (zero-copy) to main thread
|
||||
this.port.postMessage(this._buffer.buffer, [this._buffer.buffer]);
|
||||
this._buffer = new Float32Array(4096);
|
||||
this._bufferIndex = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor('audio-processor', AudioProcessor);
|
||||
registerProcessor('audio-processor', AudioProcessor);
|
||||
|
||||
Reference in New Issue
Block a user