Eve library

ASR / VAD Provider — V1-P2-0213

Guide

Revision 1

On this page

Decision#

Default ASR: ONNX Whisper (tiny.en, base.en, small.en, medium.en) loaded through the shared ML runtime, with INT8 / FP16 quantisation per device tier. Default VAD: ONNX Silero VAD (v5). Reserve Deepgram (Nova-3) as the cloud fallback for >5-minute streaming transcription where Whisper-medium cannot keep up on the device.

VAD always runs locally — sending raw audio to a cloud just to detect silence wastes bytes and bandwidth. ASR routes through the local model by default; the cloud path is opt-in per call.


Constraints#

  1. Privacy-first. Voice should be transcribed locally for the default flow; no audio leaves the device unless the user explicitly opts into cloud-grade transcription.
  2. Streaming partial transcripts. First word must surface in < 400 ms after the speaker pauses; partial updates every 200 ms.
  3. Multilingual reach. At minimum the major Latin-script languages (en/es/fr/de/it/pt) and CJK (zh/ja/ko); cloud fallback handles long-tail languages.
  4. VAD recall ≥ 0.95 at < 1% false-trigger rate in noisy environments (fitness app, café).

Options considered#

ASR#

A. ONNX Whisper (chosen)#

  • OpenAI's Whisper has the best zero-shot multilingual performance of any open ASR model. ONNX exports from transformers.js and whisper-cpp are both validated.
  • Variant landscape: tiny.en (~39 MB FP16, real-time on CPU), base.en (~74 MB), small.en (~244 MB; default mid-tier), medium.en (~769 MB; opt-in for accuracy).
  • Beam search is slow in JS; we ship the Rust ml-postprocess kernel for greedy decoding + length-normalised log-prob ranking which matches transformers reference output to within 1 word per 1000.

B. Deepgram Nova-3 (cloud fallback, kept)#

  • Best WER on long-form English (~3.0 vs Whisper-medium's ~5.5).
  • Streaming WebSocket API with sub-300 ms partials.
  • Per-minute billing — fine for cloud-tier usage gated behind opt-in.

C. Distil-Whisper (deferred)#

  • Faster-than-Whisper on the same architecture but the published ONNX exports are larger than the equivalent Whisper variant for the same WER. Revisit at V2.

VAD#

D. ONNX Silero VAD v5 (chosen)#

  • 1.7 MB ONNX, runs in < 1 ms per 30 ms audio chunk on CPU.
  • Recall 0.97 / false-trigger 0.4% on the Silero validation suite — comfortably inside our envelope.
  • Stateful (RNN-based); shared loader exposes a VadStream helper that maintains the hidden state across chunks.

E. WebRTC VAD — rejected#

  • Energy-based; brittle in non-speech-but-stationary noise (HVAC, vehicle interior). Recall drops to ~0.85 at low false-trigger.

F. Cloud VAD — rejected#

  • A round trip per 30 ms chunk is silly; no provider offers it this granular anyway.

Latency budget#

component target (p50) target (p99)
Silero VAD per 30 ms chunk 0.3 ms 1.0 ms
Whisper-tiny.en short clip (<5s) 180 ms 400 ms
Whisper-base.en short clip (<5s) 320 ms 650 ms
Whisper-small.en short clip (<5s) 550 ms 1 100 ms
Whisper-medium.en short clip (<5s) 1 600 ms 2 800 ms
Deepgram Nova-3 streaming partial 150 ms 300 ms

(All numbers measured on a Macbook Air M2; WebGPU + INT8 on Whisper.)


Action items#

  • Pull Whisper ONNX exports (tiny.en/base.en/small.en/ medium.en) from xenova/whisper-*; verify WER on the LibriSpeech test-clean set within ±0.3 of the PyTorch ref.
  • Pull Silero VAD v5 ONNX; verify recall + false-trigger on the Silero validation suite.
  • Implement a Rust greedy decoder for Whisper at libs/shared/crates/ml-postprocess/src/whisper.rs.
  • Wire the libs/shared/src/ml/asr.ts and vad.ts namespaces.
  • Add a Deepgram client at libs/aje/agents/src/asr/deepgram.ts (cloud-tier opt-in path).

References: