Eve library

Speaker Identification Provider — V1-P2-0214

Guide

Revision 1

On this page

Decision#

Use ONNX ECAPA-TDNN (192-dim embedding) as the default speaker-id model. Reserve cloud APIs (Pindrop, AWS Voice ID) for high-stakes authentication where a fraud-grade verification network behind the embedding is required. Reject Resemblyzer.

ECAPA-TDNN runs locally through the shared ML runtime, embeddings are stored in the user's local Iris memory partition, and matching is done client-side via cosine similarity. The cloud path is gated behind explicit user opt-in for any flow that authorises a financial / identity action on the speaker's voice alone.


Constraints#

  1. Privacy. Voiceprints are biometric — they cannot leave the device for the default flow.
  2. Embedding stability across sessions. Cosine distance between two utterances of the same speaker must drop below 0.25 with ≥ 90% recall.
  3. Diarisation friendly. Embedding extraction must be fast enough (≤ 50 ms per 1 s of audio) that we can extract per-segment embeddings during a multi-speaker conversation without falling behind real-time playback.

Options considered#

A. ONNX ECAPA-TDNN (chosen)#

  • 192-dim embedding, ~17 MB FP16. Inference: ~25 ms per 1 s of audio on WebGPU, ~80 ms on WASM SIMD.
  • Best EER (~0.86%) of any portable speaker-id model on VoxCeleb1-O.
  • Trained on VoxCeleb1+2 with multilingual augmentation; generalises to non-English speakers without fine-tuning.
  • ONNX export from speechbrain (spkrec-ecapa-voxceleb) is the industry reference — verified parity to the PyTorch original.

B. Resemblyzer (rejected)#

  • Older GE2E-style 256-dim embedding. EER ≈ 6%, several × worse than ECAPA-TDNN on the same eval.
  • No first-class ONNX export; runs through PyTorch only.

C. Cloud APIs (kept for high-stakes auth)#

  • Pindrop / AWS Voice ID / Azure Speaker Recognition include liveness detection, deepfake-resistance, and cross-channel voiceprint matching. Useful for "voice is the second factor in a financial login" flows.
  • Per-verification billing, so cost-bounded.
  • Privacy gate: explicit opt-in, voiceprint stored in the cloud provider's enclave, not in our database.

Output schema#

ts
export interface SpeakerEmbedding {
  /** 192-dim L2-normalised float vector. */
  readonly vector: Float32Array;
  /** Source clip duration in ms (for confidence weighting). */
  readonly clipDurationMs: number;
  /** Embedding model + version, for compat checks. */
  readonly model: 'ecapa-tdnn-v1';
}

export function speakerSimilarity(
  a: SpeakerEmbedding,
  b: SpeakerEmbedding
): number; // cosine in [-1,1]

The same SpeakerEmbedding shape is produced by the cloud path so downstream code can mix-and-match where the voiceprint store allows.


Action items#

  • Pull spkrec-ecapa-voxceleb from speechbrain; export to ONNX via torch.onnx.export; verify EER on VoxCeleb1-O within ±0.1% of the published number.
  • Quantise to FP16 + INT8; check both into the registry with SHA-256.
  • Wire the libs/shared/src/ml/speaker-id.ts namespace.
  • Implement liveness gate: reject embeddings extracted from clips < 0.5 s or with VAD-detected silence > 80%.

References: