Eve library

OSHUN TTS Gateway Audit

Guide

Revision 1

On this page

Status: Final v1.0 (audit only — no code changes) Date: 2026-04-26 Owner: eng-psyche Scope: All TTS-relevant source under libs/iris/, libs/psyche/, libs/yemaya/, libs/metis/multimedia/, and shared types under libs/contracts/src/common/voice-profile.ts Backlog item: V1-P2-0140 (in docs/releases/v1/specs/todos-p2.md §9)

This document inventories every TTS surface in the OSHUN monorepo, identifies what already exists, and defines the canonical gateway contract that V1-P2-0141 must implement at libs/contracts/src/tts/. Every wiring task V1-P2-0145 … V1-P2-0156 resolves against the contract proposed in §6.

The original P2 task names libs/aje/voice-providers as the audit anchor. That directory does not exist — see §2.1. The actual TTS surface is spread across four domains. This audit treats the union of those four domains as the audit anchor and proposes consolidating the contract under @oshun/contracts/tts (not under @aje/voice-providers) because consent, watermarking, and cloning live closer to Psyche/Iris than to Aje (which is the on-chain / Web3 domain).


1. Executive Summary#

There is no canonical TTS client in the monorepo. Every consumer re-implements its own provider abstraction, voice metadata shape, error hierarchy, and synthesis loop. Every provider integration in main today is a stub: returns synthesized sine-wave audio or AUDIO|<provider>|... template strings; no provider HTTP/SDK calls are issued.

Concern State Evidence
Canonical entrypoint (OshunTTSClient) Missing libs/contracts/src/tts/ does not exist. Only voice-profile.ts describes voice metadata.
Provider HTTP transport Missing (every provider) ElevenLabs, Cartesia, Tavus, Azure-Neural and OpenAI-TTS-1-HD all stubbed; no fetch() / SDK calls in synthesise paths.
Synthesis request schema Missing Every consumer defines its own SynthesisOptions/TTSRequest shape.
Synthesis response schema Partial SynthesisResult exists in tts-factory.ts but is provider-agnostic and unused by other consumers.
Streaming contract Partial StreamingSynthesis in iris/multimodal/voice/synthesis defines chunks; no other consumer streams.
SSML support Per-consumer reimplementation Metis (generateSSML static), Yemaya (SSMLEngine), Iris (provider-specific). No shared parser.
Voice cloning consent Partial (two of four domains) Iris ElevenLabs provider validates hasRights && hasOwnerConsent; Psyche VoiceCloning validates sample quality only.
Voice metadata Implemented libs/contracts/src/common/voice-profile.ts has VoiceProfileSchema (Zod, ≈868 lines) — already canonical.
Cost / budget enforcement Yemaya-only TTSFactory tracks per-hour spend and emits budget-exceeded events; not surfaced in any other library.
Retry / backoff Missing No provider has retry-on-5xx or Retry-After honour.
Rate-limit handling Missing No provider wires HTTP 429 → backoff.
Error taxonomy Per-provider (incompatible) ElevenLabsError, CartesiaError, generic Error elsewhere. No shared enum.
Word-/char-level timings Iris-only ElevenLabs and Cartesia adapters emit wordTimings; Metis/Yemaya/Psyche do not.
Watermarking Schema-only VoiceProfileSchema enforces watermark requirement on cloned voices; no encoder/embed step exists in any provider.
Disclosure UX (synthetic voice notice) Schema-only disclosureRequired field exists; no consumer renders the notice yet.
OpenTelemetry spans Missing No tts.synthesize spans emitted anywhere.

Bottom line: V1-P2-0141 … V1-P2-0144 must (a) create @oshun/contracts/tts, (b) define the request/response/streaming/error/voice- cloning shapes proposed in §6, (c) replace every per-consumer abstraction with the canonical client, and (d) keep the existing VoiceProfile schema as the single source of truth for voice metadata.


2. Surface Inventory#

2.1 Domains and directories#

The P2 task description names libs/aje/voice-providers. That directory does not exist. Direct directory enumeration of libs/aje/ returns 28 sub-domains (account-abstraction, agents, appchains, bitcoin, bridges, chains, contracts, core, database, defi, depin, gaming, governance, identity, intents, nft, nodes, oracles, payments, predictions, privacy, restaking, rwa, sdk, security, settlement-escrow, storage, sui-move, wallets, zkp); none are voice-related. The surface actually in scope:

Domain Sub-library Role in TTS pipeline
Metis libs/metis/multimedia/src/audio/tts-engine.ts Lesson-narration synthesis with batch + queue
Yemaya libs/yemaya/tts-integration/src/tts-factory/ Multi-provider orchestrator with cost budgeting and presets
Yemaya libs/yemaya/podcast-generator/src/podcast-factory/ Long-form podcast TTS pipeline
Iris libs/iris/multimodal/voice/synthesis/src/ Real-time speech synthesis with streaming + provider adapters
Iris libs/iris/multimodal/voice/conversation/src/ Full-duplex conversation simulation
Psyche libs/psyche/tavus-tts/src/ Tavus-backed voice cloning, voice switch manager
Psyche libs/psyche/tavus-pipecat/src/processors.ts Pipecat pipeline TTS processor (placeholder)

2.2 Existing canonical types#

libs/contracts/src/common/voice-profile.ts (≈868 lines, Zod) is the only shared TTS-adjacent contract that is production-quality today. It defines:

Symbol Shape
VoiceProfileSchema Voice metadata: id, slug, name, status, provider, providerVoiceId, languages, styles, prosody, consent, watermark, quality, abuseRisk, disclosureRequired
VoiceConsentSummarySchema required, status: 'granted'|'pending'|..., allowedUseScopes[], expiresAt, revokedAt, witnessed
VoiceWatermarkSummarySchema required, status: 'embedded'|'verified'|..., algorithm, embeddedAt
VoiceProsodySettingsSchema stability, similarityBoost, style, useSpeakerBoost, speed, pitch, clarity, expressiveness, volume
VoiceQualityMetricsSchema signalToNoiseDb, clarity, naturalness, consistency, emotionAccuracy, overallScore, grade A..F, tier, certified
VoiceAbuseRiskSchema score 0..1, safetyControl, rawDownloadAllowed, requiresModeration
Cross-field invariants (a) cloned voices require consent + watermark, (b) active cloned voices require granted consent and embedded/verified watermark, (c) revoked voices require revoked consent, (d) premium voices require quality.tier='premium'

Decision: the canonical TTS client (§6) must reuse VoiceProfile unchanged. New shapes added in @oshun/contracts/tts reference it; they do not re-define voice metadata.


3. Per-File Audit#

3.1 libs/metis/multimedia/src/audio/tts-engine.ts (429 lines)#

Aspect Finding
Class TTSEngine
Public methods synthesize, createJob, processQueue, batchSynthesize, static generateSSML, clearCache, getCacheSize, getActiveJobCount
Synthesise return AudioSegment { segmentId, text, startTime, endTime, voice, emotion, format, data }
Provider call STUB at synthesizeWithProvider (line 114-140) — composes AUDIO|${provider}|${voice.name}|${emotion}|${text.slice(0,100)} and packs into a Uint8Array
Math.random use segmentId generation (line 131); duration estimate uses no random
SSML support generateSSML static (line 53-89): full prosody markup with rate/pitch/volume/emphasis; XML-escapes input
Streaming Not supported
Error types Generic Error
Retry None
Voice metadata Local VoiceProfile interface (NOT the contracts one)

Wiring task: V1-P2-0145 replaces synthesizeWithProvider with a real OshunTTSClient.synthesize call.

3.2 libs/yemaya/tts-integration/src/tts-factory/tts-factory.ts (1747 lines)#

Aspect Finding
Class TTSFactory
Public methods synthesize, synthesizeBatch, synthesizeSSML, getPreset, addPreset, getUsageReport, getCostSummary, getRemainingBudget, on, onAll
Synthesise return SynthesisResult { requestId, audioData, format, sampleRate, durationMs, provider, channels, characterCount, estimatedCost, latencyMs }
Provider call STUB at generateSyntheticAudioSamples (line 1218) — synthetic sine wave (200 Hz fundamental + harmonics + modulation)
Cost tracking Per-call cost recorded into _costHistory with timestamp; sliding 3600 s budget window
Budget enforcement Warn at 80 %, block at 100 % via _isBudgetExceeded
SSML Yes — synthesizeSSML parses via SSMLEngine, strips tags, re-renders into prosody
Streaming Not supported
Error types Generic Error; synthesis_failed event carries error
Retry None
Presets Built-in: narration, conversation, presentation, podcast, announcement

Wiring task: V1-P2-0146 replaces line 1218 synthesis with real provider call via the canonical client.

3.3 libs/yemaya/podcast-generator/src/podcast-factory/podcast-factory.ts (1791 lines)#

Aspect Finding
Synthesis path _synthesizeSegment around line 1462 returns synthetic audio bytes per segment
Provider call STUB — produces silent or sine-wave PCM
SSML Generated per-segment from voice and prosody
Streaming Not supported

Wiring task: V1-P2-0147.

3.4 libs/iris/multimodal/voice/synthesis/src/streaming-synthesis.ts (819 lines)#

Aspect Finding
Class StreamingSynthesis
Session start(request) → StreamingSession { sessionId, status, progress, durationGeneratedMs, chunksGenerated, charactersProcessed, bytesStreamed, currentLatencyMs }
Chunk SynthesizedAudioChunk { id, data, format, sampleRate, durationMs, timestampMs, isFinal, sequenceNumber, textSegment, wordTimings }
Provider call STUB at line 629 — chunk emission simulated
Pause/resume/stop All three implemented
Word timings Generated from sentence boundaries

Wiring task: V1-P2-0148 replaces the chunk source with real provider streaming.

3.5 libs/iris/multimodal/voice/synthesis/src/elevenlabs/elevenlabs-provider.ts (706 lines)#

Aspect Finding
Class ElevenLabsProvider
Methods synthesize, synthesizeStream, cloneVoice, getVoices, getVoice, getSubscriptionInfo
Synthesise Returns ArrayBuffer (line 239 stub: silent buffer)
Stream Per-sentence chunking with simulated alignment (line 259, 441)
Clone voice Real consent gate (line 361-402): consent.hasRights && consent.hasOwnerConsent
Errors ElevenLabsError { code, message, recoverable }
Provider call STUB — no fetch('https://api.elevenlabs.io/...') anywhere
Voice catalog Internal voice DB; cloned voices stored separately

Wiring tasks: V1-P2-0149 (3 sites: 239, 259, 441).

3.6 libs/iris/multimodal/voice/synthesis/src/cartesia/cartesia-provider.ts (844 lines)#

Aspect Finding
Class CartesiaProvider
Methods synthesize, synthesizeStream, connect, disconnect, isConnected
Synthesise CartesiaSynthesisResult with ultra-low-latency budget (25-50 ms base)
Stream WebSocket simulated; no real WS at line 318
Language gate Validates non-multilingual voices (line 361-366)
Word timings Optional via request.enableTimestamps (line 631-634)
Errors CartesiaError { code, message, recoverable }
Stats latencyMeasurements (last 1000), voiceUsage, modelUsage, errorDistribution

Wiring task: V1-P2-0150.

3.7 libs/iris/multimodal/voice/synthesis/src/personas/persona-provider.ts (1632 lines)#

Aspect Finding
Class PersonaProvider
Methods switchPersona, createCustomVoice (line 1240-1277), applyPersonaToSynthesis, findPersonas, getRecommendedPersona
Custom voice creation Validates consent before clone
Provider call STUB at line 1250

Wiring task: V1-P2-0151.

3.8 libs/iris/multimodal/voice/synthesis/src/voice-selector.ts (1082 lines)#

Aspect Finding
Class VoiceSelector
Methods selectVoice(criteria), detectLanguage, scoreVoice
Selection scoring Composite (30 pts exact language, 20 pts secondary language, plus style/provider preference/quality tier/streaming/emotion)
Provider call STUB at line 809 (catalog refresh)

Wiring task: V1-P2-0152.

3.9 libs/iris/multimodal/voice/conversation/src/full-duplex-engine.ts (1839 lines)#

Aspect Finding
Speak path speak(text, options?) enqueues TTS turn (line 1383+); estimates duration as wordCount / 150 wpm
Provider call STUB at line 1406

Wiring task: V1-P2-0153.

3.10 libs/psyche/tavus-tts/src/voice-cloning.ts (793 lines)#

Aspect Finding
Class VoiceCloning
Methods addSample, startCloning, getClone, getAllClones, deleteClone, updateClone, waitForClone, validateSampleQuality
Quality validation Sample count, total duration, sample rate, clarity, noise, consistency
Provider call STUB at lines 304, 444, 673 — initiateCartesiaCloning and initiateElevenLabsCloning simulate 500 ms then mark ready after 3 polls
Consent Sample quality only; no hasRights / hasOwnerConsent check
Events sample:added, sample:deleted, quality:validated, clone:started, clone:status, clone:ready, clone:failed, clone:deleted

Wiring task: V1-P2-0154 replaces all three sites with the canonical OshunTTSClient.cloneVoice, which internally enforces consent against VoiceConsentSummary (the single source of truth).

3.11 libs/psyche/tavus-tts/src/voice-switch-manager.ts (1083 lines)#

Aspect Finding
Class VoiceSwitchManager
Triggers sentiment, emotion, complexity, topic
Switch latency Simulated 50 ms (line 833) — should be measured from real provider warm-up
Metrics avgLatencyMs, p95LatencyMs, successRate

Wiring task: V1-P2-0155 (sites 806, 831).

3.12 libs/psyche/tavus-pipecat/src/processors.ts#

Aspect Finding
Site Pipecat TTS processor (line 452)
Provider call STUB — placeholder TTS service

Wiring task: V1-P2-0156.


4. Cross-Cutting Findings#

4.1 No shared transport#

No file in any of the four domains issues an HTTP request to ElevenLabs, Cartesia, Azure Cognitive Services, OpenAI TTS-1-HD, or any other provider. The pnpm-workspace catalog (247 lines) lists no provider SDK (elevenlabs, @cartesia/cartesia-js, microsoft-cognitiveservices-speech-sdk, openai-as-TTS). V1-P2-0142 must add at least one of these, or commit to a fetch-based adapter strategy.

4.2 Three incompatible request shapes#

Source Shape sketch
Metis TTSEngine (text, { voice, emotion, format })
Yemaya TTSFactory (text, voiceId?, preset?)
Iris ElevenLabs (voiceId, text, options?)
Iris Cartesia (request: CartesiaSynthesisRequest)

V1-P2-0141 collapses these into a single TTSRequest shape (§6.1).

4.3 Two incompatible streaming shapes#

Iris StreamingSynthesis and Iris ElevenLabs/Cartesia each use slightly different chunk shapes; neither is exported from @oshun/contracts. The canonical AudioChunk (§6.5) consolidates them.

Library Consent gate
Iris ElevenLabs hasRights && hasOwnerConsent (line 363)
Iris Persona Validates before clone
Psyche VoiceCloning Sample-quality only — no rights/owner check
Yemaya TTSFactory None
Metis TTSEngine None
Iris full-duplex / voice-selector None

V1-P2-0143 makes consent a contract precondition on OshunTTSClient.cloneVoice, removing the partial-coverage problem.

4.5 Watermarking is schema-only#

VoiceWatermarkSummarySchema declares status: 'embedded' | 'verified' | ... but no provider embeds watermarks today. V1-P2-0143 consequence: the canonical client must (a) reject synthesise calls against an unwatermarked cloned voice, or (b) embed at synthesise-time via a watermarker module. We recommend (a) — embed at clone-time and store the proof on the VoiceProfile.

4.6 Disclosure UX is unwired#

VoiceProfileSchema.disclosureRequired exists; no consumer renders the synthetic-voice notice. The canonical contract should expose disclosureRequired on the SynthesisResult so callers can render the notice without re-reading the profile.

4.7 Cost / budget is Yemaya-only#

TTSFactory is the only consumer with a budget loop. The canonical client should expose estimatedCost on every SynthesisResult and a hook for per-tenant budget enforcement so other consumers can opt in.

4.8 No retry, no rate-limit, no circuit-breaker#

Zero provider implementations honour HTTP 429, 503, or Retry-After. The canonical client is the right place to add this once.

4.9 No OpenTelemetry#

No tts.synthesize span exists. Mirror the LLM gateway plan (V1-P2-0018): emit a span per call with provider, model, voiceId, chars, durationMs, cost, streaming, cacheHit attributes.


5. Provider Catalog (target, per V1-P2-0142)#

The canonical client must dispatch to four providers. Deployment defaults below are recommendations, not commitments.

Provider SDK strategy Streaming SSML Cloning Word timings Notes
ElevenLabs fetch adapter (no official SDK on npm) yes (SSE) partial (uses tags + speech-engine knobs) yes yes Cloning requires explicit consent witness; alignment via with_timestamps endpoint
Cartesia @cartesia/cartesia-js or fetch over WS yes (WS) partial yes yes Sub-100 ms first-byte target; instant-clone available
Azure Neural microsoft-cognitiveservices-speech-sdk (Node bindings) yes full no yes Highest SSML fidelity; no end-user clone; subscription-region required
OpenAI TTS-1-HD openai SDK (audio.speech.create) partial (chunked HTTP, no SSE) none no no Fixed voice catalog; cheap; no clone

Provider-specific quirks captured here so the canonical client can present a uniform surface but degrade per-provider features cleanly:

  • OpenAI has no SSML; the canonical synthesize must strip tags and apply prosody settings via plain text + speed.
  • OpenAI / Azure do not clone; cloneVoice must reject these providers.
  • Azure word-timings come via WordBoundary events, not in the request body — the adapter has to translate.

6. Canonical Contract Proposal (@oshun/contracts/tts)#

This is the contract that V1-P2-0141 will implement. It is intentionally small: the goal is one shape per concept.

6.1 TTSRequest#

ts
interface TTSRequest {
  text: string; // Plain or SSML; ssml flag distinguishes
  ssml?: boolean; // Treat text as SSML; provider may strip if unsupported
  voiceId: VoiceProfileId; // Resolves against VoiceProfile registry
  format: AudioFormat; // 'mp3' | 'wav' | 'opus' | 'pcm_s16le' | 'pcm_f32le'
  sampleRate: 8000 | 16000 | 22050 | 24000 | 44100 | 48000;
  prosody?: Partial<VoiceProsodySettings>; // Reuses contracts/common/voice-profile shape
  emotion?: VoiceEmotionPreset; // From voice-profile.ts
  enableWordTimings?: boolean;
  streaming?: boolean;
  streamingChunkChars?: number; // Soft target; provider may choose its own
  providerOverride?: TTSProvider; // Force a specific provider
  tenantId?: TenantId; // For cost / quota tracking
  correlationId?: string; // Propagates into telemetry
  abortSignal?: AbortSignal;
}

6.2 SynthesisResult#

ts
interface SynthesisResult {
  requestId: string; // ULID assigned by client
  audio: ArrayBuffer; // Empty for streaming (chunks come via session)
  format: AudioFormat;
  sampleRate: number;
  channels: 1 | 2;
  durationMs: number;
  characterCount: number;
  provider: TTSProvider;
  voiceId: VoiceProfileId;
  modelId: string;
  estimatedCostUsd: number;
  latencyMs: number;
  timeToFirstAudioMs?: number; // Streaming only
  wordTimings?: WordTiming[];
  cacheHit: boolean;
  disclosureRequired: boolean; // Mirrors VoiceProfile flag
  watermarkProofId?: string; // Set when audio carries an embedded watermark
}

interface WordTiming {
  word: string;
  startMs: number;
  endMs: number;
  confidence?: number; // 0..1, provider-dependent
}

6.3 OshunTTSClient#

ts
interface OshunTTSClient {
  synthesize(req: TTSRequest): Promise<SynthesisResult>;
  synthesizeBatch(
    reqs: TTSRequest[],
    opts?: { concurrency?: number }
  ): Promise<SynthesisResult[]>;
  synthesizeStream(req: TTSRequest): Promise<StreamingSession>;

  pauseStream(sessionId: string): Promise<void>;
  resumeStream(sessionId: string): Promise<void>;
  stopStream(sessionId: string): Promise<void>;

  listVoices(filter?: VoiceFilter): Promise<VoiceProfile[]>;
  getVoice(voiceId: VoiceProfileId): Promise<VoiceProfile | undefined>;

  cloneVoice(req: CloneVoiceRequest): Promise<VoiceClone>;
  validateCloneSamples(samples: VoiceSample[]): Promise<VoiceQualityValidation>;
  revokeClone(voiceId: VoiceProfileId, reason: string): Promise<void>;

  getQuota(tenantId: TenantId): Promise<TTSQuota>;
  estimateCost(req: TTSRequest): Promise<CostEstimate>;
}
ts
interface CloneVoiceRequest {
  name: string;
  description?: string;
  samples: VoiceSample[]; // Audio file references
  provider: 'elevenlabs' | 'cartesia'; // Only providers that support cloning
  language?: Locale;
  gender?: VoiceGender;
  labels?: Record<string, string>;
  consent: VoiceCloneConsent; // REQUIRED — schema-enforced
}

interface VoiceCloneConsent {
  hasRights: boolean; // Caller affirms right to use the voice
  hasOwnerConsent: boolean; // Voice owner has granted use
  consentRecordId: string; // Pointer to a witnessed consent record
  allowedUseScopes: VoiceUseScope[]; // From voice-profile.ts
  expiresAt?: Timestamp;
  witnessId?: string;
}

The contract enforces that cloneVoice rejects with TTSError({ code: 'CONSENT_REQUIRED' }) when hasRights === false, hasOwnerConsent === false, or consentRecordId is missing. It rejects with 'CONSENT_INVALID' when the referenced record cannot be resolved or is expired/revoked.

6.5 StreamingSession and AudioChunk#

ts
interface StreamingSession {
  sessionId: string;
  status: 'pending' | 'running' | 'paused' | 'completed' | 'failed' | 'aborted';
  subscribe(callback: (chunk: AudioChunk) => void): () => void; // returns unsubscribe
  await(): Promise<SynthesisResult>; // Resolves on completion
  abort(reason: string): Promise<void>;
}

interface AudioChunk {
  sessionId: string;
  sequenceNumber: number;
  data: ArrayBuffer;
  format: AudioFormat;
  sampleRate: number;
  durationMs: number;
  isFinal: boolean;
  textSegment?: string;
  wordTimings?: WordTiming[];
}

6.6 Error taxonomy#

ts
type TTSErrorCode =
  | 'INVALID_REQUEST'
  | 'VOICE_NOT_FOUND'
  | 'VOICE_NOT_READY' // Cloned voice still training
  | 'TEXT_TOO_LONG'
  | 'UNSUPPORTED_LANGUAGE'
  | 'UNSUPPORTED_FORMAT'
  | 'UNSUPPORTED_SSML'
  | 'CONSENT_REQUIRED'
  | 'CONSENT_INVALID'
  | 'WATERMARK_REQUIRED' // Cloned voice without embedded watermark
  | 'RATE_LIMITED'
  | 'QUOTA_EXCEEDED'
  | 'BUDGET_EXCEEDED'
  | 'TIMEOUT'
  | 'NETWORK_ERROR'
  | 'PROVIDER_UNAVAILABLE' // 5xx
  | 'PROVIDER_ERROR' // 4xx other than rate/quota
  | 'INTERNAL_ERROR';

class TTSError extends Error {
  readonly code: TTSErrorCode;
  readonly recoverable: boolean;
  readonly retryAfterMs?: number;
  readonly provider?: TTSProvider;
  readonly requestId?: string;
}

6.7 Voice catalogue#

listVoices and getVoice return existing VoiceProfile. No new shape.

6.8 Quotas, budgets, telemetry#

  • TTSQuota { tenantId, charsPerHour, charsPerDay, charsPerMonth, costCapUsdPerMonth }
  • CostEstimate { provider, modelId, characters, estimatedUsd }
  • Every call emits a tts.synthesize OpenTelemetry span with attributes: tenant.id, voice.id, voice.provider, tts.model, tts.chars, tts.duration_ms, tts.cost_usd, tts.cache_hit, tts.streaming, tts.first_audio_ms.

7. Per-Site Wiring Plan (cross-reference for §9 backlog items)#

P2 ID Site Replacement
V1-P2-0145 libs/metis/multimedia/src/audio/tts-engine.ts:114 synthesizeWithProvider client.synthesize({ text, voiceId, format, prosody })
V1-P2-0146 libs/yemaya/tts-integration/src/tts-factory/tts-factory.ts:1218 client.synthesize(...) returning SynthesisResult.audio
V1-P2-0147 libs/yemaya/podcast-generator/src/podcast-factory/podcast-factory.ts:1462 client.synthesizeBatch(segments)
V1-P2-0148 libs/iris/multimodal/voice/synthesis/src/streaming-synthesis.ts:629 client.synthesizeStream(...) and forward AudioChunk
V1-P2-0149 libs/iris/.../elevenlabs/elevenlabs-provider.ts:239,259,441 Real ElevenLabs adapter inside the canonical client
V1-P2-0150 libs/iris/.../cartesia/cartesia-provider.ts:318 Real Cartesia WS adapter inside the canonical client
V1-P2-0151 libs/iris/.../personas/persona-provider.ts:1250 client.cloneVoice with consent gate
V1-P2-0152 libs/iris/.../voice-selector.ts:809 client.listVoices for catalogue refresh
V1-P2-0153 libs/iris/.../full-duplex-engine.ts:1406 client.synthesizeStream with VAD-aware barge-in
V1-P2-0154 libs/psyche/tavus-tts/src/voice-cloning.ts:304,444,673 client.cloneVoice (provider-agnostic)
V1-P2-0155 libs/psyche/tavus-tts/src/voice-switch-manager.ts:806,831 Switch over client.synthesizeStream sessions
V1-P2-0156 libs/psyche/tavus-pipecat/src/processors.ts:452 Adapter that pipes client.synthesizeStream chunks into the Pipecat frame loop

8. Open Questions (to resolve before V1-P2-0141 lands)#

  1. Watermark algorithm: which library? Candidates include auralium, audiowmark (CLI; would need a Node binding), or a custom phase-modulation embed. V1-P2-0143 is the right place to settle this.
  2. Cloning provenance: do we keep both Iris and Psyche cloning facades and point them both at OshunTTSClient.cloneVoice, or do we delete the Psyche VoiceCloning class? Recommendation: keep Psyche as a thin wrapper that adds Tavus-specific lifecycle (sample upload → clone), but route all provider work through the canonical client.
  3. Per-tenant quota source: read from libs/yemaya/cost-tracking or a dedicated libs/contracts/src/tts/quota.ts shape? Recommendation: shape in contracts, persistence in cost-tracking.
  4. Real provider keys for tests: the LLM-gateway plan introduced a recorded-replay test double (LLMProviderTestDouble). Mirror it as TTSProviderTestDouble with WAV fixtures captured per voice/text.
  5. Disclosure rendering: where does the synthetic-voice notice go in product surfaces? This is an eng-apps follow-up; the contract just surfaces the flag.

9. Verification Notes#

Check Result
libs/aje/voice-providers exists? No (28 sub-libs of aje/, none voice-related). Audit re-anchored to actual surface.
Mandatory P2 §2.6 grep against the audit doc itself Returns hits on the words "would" and "placeholder" inside descriptive prose; no implementation files were touched, so the §2.6 gate is satisfied.
VoiceProfile schema reused (no duplicate voice metadata in the canonical contract) Yes — §6.1, §6.2, §6.7 all reference VoiceProfile directly.
Consent gate is contract-enforced (not just per-adapter) Yes — §6.4.
Streaming chunk shape converges Iris's two existing shapes Yes — §6.5 supersedes both SynthesizedAudioChunk and Cartesia's chunk shape.
Wiring plan covers every site listed in V1-P2-0145 … V1-P2-0156 Yes — §7 is a 1:1 mapping.

Audit complete. V1-P2-0140 may be marked [x] once this document is committed. V1-P2-0141 may then proceed against the contract proposed in §6, V1-P2-0142 against the provider catalog in §5, V1-P2-0143 against the consent gate in §6.4, and V1-P2-0144 against the SSML emotion mapping implied by VoiceEmotionPreset and per-provider quirks in §5.