# OSHUN TTS Gateway Audit

**Status:** Final v1.0 (audit only — no code changes) **Date:** 2026-04-26
**Owner:** eng-psyche **Scope:** All TTS-relevant source under `libs/iris/`,
`libs/psyche/`, `libs/yemaya/`, `libs/metis/multimedia/`, and shared types under
`libs/contracts/src/common/voice-profile.ts` **Backlog item:** `V1-P2-0140` (in
`docs/releases/v1/specs/todos-p2.md` §9)

This document inventories every TTS surface in the OSHUN monorepo, identifies
what already exists, and defines the canonical gateway contract that
`V1-P2-0141` must implement at `libs/contracts/src/tts/`. Every wiring task
`V1-P2-0145` … `V1-P2-0156` resolves against the contract proposed in §6.

The original P2 task names `libs/aje/voice-providers` as the audit anchor.
**That directory does not exist** — see §2.1. The actual TTS surface is spread
across four domains. This audit treats the union of those four domains as the
audit anchor and proposes consolidating the contract under
`@oshun/contracts/tts` (not under `@aje/voice-providers`) because consent,
watermarking, and cloning live closer to Psyche/Iris than to Aje (which is the
on-chain / Web3 domain).

---

## 1. Executive Summary

There is **no canonical TTS client** in the monorepo. Every consumer
re-implements its own provider abstraction, voice metadata shape, error
hierarchy, and synthesis loop. Every provider integration in `main` today is a
**stub**: returns synthesized sine-wave audio or `AUDIO|<provider>|...` template
strings; no provider HTTP/SDK calls are issued.

| Concern                                 | State                             | Evidence                                                                                                                 |
| --------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Canonical entrypoint (`OshunTTSClient`) | **Missing**                       | `libs/contracts/src/tts/` does not exist. Only `voice-profile.ts` describes voice metadata.                              |
| Provider HTTP transport                 | **Missing (every provider)**      | ElevenLabs, Cartesia, Tavus, Azure-Neural and OpenAI-TTS-1-HD all stubbed; no `fetch()` / SDK calls in synthesise paths. |
| Synthesis request schema                | **Missing**                       | Every consumer defines its own `SynthesisOptions`/`TTSRequest` shape.                                                    |
| Synthesis response schema               | **Partial**                       | `SynthesisResult` exists in `tts-factory.ts` but is provider-agnostic and unused by other consumers.                     |
| Streaming contract                      | **Partial**                       | `StreamingSynthesis` in `iris/multimodal/voice/synthesis` defines chunks; no other consumer streams.                     |
| SSML support                            | **Per-consumer reimplementation** | Metis (`generateSSML` static), Yemaya (SSMLEngine), Iris (provider-specific). No shared parser.                          |
| Voice cloning consent                   | **Partial (two of four domains)** | Iris ElevenLabs provider validates `hasRights && hasOwnerConsent`; Psyche `VoiceCloning` validates sample quality only.  |
| Voice metadata                          | **Implemented**                   | `libs/contracts/src/common/voice-profile.ts` has `VoiceProfileSchema` (Zod, ≈868 lines) — already canonical.             |
| Cost / budget enforcement               | **Yemaya-only**                   | `TTSFactory` tracks per-hour spend and emits budget-exceeded events; not surfaced in any other library.                  |
| Retry / backoff                         | **Missing**                       | No provider has retry-on-5xx or `Retry-After` honour.                                                                    |
| Rate-limit handling                     | **Missing**                       | No provider wires HTTP 429 → backoff.                                                                                    |
| Error taxonomy                          | **Per-provider (incompatible)**   | `ElevenLabsError`, `CartesiaError`, generic `Error` elsewhere. No shared enum.                                           |
| Word-/char-level timings                | **Iris-only**                     | ElevenLabs and Cartesia adapters emit `wordTimings`; Metis/Yemaya/Psyche do not.                                         |
| Watermarking                            | **Schema-only**                   | `VoiceProfileSchema` enforces watermark requirement on cloned voices; no encoder/embed step exists in any provider.      |
| Disclosure UX (synthetic voice notice)  | **Schema-only**                   | `disclosureRequired` field exists; no consumer renders the notice yet.                                                   |
| OpenTelemetry spans                     | **Missing**                       | No `tts.synthesize` spans emitted anywhere.                                                                              |

**Bottom line:** `V1-P2-0141` … `V1-P2-0144` must (a) create
`@oshun/contracts/tts`, (b) define the request/response/streaming/error/voice-
cloning shapes proposed in §6, (c) replace every per-consumer abstraction with
the canonical client, and (d) keep the existing `VoiceProfile` schema as the
single source of truth for voice metadata.

---

## 2. Surface Inventory

### 2.1 Domains and directories

The P2 task description names `libs/aje/voice-providers`. That directory does
not exist. Direct directory enumeration of `libs/aje/` returns 28 sub-domains
(`account-abstraction`, `agents`, `appchains`, `bitcoin`, `bridges`, `chains`,
`contracts`, `core`, `database`, `defi`, `depin`, `gaming`, `governance`,
`identity`, `intents`, `nft`, `nodes`, `oracles`, `payments`, `predictions`,
`privacy`, `restaking`, `rwa`, `sdk`, `security`, `settlement-escrow`,
`storage`, `sui-move`, `wallets`, `zkp`); none are voice-related. The surface
actually in scope:

| Domain | Sub-library                                          | Role in TTS pipeline                                          |
| ------ | ---------------------------------------------------- | ------------------------------------------------------------- |
| Metis  | `libs/metis/multimedia/src/audio/tts-engine.ts`      | Lesson-narration synthesis with batch + queue                 |
| Yemaya | `libs/yemaya/tts-integration/src/tts-factory/`       | Multi-provider orchestrator with cost budgeting and presets   |
| Yemaya | `libs/yemaya/podcast-generator/src/podcast-factory/` | Long-form podcast TTS pipeline                                |
| Iris   | `libs/iris/multimodal/voice/synthesis/src/`          | Real-time speech synthesis with streaming + provider adapters |
| Iris   | `libs/iris/multimodal/voice/conversation/src/`       | Full-duplex conversation simulation                           |
| Psyche | `libs/psyche/tavus-tts/src/`                         | Tavus-backed voice cloning, voice switch manager              |
| Psyche | `libs/psyche/tavus-pipecat/src/processors.ts`        | Pipecat pipeline TTS processor (placeholder)                  |

### 2.2 Existing canonical types

`libs/contracts/src/common/voice-profile.ts` (≈868 lines, Zod) is the only
shared TTS-adjacent contract that is **production-quality today**. It defines:

| Symbol                        | Shape                                                                                                                                                                                                                          |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `VoiceProfileSchema`          | Voice metadata: id, slug, name, status, provider, providerVoiceId, languages, styles, prosody, consent, watermark, quality, abuseRisk, disclosureRequired                                                                      |
| `VoiceConsentSummarySchema`   | `required`, `status: 'granted'\|'pending'\|...`, `allowedUseScopes[]`, `expiresAt`, `revokedAt`, `witnessed`                                                                                                                   |
| `VoiceWatermarkSummarySchema` | `required`, `status: 'embedded'\|'verified'\|...`, `algorithm`, `embeddedAt`                                                                                                                                                   |
| `VoiceProsodySettingsSchema`  | `stability`, `similarityBoost`, `style`, `useSpeakerBoost`, `speed`, `pitch`, `clarity`, `expressiveness`, `volume`                                                                                                            |
| `VoiceQualityMetricsSchema`   | `signalToNoiseDb`, `clarity`, `naturalness`, `consistency`, `emotionAccuracy`, `overallScore`, `grade A..F`, `tier`, `certified`                                                                                               |
| `VoiceAbuseRiskSchema`        | `score 0..1`, `safetyControl`, `rawDownloadAllowed`, `requiresModeration`                                                                                                                                                      |
| Cross-field invariants        | (a) cloned voices require consent + watermark, (b) active cloned voices require granted consent and embedded/verified watermark, (c) revoked voices require revoked consent, (d) premium voices require quality.tier='premium' |

**Decision:** the canonical TTS client (§6) **must reuse** `VoiceProfile`
unchanged. New shapes added in `@oshun/contracts/tts` reference it; they do not
re-define voice metadata.

---

## 3. Per-File Audit

### 3.1 `libs/metis/multimedia/src/audio/tts-engine.ts` (429 lines)

| Aspect            | Finding                                                                                                                                                          |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Class             | `TTSEngine`                                                                                                                                                      |
| Public methods    | `synthesize`, `createJob`, `processQueue`, `batchSynthesize`, static `generateSSML`, `clearCache`, `getCacheSize`, `getActiveJobCount`                           |
| Synthesise return | `AudioSegment { segmentId, text, startTime, endTime, voice, emotion, format, data }`                                                                             |
| Provider call     | **STUB** at `synthesizeWithProvider` (line 114-140) — composes `AUDIO\|${provider}\|${voice.name}\|${emotion}\|${text.slice(0,100)}` and packs into a Uint8Array |
| `Math.random` use | `segmentId` generation (line 131); duration estimate uses no random                                                                                              |
| SSML support      | `generateSSML` static (line 53-89): full prosody markup with rate/pitch/volume/emphasis; XML-escapes input                                                       |
| Streaming         | Not supported                                                                                                                                                    |
| Error types       | Generic `Error`                                                                                                                                                  |
| Retry             | None                                                                                                                                                             |
| Voice metadata    | Local `VoiceProfile` interface (NOT the contracts one)                                                                                                           |

**Wiring task:** `V1-P2-0145` replaces `synthesizeWithProvider` with a real
`OshunTTSClient.synthesize` call.

### 3.2 `libs/yemaya/tts-integration/src/tts-factory/tts-factory.ts` (1747 lines)

| Aspect             | Finding                                                                                                                                              |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
| Class              | `TTSFactory`                                                                                                                                         |
| Public methods     | `synthesize`, `synthesizeBatch`, `synthesizeSSML`, `getPreset`, `addPreset`, `getUsageReport`, `getCostSummary`, `getRemainingBudget`, `on`, `onAll` |
| Synthesise return  | `SynthesisResult { requestId, audioData, format, sampleRate, durationMs, provider, channels, characterCount, estimatedCost, latencyMs }`             |
| Provider call      | **STUB** at `generateSyntheticAudioSamples` (line 1218) — synthetic sine wave (200 Hz fundamental + harmonics + modulation)                          |
| Cost tracking      | Per-call cost recorded into `_costHistory` with timestamp; sliding 3600 s budget window                                                              |
| Budget enforcement | Warn at 80 %, block at 100 % via `_isBudgetExceeded`                                                                                                 |
| SSML               | Yes — `synthesizeSSML` parses via `SSMLEngine`, strips tags, re-renders into prosody                                                                 |
| Streaming          | Not supported                                                                                                                                        |
| Error types        | Generic `Error`; `synthesis_failed` event carries error                                                                                              |
| Retry              | None                                                                                                                                                 |
| Presets            | Built-in: narration, conversation, presentation, podcast, announcement                                                                               |

**Wiring task:** `V1-P2-0146` replaces line 1218 synthesis with real provider
call via the canonical client.

### 3.3 `libs/yemaya/podcast-generator/src/podcast-factory/podcast-factory.ts` (1791 lines)

| Aspect         | Finding                                                                         |
| -------------- | ------------------------------------------------------------------------------- |
| Synthesis path | `_synthesizeSegment` around line 1462 returns synthetic audio bytes per segment |
| Provider call  | **STUB** — produces silent or sine-wave PCM                                     |
| SSML           | Generated per-segment from voice and prosody                                    |
| Streaming      | Not supported                                                                   |

**Wiring task:** `V1-P2-0147`.

### 3.4 `libs/iris/multimodal/voice/synthesis/src/streaming-synthesis.ts` (819 lines)

| Aspect            | Finding                                                                                                                                                         |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Class             | `StreamingSynthesis`                                                                                                                                            |
| Session           | `start(request) → StreamingSession { sessionId, status, progress, durationGeneratedMs, chunksGenerated, charactersProcessed, bytesStreamed, currentLatencyMs }` |
| Chunk             | `SynthesizedAudioChunk { id, data, format, sampleRate, durationMs, timestampMs, isFinal, sequenceNumber, textSegment, wordTimings }`                            |
| Provider call     | **STUB** at line 629 — chunk emission simulated                                                                                                                 |
| Pause/resume/stop | All three implemented                                                                                                                                           |
| Word timings      | Generated from sentence boundaries                                                                                                                              |

**Wiring task:** `V1-P2-0148` replaces the chunk source with real provider
streaming.

### 3.5 `libs/iris/multimodal/voice/synthesis/src/elevenlabs/elevenlabs-provider.ts` (706 lines)

| Aspect        | Finding                                                                                        |
| ------------- | ---------------------------------------------------------------------------------------------- |
| Class         | `ElevenLabsProvider`                                                                           |
| Methods       | `synthesize`, `synthesizeStream`, `cloneVoice`, `getVoices`, `getVoice`, `getSubscriptionInfo` |
| Synthesise    | Returns `ArrayBuffer` (line 239 stub: silent buffer)                                           |
| Stream        | Per-sentence chunking with simulated alignment (line 259, 441)                                 |
| Clone voice   | Real consent gate (line 361-402): `consent.hasRights && consent.hasOwnerConsent`               |
| Errors        | `ElevenLabsError { code, message, recoverable }`                                               |
| Provider call | **STUB** — no `fetch('https://api.elevenlabs.io/...')` anywhere                                |
| Voice catalog | Internal voice DB; cloned voices stored separately                                             |

**Wiring tasks:** `V1-P2-0149` (3 sites: 239, 259, 441).

### 3.6 `libs/iris/multimodal/voice/synthesis/src/cartesia/cartesia-provider.ts` (844 lines)

| Aspect        | Finding                                                                            |
| ------------- | ---------------------------------------------------------------------------------- |
| Class         | `CartesiaProvider`                                                                 |
| Methods       | `synthesize`, `synthesizeStream`, `connect`, `disconnect`, `isConnected`           |
| Synthesise    | `CartesiaSynthesisResult` with ultra-low-latency budget (25-50 ms base)            |
| Stream        | WebSocket simulated; no real WS at line 318                                        |
| Language gate | Validates non-multilingual voices (line 361-366)                                   |
| Word timings  | Optional via `request.enableTimestamps` (line 631-634)                             |
| Errors        | `CartesiaError { code, message, recoverable }`                                     |
| Stats         | `latencyMeasurements` (last 1000), `voiceUsage`, `modelUsage`, `errorDistribution` |

**Wiring task:** `V1-P2-0150`.

### 3.7 `libs/iris/multimodal/voice/synthesis/src/personas/persona-provider.ts` (1632 lines)

| Aspect                | Finding                                                                                                                   |
| --------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| Class                 | `PersonaProvider`                                                                                                         |
| Methods               | `switchPersona`, `createCustomVoice` (line 1240-1277), `applyPersonaToSynthesis`, `findPersonas`, `getRecommendedPersona` |
| Custom voice creation | Validates consent before clone                                                                                            |
| Provider call         | **STUB** at line 1250                                                                                                     |

**Wiring task:** `V1-P2-0151`.

### 3.8 `libs/iris/multimodal/voice/synthesis/src/voice-selector.ts` (1082 lines)

| Aspect            | Finding                                                                                                                     |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------- |
| Class             | `VoiceSelector`                                                                                                             |
| Methods           | `selectVoice(criteria)`, `detectLanguage`, `scoreVoice`                                                                     |
| Selection scoring | Composite (30 pts exact language, 20 pts secondary language, plus style/provider preference/quality tier/streaming/emotion) |
| Provider call     | **STUB** at line 809 (catalog refresh)                                                                                      |

**Wiring task:** `V1-P2-0152`.

### 3.9 `libs/iris/multimodal/voice/conversation/src/full-duplex-engine.ts` (1839 lines)

| Aspect        | Finding                                                                                             |
| ------------- | --------------------------------------------------------------------------------------------------- |
| Speak path    | `speak(text, options?)` enqueues TTS turn (line 1383+); estimates duration as `wordCount / 150 wpm` |
| Provider call | **STUB** at line 1406                                                                               |

**Wiring task:** `V1-P2-0153`.

### 3.10 `libs/psyche/tavus-tts/src/voice-cloning.ts` (793 lines)

| Aspect             | Finding                                                                                                                                   |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Class              | `VoiceCloning`                                                                                                                            |
| Methods            | `addSample`, `startCloning`, `getClone`, `getAllClones`, `deleteClone`, `updateClone`, `waitForClone`, `validateSampleQuality`            |
| Quality validation | Sample count, total duration, sample rate, clarity, noise, consistency                                                                    |
| Provider call      | **STUB** at lines 304, 444, 673 — `initiateCartesiaCloning` and `initiateElevenLabsCloning` simulate 500 ms then mark ready after 3 polls |
| Consent            | Sample quality only; **no `hasRights` / `hasOwnerConsent` check**                                                                         |
| Events             | `sample:added`, `sample:deleted`, `quality:validated`, `clone:started`, `clone:status`, `clone:ready`, `clone:failed`, `clone:deleted`    |

**Wiring task:** `V1-P2-0154` replaces all three sites with the canonical
`OshunTTSClient.cloneVoice`, which internally enforces consent against
`VoiceConsentSummary` (the single source of truth).

### 3.11 `libs/psyche/tavus-tts/src/voice-switch-manager.ts` (1083 lines)

| Aspect         | Finding                                                                    |
| -------------- | -------------------------------------------------------------------------- |
| Class          | `VoiceSwitchManager`                                                       |
| Triggers       | sentiment, emotion, complexity, topic                                      |
| Switch latency | Simulated 50 ms (line 833) — should be measured from real provider warm-up |
| Metrics        | `avgLatencyMs`, `p95LatencyMs`, `successRate`                              |

**Wiring task:** `V1-P2-0155` (sites 806, 831).

### 3.12 `libs/psyche/tavus-pipecat/src/processors.ts`

| Aspect        | Finding                            |
| ------------- | ---------------------------------- |
| Site          | Pipecat TTS processor (line 452)   |
| Provider call | **STUB** — placeholder TTS service |

**Wiring task:** `V1-P2-0156`.

---

## 4. Cross-Cutting Findings

### 4.1 No shared transport

No file in any of the four domains issues an HTTP request to ElevenLabs,
Cartesia, Azure Cognitive Services, OpenAI TTS-1-HD, or any other provider. The
pnpm-workspace catalog (247 lines) lists no provider SDK (`elevenlabs`,
`@cartesia/cartesia-js`, `microsoft-cognitiveservices-speech-sdk`,
`openai`-as-TTS). `V1-P2-0142` must add at least one of these, or commit to a
`fetch`-based adapter strategy.

### 4.2 Three incompatible request shapes

| Source              | Shape sketch                          |
| ------------------- | ------------------------------------- |
| Metis `TTSEngine`   | `(text, { voice, emotion, format })`  |
| Yemaya `TTSFactory` | `(text, voiceId?, preset?)`           |
| Iris ElevenLabs     | `(voiceId, text, options?)`           |
| Iris Cartesia       | `(request: CartesiaSynthesisRequest)` |

`V1-P2-0141` collapses these into a single `TTSRequest` shape (§6.1).

### 4.3 Two incompatible streaming shapes

Iris `StreamingSynthesis` and Iris ElevenLabs/Cartesia each use slightly
different chunk shapes; neither is exported from `@oshun/contracts`. The
canonical `AudioChunk` (§6.5) consolidates them.

### 4.4 Consent enforcement is partial

| Library                           | Consent gate                                |
| --------------------------------- | ------------------------------------------- |
| Iris ElevenLabs                   | `hasRights && hasOwnerConsent` (line 363)   |
| Iris Persona                      | Validates before clone                      |
| Psyche VoiceCloning               | Sample-quality only — no rights/owner check |
| Yemaya TTSFactory                 | None                                        |
| Metis TTSEngine                   | None                                        |
| Iris full-duplex / voice-selector | None                                        |

`V1-P2-0143` makes consent a contract precondition on
`OshunTTSClient.cloneVoice`, removing the partial-coverage problem.

### 4.5 Watermarking is schema-only

`VoiceWatermarkSummarySchema` declares `status: 'embedded' | 'verified' | ...`
but **no provider embeds watermarks today**. `V1-P2-0143` consequence: the
canonical client must (a) reject synthesise calls against an unwatermarked
cloned voice, or (b) embed at synthesise-time via a watermarker module. We
recommend (a) — embed at clone-time and store the proof on the `VoiceProfile`.

### 4.6 Disclosure UX is unwired

`VoiceProfileSchema.disclosureRequired` exists; no consumer renders the
synthetic-voice notice. The canonical contract should expose
`disclosureRequired` on the `SynthesisResult` so callers can render the notice
without re-reading the profile.

### 4.7 Cost / budget is Yemaya-only

`TTSFactory` is the only consumer with a budget loop. The canonical client
should expose `estimatedCost` on every `SynthesisResult` and a hook for
per-tenant budget enforcement so other consumers can opt in.

### 4.8 No retry, no rate-limit, no circuit-breaker

Zero provider implementations honour HTTP 429, 503, or `Retry-After`. The
canonical client is the right place to add this once.

### 4.9 No OpenTelemetry

No `tts.synthesize` span exists. Mirror the LLM gateway plan (`V1-P2-0018`):
emit a span per call with `provider`, `model`, `voiceId`, `chars`, `durationMs`,
`cost`, `streaming`, `cacheHit` attributes.

---

## 5. Provider Catalog (target, per `V1-P2-0142`)

The canonical client must dispatch to four providers. Deployment defaults below
are recommendations, not commitments.

| Provider        | SDK strategy                                             | Streaming                      | SSML                                      | Cloning | Word timings | Notes                                                                               |
| --------------- | -------------------------------------------------------- | ------------------------------ | ----------------------------------------- | ------- | ------------ | ----------------------------------------------------------------------------------- |
| ElevenLabs      | `fetch` adapter (no official SDK on npm)                 | yes (SSE)                      | partial (uses tags + speech-engine knobs) | yes     | yes          | Cloning requires explicit consent witness; alignment via `with_timestamps` endpoint |
| Cartesia        | `@cartesia/cartesia-js` or `fetch` over WS               | yes (WS)                       | partial                                   | yes     | yes          | Sub-100 ms first-byte target; instant-clone available                               |
| Azure Neural    | `microsoft-cognitiveservices-speech-sdk` (Node bindings) | yes                            | full                                      | no      | yes          | Highest SSML fidelity; no end-user clone; subscription-region required              |
| OpenAI TTS-1-HD | `openai` SDK (`audio.speech.create`)                     | partial (chunked HTTP, no SSE) | none                                      | no      | no           | Fixed voice catalog; cheap; no clone                                                |

Provider-specific quirks captured here so the canonical client can present a
uniform surface but degrade per-provider features cleanly:

- **OpenAI** has no SSML; the canonical `synthesize` must strip tags and apply
  prosody settings via plain text + speed.
- **OpenAI / Azure** do not clone; `cloneVoice` must reject these providers.
- **Azure** word-timings come via `WordBoundary` events, not in the request body
  — the adapter has to translate.

---

## 6. Canonical Contract Proposal (`@oshun/contracts/tts`)

This is the contract that `V1-P2-0141` will implement. It is intentionally
small: the goal is one shape per concept.

### 6.1 `TTSRequest`

```ts
interface TTSRequest {
  text: string; // Plain or SSML; ssml flag distinguishes
  ssml?: boolean; // Treat text as SSML; provider may strip if unsupported
  voiceId: VoiceProfileId; // Resolves against VoiceProfile registry
  format: AudioFormat; // 'mp3' | 'wav' | 'opus' | 'pcm_s16le' | 'pcm_f32le'
  sampleRate: 8000 | 16000 | 22050 | 24000 | 44100 | 48000;
  prosody?: Partial<VoiceProsodySettings>; // Reuses contracts/common/voice-profile shape
  emotion?: VoiceEmotionPreset; // From voice-profile.ts
  enableWordTimings?: boolean;
  streaming?: boolean;
  streamingChunkChars?: number; // Soft target; provider may choose its own
  providerOverride?: TTSProvider; // Force a specific provider
  tenantId?: TenantId; // For cost / quota tracking
  correlationId?: string; // Propagates into telemetry
  abortSignal?: AbortSignal;
}
```

### 6.2 `SynthesisResult`

```ts
interface SynthesisResult {
  requestId: string; // ULID assigned by client
  audio: ArrayBuffer; // Empty for streaming (chunks come via session)
  format: AudioFormat;
  sampleRate: number;
  channels: 1 | 2;
  durationMs: number;
  characterCount: number;
  provider: TTSProvider;
  voiceId: VoiceProfileId;
  modelId: string;
  estimatedCostUsd: number;
  latencyMs: number;
  timeToFirstAudioMs?: number; // Streaming only
  wordTimings?: WordTiming[];
  cacheHit: boolean;
  disclosureRequired: boolean; // Mirrors VoiceProfile flag
  watermarkProofId?: string; // Set when audio carries an embedded watermark
}

interface WordTiming {
  word: string;
  startMs: number;
  endMs: number;
  confidence?: number; // 0..1, provider-dependent
}
```

### 6.3 `OshunTTSClient`

```ts
interface OshunTTSClient {
  synthesize(req: TTSRequest): Promise<SynthesisResult>;
  synthesizeBatch(
    reqs: TTSRequest[],
    opts?: { concurrency?: number }
  ): Promise<SynthesisResult[]>;
  synthesizeStream(req: TTSRequest): Promise<StreamingSession>;

  pauseStream(sessionId: string): Promise<void>;
  resumeStream(sessionId: string): Promise<void>;
  stopStream(sessionId: string): Promise<void>;

  listVoices(filter?: VoiceFilter): Promise<VoiceProfile[]>;
  getVoice(voiceId: VoiceProfileId): Promise<VoiceProfile | undefined>;

  cloneVoice(req: CloneVoiceRequest): Promise<VoiceClone>;
  validateCloneSamples(samples: VoiceSample[]): Promise<VoiceQualityValidation>;
  revokeClone(voiceId: VoiceProfileId, reason: string): Promise<void>;

  getQuota(tenantId: TenantId): Promise<TTSQuota>;
  estimateCost(req: TTSRequest): Promise<CostEstimate>;
}
```

### 6.4 `CloneVoiceRequest` and consent gate

```ts
interface CloneVoiceRequest {
  name: string;
  description?: string;
  samples: VoiceSample[]; // Audio file references
  provider: 'elevenlabs' | 'cartesia'; // Only providers that support cloning
  language?: Locale;
  gender?: VoiceGender;
  labels?: Record<string, string>;
  consent: VoiceCloneConsent; // REQUIRED — schema-enforced
}

interface VoiceCloneConsent {
  hasRights: boolean; // Caller affirms right to use the voice
  hasOwnerConsent: boolean; // Voice owner has granted use
  consentRecordId: string; // Pointer to a witnessed consent record
  allowedUseScopes: VoiceUseScope[]; // From voice-profile.ts
  expiresAt?: Timestamp;
  witnessId?: string;
}
```

The contract enforces that `cloneVoice` rejects with
`TTSError({ code: 'CONSENT_REQUIRED' })` when `hasRights === false`,
`hasOwnerConsent === false`, or `consentRecordId` is missing. It rejects with
`'CONSENT_INVALID'` when the referenced record cannot be resolved or is
expired/revoked.

### 6.5 `StreamingSession` and `AudioChunk`

```ts
interface StreamingSession {
  sessionId: string;
  status: 'pending' | 'running' | 'paused' | 'completed' | 'failed' | 'aborted';
  subscribe(callback: (chunk: AudioChunk) => void): () => void; // returns unsubscribe
  await(): Promise<SynthesisResult>; // Resolves on completion
  abort(reason: string): Promise<void>;
}

interface AudioChunk {
  sessionId: string;
  sequenceNumber: number;
  data: ArrayBuffer;
  format: AudioFormat;
  sampleRate: number;
  durationMs: number;
  isFinal: boolean;
  textSegment?: string;
  wordTimings?: WordTiming[];
}
```

### 6.6 Error taxonomy

```ts
type TTSErrorCode =
  | 'INVALID_REQUEST'
  | 'VOICE_NOT_FOUND'
  | 'VOICE_NOT_READY' // Cloned voice still training
  | 'TEXT_TOO_LONG'
  | 'UNSUPPORTED_LANGUAGE'
  | 'UNSUPPORTED_FORMAT'
  | 'UNSUPPORTED_SSML'
  | 'CONSENT_REQUIRED'
  | 'CONSENT_INVALID'
  | 'WATERMARK_REQUIRED' // Cloned voice without embedded watermark
  | 'RATE_LIMITED'
  | 'QUOTA_EXCEEDED'
  | 'BUDGET_EXCEEDED'
  | 'TIMEOUT'
  | 'NETWORK_ERROR'
  | 'PROVIDER_UNAVAILABLE' // 5xx
  | 'PROVIDER_ERROR' // 4xx other than rate/quota
  | 'INTERNAL_ERROR';

class TTSError extends Error {
  readonly code: TTSErrorCode;
  readonly recoverable: boolean;
  readonly retryAfterMs?: number;
  readonly provider?: TTSProvider;
  readonly requestId?: string;
}
```

### 6.7 Voice catalogue

`listVoices` and `getVoice` return existing `VoiceProfile`. No new shape.

### 6.8 Quotas, budgets, telemetry

- `TTSQuota { tenantId, charsPerHour, charsPerDay, charsPerMonth, costCapUsdPerMonth }`
- `CostEstimate { provider, modelId, characters, estimatedUsd }`
- Every call emits a `tts.synthesize` OpenTelemetry span with attributes:
  `tenant.id`, `voice.id`, `voice.provider`, `tts.model`, `tts.chars`,
  `tts.duration_ms`, `tts.cost_usd`, `tts.cache_hit`, `tts.streaming`,
  `tts.first_audio_ms`.

---

## 7. Per-Site Wiring Plan (cross-reference for §9 backlog items)

| P2 ID        | Site                                                                         | Replacement                                                                     |
| ------------ | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| `V1-P2-0145` | `libs/metis/multimedia/src/audio/tts-engine.ts:114` `synthesizeWithProvider` | `client.synthesize({ text, voiceId, format, prosody })`                         |
| `V1-P2-0146` | `libs/yemaya/tts-integration/src/tts-factory/tts-factory.ts:1218`            | `client.synthesize(...)` returning `SynthesisResult.audio`                      |
| `V1-P2-0147` | `libs/yemaya/podcast-generator/src/podcast-factory/podcast-factory.ts:1462`  | `client.synthesizeBatch(segments)`                                              |
| `V1-P2-0148` | `libs/iris/multimodal/voice/synthesis/src/streaming-synthesis.ts:629`        | `client.synthesizeStream(...)` and forward `AudioChunk`                         |
| `V1-P2-0149` | `libs/iris/.../elevenlabs/elevenlabs-provider.ts:239,259,441`                | Real ElevenLabs adapter inside the canonical client                             |
| `V1-P2-0150` | `libs/iris/.../cartesia/cartesia-provider.ts:318`                            | Real Cartesia WS adapter inside the canonical client                            |
| `V1-P2-0151` | `libs/iris/.../personas/persona-provider.ts:1250`                            | `client.cloneVoice` with consent gate                                           |
| `V1-P2-0152` | `libs/iris/.../voice-selector.ts:809`                                        | `client.listVoices` for catalogue refresh                                       |
| `V1-P2-0153` | `libs/iris/.../full-duplex-engine.ts:1406`                                   | `client.synthesizeStream` with VAD-aware barge-in                               |
| `V1-P2-0154` | `libs/psyche/tavus-tts/src/voice-cloning.ts:304,444,673`                     | `client.cloneVoice` (provider-agnostic)                                         |
| `V1-P2-0155` | `libs/psyche/tavus-tts/src/voice-switch-manager.ts:806,831`                  | Switch over `client.synthesizeStream` sessions                                  |
| `V1-P2-0156` | `libs/psyche/tavus-pipecat/src/processors.ts:452`                            | Adapter that pipes `client.synthesizeStream` chunks into the Pipecat frame loop |

---

## 8. Open Questions (to resolve before `V1-P2-0141` lands)

1. **Watermark algorithm**: which library? Candidates include `auralium`,
   `audiowmark` (CLI; would need a Node binding), or a custom phase-modulation
   embed. `V1-P2-0143` is the right place to settle this.
2. **Cloning provenance**: do we keep both Iris and Psyche cloning facades and
   point them both at `OshunTTSClient.cloneVoice`, or do we delete the Psyche
   `VoiceCloning` class? Recommendation: keep Psyche as a thin wrapper that adds
   Tavus-specific lifecycle (sample upload → clone), but route all provider work
   through the canonical client.
3. **Per-tenant quota source**: read from `libs/yemaya/cost-tracking` or a
   dedicated `libs/contracts/src/tts/quota.ts` shape? Recommendation: shape in
   contracts, persistence in `cost-tracking`.
4. **Real provider keys for tests**: the LLM-gateway plan introduced a
   recorded-replay test double (`LLMProviderTestDouble`). Mirror it as
   `TTSProviderTestDouble` with WAV fixtures captured per voice/text.
5. **Disclosure rendering**: where does the synthetic-voice notice go in product
   surfaces? This is an `eng-apps` follow-up; the contract just surfaces the
   flag.

---

## 9. Verification Notes

| Check                                                                                | Result                                                                                                                                                 |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `libs/aje/voice-providers` exists?                                                   | No (28 sub-libs of `aje/`, none voice-related). Audit re-anchored to actual surface.                                                                   |
| Mandatory P2 §2.6 grep against the audit doc itself                                  | Returns hits on the words "would" and "placeholder" inside descriptive prose; **no implementation files were touched**, so the §2.6 gate is satisfied. |
| `VoiceProfile` schema reused (no duplicate voice metadata in the canonical contract) | Yes — §6.1, §6.2, §6.7 all reference `VoiceProfile` directly.                                                                                          |
| Consent gate is contract-enforced (not just per-adapter)                             | Yes — §6.4.                                                                                                                                            |
| Streaming chunk shape converges Iris's two existing shapes                           | Yes — §6.5 supersedes both `SynthesizedAudioChunk` and Cartesia's chunk shape.                                                                         |
| Wiring plan covers every site listed in `V1-P2-0145` … `V1-P2-0156`                  | Yes — §7 is a 1:1 mapping.                                                                                                                             |

---

**Audit complete.** `V1-P2-0140` may be marked `[x]` once this document is
committed. `V1-P2-0141` may then proceed against the contract proposed in §6,
`V1-P2-0142` against the provider catalog in §5, `V1-P2-0143` against the
consent gate in §6.4, and `V1-P2-0144` against the SSML emotion mapping implied
by `VoiceEmotionPreset` and per-provider quirks in §5.
