# Audio API Reference

Complete reference for Lilith's audio APIs including Text-to-Speech (TTS),
Speech-to-Text (STT), voice sessions, voice cloning, and audio quality
assurance.

---

## Text-to-Speech (TTS)

### Synthesize Speech

```http
POST /v1/tts:synthesize
POST /v1/tts/synthesize
```

Convert text to speech with customizable voice and SSML support.

**Request Body:**

```json
{
  "text": "Hello, welcome to meditation.",
  "voice_id": "zen-master-01",
  "format": "opus",
  "ssml": "<speak><prosody rate='slow'>Hello</prosody></speak>",
  "stream": false,
  "language": "en-US",
  "style_tokens": ["calm", "soothing"]
}
```

**Response:**

```json
{
  "synthesis_id": "syn_abc123",
  "audio_url": "https://cdn.lilith.app/audio/syn_abc123.opus",
  "duration_ms": 2500,
  "format": "opus",
  "sample_rate": 48000
}
```

### Stream TTS

```http
POST /v1/tts/synthesize/stream
```

Stream synthesized audio in real-time for low-latency applications.

**Response:** `audio/opus` stream with chunked transfer encoding

### Async TTS Job

```http
POST /v1/tts/synthesize/async
```

Queue long-form content for background synthesis.

**Request Body:**

```json
{
  "text": "Long meditation script...",
  "voice_id": "guide-01",
  "callback_url": "https://your-app.com/webhook"
}
```

**Response:**

```json
{
  "job_id": "job_xyz789",
  "status": "queued",
  "estimated_completion": "2026-01-03T12:00:00Z"
}
```

### TTS Job Management

```http
GET /v1/tts/jobs
GET /v1/tts/jobs/{jobId}
GET /v1/tts/jobs/{jobId}/audio
DELETE /v1/tts/jobs/{jobId}
```

---

## Voice Management

### List Voices

```http
GET /v1/voices
```

**Query Parameters:** | Parameter | Type | Description |
|-----------|------|-------------| | `language` | string | Filter by language
(BCP-47) | | `gender` | string | `male`, `female`, `neutral` | | `style` |
string | `calm`, `authoritative`, `soothing` | | `limit` | number | Max results
(default: 50) |

**Response:**

```json
{
  "voices": [
    {
      "voice_id": "zen-master-01",
      "name": "Zen Master Takeshi",
      "language": "en-US",
      "gender": "male",
      "style": ["calm", "wise"],
      "preview_url": "https://cdn.lilith.app/voices/preview/zen-master-01.mp3"
    }
  ],
  "total": 100
}
```

### Get Voice Details

```http
GET /v1/voices/{voiceId}
```

### Voice Preview

```http
GET /v1/voices/{voiceId}/preview
```

Returns a sample audio clip demonstrating the voice.

### Voice Lexicons

Custom pronunciation dictionaries per voice.

```http
GET /v1/voices/{voiceId}/lexicons
POST /v1/voices/{voiceId}/lexicons
PATCH /v1/voices/{voiceId}/lexicons/{lexiconId}
DELETE /v1/voices/{voiceId}/lexicons/{lexiconId}
```

**Lexicon Entry:**

```json
{
  "word": "Dhyana",
  "phoneme": "ˈdjɑːnə",
  "alias": "dhya-na"
}
```

---

## Voice Cloning

### Consent Management

```http
POST /v1/voice-cloning/consent
GET /v1/voice-cloning/consent/{userId}
```

Voice cloning requires explicit user consent with legal acknowledgment.

**Consent Request:**

```json
{
  "user_id": "user_123",
  "scope": ["personal_use", "meditation_guides"],
  "duration_days": 365,
  "acknowledgment": "I consent to voice cloning..."
}
```

### Clone Voice

```http
POST /v1/voices/clone
POST /v1/voice-cloning/initiate
```

**Request:** `multipart/form-data`

- `audio_sample`: Audio file (min 10s, WAV/MP3/FLAC)
- `consent_token`: Consent verification token
- `voice_name`: Custom name for the cloned voice

### Cloning Workflows

```http
POST /v1/voice-cloning/workflows
GET /v1/voice-cloning/workflows/{workflowId}
GET /v1/voice-cloning/workflows/{workflowId}/progress
POST /v1/voice-cloning/workflows/{workflowId}/approve
```

**Workflow States:**

- `pending_consent` - Awaiting consent verification
- `processing` - Audio analysis in progress
- `review` - Ready for quality review
- `approved` - Voice available for use
- `rejected` - Failed quality checks

### List Voice Clones

```http
GET /v1/voice-clones
```

---

## Voice Library

### Search Voices

```http
GET /v1/voice-library/search
```

**Query Parameters:** | Parameter | Type | Description |
|-----------|------|-------------| | `q` | string | Search query | | `category`
| string | Category filter | | `tags` | string[] | Tag filters | |
`style_tokens` | string[] | Style filters |

### Categories & Tags

```http
GET /v1/voice-library/categories
GET /v1/voice-library/tags
GET /v1/voice-library/style-tokens
```

### Collections

```http
GET /v1/voice-library/collections
POST /v1/voice-library/collections
```

### Voice Reviews

```http
POST /v1/voice-library/voices/{voiceId}/reviews
GET /v1/voice-library/voices/{voiceId}/reviews
```

### Analytics

```http
GET /v1/voice-library/analytics/usage
```

---

## Speech-to-Text (STT)

### Transcribe Audio

```http
POST /v1/stt/transcribe
```

**Request:** `multipart/form-data`

- `audio`: Audio file (WAV, MP3, FLAC, OGG)
- `language`: Language code (optional, auto-detect)
- `model`: Model selection (`whisper-large`, `whisper-medium`)

**Response:**

```json
{
  "transcription_id": "tr_abc123",
  "text": "Hello, this is a meditation...",
  "language": "en",
  "confidence": 0.95,
  "words": [{ "word": "Hello", "start": 0.0, "end": 0.5, "confidence": 0.98 }],
  "duration_ms": 5000
}
```

### Streaming Transcription

```http
POST /v1/stt/transcribe/stream
```

WebSocket endpoint for real-time transcription with partial hypotheses.

**WebSocket Events:**

- `partial` - Interim transcription
- `final` - Completed segment
- `end` - Stream complete

### Available Models

```http
GET /v1/stt/models
```

---

## Real-Time Voice Sessions

### Session Management

```http
POST /v1/voice/sessions
GET /v1/voice/sessions/{sessionId}
POST /v1/voice/sessions/{sessionId}/start
DELETE /v1/voice/sessions/{sessionId}
```

**Create Session:**

```json
{
  "persona_id": "zen-master",
  "voice_id": "zen-master-01",
  "mode": "duplex",
  "vad_enabled": true,
  "barge_in_enabled": true
}
```

### Audio Input/Output

```http
POST /v1/voice/sessions/{sessionId}/audio-input
GET /v1/voice/sessions/{sessionId}/audio
POST /v1/voice/sessions/{sessionId}/process-text
GET /v1/voice/sessions/{sessionId}/stream-tts
```

### Push-to-Talk

```http
POST /v1/voice/sessions/{sessionId}/push-to-talk/start
POST /v1/voice/sessions/{sessionId}/push-to-talk/stop
```

### Voice Activity Detection (VAD)

```http
POST /v1/voice/sessions/{sessionId}/vad/activity
POST /v1/voice/sessions/{sessionId}/vad/silence
POST /v1/voice/vad/configure
```

### Barge-In (Interruption)

```http
POST /v1/voice/sessions/{sessionId}/start-ai-response
POST /v1/voice/sessions/{sessionId}/barge-in
```

### Diagnostics

```http
GET /v1/voice/sessions/{sessionId}/diagnostics
GET /v1/voice/sessions/{sessionId}/diagnostics/detailed
```

---

## Audio Quality Assurance

### MOS (Mean Opinion Score)

```http
POST /v1/audio/qa/mos/calculate
POST /v1/audio/qa/mos/panel/create
POST /v1/audio/qa/mos/panel/{panelId}/evaluate
GET /v1/audio/qa/mos/panel/{panelId}/results
```

### Audio Analysis

```http
POST /v1/audio/qa/clipping/analyze
POST /v1/audio/qa/artifacts/detect
POST /v1/audio/qa/snr/analyze
POST /v1/audio/qa/frequency/analyze
```

### LUFS Normalization

```http
POST /v1/audio/qa/lufs/measure
POST /v1/audio/qa/lufs/validate
POST /v1/audio/qa/lufs/normalize
POST /v1/audio/qa/lufs/batch-process
GET /v1/audio/qa/lufs/batch/{jobId}/status
```

**Target:** -16 LUFS for consistent loudness

### Quality Reports

```http
POST /v1/audio/qa/metrics/analyze
POST /v1/audio/qa/compare
POST /v1/audio/qa/reports/generate
```

### Real-Time Monitoring

```http
POST /v1/audio/qa/monitor/start
GET /v1/audio/qa/monitor/{sessionId}/alerts
POST /v1/audio/qa/monitor/{sessionId}/stop
```

### Standards Configuration

```http
POST /v1/audio/qa/standards/configure
POST /v1/audio/qa/validate
```

---

## Voice Pipeline

### Configuration

```http
GET /v1/voice/pipeline/config
POST /v1/voice/pipeline/config
GET /v1/voice/pipeline/health
GET /v1/voice/pipeline/audio-config
GET /v1/voice/pipeline/metrics
```

### Circuit Breakers

```http
GET /v1/voice/pipeline/circuit-breakers
POST /v1/voice/pipeline/reset-overload
```

### Optimization

```http
POST /v1/voice/pipeline/optimize
POST /v1/voice/pipeline/priority-test
```

---

## HLS Streaming

For lecture and meditation audio delivery:

```http
GET /v1/tts/hls/{synthesisId}/manifest.m3u8
GET /v1/tts/hls/{synthesisId}/playlist.m3u8
GET /v1/tts/hls/{synthesisId}/segments/{segmentFile}
```

---

## Usage & Billing

### User Usage

```http
GET /v1/tts/usage/{userId}
GET /v1/tts/usage/{userId}/stats
```

### Token Management

```http
POST /v1/tts/usage/tokens/initialize
POST /v1/tts/usage/tokens/topup
```

### Consent History

```http
GET /v1/tts/consent/{userId}/history
POST /v1/tts/consent/grant
POST /v1/tts/consent/revoke
```

---

## Watermarking & Abuse Prevention

### Watermark Detection

```http
GET /v1/tts/watermarks/{watermarkId}
POST /v1/tts/watermarks/detect
```

### Abuse Metrics

```http
GET /v1/tts/abuse/metrics/{userId}
```

---

## Audio Handoff

For cross-device audio session transfer:

```http
WS /v1/audio-handoff/websocket
```

Enables seamless audio handoff between devices during meditation sessions.

---

## Error Codes

| Code                       | Description                    |
| -------------------------- | ------------------------------ |
| `audio.invalid_format`     | Unsupported audio format       |
| `tts.voice_not_found`      | Voice ID not found             |
| `tts.consent_required`     | Voice cloning requires consent |
| `tts.rate_limited`         | TTS quota exceeded             |
| `stt.transcription_failed` | STT processing error           |
| `voice.session_expired`    | Voice session timed out        |

---

## Rate Limits

| Endpoint          | Limit         |
| ----------------- | ------------- |
| TTS Synthesis     | 100 req/min   |
| STT Transcription | 50 req/min    |
| Voice Sessions    | 10 concurrent |
| Voice Cloning     | 5 req/day     |

---

## Related Documentation

- [Audio Streaming Guide](/docs/guides/audio-streaming.md)
- [SSML Reference](/docs/internal-handbooks/tts/ssml-guide.md)
- [Voice Personality Library](/docs/VOICE_PERSONALITY_LIBRARY.md)
