# ElevenLabs Integration Documentation

This directory contains comprehensive documentation for integrating ElevenLabs
text-to-speech services into the Oshun platform.

## Quick Links

| Document                                                                               | Description                                        |
| -------------------------------------------------------------------------------------- | -------------------------------------------------- |
| [API Reference](../../domains/lilith/extras/reference/elevenlabs-integration.md)       | Complete API documentation with types and examples |
| [Integration Guide](../../../apps/lilith/svc-ai/docs/guides/elevenlabs-integration.md) | Step-by-step integration guide and best practices  |
| [Troubleshooting](./troubleshooting.md)                                                | Common issues and solutions                        |
| [Migration Guide](./migration.md)                                                      | Migrate from other TTS providers                   |

## Overview

ElevenLabs provides high-quality text-to-speech capabilities including:

- **Text-to-Speech**: Convert text to natural-sounding speech
- **Voice Cloning**: Create custom voices from audio samples
- **Streaming**: Real-time audio streaming for low-latency applications
- **Voice Library**: Access to pre-made and community voices
- **Speech-to-Speech**: Voice conversion with emotion preservation
- **Sound Effects**: AI-generated sound effects
- **Audio Isolation**: Remove background noise from audio

## Architecture

```
┌─────────────────────────────────────────────────────────────────────────────┐
│                     Oshun ElevenLabs Integration                             │
├─────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Application Layer                                                           │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  Lilith Consciousness    │    Yemaya Creative    │    Isis Factory  │    │
│  │  - Conversational AI     │    - Voice Design     │    - Batch TTS   │    │
│  │  - Streaming responses   │    - Voice cloning    │    - Dubbing     │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                      │                                       │
│  Service Layer                       ▼                                       │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │                    ElevenLabs Provider                               │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │    │
│  │  │ TTS Provider │  │ Voice Clone  │  │ STS Provider │              │    │
│  │  │              │  │ Provider     │  │              │              │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘              │    │
│  │                                                                      │    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │    │
│  │  │ Voice Lib    │  │ Sound FX     │  │ Audio        │              │    │
│  │  │ Provider     │  │ Provider     │  │ Isolation    │              │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘              │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                      │                                       │
│  Infrastructure                      ▼                                       │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │    │
│  │  │ Rate Limiter │  │ Circuit      │  │ Retry        │              │    │
│  │  │              │  │ Breaker      │  │ Handler      │              │    │
│  │  └──────────────┘  └──────────────┘  └──────────────┘              │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                      │                                       │
│                                      ▼                                       │
│  ┌─────────────────────────────────────────────────────────────────────┐    │
│  │                     ElevenLabs API                                   │    │
│  └─────────────────────────────────────────────────────────────────────┘    │
│                                                                              │
└─────────────────────────────────────────────────────────────────────────────┘
```

## Getting Started

### Prerequisites

1. ElevenLabs account with API key
2. Node.js 18+ environment
3. Network access to ElevenLabs API

### Quick Start

```typescript
import { ElevenLabsProvider } from '@oshun/elevenlabs-provider';

// Initialize provider
const elevenlabs = new ElevenLabsProvider({
  apiKey: process.env.ELEVENLABS_API_KEY,
});

// Text-to-speech
const audio = await elevenlabs.textToSpeech({
  voiceId: 'Rachel',
  text: 'Hello, world!',
  modelId: 'eleven_multilingual_v2',
});

// Streaming
const stream = elevenlabs.textToSpeechStream({
  voiceId: 'Rachel',
  text: 'Hello, world!',
});

for await (const chunk of stream) {
  // Process audio chunk
}
```

## Available Models

| Model ID                 | Use Case                         | Languages | Latency | Quality   |
| ------------------------ | -------------------------------- | --------- | ------- | --------- |
| `eleven_multilingual_v2` | General purpose, highest quality | 29        | Medium  | Excellent |
| `eleven_turbo_v2_5`      | Low latency, real-time apps      | 32        | Low     | Very Good |
| `eleven_monolingual_v1`  | English only, legacy             | 1         | Medium  | Good      |
| `eleven_english_sts_v2`  | Speech-to-speech                 | 1         | Medium  | Very Good |

## Voice Selection

### Pre-made Voices

ElevenLabs provides a variety of pre-made voices:

| Voice  | Gender | Age         | Accent   | Use Case        |
| ------ | ------ | ----------- | -------- | --------------- |
| Rachel | Female | Young Adult | American | Narration       |
| Domi   | Female | Young Adult | American | Conversation    |
| Bella  | Female | Young Adult | American | Soft, gentle    |
| Antoni | Male   | Young Adult | American | Warm, friendly  |
| Adam   | Male   | Middle Aged | American | Deep, narration |
| Arnold | Male   | Middle Aged | American | Crisp, formal   |

### Voice Cloning

Create custom voices from audio samples:

```typescript
const customVoice = await elevenlabs.cloneVoice({
  name: 'My Custom Voice',
  files: [audioSample1, audioSample2],
  description: 'Professional narrator voice',
});
```

## Best Practices

1. **Choose the right model**: Use Turbo for real-time, Multilingual for quality
2. **Tune voice settings**: Adjust stability and similarity for best results
3. **Handle rate limits**: Implement exponential backoff
4. **Cache audio**: Store generated audio to avoid redundant API calls
5. **Monitor usage**: Track character consumption against quotas
6. **Use streaming**: For long content, stream audio progressively

## Error Handling

The integration includes comprehensive error handling:

```typescript
try {
  const audio = await elevenlabs.textToSpeech({...});
} catch (error) {
  if (error instanceof RateLimitError) {
    // Handle rate limiting
  } else if (error instanceof QuotaExceededError) {
    // Handle quota issues
  } else if (error instanceof VoiceNotFoundError) {
    // Handle missing voice
  }
}
```

See [Troubleshooting](./troubleshooting.md) for detailed error resolution.

## Monitoring

Key metrics to monitor:

- **Character usage**: Track against subscription quota
- **API latency**: Monitor response times
- **Error rates**: Track failures by error type
- **Cache hit rate**: Optimize caching strategy

## Support

- [ElevenLabs Documentation](https://docs.elevenlabs.io/)
- [ElevenLabs Status](https://status.elevenlabs.io/)
- Internal: #voice-ai Slack channel

## Related Documentation

- **Voice Analytics** — ElevenLabs capability surfaced through the shared Isis
  audio providers (`libs/isis/ai-providers`); no dedicated per-feature module
- **Pronunciation Dictionaries** — applied via the shared Isis TTS provider
  (`libs/isis/ai-providers`, `tts` family)
- **Audio Enhancement** — applied via the shared Isis audio providers
  (`libs/isis/ai-providers`)
- **Dubbing Studio** — ElevenLabs dubbing surfaced through the shared Isis
  providers (`libs/isis/ai-providers`, `conversational-ai`/`tts`)