# Voice Library Enhanced Schema and API Design

## Current State Analysis

### Existing Infrastructure

- **TTS Service**: Voice management, cloning endpoints, permission-based access
- **Consent Management**: Voice cloning consent with granular permissions
- **Voice Types**: 'standard' and 'cloned' voices with metadata
- **Features**: Usage tracking, watermarking, consent validation

## Enhanced Voice Library Schema

### 1. Voice Library Categories

```javascript
{
  id: "category-meditation-001",
  name: "Meditation & Mindfulness",
  description: "Calm, soothing voices perfect for meditation content",
  parent_category: null, // For hierarchical categories
  display_order: 1,
  icon_url: "https://cdn.lilith.example.com/icons/meditation.svg",
  color_theme: "#4A90E2",
  voice_count: 15,
  created_at: "2024-01-01T00:00:00Z",
  updated_at: "2024-01-01T00:00:00Z"
}
```

### 2. Enhanced Voice Metadata

```javascript
{
  // Existing fields from current implementation
  id: "voice-enhanced-001",
  name: "Serene Pro",
  type: "standard|cloned|premium|community",
  languages: ["en", "es", "fr"],
  style_tokens: ["calm", "soothing", "gentle"],
  sample_url: "https://cdn.lilith.example.com/voices/samples/serene-pro.mp3",
  description: "Enhanced meditation voice with improved prosody",

  // New enhanced fields
  category_id: "category-meditation-001",
  tags: ["meditation", "female", "adult", "warm"],
  quality_score: 4.8, // 1-5 rating from reviews
  accent: "neutral", // "neutral", "british", "american", "australian"
  age_range: "adult", // "child", "young_adult", "adult", "senior"
  gender: "female", // "male", "female", "neutral", "other"
  emotion_range: ["calm", "gentle", "soothing", "peaceful"],
  prosody_capabilities: {
    emphasis: true,
    speed_variation: true,
    pitch_modulation: true,
    emotion_expression: ["calm", "gentle"]
  },
  technical_specs: {
    sample_rate: 24000,
    bit_depth: 16,
    format_support: ["opus", "mp3", "wav"],
    streaming_optimized: true,
    latency_ms: 150
  },
  usage_stats: {
    total_synthesized_minutes: 15420,
    unique_users: 1205,
    average_rating: 4.8,
    total_reviews: 342
  },
  availability: {
    pricing_tier: "free|standard|premium",
    usage_limits: {
      monthly_minutes: 1000, // null for unlimited
      commercial_use: true,
      api_access: true
    },
    geographic_restrictions: [], // ISO country codes
    license_type: "creative_commons|proprietary|custom"
  },
  created_at: "2024-01-01T00:00:00Z",
  updated_at: "2024-01-01T00:00:00Z"
}
```

### 3. Voice Reviews and Ratings

```javascript
{
  id: "review-001",
  voice_id: "voice-enhanced-001",
  user_id: "user-12345",
  rating: 5, // 1-5 stars
  review_text: "Perfect for meditation sessions. Very soothing and clear.",
  aspects: {
    clarity: 5,
    naturalness: 4,
    emotion: 5,
    pronunciation: 4,
    overall_quality: 5
  },
  usage_context: "meditation", // "meditation", "lectures", "casual", "professional"
  verified_usage: true, // User has actually used this voice
  helpful_votes: 23,
  reported_issues: [],
  created_at: "2024-01-15T00:00:00Z",
  updated_at: "2024-01-15T00:00:00Z"
}
```

### 4. Voice Collections and Playlists

```javascript
{
  id: "collection-mindfulness-001",
  name: "Mindfulness Masters",
  description: "Curated collection of the best meditation voices",
  type: "curated|user_created|algorithm_generated",
  curator_id: "curator-001", // null for system collections
  voice_ids: ["voice-001", "voice-002", "voice-003"],
  category_ids: ["category-meditation-001"],
  tags: ["featured", "staff_picks", "meditation"],
  visibility: "public|private|unlisted",
  usage_stats: {
    views: 5420,
    voices_tried: 3240,
    saves: 1205
  },
  created_at: "2024-01-01T00:00:00Z",
  updated_at: "2024-01-01T00:00:00Z"
}
```

### 5. Enhanced Voice Cloning Workflow

```javascript
{
  id: "clone-job-001",
  user_id: "user-12345",
  name: "My Personal Voice",
  status: "pending|processing|training|validation|ready|failed",
  workflow_step: "sample_upload|consent_collection|quality_check|training|validation|deployment",

  // Voice samples management
  voice_samples: [
    {
      id: "sample-001",
      file_url: "https://cdn.lilith.example.com/samples/user-12345/sample-001.wav",
      duration_sec: 30,
      quality_score: 0.92,
      text_content: "This is a sample of my voice for cloning purposes.",
      phonetic_coverage: 0.85, // 0-1, how well it covers phonemes
      noise_level: 0.1, // 0-1, lower is better
      validated: true,
      uploaded_at: "2024-01-01T10:00:00Z"
    }
  ],

  // Consent integration
  consent_tokens: ["consent-voice-cloning-token-001"],
  granular_permissions: {
    voice_model_creation: true,
    voice_model_usage: true,
    voice_model_improvement: false,
    third_party_voice_usage: false,
    voice_model_sharing: false
  },

  // Training progress
  training_progress: {
    current_step: "feature_extraction",
    completion_percentage: 65,
    estimated_completion_time: "2024-01-01T18:00:00Z",
    quality_metrics: {
      similarity_score: 0.89,
      naturalness_score: 0.82,
      clarity_score: 0.91
    }
  },

  // Validation and testing
  validation_tests: [
    {
      test_id: "validation-001",
      test_text: "Hello, this is a test of my cloned voice quality.",
      generated_audio_url: "https://cdn.lilith.example.com/validation/clone-job-001/test-001.wav",
      user_approval: null, // true|false|null (pending)
      quality_metrics: {
        similarity_score: 0.91,
        naturalness_score: 0.85,
        mispronunciation_count: 0
      }
    }
  ],

  // Final voice model
  output_voice: {
    voice_id: "voice-cloned-user-12345-001",
    model_version: "1.0",
    quality_tier: "standard|premium",
    deployment_status: "active|inactive|archived"
  },

  created_at: "2024-01-01T00:00:00Z",
  updated_at: "2024-01-01T12:30:00Z",
  completed_at: null
}
```

## Enhanced API Endpoints

### Voice Library Discovery APIs

#### GET /v1/voice-library/categories

List all voice categories with metadata.

**Query Parameters:**

- `include_voice_count`: boolean - Include voice count per category
- `hierarchy`: boolean - Return hierarchical structure
- `locale`: string - Localization preference

**Response:**

```javascript
{
  categories: [
    {
      id: "category-meditation-001",
      name: "Meditation & Mindfulness",
      description: "Calm, soothing voices perfect for meditation content",
      voice_count: 15,
      subcategories: []
    }
  ],
  total_categories: 8
}
```

#### GET /v1/voice-library/search

Advanced voice search and discovery.

**Query Parameters:**

- `q`: string - Search query
- `category_id`: string - Filter by category
- `tags`: string[] - Filter by tags
- `language`: string[] - Filter by languages
- `gender`: string - Filter by gender
- `accent`: string - Filter by accent
- `quality_min`: number - Minimum quality score
- `pricing_tier`: string - Filter by pricing tier
- `sort`: string - Sort by (quality|popularity|newest|name)
- `limit`: number - Results limit
- `offset`: number - Pagination offset

**Response:**

```javascript
{
  voices: [
    {
      id: "voice-enhanced-001",
      name: "Serene Pro",
      category: "Meditation & Mindfulness",
      quality_score: 4.8,
      tags: ["meditation", "female", "adult", "warm"],
      sample_url: "https://cdn.lilith.example.com/voices/samples/serene-pro.mp3",
      pricing_tier: "free"
    }
  ],
  total_results: 145,
  facets: {
    categories: { "meditation": 45, "lectures": 32 },
    languages: { "en": 120, "es": 45, "fr": 38 },
    gender: { "female": 78, "male": 67 }
  }
}
```

#### GET /v1/voice-library/collections

List voice collections and curated playlists.

**Query Parameters:**

- `type`: string - Collection type filter
- `featured`: boolean - Only featured collections
- `user_id`: string - Collections by specific curator

### Voice Management APIs

#### POST /v1/voice-library/voices/:voiceId/review

Submit a voice review and rating.

**Request Body:**

```javascript
{
  rating: 5,
  review_text: "Perfect for meditation sessions.",
  aspects: {
    clarity: 5,
    naturalness: 4,
    emotion: 5
  },
  usage_context: "meditation"
}
```

#### GET /v1/voice-library/voices/:voiceId/analytics

Get detailed voice usage analytics and performance metrics.

### Enhanced Voice Cloning APIs

#### POST /v1/voice-cloning/workflows

Start a new voice cloning workflow with consent collection.

**Request Body:**

```javascript
{
  name: "My Personal Voice",
  description: "Personal voice for meditation content",
  usage_scope: "personal",
  consent_flow_preferences: {
    granular_permissions: {
      voice_model_improvement: false,
      third_party_usage: false
    }
  }
}
```

**Response:**

```javascript
{
  workflow_id: "clone-job-001",
  status: "consent_pending",
  consent_flow_url: "/consent-flows/voice_cloning?workflow_id=clone-job-001",
  next_steps: [
    "Complete consent collection",
    "Upload voice samples",
    "Review and approve training"
  ]
}
```

#### POST /v1/voice-cloning/workflows/:workflowId/samples

Upload voice samples for cloning.

**Request Body:** Multipart form with audio files and metadata.

#### GET /v1/voice-cloning/workflows/:workflowId/progress

Get real-time progress of voice cloning workflow.

#### POST /v1/voice-cloning/workflows/:workflowId/validation

Submit validation feedback during voice cloning process.

### Voice Library Analytics APIs

#### GET /v1/voice-library/analytics/usage

Get voice library usage analytics.

**Query Parameters:**

- `timeframe`: string - daily|weekly|monthly
- `voice_ids`: string[] - Specific voices to analyze
- `user_segment`: string - User segment filter

**Response:**

```javascript
{
  timeframe: "monthly",
  summary: {
    total_synthesis_minutes: 450320,
    unique_voices_used: 89,
    most_popular_voice: "voice-enhanced-001",
    average_session_duration: 12.5,
    user_satisfaction_score: 4.7
  },
  trends: {
    synthesis_minutes_trend: "+15%",
    new_voice_adoption_rate: "23%",
    user_retention_rate: "87%"
  },
  voice_performance: [
    {
      voice_id: "voice-enhanced-001",
      name: "Serene Pro",
      usage_minutes: 45680,
      unique_users: 2340,
      rating: 4.8,
      trend: "+8%"
    }
  ]
}
```

## Integration Points

### 1. Consent Management Integration

- Seamless consent collection during voice cloning workflow
- Granular permission validation for voice usage
- Consent renewal notifications for long-term voice models

### 2. Media Service Integration

- Voice sample storage and processing
- Audio quality analysis and validation
- CDN distribution for voice samples and models

### 3. User Management Integration

- User preferences for voice discovery
- Voice library personalization based on usage history
- Social features (reviews, favorites, sharing)

### 4. Analytics and Observability

- Voice library usage tracking
- Quality metrics and performance monitoring
- User engagement and satisfaction analytics

## Implementation Phases

### Phase 1: Enhanced Voice Discovery

- Voice categorization and tagging
- Advanced search and filtering
- Voice collections and curation

### Phase 2: Reviews and Quality System

- Voice rating and review system
- Quality scoring algorithms
- Community feedback integration

### Phase 3: Advanced Voice Cloning

- Multi-step workflow management
- Real-time progress tracking
- Quality validation and user approval

### Phase 4: Analytics and Insights

- Comprehensive usage analytics
- Voice performance metrics
- User behavior insights

This design builds upon the existing robust foundation while adding
enterprise-grade voice library management capabilities.
