Eve library

Psyche Tavus Integration Service

Other

Revision 1

On this page

AI avatar management service for the Psyche AI Virtual Assistant Platform.

Part of the Oshun Platform.

Overview#

The Tavus Integration Service provides comprehensive integration with Tavus's SOTA avatar models:

  • Phoenix-4: Gaussian-diffusion rendering model for realistic avatars
  • Sparrow-1: Turn-taking and dialogue model for natural conversations
  • Raven-1: Multimodal perception model

This service enables Psyche AI assistants to conduct real-time video conversations with human-like avatars.

Features#

Replica Management#

  • Create digital twins from training videos
  • Training status monitoring with progress callbacks
  • Video upload pipeline with validation
  • S3 and local storage support
  • Quality assessment for training videos

Persona Management#

  • System prompt configuration builder
  • Template library for common personas (Sales, Support, Education, etc.)
  • Guardrails and objectives system
  • Persona-replica associations
  • Template-based persona creation

Conversation Management#

  • Real-time video conversation creation
  • Audio-only mode support
  • Custom greetings and context
  • Session management and monitoring
  • Recording and captions support

Video Generation#

  • Script-to-video generation
  • Audio-to-video synchronization
  • Background customization
  • Batch video processing

Webhook Handling#

  • Signature verification
  • Event parsing and routing
  • Retry logic with exponential backoff
  • Dead letter queue for failed events
  • Priority-based handler registration

Tech Stack#

  • Language: Python 3.11+
  • Framework: FastAPI + Uvicorn
  • HTTP Client: aiohttp (async)
  • Encryption: cryptography (Fernet)
  • S3 Integration: aiobotocore (optional)
  • Testing: pytest + pytest-asyncio

Project Structure#

text
src/tavus_integration/
├── __init__.py              # Package exports
├── main.py                  # FastAPI application
├── api/
│   ├── __init__.py          # API module exports
│   ├── client.py            # Tavus API client with rate limiting
│   ├── auth.py              # API key management and encryption
│   └── webhooks.py          # Webhook receiver and handlers
├── replica/
│   ├── __init__.py          # Replica module exports
│   └── manager.py           # Replica management and video upload
└── persona/
    ├── __init__.py          # Persona module exports
    └── manager.py           # Persona management and templates

API Endpoints#

Health#

  • GET /health - Health check
  • GET /ready - Readiness check
  • GET /stats - Service statistics

Replicas#

  • POST /replicas - Create a new replica
  • GET /replicas - List all replicas
  • GET /replicas/{id} - Get replica by ID
  • DELETE /replicas/{id} - Delete replica

Personas#

  • POST /personas - Create a new persona
  • GET /personas - List all personas
  • GET /personas/{id} - Get persona by ID
  • DELETE /personas/{id} - Delete persona
  • GET /personas/templates - List persona templates

Conversations#

  • POST /conversations - Create conversation
  • GET /conversations - List conversations
  • GET /conversations/{id} - Get conversation
  • POST /conversations/{id}/end - End conversation

Videos#

  • POST /videos - Generate video
  • GET /videos - List videos
  • GET /videos/{id} - Get video
  • DELETE /videos/{id} - Delete video

Webhooks#

  • POST /webhooks/tavus - Receive Tavus webhooks
  • GET /webhooks/stats - Webhook statistics

Development#

Using Nx#

bash
# Install dependencies
nx install psyche-tavus-integration

# Development server
nx serve psyche-tavus-integration

# Production server
nx serve-prod psyche-tavus-integration

# Build
nx build psyche-tavus-integration

# Linting
nx lint psyche-tavus-integration
nx lint-fix psyche-tavus-integration

# Formatting
nx format psyche-tavus-integration
nx format-check psyche-tavus-integration

# Type checking
nx typecheck psyche-tavus-integration

# Testing
nx test psyche-tavus-integration
nx test-cov psyche-tavus-integration

# Docker
nx docker-build psyche-tavus-integration
nx docker-run psyche-tavus-integration

Direct Commands#

bash
cd apps/psyche/tavus-integration

# Install dependencies
poetry install

# With S3 support
poetry install --extras s3

# Development server
poetry run uvicorn tavus_integration.main:app --reload --port 8013

# Run tests
poetry run pytest

# Lint
poetry run ruff check src tests

Configuration#

Environment Variables#

Variable Description Default
TAVUS_API_KEY Tavus API key Required
TAVUS_ENVIRONMENT API environment (production/sandbox) production
TAVUS_WEBHOOK_SECRET Webhook signature verification secret -
TAVUS_MASTER_KEY Master key for API key encryption -
CORS_ORIGINS Allowed CORS origins *
PORT Service port 8013
HOST Service host 0.0.0.0

S3 Video Upload#

For S3-based video uploads, configure:

Variable Description
AWS_ACCESS_KEY_ID AWS access key
AWS_SECRET_ACCESS_KEY AWS secret key
AWS_DEFAULT_REGION AWS region
TAVUS_S3_BUCKET S3 bucket for training videos

Usage Examples#

Create a Replica#

python
import httpx

async with httpx.AsyncClient() as client:
    response = await client.post(
        "http://localhost:8013/replicas",
        json={
            "name": "My Avatar",
            "video_url": "https://example.com/training-video.mp4",
            "callback_url": "https://my-app.com/webhooks/tavus",
        }
    )
    replica = response.json()
    print(f"Created replica: {replica['replica_id']}")

Create a Persona#

python
response = await client.post(
    "http://localhost:8013/personas",
    json={
        "name": "Sales Assistant",
        "role": "Sales Representative",
        "organization": "Acme Corp",
        "tone": "professional",
        "personality_traits": ["helpful", "knowledgeable"],
        "expertise_areas": ["product features", "pricing"],
        "default_replica_id": "replica_123",
    }
)

Start a Conversation#

python
response = await client.post(
    "http://localhost:8013/conversations",
    json={
        "persona_id": "persona_456",
        "replica_id": "replica_123",
        "conversation_name": "Sales Demo",
        "custom_greeting": "Hi! I'm here to help you learn about our products.",
    }
)
conversation = response.json()
print(f"Join URL: {conversation['conversation_url']}")

Generate a Video#

python
response = await client.post(
    "http://localhost:8013/videos",
    json={
        "replica_id": "replica_123",
        "script": "Welcome to our product demonstration...",
        "video_name": "Product Demo",
    }
)

Persona Templates#

Built-in templates for common use cases:

  • sales_representative - Sales and business development
  • customer_support - Customer support and issue resolution
  • interview_screener - Job interview screening
  • educational_tutor - Educational tutoring

Docker#

Build#

bash
nx docker-build psyche-tavus-integration

Run#

bash
nx docker-run psyche-tavus-integration

The container:

  • Runs on port 8013
  • Uses Python 3.11 slim image
  • Includes health checks
  • Non-root user (psyche)

License#

Proprietary - Oshun Platform