AI avatar management service for the Psyche AI Virtual Assistant Platform.
Part of the Oshun Platform.
Overview#
The Tavus Integration Service provides comprehensive integration with Tavus's SOTA avatar models:
- Phoenix-4: Gaussian-diffusion rendering model for realistic avatars
- Sparrow-1: Turn-taking and dialogue model for natural conversations
- Raven-1: Multimodal perception model
This service enables Psyche AI assistants to conduct real-time video conversations with human-like avatars.
Features#
Replica Management#
- Create digital twins from training videos
- Training status monitoring with progress callbacks
- Video upload pipeline with validation
- S3 and local storage support
- Quality assessment for training videos
Persona Management#
- System prompt configuration builder
- Template library for common personas (Sales, Support, Education, etc.)
- Guardrails and objectives system
- Persona-replica associations
- Template-based persona creation
Conversation Management#
- Real-time video conversation creation
- Audio-only mode support
- Custom greetings and context
- Session management and monitoring
- Recording and captions support
Video Generation#
- Script-to-video generation
- Audio-to-video synchronization
- Background customization
- Batch video processing
Webhook Handling#
- Signature verification
- Event parsing and routing
- Retry logic with exponential backoff
- Dead letter queue for failed events
- Priority-based handler registration
Tech Stack#
- Language: Python 3.11+
- Framework: FastAPI + Uvicorn
- HTTP Client: aiohttp (async)
- Encryption: cryptography (Fernet)
- S3 Integration: aiobotocore (optional)
- Testing: pytest + pytest-asyncio
Project Structure#
text
src/tavus_integration/
├── __init__.py # Package exports
├── main.py # FastAPI application
├── api/
│ ├── __init__.py # API module exports
│ ├── client.py # Tavus API client with rate limiting
│ ├── auth.py # API key management and encryption
│ └── webhooks.py # Webhook receiver and handlers
├── replica/
│ ├── __init__.py # Replica module exports
│ └── manager.py # Replica management and video upload
└── persona/
├── __init__.py # Persona module exports
└── manager.py # Persona management and templates
API Endpoints#
Health#
GET /health- Health checkGET /ready- Readiness checkGET /stats- Service statistics
Replicas#
POST /replicas- Create a new replicaGET /replicas- List all replicasGET /replicas/{id}- Get replica by IDDELETE /replicas/{id}- Delete replica
Personas#
POST /personas- Create a new personaGET /personas- List all personasGET /personas/{id}- Get persona by IDDELETE /personas/{id}- Delete personaGET /personas/templates- List persona templates
Conversations#
POST /conversations- Create conversationGET /conversations- List conversationsGET /conversations/{id}- Get conversationPOST /conversations/{id}/end- End conversation
Videos#
POST /videos- Generate videoGET /videos- List videosGET /videos/{id}- Get videoDELETE /videos/{id}- Delete video
Webhooks#
POST /webhooks/tavus- Receive Tavus webhooksGET /webhooks/stats- Webhook statistics
Development#
Using Nx#
bash
# Install dependencies
nx install psyche-tavus-integration
# Development server
nx serve psyche-tavus-integration
# Production server
nx serve-prod psyche-tavus-integration
# Build
nx build psyche-tavus-integration
# Linting
nx lint psyche-tavus-integration
nx lint-fix psyche-tavus-integration
# Formatting
nx format psyche-tavus-integration
nx format-check psyche-tavus-integration
# Type checking
nx typecheck psyche-tavus-integration
# Testing
nx test psyche-tavus-integration
nx test-cov psyche-tavus-integration
# Docker
nx docker-build psyche-tavus-integration
nx docker-run psyche-tavus-integration
Direct Commands#
bash
cd apps/psyche/tavus-integration
# Install dependencies
poetry install
# With S3 support
poetry install --extras s3
# Development server
poetry run uvicorn tavus_integration.main:app --reload --port 8013
# Run tests
poetry run pytest
# Lint
poetry run ruff check src tests
Configuration#
Environment Variables#
| Variable | Description | Default |
|---|---|---|
TAVUS_API_KEY |
Tavus API key | Required |
TAVUS_ENVIRONMENT |
API environment (production/sandbox) | production |
TAVUS_WEBHOOK_SECRET |
Webhook signature verification secret | - |
TAVUS_MASTER_KEY |
Master key for API key encryption | - |
CORS_ORIGINS |
Allowed CORS origins | * |
PORT |
Service port | 8013 |
HOST |
Service host | 0.0.0.0 |
S3 Video Upload#
For S3-based video uploads, configure:
| Variable | Description |
|---|---|
AWS_ACCESS_KEY_ID |
AWS access key |
AWS_SECRET_ACCESS_KEY |
AWS secret key |
AWS_DEFAULT_REGION |
AWS region |
TAVUS_S3_BUCKET |
S3 bucket for training videos |
Usage Examples#
Create a Replica#
python
import httpx
async with httpx.AsyncClient() as client:
response = await client.post(
"http://localhost:8013/replicas",
json={
"name": "My Avatar",
"video_url": "https://example.com/training-video.mp4",
"callback_url": "https://my-app.com/webhooks/tavus",
}
)
replica = response.json()
print(f"Created replica: {replica['replica_id']}")
Create a Persona#
python
response = await client.post(
"http://localhost:8013/personas",
json={
"name": "Sales Assistant",
"role": "Sales Representative",
"organization": "Acme Corp",
"tone": "professional",
"personality_traits": ["helpful", "knowledgeable"],
"expertise_areas": ["product features", "pricing"],
"default_replica_id": "replica_123",
}
)
Start a Conversation#
python
response = await client.post(
"http://localhost:8013/conversations",
json={
"persona_id": "persona_456",
"replica_id": "replica_123",
"conversation_name": "Sales Demo",
"custom_greeting": "Hi! I'm here to help you learn about our products.",
}
)
conversation = response.json()
print(f"Join URL: {conversation['conversation_url']}")
Generate a Video#
python
response = await client.post(
"http://localhost:8013/videos",
json={
"replica_id": "replica_123",
"script": "Welcome to our product demonstration...",
"video_name": "Product Demo",
}
)
Persona Templates#
Built-in templates for common use cases:
- sales_representative - Sales and business development
- customer_support - Customer support and issue resolution
- interview_screener - Job interview screening
- educational_tutor - Educational tutoring
Docker#
Build#
bash
nx docker-build psyche-tavus-integration
Run#
bash
nx docker-run psyche-tavus-integration
The container:
- Runs on port 8013
- Uses Python 3.11 slim image
- Includes health checks
- Non-root user (psyche)
License#
Proprietary - Oshun Platform