Eve library

Psyche Integration Guide

Guide

Revision 1

On this page

This guide provides comprehensive documentation for integrating the Psyche AI Virtual Assistant Platform into your applications. It covers SDK installation, authentication, conferencing platform setup, custom avatar creation, and advanced integration patterns.


Table of Contents#

  1. Overview
  2. Quick Start
  3. SDK Installation
  4. Authentication
  5. Session Management
  6. Conferencing Platform Integration
  7. Custom Avatar Creation
  8. Persona Configuration
  9. Knowledge Base Integration
  10. Tool & Function Calling
  11. WebSocket Streaming
  12. Webhook Integration
  13. Error Handling
  14. Performance Optimization
  15. Security Best Practices
  16. Troubleshooting

Overview#

Psyche provides hyper-realistic AI virtual assistants powered by:

  • 3D Gaussian Splatting (3DGS) - Photorealistic avatar rendering at 30 FPS
  • FLAME Morphable Model - Expressive facial animations with 50 blend shapes
  • Multi-LLM Orchestration - Claude, GPT-4, and custom models
  • Real-time Voice - Sub-100ms TTS/STT latency
  • RAG Knowledge Base - Domain-specific knowledge retrieval

Architecture Overview#

text
┌─────────────────────────────────────────────────────────────────┐
│                        Your Application                         │
├─────────────────────────────────────────────────────────────────┤
│  SDK (Python/TypeScript)  │  REST API  │  WebSocket  │  gRPC   │
├─────────────────────────────────────────────────────────────────┤
│                      Psyche API Gateway                         │
├──────────┬──────────┬──────────┬──────────┬─────────────────────┤
│ Sessions │ Personas │ Avatars  │Knowledge │ Tools               │
├──────────┴──────────┴──────────┴──────────┴─────────────────────┤
│                    Core Services Layer                          │
│  ┌──────────┐ ┌───────────┐ ┌──────────┐ ┌──────────────────┐  │
│  │ Avatar   │ │ Voice     │ │ LLM      │ │ Video            │  │
│  │ Engine   │ │ Pipeline  │ │ Gateway  │ │ Conferencing     │  │
│  └──────────┘ └───────────┘ └──────────┘ └──────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Integration Methods#

Method Use Case Latency
REST API CRUD operations, batch processing < 50ms
WebSocket Real-time events, streaming < 20ms
gRPC High-performance internal calls < 10ms
SDK Full-featured client integration Varies

Quick Start#

Get a Psyche assistant running in 5 minutes.

Step 1: Get API Credentials#

bash
# Sign up at https://console.serwaa.ai
# Create an API key with required scopes
export SERWAA_API_KEY="sk_live_your_api_key_here"

Step 2: Install SDK#

bash
# Python
pip install serwaa[async]

# TypeScript/Node.js
pnpm add @serwaa/sdk

Step 3: Create Your First Session#

Python:

python
import asyncio
from serwaa import AsyncSerwaaClient

async def main():
    async with AsyncSerwaaClient() as client:
        # Create a persona
        persona = await client.personas.create(
            name="Alex",
            description="Friendly customer support assistant",
            system_prompt="You are Alex, a helpful support agent.",
        )

        # Create a session
        session = await client.sessions.create(
            persona_id=persona.id,
            config={"auto_start": True},
        )

        # Send a message
        response = await client.sessions.send_message(
            session_id=session.id,
            content="Hello! How can I help you today?",
        )

        print(f"Session ID: {session.id}")
        print(f"Response: {response.content}")

asyncio.run(main())

TypeScript:

typescript
import { SerwaaClient } from '@serwaa/sdk';

async function main() {
  const client = new SerwaaClient();

  // Create a persona
  const persona = await client.personas.create({
    name: 'Alex',
    description: 'Friendly customer support assistant',
    systemPrompt: 'You are Alex, a helpful support agent.',
  });

  // Create a session
  const session = await client.sessions.create({
    personaId: persona.id,
    config: { autoStart: true },
  });

  // Send a message
  const response = await client.sessions.sendMessage({
    sessionId: session.id,
    content: 'Hello! How can I help you today?',
  });

  console.log(`Session ID: ${session.id}`);
  console.log(`Response: ${response.content}`);
}

main();

SDK Installation#

Python SDK#

Requirements:

  • Python 3.11+
  • pip or Poetry

Installation:

bash
# Basic installation
pip install serwaa

# With async support (recommended)
pip install serwaa[async]

# With all optional dependencies
pip install serwaa[all]

# Using Poetry
poetry add serwaa
poetry add serwaa -E async

SDK Features:

  • Full async/await support with AsyncSerwaaClient
  • Automatic retry with exponential backoff
  • Type hints for complete type safety
  • Auto-pagination for list endpoints
  • Context manager support for resource cleanup

TypeScript SDK#

Requirements:

  • Node.js 18+ or modern browser
  • pnpm, npm, or yarn

Installation:

bash
# pnpm (recommended)
pnpm add @serwaa/sdk

# npm
npm install @serwaa/sdk

# yarn
yarn add @serwaa/sdk

SDK Features:

  • Full TypeScript types with strict mode
  • Event streaming via WebSocket
  • Browser and Node.js compatible
  • Automatic reconnection handling
  • Promise-based async API

SDK Configuration#

python
from serwaa import AsyncSerwaaClient

client = AsyncSerwaaClient(
    api_key="sk_live_...",              # Required (or SERWAA_API_KEY env var)
    base_url="https://api.serwaa.ai",   # Optional: custom endpoint
    timeout=30.0,                        # Request timeout (seconds)
    max_retries=3,                       # Retry attempts for transient errors
    retry_delay=1.0,                     # Initial retry delay (seconds)
    retry_max_delay=30.0,                # Maximum retry delay
    retry_multiplier=2.0,                # Exponential backoff multiplier
)
typescript
import { SerwaaClient } from '@serwaa/sdk';

const client = new SerwaaClient({
  apiKey: 'sk_live_...', // Required (or SERWAA_API_KEY env var)
  baseUrl: 'https://api.serwaa.ai', // Optional: custom endpoint
  timeout: 30000, // Request timeout (milliseconds)
  maxRetries: 3, // Retry attempts
  debug: false, // Enable debug logging
});

Authentication#

Psyche supports multiple authentication methods for different use cases.

API Key Authentication#

Best for: Server-to-server integrations, backend services

bash
# Environment variable (recommended)
export SERWAA_API_KEY="sk_live_your_api_key"

# Header format
Authorization: Bearer sk_live_your_api_key
# or
X-API-Key: sk_live_your_api_key

API Key Prefixes:

Prefix Environment Usage
sk_live_ Production Live traffic
sk_test_ Staging Testing
sk_dev_ Development Local dev

API Key Scopes:

Scope Description
sessions:read Read session data
sessions:write Create/update sessions
personas:read Read persona data
personas:write Create/update personas
knowledge:read Query knowledge base
knowledge:write Upload/manage documents
tools:read Read tool configurations
tools:write Register/update tools
webhooks:read Read webhook configurations
webhooks:write Manage webhooks
admin Full administrative access

JWT Bearer Token Authentication#

Best for: User-authenticated applications, browser clients

python
# Login to get tokens
response = await client.auth.login(
    email="user@example.com",
    password="secure_password",
)

access_token = response.access_token      # Valid for 30 minutes
refresh_token = response.refresh_token    # Valid for 7 days

# Use access token for requests
client = AsyncSerwaaClient(api_key=access_token)

# Refresh when expired
new_tokens = await client.auth.refresh(refresh_token=refresh_token)

OAuth 2.0 Authorization Code Flow#

Best for: Third-party integrations, user consent flows

python
# Step 1: Generate authorization URL
auth_url = client.oauth.get_authorization_url(
    client_id="your_client_id",
    redirect_uri="https://yourapp.com/callback",
    scopes=["openid", "profile", "sessions:write"],
    state="random_state_string",
)

# Step 2: User authorizes and you receive callback with code
# GET https://yourapp.com/callback?code=auth_code&state=random_state_string

# Step 3: Exchange code for tokens
tokens = await client.oauth.exchange_code(
    client_id="your_client_id",
    client_secret="your_client_secret",
    code="auth_code",
    redirect_uri="https://yourapp.com/callback",
)

Webhook Signature Verification#

python
import hmac
import hashlib
import time

def verify_webhook_signature(
    payload: bytes,
    signature: str,
    timestamp: str,
    secret: str,
    tolerance_seconds: int = 300,
) -> bool:
    """
    Verify HMAC-SHA256 webhook signature.

    Args:
        payload: Raw request body bytes
        signature: X-Serwaa-Signature header value
        timestamp: X-Serwaa-Timestamp header value
        secret: Your webhook signing secret
        tolerance_seconds: Maximum age of request (default 5 minutes)

    Returns:
        True if signature is valid, False otherwise
    """
    # Check timestamp is within tolerance
    try:
        ts = int(timestamp)
        if abs(time.time() - ts) > tolerance_seconds:
            return False
    except ValueError:
        return False

    # Construct signed payload
    signed_payload = f"{timestamp}.{payload.decode('utf-8')}"

    # Compute expected signature
    expected = hmac.new(
        secret.encode("utf-8"),
        signed_payload.encode("utf-8"),
        hashlib.sha256,
    ).hexdigest()

    # Constant-time comparison
    return hmac.compare_digest(signature, expected)

Session Management#

Sessions represent active AI assistant interactions.

Session Lifecycle#

text
┌─────────┐    create    ┌─────────┐    start    ┌─────────┐
│ (none)  │ ─────────────▶│ created │ ────────────▶│ active  │
└─────────┘              └─────────┘              └────┬────┘
                                                       │
                              ┌────────────────────────┤
                              │                        │
                              ▼                        ▼
                         ┌─────────┐    resume    ┌─────────┐
                         │ paused  │ ◀───────────▶│ active  │
                         └─────────┘              └────┬────┘
                                                       │
                                                       │ end
                                                       ▼
                                                  ┌─────────┐
                                                  │  ended  │
                                                  └─────────┘

Creating Sessions#

python
from serwaa import AsyncSerwaaClient
from serwaa.types import SessionConfig, MediaConfig

async with AsyncSerwaaClient() as client:
    session = await client.sessions.create(
        persona_id="persona_abc123",
        config=SessionConfig(
            auto_start=False,           # Don't start immediately
            idle_timeout_seconds=300,   # End after 5 min idle
            max_duration_seconds=3600,  # Maximum 1 hour
            recording_enabled=True,     # Record session
        ),
        media=MediaConfig(
            video_enabled=True,
            audio_enabled=True,
            resolution="1080p",
            frame_rate=30,
        ),
        metadata={
            "user_id": "user_123",
            "channel": "web",
        },
    )

    # Start the session when ready
    await client.sessions.start(session.id)

    # Send messages
    response = await client.sessions.send_message(
        session_id=session.id,
        content="Hello!",
    )

    # Pause if needed
    await client.sessions.pause(session.id)

    # Resume later
    await client.sessions.resume(session.id)

    # End the session
    await client.sessions.end(session.id)

Session with Video Conferencing#

python
# Create session and join a meeting
session = await client.sessions.create(
    persona_id="persona_abc123",
    config={"auto_start": True},
)

# Join Zoom meeting
await client.sessions.join_meeting(
    session_id=session.id,
    meeting_url="https://zoom.us/j/1234567890",
    display_name="Alex (AI Assistant)",
    join_options={
        "mute_on_join": False,
        "video_on_join": True,
        "waiting_room": False,
    },
)

Conferencing Platform Integration#

Psyche integrates with major video conferencing platforms to provide AI assistants in meetings.

Zoom Integration#

Prerequisites#

  1. Zoom App Credentials:

    • Create a Server-to-Server OAuth app at https://marketplace.zoom.us/
    • Or create an SDK app for client-side integration
    • Required scopes: meeting:read, meeting:write, recording:read
  2. Environment Variables:

    bash
    ZOOM_CLIENT_ID="your_client_id"
    ZOOM_CLIENT_SECRET="your_client_secret"
    ZOOM_ACCOUNT_ID="your_account_id"  # For S2S OAuth
    

SDK JWT Authentication (Meeting SDK)#

python
from video_conferencing.zoom.auth import SDKJWTGenerator

# Generate JWT for Zoom Meeting SDK
generator = SDKJWTGenerator(
    client_id="your_client_id",
    client_secret="your_client_secret",
)

token = generator.generate(
    meeting_number="1234567890",
    role=0,  # 0 = participant, 1 = host
    expiration_seconds=7200,  # 2 hours
)

# Use token to join meeting via Meeting SDK

Server-to-Server OAuth (REST API)#

python
from video_conferencing.zoom.auth import ZoomAuthManager
from video_conferencing.zoom.types import ZoomCredentials, ZoomAuthType

auth = ZoomAuthManager(
    ZoomCredentials(
        client_id="your_client_id",
        client_secret="your_client_secret",
        account_id="your_account_id",
        auth_type=ZoomAuthType.OAUTH_S2S,
    )
)

# Get access token for API calls
access_token = await auth.get_api_token()

Zoom REST API Operations#

python
from video_conferencing.zoom.api import ZoomAPIClient
from video_conferencing.zoom.types import (
    CreateMeetingRequest,
    ZoomMeetingType,
    ZoomRecordingType,
)

async with ZoomAPIClient(credentials) as client:
    # Create a meeting
    meeting = await client.create_meeting(
        user_id="me",
        request=CreateMeetingRequest(
            topic="AI Assistant Demo",
            type=ZoomMeetingType.SCHEDULED,
            start_time="2024-01-15T14:00:00Z",
            duration=60,
            timezone="America/New_York",
            auto_recording=ZoomRecordingType.CLOUD,
            settings={
                "host_video": True,
                "participant_video": True,
                "join_before_host": True,
                "waiting_room": False,
            },
        ),
    )

    print(f"Join URL: {meeting.join_url}")

    # Get meeting details
    details = await client.get_meeting(meeting.id)

    # List participants
    participants = await client.get_meeting_participants(meeting.id)

    # Get recordings
    recordings = await client.get_recordings(user_id="me")

    # End meeting
    await client.end_meeting(meeting.id)

Join Meeting via Psyche SDK#

python
# High-level SDK integration
session = await client.sessions.create(persona_id="persona_abc123")

await client.sessions.join_meeting(
    session_id=session.id,
    platform="zoom",
    meeting_url="https://zoom.us/j/1234567890?pwd=meeting_password",
    display_name="Alex (AI)",
    join_options={
        "mute_on_join": False,
        "video_on_join": True,
    },
)

# Monitor meeting events
async for event in client.sessions.stream(session.id):
    if event.type == "participant.joined":
        print(f"{event.participant.name} joined")
    elif event.type == "participant.left":
        print(f"{event.participant.name} left")
    elif event.type == "transcript":
        print(f"{event.speaker}: {event.text}")

Google Meet Integration#

Prerequisites#

  1. Google Cloud Project:

    • Enable Google Meet API
    • Create OAuth 2.0 credentials
    • Required scopes: meet.googleapis.com/meetings.*
  2. Service Account (for bots):

    • Create service account with domain-wide delegation
    • Download JSON key file
  3. Environment Variables:

    bash
    GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
    GOOGLE_MEET_DOMAIN="your-domain.com"
    

WebRTC Configuration#

python
from video_conferencing.meet.webrtc import MeetWebRTCClient, SDPBuilder
from video_conferencing.meet.types import MeetSDPConfig, MeetMediaConfig

# Configure SDP for WebRTC connection
sdp_config = MeetSDPConfig(
    session_id="unique_session_id",
    version=0,
    ice_ufrag="generated_ufrag",
    ice_pwd="generated_password",
)

sdp_builder = SDPBuilder(sdp_config)

# Build SDP offer
offer = sdp_builder.build_offer(
    receive_audio=True,
    receive_video=True,
    send_audio=True,     # AI can speak
    send_video=True,     # AI has avatar video
)

# Create WebRTC client
webrtc_client = MeetWebRTCClient(
    api_credentials=credentials,
    media_config=MeetMediaConfig(
        audio_codec="opus",
        video_codec="vp9",
        max_bitrate=2500000,  # 2.5 Mbps
    ),
)

# Connect to meeting
await webrtc_client.connect(
    meeting_code="abc-defg-hij",
    local_sdp=offer,
)

# Handle incoming media
async for frame in webrtc_client.receive_video():
    # Process video frames
    pass

async for audio in webrtc_client.receive_audio():
    # Process audio chunks
    pass

# Send AI audio/video
await webrtc_client.send_audio(audio_data)
await webrtc_client.send_video(video_frame)

Google Meet API Operations#

python
from video_conferencing.meet.api import MeetAPIClient

async with MeetAPIClient(credentials) as client:
    # Create a meeting space
    space = await client.create_space(
        display_name="AI Assistant Meeting",
        config={
            "access_type": "OPEN",  # or "TRUSTED"
            "entry_point_access": "ALL",
        },
    )

    print(f"Meeting URL: {space.meeting_uri}")

    # Get meeting details
    meeting = await client.get_meeting(space.name)

    # List participants
    participants = await client.list_participants(space.name)

    # End meeting
    await client.end_meeting(space.name)

Microsoft Teams Integration#

Prerequisites#

  1. Azure AD App Registration:

    • Register app in Azure Portal
    • Grant Microsoft Graph permissions
    • Required permissions: OnlineMeetings.ReadWrite, User.Read
  2. Bot Framework Registration:

    • Register bot at https://dev.botframework.com/
    • Configure messaging endpoint
  3. Environment Variables:

    bash
    AZURE_TENANT_ID="your_tenant_id"
    AZURE_CLIENT_ID="your_client_id"
    AZURE_CLIENT_SECRET="your_client_secret"
    TEAMS_BOT_ID="your_bot_id"
    

Teams Integration#

python
from video_conferencing.teams.api import TeamsAPIClient
from video_conferencing.teams.types import TeamsCredentials

async with TeamsAPIClient(
    TeamsCredentials(
        tenant_id="your_tenant_id",
        client_id="your_client_id",
        client_secret="your_client_secret",
    )
) as client:
    # Create online meeting
    meeting = await client.create_meeting(
        subject="AI Assistant Demo",
        start_time="2024-01-15T14:00:00Z",
        end_time="2024-01-15T15:00:00Z",
        participants=["user@domain.com"],
    )

    print(f"Join URL: {meeting.join_url}")

    # Join as bot
    await client.join_meeting_as_bot(
        meeting_id=meeting.id,
        display_name="Alex (AI)",
        media_config={
            "audio_enabled": True,
            "video_enabled": True,
        },
    )

Cisco WebEx Integration#

Prerequisites#

  1. WebEx Developer Account:

    • Create integration at https://developer.webex.com/
    • Required scopes: meeting:schedules_write, meeting:recordings_read
  2. Environment Variables:

    bash
    WEBEX_ACCESS_TOKEN="your_access_token"
    WEBEX_BOT_ID="your_bot_id"
    

WebEx API Operations#

python
from video_conferencing.webex.api import WebexAPIClient
from video_conferencing.webex.types import WebexCredentials

async with WebexAPIClient(
    WebexCredentials(
        access_token="your_access_token",
        bot_id="your_bot_id",
    )
) as client:
    # Create meeting
    meeting = await client.create_meeting(
        title="AI Assistant Demo",
        start_time="2024-01-15T14:00:00Z",
        duration_minutes=60,
        invitees=["user@domain.com"],
    )

    print(f"Join Link: {meeting.web_link}")

    # Get meeting info
    details = await client.get_meeting(meeting.id)

    # List participants
    participants = await client.list_participants(meeting.id)

    # Get recordings
    recordings = await client.list_recordings(meeting_id=meeting.id)

Platform Comparison#

Feature Zoom Google Meet Teams WebEx
WebRTC Native No Yes No No
Bot SDK Yes Limited Yes Yes
REST API Yes Yes Yes Yes
Real-time Transcript Yes Yes Yes Yes
Cloud Recording Yes Yes Yes Yes
Breakout Rooms Yes Yes Yes Yes
Max Participants 1000 500 1000 1000
AI Avatar Support Full Full Full Full

Custom Avatar Creation#

Create photorealistic AI avatars using 3D Gaussian Splatting and FLAME models.

Avatar Creation Pipeline#

text
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Video Input  │────▶│ Face Extract │────▶│ FLAME Fit    │
│ (30-60 sec)  │     │ + Align      │     │ + Params     │
└──────────────┘     └──────────────┘     └──────────────┘
                                                  │
                                                  ▼
┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ Avatar Ready │◀────│ Optimization │◀────│ 3DGS Init    │
│ (Deployable) │     │ (30K iters)  │     │ + Splats     │
└──────────────┘     └──────────────┘     └──────────────┘

Recording Requirements#

For best avatar quality:

Requirement Specification
Duration 30-60 seconds minimum
Resolution 1080p or higher
Frame Rate 30 FPS minimum
Lighting Even, diffused lighting
Background Solid color, no clutter
Head Movement Slow rotations, all angles
Expressions Neutral, smile, talk
Audio Not required for avatar

Create Avatar from Video#

python
from avatar_engine.engine import AvatarEngine
from avatar_engine.config import AvatarConfig
from avatar_engine.models.avatar import AvatarMetadata

# Initialize avatar engine
config = AvatarConfig(
    device="cuda",                    # GPU device
    model_path="/models/flame",       # FLAME model path
    default_resolution=(1920, 1080),  # Output resolution
)
engine = AvatarEngine(config)
await engine.initialize()

# Create avatar from video
avatar = await engine.create_avatar(
    video_path="/path/to/capture.mp4",
    metadata=AvatarMetadata(
        name="Custom Avatar",
        owner_id="user_123",
        description="Executive avatar for presentations",
        gender="male",
        age_range="40-50",
        ethnicity="caucasian",
        tags=["executive", "formal"],
    ),
    training_config={
        "iterations": 30000,          # Training iterations
        "checkpoint_interval": 5000,  # Save checkpoints
        "early_stopping": True,       # Stop if converged
    },
)

print(f"Avatar ID: {avatar.id}")
print(f"Status: {avatar.status}")

Training Configuration#

python
from avatar_engine.training.trainer import TrainingConfig

config = TrainingConfig(
    # Basic training parameters
    iterations=30000,                 # Total training iterations
    batch_size=1,                     # Images per batch (usually 1 for 3DGS)

    # Learning rates
    position_lr_init=0.00016,         # Initial position learning rate
    position_lr_final=0.0000016,      # Final position learning rate
    position_lr_delay_mult=0.01,      # LR delay multiplier
    position_lr_max_steps=30000,      # Steps for LR decay

    feature_lr=0.0025,                # Feature learning rate
    opacity_lr=0.05,                  # Opacity learning rate
    scaling_lr=0.005,                 # Scale learning rate
    rotation_lr=0.001,                # Rotation learning rate

    # Blendshape training (FLAME expressions)
    train_blendshapes=True,           # Enable expression learning
    blendshape_lr=0.001,              # Blendshape learning rate
    num_blendshapes=50,               # FLAME expression coefficients

    # Regularization
    lambda_dssim=0.2,                 # SSIM loss weight
    lambda_l1=0.8,                    # L1 loss weight
    lambda_opacity=0.01,              # Opacity regularization

    # Densification
    densify_from_iter=500,            # Start densifying
    densify_until_iter=15000,         # Stop densifying
    densification_interval=100,       # Densify every N iters
    densify_grad_threshold=0.0002,    # Gradient threshold

    # Checkpointing
    checkpoint_interval=5000,         # Save every N iterations
    early_stopping_patience=5000,     # Stop if no improvement

    # Hardware
    device="cuda",                    # GPU device
)

FLAME Model Parameters#

The FLAME model provides parametric control over avatar appearance:

python
from avatar_engine.models.flame import FLAMEModel, FLAMEParams

flame = FLAMEModel(model_path="/models/flame")

# FLAME parameters
params = FLAMEParams(
    # Shape parameters (identity) - 100 coefficients
    shape=torch.zeros(100),

    # Expression parameters - 50 coefficients
    expression=torch.zeros(50),

    # Pose parameters
    jaw_pose=torch.zeros(3),          # Jaw rotation
    neck_pose=torch.zeros(3),         # Neck rotation
    eye_pose=torch.zeros(6),          # Left/right eye gaze

    # Global transformation
    global_rotation=torch.zeros(3),   # Head rotation
    translation=torch.zeros(3),       # Head position
)

# Generate mesh vertices
vertices = flame.forward(params)

Expression Mapping#

Map emotions to FLAME expression coefficients:

python
# Predefined expression presets
EXPRESSION_PRESETS = {
    "neutral": torch.zeros(50),
    "smile": torch.tensor([0.8, 0.2, ...]),      # 50 coefficients
    "surprise": torch.tensor([0.1, 0.9, ...]),
    "sad": torch.tensor([-0.3, 0.1, ...]),
    "angry": torch.tensor([-0.5, -0.2, ...]),
    "fear": torch.tensor([0.2, 0.7, ...]),
    "disgust": torch.tensor([-0.4, 0.3, ...]),
    "contempt": torch.tensor([0.1, -0.1, ...]),
}

# Blend expressions
def blend_expressions(expr1, expr2, weight):
    return expr1 * (1 - weight) + expr2 * weight

# Animate between expressions
current = EXPRESSION_PRESETS["neutral"]
target = EXPRESSION_PRESETS["smile"]

for t in range(30):  # 30 frames = 1 second at 30 FPS
    weight = t / 30.0
    expression = blend_expressions(current, target, weight)
    frame = engine.render(avatar, expression_params=expression)

Rendering Pipeline#

python
from avatar_engine.rendering.rasterizer import GaussianRasterizer

rasterizer = GaussianRasterizer(
    image_height=1080,
    image_width=1920,
    tanfovx=0.5,                      # Horizontal FOV tangent
    tanfovy=0.5 * 1080/1920,          # Vertical FOV tangent
    bg=torch.zeros(3),                # Background color (black)
    scale_modifier=1.0,
    viewmatrix=view_matrix,           # 4x4 view matrix
    projmatrix=proj_matrix,           # 4x4 projection matrix
    sh_degree=3,                      # Spherical harmonics degree
    campos=camera_position,           # Camera position
    prefiltered=False,
    debug=False,
)

# Render frame
rendered = rasterizer.forward(
    means3D=gaussian_means,           # Gaussian positions
    means2D=screen_space_points,      # 2D projections
    shs=spherical_harmonics,          # Color SH coefficients
    colors_precomp=None,              # Or precomputed colors
    opacities=opacities,              # Gaussian opacities
    scales=scales,                    # Gaussian scales
    rotations=rotations,              # Gaussian rotations
    cov3D_precomp=None,               # Or precomputed covariance
)

# Get RGB image
image = rendered["render"]            # [3, H, W] tensor

Avatar API#

python
# Via Psyche SDK
async with AsyncSerwaaClient() as client:
    # Create avatar from video
    avatar = await client.avatars.create(
        video_url="https://storage.example.com/capture.mp4",
        name="Executive Avatar",
        metadata={
            "gender": "male",
            "style": "formal",
        },
    )

    # Check training status
    status = await client.avatars.get_status(avatar.id)
    print(f"Progress: {status.progress}%")

    # Wait for completion
    avatar = await client.avatars.wait_for_ready(
        avatar.id,
        timeout=3600,  # 1 hour max
    )

    # Use avatar in session
    session = await client.sessions.create(
        persona_id="persona_abc123",
        avatar_id=avatar.id,
    )

    # List avatars
    avatars = await client.avatars.list(owner_id="user_123")

    # Delete avatar
    await client.avatars.delete(avatar.id)

Persona Configuration#

Personas define the AI assistant's personality, knowledge, and behavior.

Persona Structure#

python
from serwaa.types import (
    PersonaConfig,
    PersonalityConfig,
    VoiceConfig,
    AppearanceConfig,
)

persona = await client.personas.create(
    name="Alex",
    description="Senior customer support specialist",

    # System prompt (core behavior)
    system_prompt="""You are Alex, a senior customer support specialist
    at TechCorp. You have 10 years of experience helping customers with
    technical issues. You are patient, knowledgeable, and always maintain
    a professional yet friendly demeanor.

    Key behaviors:
    - Always greet customers warmly
    - Ask clarifying questions before providing solutions
    - Explain technical concepts in simple terms
    - Offer to escalate complex issues to human agents
    - End conversations with a satisfaction check
    """,

    # Personality configuration (Big Five model)
    personality=PersonalityConfig(
        # Big Five traits (0.0 to 1.0)
        openness=0.7,           # Curious, creative
        conscientiousness=0.8,  # Organized, dependable
        extraversion=0.6,       # Sociable, energetic
        agreeableness=0.9,      # Cooperative, trusting
        neuroticism=0.2,        # Emotionally stable

        # Communication style
        formality=0.6,          # 0=casual, 1=formal
        friendliness=0.8,       # 0=reserved, 1=warm
        verbosity=0.5,          # 0=terse, 1=verbose
        humor=0.3,              # 0=serious, 1=humorous
        empathy=0.9,            # 0=detached, 1=empathetic
        assertiveness=0.5,      # 0=passive, 1=assertive
    ),

    # Voice configuration
    voice=VoiceConfig(
        provider="elevenlabs",
        voice_id="rachel",
        language="en-US",
        speed=1.0,              # Speech rate multiplier
        pitch=0.0,              # Pitch adjustment
        stability=0.75,         # Voice consistency
        similarity_boost=0.75,  # Voice clarity
    ),

    # Avatar appearance
    appearance=AppearanceConfig(
        avatar_id="avatar_abc123",  # Custom avatar
        # Or use preset
        preset="professional_female",
        background="office",
    ),

    # Metadata
    tags=["support", "technical", "tier-2"],
    metadata={
        "department": "Customer Support",
        "tier": 2,
        "languages": ["en", "es"],
    },
)

Personality Presets#

python
# Available presets
PERSONALITY_PRESETS = {
    "warm_professional": {
        "openness": 0.6,
        "conscientiousness": 0.8,
        "extraversion": 0.6,
        "agreeableness": 0.8,
        "neuroticism": 0.2,
        "formality": 0.6,
        "friendliness": 0.8,
    },
    "analytical": {
        "openness": 0.8,
        "conscientiousness": 0.9,
        "extraversion": 0.4,
        "agreeableness": 0.6,
        "neuroticism": 0.3,
        "formality": 0.7,
        "verbosity": 0.6,
    },
    "friendly_casual": {
        "openness": 0.7,
        "conscientiousness": 0.6,
        "extraversion": 0.8,
        "agreeableness": 0.9,
        "neuroticism": 0.2,
        "formality": 0.3,
        "humor": 0.6,
    },
}

# Use preset
persona = await client.personas.create(
    name="Alex",
    personality_preset="warm_professional",
    ...
)

Memory System#

python
# Add facts (persistent knowledge)
await client.personas.add_fact(
    persona_id=persona.id,
    key="company_name",
    value="TechCorp",
    category="organization",
)

await client.personas.add_fact(
    persona_id=persona.id,
    key="support_hours",
    value="24/7",
    category="policy",
)

# Add opinions (subjective preferences)
await client.personas.add_opinion(
    persona_id=persona.id,
    topic="remote_work",
    opinion="I believe remote work improves productivity when done right.",
    strength=0.7,  # How strongly held
)

# Get all memories
memories = await client.personas.get_memories(persona.id)

# Clear memories
await client.personas.clear_memories(
    persona_id=persona.id,
    categories=["temporary"],  # Only clear specific categories
)

Expertise Domains#

python
# Add domain expertise
await client.personas.add_expertise(
    persona_id=persona.id,
    domain="technical_support",
    config={
        "product_categories": ["software", "hardware"],
        "escalation_threshold": 3,  # Escalate after 3 failed attempts
        "knowledge_base_ids": ["kb_abc123", "kb_def456"],
    },
)

await client.personas.add_expertise(
    persona_id=persona.id,
    domain="sales",
    config={
        "products": ["enterprise", "pro", "basic"],
        "discount_authority": 0.15,  # Can offer up to 15% discount
        "upsell_enabled": True,
    },
)

Knowledge Base Integration#

Integrate domain-specific knowledge for RAG-powered responses.

Document Upload#

python
from pathlib import Path

# Upload file
document = await client.knowledge.upload(
    file=Path("product_manual.pdf"),
    title="Product Manual v2.0",
    description="Complete user guide for ProductX",
    persona_ids=["persona_abc123"],
    tags=["manual", "product", "v2"],
    metadata={
        "version": "2.0",
        "author": "TechCorp",
        "publish_date": "2024-01-01",
    },
)

print(f"Document ID: {document.id}")
print(f"Status: {document.status}")  # processing, ready, failed

# Wait for processing
document = await client.knowledge.wait_for_ready(document.id)

Supported File Types#

Format Max Size Notes
PDF 100 MB Text + images
DOCX 50 MB Word documents
TXT 10 MB Plain text
MD 10 MB Markdown
HTML 10 MB Web pages
CSV 50 MB Structured data
JSON 50 MB Structured data
XLSX 50 MB Excel spreadsheets

Text Content Creation#

python
# Create from text
document = await client.knowledge.create(
    title="Product FAQ",
    content="""
    ## Frequently Asked Questions

    ### Q: What is your return policy?
    A: We offer a 30-day money-back guarantee on all products.

    ### Q: How do I contact support?
    A: You can reach us via email at support@example.com or
    call 1-800-SUPPORT.

    ### Q: What payment methods do you accept?
    A: We accept Visa, MasterCard, American Express, and PayPal.
    """,
    document_type="text",
    persona_ids=["persona_abc123"],
    tags=["faq", "support"],
)

URL Import#

python
# Import from URL
document = await client.knowledge.import_url(
    url="https://docs.example.com/api-reference",
    title="API Reference",
    persona_ids=["persona_abc123"],
    crawl_depth=2,  # Follow links 2 levels deep
    include_patterns=["docs.example.com/*"],
    exclude_patterns=["*/login", "*/admin/*"],
)
python
# Search knowledge base
results = await client.knowledge.search(
    query="How do I return a product?",
    persona_id="persona_abc123",
    top_k=5,                    # Number of results
    min_score=0.5,              # Minimum relevance score
    tags=["faq", "support"],    # Filter by tags
)

for result in results:
    print(f"Score: {result.score}")
    print(f"Document: {result.document.title}")
    print(f"Content: {result.content[:200]}...")

Collection Management#

python
# Create collection
collection = await client.knowledge.create_collection(
    name="Product Documentation",
    description="All product-related documentation",
    persona_ids=["persona_abc123"],
)

# Add documents to collection
await client.knowledge.add_to_collection(
    collection_id=collection.id,
    document_ids=["doc_1", "doc_2", "doc_3"],
)

# Search within collection
results = await client.knowledge.search(
    query="installation guide",
    collection_id=collection.id,
)

# List collections
collections = await client.knowledge.list_collections()

# Delete collection
await client.knowledge.delete_collection(collection.id)

Tool & Function Calling#

Extend Psyche with custom tools and function calling.

Tool Types#

Type Description Execution
API Call external REST API Server-side
Webhook Send data to your endpoint Server-side
Function Execute in your application Client-side

API Tool#

python
# Register API tool
tool = await client.tools.create(
    name="check_order_status",
    description="Check the status of a customer order by order ID",
    tool_type="api",
    parameters=[
        {
            "name": "order_id",
            "type": "string",
            "description": "The order ID to check",
            "required": True,
        },
    ],
    endpoint_url="https://api.yourcompany.com/orders/{order_id}/status",
    http_method="GET",
    auth_type="bearer",
    auth_token="your_api_token",
    headers={
        "X-API-Version": "2024-01",
    },
    timeout_seconds=10,
    persona_ids=["persona_abc123"],
)

Webhook Tool#

python
# Register webhook tool
tool = await client.tools.create(
    name="create_support_ticket",
    description="Create a support ticket for the customer",
    tool_type="webhook",
    parameters=[
        {
            "name": "subject",
            "type": "string",
            "description": "Ticket subject",
            "required": True,
        },
        {
            "name": "description",
            "type": "string",
            "description": "Detailed description of the issue",
            "required": True,
        },
        {
            "name": "priority",
            "type": "string",
            "description": "Ticket priority",
            "required": False,
            "enum": ["low", "medium", "high", "critical"],
            "default": "medium",
        },
    ],
    webhook_url="https://api.yourcompany.com/webhooks/create-ticket",
    webhook_secret="whsec_your_secret",
    persona_ids=["persona_abc123"],
)

Function Tool (Client-Side)#

python
# Register function tool
tool = await client.tools.create(
    name="calculate_shipping",
    description="Calculate shipping cost based on weight and destination",
    tool_type="function",
    parameters=[
        {
            "name": "weight_kg",
            "type": "number",
            "description": "Package weight in kilograms",
            "required": True,
        },
        {
            "name": "destination",
            "type": "string",
            "description": "Destination country code (ISO 3166-1)",
            "required": True,
        },
        {
            "name": "service",
            "type": "string",
            "description": "Shipping service level",
            "required": False,
            "enum": ["standard", "express", "overnight"],
            "default": "standard",
        },
    ],
    persona_ids=["persona_abc123"],
)

# Handle function calls in your application
async for event in client.sessions.stream(session_id):
    if event.type == "tool_call":
        if event.tool_name == "calculate_shipping":
            # Execute your function
            result = calculate_shipping(
                weight_kg=event.arguments["weight_kg"],
                destination=event.arguments["destination"],
                service=event.arguments.get("service", "standard"),
            )

            # Send result back
            await client.sessions.send_tool_result(
                session_id=session_id,
                tool_call_id=event.tool_call_id,
                result={
                    "cost": result.cost,
                    "currency": result.currency,
                    "estimated_days": result.estimated_days,
                },
            )

Tool Execution Flow#

text
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ User Input  │────▶│ LLM Decides │────▶│ Tool Call   │
│             │     │ to Use Tool │     │ Event       │
└─────────────┘     └─────────────┘     └──────┬──────┘
                                               │
                    ┌──────────────────────────┤
                    │                          │
                    ▼                          ▼
              ┌───────────┐            ┌───────────────┐
              │ API/      │            │ Function Tool │
              │ Webhook   │            │ (Your Code)   │
              │ (Auto)    │            │               │
              └─────┬─────┘            └───────┬───────┘
                    │                          │
                    │    send_tool_result      │
                    │◀─────────────────────────┤
                    │                          │
                    ▼                          │
              ┌───────────┐                    │
              │ LLM       │◀───────────────────┘
              │ Response  │
              └───────────┘

WebSocket Streaming#

Real-time event streaming for live interactions.

Connection#

python
import websockets
import json

async def connect_websocket(session_id: str, api_key: str):
    url = f"wss://api.serwaa.ai/api/v1/ws/sessions/{session_id}"
    headers = {"Authorization": f"Bearer {api_key}"}

    async with websockets.connect(url, extra_headers=headers) as ws:
        async for message in ws:
            event = json.loads(message)
            await handle_event(event)

Event Types#

Event Description
session.started Session has started
session.paused Session was paused
session.resumed Session was resumed
session.ended Session has ended
transcript Speech-to-text transcript
response.started AI started generating response
response.chunk Partial response text
response.completed Full response available
audio.started Audio playback started
audio.chunk Audio data chunk (base64)
audio.completed Audio playback completed
video.frame Avatar video frame
tool_call Tool invocation request
tool_result Tool execution result
participant.joined Participant joined (meetings)
participant.left Participant left (meetings)
error Error occurred
metrics Performance metrics

Event Handling#

python
async def handle_event(event: dict):
    event_type = event["type"]
    data = event.get("data", {})

    if event_type == "transcript":
        # Real-time speech transcription
        print(f"[{data['speaker']}]: {data['text']}")
        if data["is_final"]:
            # Final transcript, process it
            await process_user_input(data["text"])

    elif event_type == "response.chunk":
        # Streaming response
        print(data["text"], end="", flush=True)

    elif event_type == "response.completed":
        # Full response available
        print(f"\n[AI]: {data['full_text']}")

    elif event_type == "audio.chunk":
        # Audio data for playback
        audio_bytes = base64.b64decode(data["audio"])
        await play_audio(audio_bytes)

    elif event_type == "video.frame":
        # Avatar video frame
        frame_bytes = base64.b64decode(data["frame"])
        await display_frame(frame_bytes)

    elif event_type == "tool_call":
        # Function tool invocation
        result = await execute_tool(
            data["tool_name"],
            data["arguments"],
        )
        await send_tool_result(data["tool_call_id"], result)

    elif event_type == "error":
        print(f"Error: {data['message']}")

    elif event_type == "metrics":
        # Performance metrics
        print(f"Latency: {data['latency_ms']}ms")

Control Commands#

python
async def send_command(ws, command: str, data: dict = None):
    message = {"command": command}
    if data:
        message["data"] = data
    await ws.send(json.dumps(message))

# Interrupt AI response
await send_command(ws, "interrupt")

# Mute/unmute
await send_command(ws, "mute", {"audio": True, "video": False})

# Change persona mid-session
await send_command(ws, "switch_persona", {"persona_id": "new_persona"})

# Send text message
await send_command(ws, "message", {"content": "Hello!"})

# Request metrics
await send_command(ws, "get_metrics")

SDK Streaming#

python
# Using SDK for simpler streaming
async for event in client.sessions.stream(session_id):
    if event.type == "transcript" and event.is_final:
        print(f"User: {event.text}")

    elif event.type == "response.completed":
        print(f"AI: {event.full_text}")

    elif event.type == "tool_call":
        # Handle tool call
        pass

Webhook Integration#

Receive events via webhooks for server-to-server integration.

Webhook Registration#

python
webhook = await client.webhooks.create(
    url="https://api.yourcompany.com/webhooks/serwaa",
    description="Production webhook endpoint",
    events=[
        "session.started",
        "session.ended",
        "conversation.message",
        "tool.called",
        "tool.completed",
    ],
    secret="whsec_your_signing_secret",
    active=True,
    metadata={
        "environment": "production",
    },
)

Event Categories#

Session Events:

  • session.created - Session created
  • session.started - Session started
  • session.paused - Session paused
  • session.resumed - Session resumed
  • session.ended - Session ended
  • session.error - Session error

Conversation Events:

  • conversation.message - New message (user or AI)
  • conversation.transcript - Speech transcript

Participant Events:

  • participant.joined - Participant joined
  • participant.left - Participant left

Tool Events:

  • tool.called - Tool invocation started
  • tool.completed - Tool execution completed
  • tool.failed - Tool execution failed

Knowledge Events:

  • document.processed - Document processing complete
  • document.failed - Document processing failed
  • knowledge.retrieved - Knowledge retrieved for response

Analytics Events:

  • analytics.metrics - Real-time metrics
  • analytics.summary - Session summary

Webhook Handler#

python
from fastapi import FastAPI, Request, HTTPException, Header
import hmac
import hashlib
import time

app = FastAPI()
WEBHOOK_SECRET = "whsec_your_signing_secret"

@app.post("/webhooks/serwaa")
async def handle_webhook(
    request: Request,
    x_serwaa_signature: str = Header(...),
    x_serwaa_timestamp: str = Header(...),
):
    # Get raw body
    body = await request.body()

    # Verify signature
    if not verify_signature(body, x_serwaa_signature, x_serwaa_timestamp):
        raise HTTPException(status_code=401, detail="Invalid signature")

    # Parse event
    event = await request.json()
    event_type = event["type"]
    data = event["data"]

    # Handle event
    if event_type == "session.ended":
        session_id = data["session_id"]
        duration = data["duration_seconds"]
        await log_session_ended(session_id, duration)

    elif event_type == "conversation.message":
        await store_message(
            session_id=data["session_id"],
            role=data["role"],
            content=data["content"],
            timestamp=data["timestamp"],
        )

    elif event_type == "tool.completed":
        await log_tool_usage(
            tool_name=data["tool_name"],
            duration_ms=data["duration_ms"],
            success=data["success"],
        )

    # Return 200 to acknowledge receipt
    return {"status": "ok"}


def verify_signature(
    payload: bytes,
    signature: str,
    timestamp: str,
) -> bool:
    """Verify webhook signature."""
    # Check timestamp (5 minute tolerance)
    try:
        ts = int(timestamp)
        if abs(time.time() - ts) > 300:
            return False
    except ValueError:
        return False

    # Compute expected signature
    signed_payload = f"{timestamp}.{payload.decode('utf-8')}"
    expected = hmac.new(
        WEBHOOK_SECRET.encode("utf-8"),
        signed_payload.encode("utf-8"),
        hashlib.sha256,
    ).hexdigest()

    return hmac.compare_digest(signature, expected)

Retry Policy#

Attempt Delay Total Time
1 Immediate 0s
2 1 minute 1m
3 5 minutes 6m
4 30 minutes 36m
5 2 hours 2h 36m
6 6 hours 8h 36m
7 12 hours 20h 36m
8 24 hours 44h 36m

After 8 failed attempts, the webhook is disabled and requires manual re-activation.

Webhook Management#

python
# List webhooks
webhooks = await client.webhooks.list()

# Update webhook
await client.webhooks.update(
    webhook_id=webhook.id,
    events=["session.ended"],  # Change subscribed events
    active=True,
)

# Test webhook (sends test event)
await client.webhooks.test(webhook.id)

# Get delivery history
deliveries = await client.webhooks.list_deliveries(
    webhook_id=webhook.id,
    limit=100,
)

# Retry failed delivery
await client.webhooks.retry_delivery(delivery_id="del_abc123")

# Delete webhook
await client.webhooks.delete(webhook.id)

Error Handling#

Exception Types#

python
from serwaa.exceptions import (
    SerwaaError,           # Base exception
    AuthenticationError,   # 401 - Invalid credentials
    AuthorizationError,    # 403 - Insufficient permissions
    NotFoundError,         # 404 - Resource not found
    ValidationError,       # 400 - Invalid request
    RateLimitError,        # 429 - Rate limited
    ConflictError,         # 409 - Resource conflict
    ServerError,           # 5xx - Server error
    ConnectionError,       # Network error
    TimeoutError,          # Request timeout
)

Error Handling Patterns#

python
from serwaa.exceptions import (
    SerwaaError,
    AuthenticationError,
    NotFoundError,
    RateLimitError,
    ValidationError,
)

async def safe_operation():
    try:
        session = await client.sessions.get("session_abc123")
        return session

    except AuthenticationError as e:
        # Invalid or expired API key
        logger.error(f"Auth failed: {e.message}")
        # Refresh credentials or re-authenticate
        raise

    except NotFoundError as e:
        # Resource doesn't exist
        logger.warning(f"Session not found: {e.resource_id}")
        return None

    except RateLimitError as e:
        # Rate limited - retry after delay
        logger.warning(f"Rate limited, retry after {e.retry_after}s")
        await asyncio.sleep(e.retry_after)
        return await safe_operation()  # Retry

    except ValidationError as e:
        # Invalid request parameters
        logger.error(f"Validation error: {e.message}")
        for error in e.errors:
            logger.error(f"  - {error['field']}: {error['message']}")
        raise

    except SerwaaError as e:
        # Generic API error
        logger.error(f"API error [{e.code}]: {e.message}")
        raise

Error Response Format#

json
{
  "error": {
    "code": "RESOURCE_NOT_FOUND",
    "message": "Session with ID 'session_abc123' not found",
    "details": {
      "resource_type": "session",
      "resource_id": "session_abc123"
    },
    "request_id": "req_xyz789"
  }
}

Common Error Codes#

Code HTTP Description
INVALID_API_KEY 401 API key is invalid or expired
INSUFFICIENT_SCOPE 403 API key lacks required scope
RESOURCE_NOT_FOUND 404 Requested resource not found
VALIDATION_ERROR 400 Request validation failed
RATE_LIMIT_EXCEEDED 429 Too many requests
SESSION_EXPIRED 400 Session has ended
PERSONA_UNAVAILABLE 503 Persona is being updated
AVATAR_TRAINING 503 Avatar still training
QUOTA_EXCEEDED 402 Account quota exceeded
INTERNAL_ERROR 500 Internal server error

Performance Optimization#

Latency Requirements#

Metric Target Maximum
Avatar rendering < 20ms 33ms
Voice synthesis (TTS) < 50ms 100ms
Speech recognition < 100ms 200ms
LLM response start < 300ms 500ms
End-to-end response < 500ms 1000ms
WebSocket latency < 10ms 20ms
REST API latency < 30ms 50ms

Client-Side Optimization#

python
# 1. Use connection pooling
client = AsyncSerwaaClient(
    connection_pool_size=10,
    keepalive_timeout=30,
)

# 2. Enable response streaming
async for chunk in client.sessions.stream_response(session_id, message):
    # Process chunks as they arrive
    process_chunk(chunk)

# 3. Preload persona and avatar
await client.sessions.preload(
    persona_id="persona_abc123",
    avatar_id="avatar_abc123",
)

# 4. Use batch operations
documents = await client.knowledge.upload_batch([
    {"file": file1, "title": "Doc 1"},
    {"file": file2, "title": "Doc 2"},
])

# 5. Cache persona configurations
@lru_cache(maxsize=100)
async def get_persona(persona_id: str):
    return await client.personas.get(persona_id)

Network Optimization#

python
# Use regional endpoints
client = AsyncSerwaaClient(
    base_url="https://us-east.api.serwaa.ai",  # Nearest region
)

# Enable compression
client = AsyncSerwaaClient(
    compression=True,  # gzip for requests/responses
)

# Configure timeouts appropriately
client = AsyncSerwaaClient(
    connect_timeout=5.0,    # Connection timeout
    read_timeout=30.0,      # Read timeout
    write_timeout=30.0,     # Write timeout
)

Resource Management#

python
# Always use context managers
async with AsyncSerwaaClient() as client:
    # Client connection automatically cleaned up
    pass

# Explicitly close if not using context manager
client = AsyncSerwaaClient()
try:
    await client.sessions.create(...)
finally:
    await client.close()

# Limit concurrent sessions
semaphore = asyncio.Semaphore(10)

async def create_session():
    async with semaphore:
        return await client.sessions.create(...)

Security Best Practices#

API Key Security#

python
# DO: Use environment variables
import os
api_key = os.environ.get("SERWAA_API_KEY")

# DON'T: Hardcode API keys
api_key = "sk_live_abc123"  # NEVER DO THIS

# DO: Use separate keys for environments
PRODUCTION_KEY = os.environ.get("SERWAA_PROD_KEY")
STAGING_KEY = os.environ.get("SERWAA_STAGING_KEY")

# DO: Rotate keys regularly
# Use the dashboard to create new keys and deprecate old ones

# DO: Use minimum required scopes
# If you only need to read sessions, use sessions:read, not admin

Webhook Security#

python
# ALWAYS verify webhook signatures
@app.post("/webhooks/serwaa")
async def webhook(request: Request):
    signature = request.headers.get("X-Serwaa-Signature")
    timestamp = request.headers.get("X-Serwaa-Timestamp")

    if not verify_signature(await request.body(), signature, timestamp):
        raise HTTPException(401)

    # Process verified webhook
    ...

# Use HTTPS for webhook endpoints
# Never accept webhooks over HTTP

# Implement idempotency
async def handle_webhook(event):
    event_id = event["id"]
    if await is_event_processed(event_id):
        return  # Already processed

    await process_event(event)
    await mark_event_processed(event_id)

Data Protection#

python
# Sanitize user input before sending to AI
def sanitize_input(text: str) -> str:
    # Remove potential prompt injection attempts
    # Log suspicious inputs
    return text

# Don't log sensitive data
logger.info(f"Session created for user {user_id}")  # OK
logger.info(f"User message: {message}")  # AVOID - may contain PII

# Use session-level encryption for sensitive data
session = await client.sessions.create(
    persona_id="persona_abc123",
    encryption={
        "enabled": True,
        "key_id": "key_abc123",  # Your encryption key
    },
)

Network Security#

python
# Use TLS 1.3
client = AsyncSerwaaClient(
    ssl_context=create_ssl_context(
        min_version=ssl.TLSVersion.TLSv1_3,
    ),
)

# Configure IP allowlist for webhooks
# Only accept webhooks from Serwaa IP ranges

# Use VPC peering for high-security deployments
# Contact support for private connectivity options

Troubleshooting#

Common Issues#

Connection Errors#

python
# Problem: Connection refused
# Cause: Wrong endpoint or firewall blocking
# Solution:
client = AsyncSerwaaClient(
    base_url="https://api.serwaa.ai",  # Verify correct endpoint
    timeout=30.0,  # Increase timeout if needed
)

# Problem: SSL certificate error
# Cause: Outdated CA certificates
# Solution: Update certifi package
# pip install --upgrade certifi

Authentication Errors#

python
# Problem: 401 Unauthorized
# Cause: Invalid or expired API key
# Solution:
# 1. Verify API key is correct
# 2. Check key hasn't expired
# 3. Ensure key has required scopes

# Debug authentication
try:
    await client.auth.verify()
except AuthenticationError as e:
    print(f"Auth error: {e.message}")
    print(f"Key prefix: {api_key[:10]}...")

Session Errors#

python
# Problem: Session not starting
# Cause: Persona or avatar not ready
# Solution:
persona = await client.personas.get("persona_abc123")
if persona.status != "ready":
    print(f"Persona status: {persona.status}")
    # Wait for persona to be ready

# Problem: Session timing out
# Cause: Idle timeout too short
# Solution:
session = await client.sessions.create(
    persona_id="persona_abc123",
    config={"idle_timeout_seconds": 600},  # Increase timeout
)

Avatar Issues#

python
# Problem: Avatar training failed
# Cause: Poor quality input video
# Solution:
# 1. Check video requirements (resolution, duration, lighting)
# 2. Re-record with better conditions

status = await client.avatars.get_status("avatar_abc123")
if status.status == "failed":
    print(f"Error: {status.error_message}")
    print(f"Details: {status.error_details}")

# Problem: Avatar rendering slow
# Cause: High resolution or complex scene
# Solution:
session = await client.sessions.create(
    persona_id="persona_abc123",
    media={
        "resolution": "720p",  # Lower resolution
        "frame_rate": 24,      # Lower frame rate
    },
)

WebSocket Issues#

python
# Problem: WebSocket disconnecting
# Cause: Network instability or idle timeout
# Solution:
async def connect_with_reconnect(session_id: str):
    while True:
        try:
            async for event in client.sessions.stream(session_id):
                await handle_event(event)
        except ConnectionError:
            print("Disconnected, reconnecting...")
            await asyncio.sleep(1)
            continue

# Problem: Missing events
# Cause: Event buffer overflow
# Solution:
# Process events faster or use webhooks for reliable delivery

Debug Mode#

python
# Enable debug logging
import logging
logging.basicConfig(level=logging.DEBUG)

client = AsyncSerwaaClient(debug=True)

# This will log:
# - All HTTP requests and responses
# - WebSocket messages
# - Retry attempts
# - Performance metrics

Support Resources#

  • Documentation: https://docs.serwaa.ai/
  • API Status: https://status.serwaa.ai/
  • Community: https://community.serwaa.ai/
  • Support: support@serwaa.ai
  • Enterprise: enterprise@serwaa.ai

Appendix#

API Rate Limits#

Endpoint Category Limit Window
Sessions 100 req/min Per key
Personas 60 req/min Per key
Knowledge 30 req/min Per key
Tools 60 req/min Per key
Webhooks 30 req/min Per key
Streaming 10 connections Per key

SDK Versions#

SDK Version Node/Python Status
Python 1.0.x 3.11+ Stable
TypeScript 1.0.x 18+ Stable
Go 0.9.x 1.21+ Beta
Java 0.9.x 17+ Beta

Regional Endpoints#

Region Endpoint
US East https://us-east.api.serwaa.ai
US West https://us-west.api.serwaa.ai
EU West https://eu-west.api.serwaa.ai
Asia Pacific https://ap-south.api.serwaa.ai
Global (Auto) https://api.serwaa.ai

Changelog#

See CHANGELOG.md for version history and breaking changes.