Eve library

Psyche -- Product Features

Guide

Revision 1

On this page

Canonical source: the canonical features document for this domain is deep-dive/features.md, rendered in the docs center. This file is its operational companion (service topology, APIs, schemas, runtime detail) — reconciled by linking, not duplicated (§2, §13).

Hyper-Realistic AI Virtual Assistant Platform

Psyche creates photorealistic digital humans that join video conferences, hold real-time conversations, and perform tasks on your behalf. This document describes every capability available to users of the Psyche platform.


Table of Contents#

  1. Digital Human Personas
  2. Hyper-Realistic Avatar System
  3. Natural Voice & Speech
  4. Real-Time Conversation
  5. Knowledge Base & RAG
  6. Video Conferencing Integration
  7. Computer Use & Task Execution
  8. Emotion & Perception Intelligence
  9. Memory & Continuity
  10. Session Management
  11. Admin Dashboard
  12. Analytics & Reporting
  13. Accessibility
  14. Integrations & Webhooks
  15. Security & Compliance
  16. Plans & Organization Management

1. Digital Human Personas#

Personas are the core of Psyche. Each persona is a fully configured virtual assistant with a distinct identity, personality, voice, appearance, expertise, and memory.

1.1 Identity Creation#

Feature Description
Full name Assign a realistic first and last name
Job title Set a professional title (e.g., "Senior Sales Consultant")
Company / organization Link the persona to a specific company or division
Short bio Write a 2-3 sentence professional summary visible to meeting participants
Background story Provide a detailed career narrative (up to 2,000 characters) that shapes conversational authenticity
Profile photo Upload a reference headshot (JPEG, PNG, WebP; up to 10 MB) used to generate the avatar

1.2 Personality Engineering (OCEAN Model)#

Configure personality using the scientifically validated Big Five model. Each trait is set on a 0.0 - 1.0 scale:

Trait Low (0.0 - 0.3) Mid (0.4 - 0.6) High (0.7 - 1.0)
Openness Practical, routine, conventional Balanced curiosity Creative, curious, embraces novel ideas
Conscientiousness Flexible, spontaneous Reasonably organized Meticulous, detail-oriented, thorough
Extraversion Reserved, calm, thoughtful Situationally social Outgoing, energetic, enthusiastic
Agreeableness Direct, assertive, competitive Balanced cooperation Warm, cooperative, trusting
Neuroticism Emotionally stable, calm Normal range Sensitive, empathetic to stress

1.3 Communication Style & Tone#

Setting Options
Communication style Formal, Professional, Casual, Friendly, Technical
Conversational tone Warm, Neutral, Enthusiastic, Calm, Assertive
Custom instructions Free-form behavioral rules (e.g., "Always greet by name," "Avoid jargon unless user demonstrates expertise")

1.4 Personality Presets#

Quick-start configurations for common use cases:

Preset Best For Key Traits
Warm Professional Customer success, onboarding High agreeableness, professional style, warm tone
Analytical Expert Technical support, research High conscientiousness, technical style, neutral tone
Friendly Guide Retail, hospitality, consumer High extraversion, casual style, enthusiastic tone
Calm Advisor Healthcare, finance, crisis support Low neuroticism, professional style, calm tone

1.5 Expertise Configuration#

Feature Description
Domain tagging Assign expertise areas (e.g., "Product Support," "API Integration")
Proficiency levels Set per-domain depth: Beginner, Intermediate, Advanced, Expert
Topic boundaries Define what the persona should NOT attempt and where to redirect
Escalation triggers Auto-escalate after repeated failures, detected frustration, or out-of-scope requests

1.6 Tool Assignment#

Assign registered tools the persona can invoke during conversations:

Tool Type Description
MCP tools Model Context Protocol server integrations
API tools REST API endpoint invocations
Webhook tools External webhook triggers
Built-in tools Platform-native capabilities (search, calendar, etc.)

1.7 Persona Lifecycle#

Status Meaning
Draft Under construction, not yet usable
Training Avatar/voice models being generated
Active Live and available for sessions
Inactive Temporarily disabled
Archived Preserved for records, not available

2. Hyper-Realistic Avatar System#

2.1 3D Gaussian Splatting Avatars#

Feature Description
Photorealistic rendering 3D Gaussian Splatting produces lifelike digital humans from a short training video
FLAME face model Industry-standard parametric face model with 50 blend shapes for expressions
Real-time rendering 30+ FPS at up to 1080p with sub-33ms frame latency
Training from video Upload a 30-60 second recording; the system extracts face geometry, fits the FLAME model, trains the 3D Gaussian Splatting model (~30,000 iterations), and learns expression blend shapes
Training time 1-4 hours depending on video quality

2.2 Avatar Quality Settings#

Quality Resolution Frame Rate Latency GPU Load
480p 854 x 480 24 FPS < 20 ms Minimal
720p 1280 x 720 30 FPS < 25 ms Standard
1080p 1920 x 1080 30 FPS < 30 ms High
4K 3840 x 2160 30 FPS < 40 ms Very High

2.3 Avatar Styles#

Style Description Use Case
Photorealistic Lifelike human appearance Customer-facing video calls
Animated Stylized cartoon character Casual, internal meetings
Stylized Artistic interpretation Brand-specific environments
None (audio only) No video, voice only Voice-only sessions

2.4 Training Video Guidelines#

Users record a short video for avatar training:

  • Duration: 30-60 seconds
  • Resolution: 1080p minimum (4K preferred)
  • Framing: Head and shoulders, centered
  • Movement: Slow head rotations covering all angles
  • Expression variety: Neutral, smile, talking
  • Lighting: Even, soft, diffused (no harsh shadows)
  • Background: Solid color, preferably green or gray

2.5 Facial Expression System#

Feature Description
Automatic emotion mapping Expressions change based on speech content and detected emotion
FACS control Facial Action Coding System for precise individual muscle unit control
Micro-expressions Subtle, involuntary facial expressions that add realism
Lip synchronization Real-time lip sync driven by TTS audio output
Gaze control Natural eye gaze direction that follows the conversation
Head movement Realistic head tilts, nods, and turns during conversation
Posture system Body posture management for upper-body presence
Gesture system Hand and body gestures to accompany speech
Uncanny valley mitigation Automatic adjustments to prevent unsettling appearance artifacts

2.6 Avatar Caching & Performance#

Feature Description
Model preloading Avatar models are cached for instant startup
Temporal anti-aliasing TAA removes rendering flicker and edge artifacts
Quality monitoring Continuous quality metrics track rendering fidelity
TensorRT acceleration GPU-optimized inference for faster rendering

3. Natural Voice & Speech#

3.1 Text-to-Speech (Voice Synthesis)#

Feature Description
Multiple TTS providers ElevenLabs (primary), Cartesia, OpenAI TTS, Deepgram Aura, XTTS v2, Orpheus, F5-TTS, Piper
Ultra-low latency ~75 ms to first audio byte via WebSocket streaming
Voice selection Choose from provider voice libraries (e.g., Rachel, Josh, Emily, Sam, Bella, Adam)
Custom voice cloning Create a unique voice from 1-3 minutes of audio samples
Provider failover Automatic fallback to secondary providers if primary is unavailable

3.2 Voice Parameters#

Parameter Range Description
Speed 0.5x - 2.0x Speaking rate (1.0x is natural)
Pitch -50% to +50% Voice pitch (0% is natural)
Stability 0.0 - 1.0 How consistent the voice sounds (higher = less variation)
Clarity / similarity boost 0.0 - 1.0 How close to the original voice model
Style expressiveness 0.0 - 1.0 Emotional range in delivery
Speaker boost On / Off Enhanced voice presence

3.3 Speech Recognition#

Feature Description
Primary STT provider Deepgram Nova-2 for real-time WebSocket transcription
Fallback providers OpenAI Whisper (local or API), AssemblyAI
Interim results Partial transcriptions for responsive turn-taking
Speaker diarization Identify which participant is speaking
Smart formatting Automatic punctuation and formatting
Noise handling Audio preprocessing for noisy environments
Word-level confidence Per-word confidence scores for accuracy tracking

3.4 Voice Consistency#

Feature Description
Voice consistency monitoring Tracks voice quality across sessions to prevent drift
Viseme generation Generates visual phoneme data that drives lip sync animation

4. Real-Time Conversation#

4.1 AI-Powered Dialogue#

Feature Description
Claude (Anthropic) as primary LLM Conversational reasoning powered by Claude 3.5 Sonnet (or configurable)
Streaming responses Real-time token streaming for natural conversation pacing
Multi-LLM failover Automatic fallback chain: Claude -> OpenAI GPT-4 -> Google Gemini -> Local LLM (Ollama/vLLM)
Context management Up to 200K token context window with intelligent truncation
Tool calling The LLM can invoke registered tools mid-conversation
System prompt per persona Each persona's personality, expertise, and rules are injected as a system prompt

4.2 Turn-Taking#

Feature Description
Wait for completion The assistant waits for the user to finish speaking before responding
Acknowledgment phrases Natural filler responses ("I understand," "Got it") while processing
Clarification behavior Asks clarifying questions when confidence is below a configurable threshold
Smooth topic transitions Natural phrases for changing or returning to topics
Proactive suggestions After resolving an issue, offers related help or resources

4.3 Dialogue Management#

Feature Description
Multi-turn context Maintains full conversation history within a session
Response length control Configurable short (50 words), medium (150 words), or long (500 words) responses
Response formatting Markdown, bullet points, numbered lists, code blocks
Error handling Graceful responses for unknown queries, repeated failures, and out-of-scope requests
Escalation to human Automatic handoff after configurable failure threshold (default: 3 attempts)

4.4 Behavior Coordination#

Feature Description
Multi-channel orchestration Coordinates speech, facial expression, gaze, gestures, and head movement simultaneously
Behavioral anomaly detection Identifies and corrects unnatural behavior combinations

5. Knowledge Base & RAG#

5.1 Document Ingestion#

Feature Description
Supported document types PDF, DOCX, TXT, Markdown, HTML, CSV, JSON, web URLs
Upload interface Drag-and-drop document upload in the admin dashboard
Automatic chunking Documents are split into semantically meaningful chunks
Status tracking Upload progress: Uploading -> Processing -> Ready (or Error)

5.2 Vector Search & Retrieval#

Feature Description
Embedding generation Voyage AI (primary), Cohere, OpenAI (fallback) for vector embeddings
Vector database Qdrant for high-performance semantic search
Retrieval-Augmented Generation Relevant knowledge chunks are injected into the LLM context during conversations
Configurable retrieval Set top-K results (default: 5) and minimum relevance score (default: 0.7)
Collection priority Assign high/medium/low priority to different knowledge collections

5.3 Knowledge Sources#

Source Type Description
Manual upload One-time document uploads
Sync source Scheduled synchronization from external systems
API source Programmatic content updates via API
Sync scheduling Configure automatic refresh intervals

5.4 Knowledge-Persona Linking#

Feature Description
Per-persona knowledge Link specific knowledge collections to individual personas
Collection assignment Each persona can access multiple knowledge collections with priority ranking
Fact memory Store persistent facts the persona always knows (company name, policies, promotions)
Opinion memory Define the persona's stance on topics with configurable conviction strength

6. Video Conferencing Integration#

6.1 Supported Platforms#

Platform Integration Method Key Capabilities
Zoom API + Browser automation Create meetings, join meetings, list participants, access recordings
Microsoft Teams Graph API + Bot Framework Create meetings, start calls, send messages, manage participants, presence detection
Google Meet OAuth + Browser automation (WebRTC) Join meetings via meeting code, real-time audio/video
Webex REST API Create meetings, manage participants, send messages
Custom WebRTC Direct WebRTC STUN/TURN server integration for custom conferencing

6.2 Meeting Capabilities#

Feature Description
Auto-join meetings The virtual assistant automatically joins scheduled meetings
Participant management Track who is in the meeting, detect joins/leaves
Audio/video streaming Stream the avatar's video and synthesized speech into the meeting
Meeting state sync Synchronize conferencing state across all components
Recording support Access meeting recordings from supported platforms
Chat interaction Send and receive text messages in the meeting chat (Teams, Webex)

6.3 Tavus Integration (Optional Cloud Avatars)#

Feature Description
Tavus replica creation Create and train cloud-hosted avatar replicas
Tavus conversations Real-time conversational AI via Tavus Phoenix rendering
Hybrid rendering Seamlessly switch between local 3D Gaussian Splatting and Tavus cloud rendering
Tavus perception Use Tavus Raven model for participant understanding
Tavus LLM orchestration Route LLM calls through Tavus Sparrow for dialogue
Tavus turn-taking Manage conversational turn-taking via Tavus pipeline
Tavus expression control Drive avatar expressions through Tavus API
Tavus tool integration Execute tools and knowledge queries within Tavus sessions
Tavus Pipecat integration Low-latency audio/video pipeline via Pipecat framework
Webhook verification Secure HMAC-SHA256 verification of Tavus webhook payloads

7. Computer Use & Task Execution#

7.1 Browser Automation#

Feature Description
Sandboxed browser A headless Chromium browser that the AI can control during a video call
Screen understanding The AI takes screenshots and uses Claude's vision capability to understand what is on screen
Observe-Reason-Act loop Iterative cycle: take screenshot, ask Claude what to do next, execute the action, verify result
Action types Click, fill text, navigate, scroll, select, upload, download
Action verification After each action, the system verifies the outcome before proceeding
Recovery Automatic recovery if an action fails or produces unexpected results

7.2 Task Safety#

Feature Description
Action safety validation Every action is checked against safety rules before execution
Confirmation prompts High-risk actions (purchases, deletions) require human confirmation
Iteration limits Maximum 50 actions per task (configurable) to prevent runaway automation
Sandbox isolation Browser runs in a secure, isolated environment

7.3 Screen Analysis#

Feature Description
Screen content understanding AI interprets screen content to guide actions
OCR and element detection Identifies text, buttons, forms, and interactive elements
Multi-step tasks Complete complex workflows spanning multiple pages and steps

8. Emotion & Perception Intelligence#

8.1 Emotion Recognition (Inbound)#

Feature Description
Face detection Detect faces in the participant's video stream
Facial emotion analysis Recognize emotions from facial expressions
Voice sentiment detection Analyze vocal tone and pace for emotional cues
Engagement measurement Quantify how engaged participants are in the conversation
Attention tracking Detect whether participants are looking at the camera or distracted
Head pose estimation Determine where participants are looking
Gaze estimation Track eye gaze direction of meeting participants

8.2 Emotion Expression (Outbound)#

Emotion Trigger
Neutral Default resting state
Happy Positive language, greetings, good news
Concerned Problem acknowledgment, empathy situations
Thoughtful Processing questions, complex explanations
Surprise Unexpected input, exclamations

8.3 Participation Analytics#

Feature Description
Participation tracking Measure how much each person contributes to the meeting
Meeting engagement scoring Overall meeting engagement level across all participants

9. Memory & Continuity#

9.1 Working Memory#

Feature Description
In-context memory Full conversation history maintained within a single session
Short-term working memory Key facts and context from the current conversation held in active memory

9.2 Long-Term Memory#

Feature Description
Memory persistence Conversations are saved to the database for future retrieval
Memory embeddings Conversation memories are vectorized for semantic search
Memory retrieval Past conversation context retrieved via semantic similarity when relevant
Memory consolidation Older memories are summarized and compressed to save storage
Memory archival Long-term storage of consolidated memories

9.3 Cross-Session Memory#

Feature Description
Returning user recognition Recognize returning participants across sessions (requires user identification)
Interaction history Remember past interactions, preferences, and unresolved issues
Configurable retention Set memory retention period (default: 90 days)
Privacy controls Users can request deletion of their memory data; automatic anonymization after configurable period

9.4 Fact & Opinion Memory#

Feature Description
Fact storage Persistent key-value facts the persona always knows (company name, support hours, policies)
Opinion definitions Define the persona's stance on topics with conviction strength (0.0 - 1.0)
Session context rules Configure what is remembered within a session vs. what is forgotten afterward

10. Session Management#

10.1 Session Lifecycle#

State Description
Pending Session created, waiting to start
Connecting Joining the video conference
Active Conversation in progress
Paused Temporarily paused
Ended Session completed
Error Session encountered an error

10.2 Session Configuration#

Feature Description
Persona assignment Select which persona participates in the session
Platform selection Choose the video conferencing platform (Zoom, Teams, Meet, Webex, Custom)
Meeting ID linkage Connect the session to a specific meeting ID or URL
Session naming Assign descriptive names to sessions for easy identification

10.3 Session Metrics#

Metric Description
Messages exchanged Total messages sent and received
Tools executed Number of tool invocations during the session
Average response time Mean time between user speech and assistant response
Satisfaction score Post-session satisfaction rating (if collected)
Duration Total session length
Participant count Number of human participants in the meeting

11. Admin Dashboard#

11.1 Dashboard Overview#

Feature Description
At-a-glance metrics Total sessions, active sessions, total messages, average response time, satisfaction score, error rate
Period selection View metrics by day, week, month, quarter, or year
Time-series charts Visualize trends over time

11.2 Persona Management#

Feature Description
Create new persona Multi-step form: Identity -> Personality -> Expertise -> Voice -> Avatar
Edit persona Modify any aspect of an existing persona
Persona list Browse all personas with status, training progress, and quality scores
Persona detail view Deep view of a single persona's configuration, history, and metrics
Avatar capture guide Built-in dialog with recording guidelines for training video

11.3 Knowledge Base Management#

Feature Description
Document browser View all uploaded documents with type, status, chunk count, and metadata
Document upload Drag-and-drop upload with type and size validation
Source management Create and manage sync sources, API sources, and manual collections
Source detail view View individual source with document count, sync status, and schedule

11.4 Session Management#

Feature Description
Session list Browse all sessions with status, platform, duration, and participant count
Session detail View individual session metrics, transcript, and tool usage
Session settings Configure default session parameters and platform integrations
Integration settings Per-platform configuration (Zoom, Teams, Meet, Webex)

11.5 Settings#

Feature Description
Organization settings Manage organization name, logo, and plan
User management Invite and manage team members with role-based access
API key management Generate and revoke API keys for programmatic access

12. Analytics & Reporting#

12.1 Usage Analytics#

Metric Description
Total sessions Cumulative session count over selected period
Active sessions Currently running sessions
Total messages Total conversational exchanges
Average response time Platform-wide response latency
Satisfaction score Aggregate satisfaction across all sessions
Error rate Percentage of sessions encountering errors

12.2 Latency Analysis#

Feature Description
End-to-end latency tracking Measure total time from user speech to assistant response
Component-level breakdown Latency per component: STT, LLM, TTS, avatar rendering
SLO monitoring Track performance against defined service level objectives
SLO alerting Automatic alerts when latency or error rate exceeds thresholds

12.3 Quality Metrics#

Metric Description
Net Promoter Score (NPS) Track NPS across sessions and personas
User satisfaction Per-session satisfaction measurement
Human evaluation Framework for human reviewers to evaluate conversation quality
Task completion rate Percentage of issues resolved without escalation
Escalation rate Percentage of conversations requiring human handoff

13. Accessibility#

13.1 Admin Dashboard Accessibility#

Feature Description
ARIA labels Full ARIA annotation across all dashboard components
Skip navigation links Keyboard users can skip to main content
Screen reader announcements Live region announcements for dynamic content changes
Keyboard shortcuts Navigate and interact with the dashboard entirely via keyboard
Focus management Proper focus trapping in modals and dialogs
High contrast support Accessible color schemes for low-vision users

13.2 Session Accessibility#

Feature Description
Caption overlay Real-time captions displayed over the video feed
Caption history Scrollable transcript of the full conversation
Caption controls Toggle captions, adjust font size and position
Audio description Screen reader-friendly descriptions of visual elements and avatar actions
Voice command interface Control the session via voice commands
Voice command help In-session help overlay listing available voice commands

14. Integrations & Webhooks#

14.1 Webhook Delivery#

Feature Description
Session event webhooks Receive notifications when sessions start, end, or encounter errors
Configurable endpoints Set webhook URLs per organization
Retry logic Automatic retry with exponential backoff on delivery failure
Delivery verification HMAC signature verification for secure webhook payloads

14.2 Cross-Domain Integration#

Integration Description
Sophia integration Connect to the Sophia knowledge and search platform for enhanced RAG
Sophia semantic search Bridge to Sophia's semantic search for broader knowledge queries
Recall integration Connect to the Recall memory system for cross-platform memory sharing
Isis avatar generation Generate avatar assets via the Isis generative factory

14.3 API Access#

Feature Description
REST API Full REST API for programmatic persona, session, and knowledge management
WebSocket API Real-time streaming for session audio and status
gRPC support High-throughput streaming for internal service communication
API documentation Auto-generated OpenAPI / Swagger documentation

15. Security & Compliance#

15.1 Authentication & Authorization#

Feature Description
Role-based access control Admin, Manager, Member, Viewer roles with granular permissions
Organization isolation Multi-tenant architecture with strict data isolation
JWT authentication Secure token-based auth for API access

15.2 Data Security#

Feature Description
Sandboxed computer use Browser automation runs in an isolated security sandbox
Action safety validation All AI-initiated actions are validated against safety rules before execution
Kubernetes security policies Network policies, pod security, and RBAC for production deployment
Secure model storage ML models and avatar data stored in encrypted S3-compatible storage

15.3 Privacy#

Feature Description
Memory deletion Users can request deletion of all stored memory and conversation data
Automatic anonymization Personal data anonymized after configurable retention period
Session data retention policies Configurable data retention and automatic cleanup

16. Plans & Organization Management#

16.1 Organization Plans#

Plan Personas Sessions Knowledge Size Features
Free Limited Limited Limited Core features only
Starter Moderate Moderate Moderate Core + analytics
Professional Expanded Expanded Expanded All features
Enterprise Unlimited Unlimited Unlimited All features + SLA + priority support

16.2 Team Management#

Feature Description
Invite members Add team members by email with role assignment
Role management Change roles: Admin, Manager, Member, Viewer
Organization branding Upload logo and customize organization profile

Feature Summary#

Category Feature Count Highlights
Personas 25+ OCEAN personality, expertise domains, voice cloning, tool assignment
Avatar 20+ 3D Gaussian Splatting, FLAME model, FACS expressions, micro-expressions, gaze, gestures
Voice & Speech 15+ Multi-provider TTS/STT, voice cloning, noise handling, viseme generation
Conversation 15+ Claude-powered dialogue, multi-LLM failover, tool calling, turn-taking
Knowledge Base 10+ RAG with vector search, multi-format ingestion, collection priority
Video Conferencing 10+ Zoom, Teams, Meet, Webex, custom WebRTC
Computer Use 10+ Sandboxed browser, screen understanding, task safety, recovery
Emotion & Perception 10+ Emotion recognition, engagement detection, attention tracking
Memory 10+ Working memory, long-term archival, cross-session continuity
Admin Dashboard 15+ Persona management, knowledge management, session management
Analytics 10+ Usage metrics, latency analysis, NPS, satisfaction tracking
Accessibility 10+ Captions, audio description, voice commands, ARIA, keyboard navigation
Security 10+ RBAC, sandbox isolation, action safety, encryption, privacy controls