This directory contains Python-based services for the Psyche domain (AI Virtual Assistant Platform).
Poetry Workspace#
This directory is configured as a Poetry workspace, allowing unified development
across all Psyche Python services. The root pyproject.toml provides:
- Centralized Dependencies: Shared dependency constraints ensure consistent versions across all services
- Local Package References: All services and the common library are included as path dependencies
- Unified Tool Configuration: Shared ruff, mypy, black, and pytest settings
- Optional Extras: Install only what you need (minimal, ml, voice, vision, etc.)
Quick Start#
bash
# Install all services with core dependencies
cd apps/psyche
poetry install
# Install with ML capabilities (torch, transformers)
poetry install --extras ml
# Install with voice processing
poetry install --extras voice
# Install everything
poetry install --extras all
Available Extras#
| Extra | Description |
|---|---|
minimal |
Core API dependencies only |
ml |
PyTorch, NumPy, Transformers |
voice |
Audio processing and TTS/STT |
vision |
Computer vision (OpenCV, MediaPipe) |
avatar |
Avatar rendering dependencies |
aws |
AWS SDK (boto3, aiobotocore) |
all |
Everything |
Overview#
Psyche services are migrated from the Serwaa codebase and provide:
- Avatar Engine: 3D Gaussian Splatting rendering for hyper-realistic avatars
- Voice Engine: TTS/STT, dialogue management, and lip-sync generation
- Behavior Engine: FACS expressions, micro-expressions, and behavioral realism
- Perception Engine: Face analysis, emotion detection, and gaze tracking
- Computer Use: Claude Computer Use agent integration
- Conferencing Integration: Zoom, Teams, Meet, Webex virtual camera injection
- Knowledge Base: RAG, embeddings, and persistent memory
- Persona Service: Identity and personality management
- Learning System: Skill acquisition and memory consolidation
Technology Stack#
- Language: Python 3.11+
- Framework: FastAPI
- ML: PyTorch 2.2+ (CUDA 12.x)
- 3D Graphics: pytorch3d, 3D Gaussian Splatting
- Database: PostgreSQL (asyncpg), Redis, Qdrant
- Messaging: gRPC, WebSockets
Directory Structure#
text
apps/psyche/
├── api-gateway/ # REST API Gateway (FastAPI)
├── avatar-engine/ # Avatar generation and rendering
├── voice-engine/ # TTS, STT, dialogue management
├── behavior-engine/ # Expression and behavior generation
├── perception-engine/ # Face analysis and emotion detection
├── computer-use/ # Claude Computer Use integration
├── conferencing/ # Video platform integrations
├── knowledge-base/ # RAG and document processing
├── persona-service/ # Persona management
├── learning-system/ # Memory and skill acquisition
└── orchestrator/ # Session coordination
Development#
Prerequisites#
- Python 3.11+
- Poetry 1.8+ for dependency management
- CUDA 12.x for GPU services
- Docker for local development
Workspace Development#
bash
# Navigate to workspace
cd apps/psyche
# Install all services in development mode
poetry install
# Install with specific extras
poetry install --extras "ml voice"
# Update all dependencies
poetry update
# Add a new dependency (updates constraint file)
poetry add package-name
# Add to a specific service (run in service directory)
cd avatar-engine && poetry add new-package
Running Services#
bash
# Via Poetry (workspace)
poetry run uvicorn api_gateway.main:app --reload --port 8000
# Via Nx (recommended)
nx serve psyche-api-gateway
nx serve psyche-avatar-engine
Testing#
bash
# Run all tests in workspace
poetry run pytest
# Run tests for a specific service
poetry run pytest api-gateway/tests/
# Run with coverage (all services)
poetry run pytest --cov --cov-report=html
# Via Nx
nx test psyche-api-gateway
nx test-cov psyche-avatar-engine
Code Quality#
bash
# Lint all services
poetry run ruff check .
# Format all services
poetry run black .
# Type check all services
poetry run mypy .
# Fix linting issues
poetry run ruff check --fix .
# Via Nx (single service)
nx lint psyche-api-gateway
nx lint-fix psyche-api-gateway
nx format psyche-api-gateway
nx typecheck psyche-api-gateway
Nx Integration#
Each service has an project.json for Nx integration:
bash
# Build a service
nx build psyche-avatar-engine
# Test a service
nx test psyche-avatar-engine
# Lint a service
nx lint psyche-avatar-engine
GPU Requirements#
| Service | Min VRAM | Recommended |
|---|---|---|
| Avatar Engine | 10 GB | 24+ GB |
| Voice Engine | 4 GB | 8+ GB |
| Perception Engine | 4 GB | 8+ GB |
| Behavior Engine | 2 GB | 4+ GB |
Environment Variables#
See .env.example in each service directory for required configuration.
Key variables:
DATABASE_URL: PostgreSQL connectionREDIS_URL: Redis connectionQDRANT_URL: Vector databaseANTHROPIC_API_KEY: Claude APIELEVENLABS_API_KEY: Voice synthesis