Behavioral realism and FACS expression engine for AI virtual assistants.
Part of the Psyche AI Virtual Assistant Platform.
Overview#
The Behavior Engine provides realistic facial expression generation using the Facial Action Coding System (FACS). It coordinates with the Avatar Engine to produce photorealistic micro-expressions and macro emotions.
Note: The core behavioral logic is integrated into the Avatar Engine
(apps/psyche/avatar-engine/src/avatar_engine/behavior/). This service provides
an API interface for behavior control.
Features#
- FACS Implementation: 27 Action Units mapped to ARKit blendshapes
- Micro-Expression Generation: 7 universal micro-expressions (40-200ms)
- Emotion Transitions: 8 primary emotions with blending
- Expression Leakage: Suppression-driven micro-expression simulation
- Personality Modulation: Expression intensity based on personality
- Asymmetry Support: Realistic bilateral facial asymmetry
FACS Action Units#
| AU | Name | Description |
|---|---|---|
| AU1 | Inner Brow Raiser | Eyebrow inner raise |
| AU2 | Outer Brow Raiser | Eyebrow outer raise |
| AU4 | Brow Lowerer | Eyebrow lower (frown) |
| AU5 | Upper Lid Raiser | Eyes wide |
| AU6 | Cheek Raiser | Duchenne smile |
| AU7 | Lid Tightener | Eye squint |
| AU9 | Nose Wrinkler | Disgust nose |
| AU10 | Upper Lip Raiser | Sneer |
| AU12 | Lip Corner Puller | Smile |
| AU15 | Lip Corner Depressor | Frown |
| AU17 | Chin Raiser | Chin push up |
| AU18 | Lip Pucker | Kiss/pucker |
| AU20 | Lip Stretcher | Stretch |
| AU22 | Lip Funneler | Funnel |
| AU23 | Lip Tightener | Tighten |
| AU24 | Lip Pressor | Press |
| AU25 | Lips Part | Slight open |
| AU26 | Jaw Drop | Jaw open |
| AU27 | Mouth Stretch | Wide open |
| AU28 | Lip Suck | Roll inward |
| AU43 | Eyes Closed | Close |
| AU45 | Blink | Blink |
Primary Emotions#
| Emotion | Primary AUs | Timing |
|---|---|---|
| Happiness | AU6, AU12 | 0.4s onset, 3s hold |
| Sadness | AU1, AU4, AU15 | 0.8s onset, 5s hold |
| Anger | AU4, AU5, AU7 | 0.2s onset, 2s hold |
| Fear | AU1, AU2, AU5 | 0.15s onset, 1.5s hold |
| Surprise | AU1, AU2, AU5 | 0.1s onset, 0.5s hold |
| Disgust | AU9, AU10 | 0.3s onset, 2s hold |
| Contempt | AU12R, AU14R | 0.5s onset, 3s hold |
| Neutral | None | Indefinite |
Micro-Expression Timing#
| Expression | Duration | Onset | Peak |
|---|---|---|---|
| Happiness | 40-150ms | 20ms | 90-110ms |
| Sadness | 50-200ms | 30ms | 110-130ms |
| Anger | 40-120ms | 20ms | 60-80ms |
| Fear | 40-150ms | 20ms | 90-110ms |
| Surprise | 40-100ms | 10ms | 30-50ms |
| Disgust | 50-180ms | 20ms | 100-120ms |
| Contempt | 60-200ms | 30ms | 130-160ms |
Quick Start#
Installation#
bash
# Using Nx
nx install psyche-behavior-engine
# Or directly with Poetry
cd apps/psyche/behavior-engine
poetry install
Environment Variables#
bash
# Service Config
BEHAVIOR_ENGINE_PORT=8003
BEHAVIOR_ENGINE_HOST=0.0.0.0
# Avatar Engine Connection (for delegation)
AVATAR_ENGINE_URL=http://localhost:8001
API Endpoints#
Expressions#
POST /expressions/set- Set active expressionPOST /expressions/blend- Blend multiple expressionsGET /expressions/current- Get current expression state
Micro-Expressions#
POST /micro/trigger- Trigger a micro-expressionGET /micro/active- Get active micro-expressions
Emotions#
POST /emotions/transition- Transition to emotionPOST /emotions/blend- Blend emotionsGET /emotions/current- Get current emotional state
Behaviors#
POST /behaviors/activate- Activate a behaviorPOST /behaviors/deactivate- Deactivate a behaviorGET /behaviors/active- Get active behaviors
Development#
Running the Server#
bash
# Development mode with auto-reload
nx serve psyche-behavior-engine
# Production mode
nx serve-prod psyche-behavior-engine
Running Tests#
bash
nx test psyche-behavior-engine
nx test-cov psyche-behavior-engine
Linting & Formatting#
bash
nx lint psyche-behavior-engine
nx format psyche-behavior-engine
Docker#
bash
nx docker-build psyche-behavior-engine
nx docker-run psyche-behavior-engine
Nx Integration#
bash
# Available targets
nx serve psyche-behavior-engine # Development server
nx serve-prod psyche-behavior-engine # Production server
nx build psyche-behavior-engine # Build package
nx install psyche-behavior-engine # Install dependencies
nx lint psyche-behavior-engine # Run linters
nx format psyche-behavior-engine # Format code
nx test psyche-behavior-engine # Run tests
nx test-cov psyche-behavior-engine # Tests with coverage
nx docker-build psyche-behavior-engine # Build Docker image
nx docker-run psyche-behavior-engine # Run Docker container
Architecture#
The behavior engine follows a delegation pattern:
- API Layer - FastAPI endpoints for behavior control
- Orchestrator - Coordinates behavior priorities and blending
- Avatar Engine - Actual FACS/expression implementation
Research References#
- Paul Ekman's micro-expression research
- FACS Manual (Ekman & Friesen)
- Russell Circumplex Model of Affect
License#
Proprietary - Oshun Platform