Video conferencing platform integrations with virtual camera/audio injection.
Part of the Psyche AI Virtual Assistant Platform.
Overview#
The Conferencing service provides unified access to major video conferencing platforms, enabling AI avatar injection into live meetings.
Supported Platforms#
| Platform | Integration Type | Features |
|---|---|---|
| Zoom | SDK + Browser Hybrid | Audio, video, screen share, breakout rooms, webinars |
| Teams | Graph API + Bot Framework | Audio, video, screen share, presence, calendar |
| Meet | WebRTC + Browser | Audio, video, screen share, chat |
| Webex | SDK | Audio, video, screen share |
Architecture#
text
video_conferencing/
├── gateway/ # Unified meeting adapter interface
│ ├── interface.py # Abstract MeetingAdapter base
│ └── session.py # Session lifecycle management
├── zoom/ # Zoom integration (29 files)
│ ├── sdk.py # Native Zoom SDK
│ ├── browser.py # Browser-based fallback
│ ├── hybrid_adapter.py # SDK-preferred hybrid approach
│ ├── api.py # REST API client
│ ├── auth.py # OAuth authentication
│ ├── phone.py # Zoom Phone integration
│ ├── webinar.py # Webinar support
│ └── ...
├── teams/ # Microsoft Teams (16 files)
│ ├── adapter.py # Teams adapter
│ ├── graph.py # Graph API client
│ ├── auth.py # Azure AD authentication
│ ├── bot/ # Bot framework
│ │ ├── manifest.py # App manifest generation
│ │ └── registration.py # Bot registration
│ └── media/ # Application-hosted media
│ ├── streaming.py # RTP/RTCP streaming
│ ├── sdp.py # SDP negotiation
│ ├── codecs.py # Opus, VP8, H.264
│ └── ...
├── meet/ # Google Meet (10 files)
│ ├── adapter.py # Meet adapter
│ ├── webrtc.py # WebRTC Media API
│ ├── browser.py # Browser automation
│ └── ...
├── webex/ # Webex (5 files)
│ ├── adapter.py # Webex adapter
│ └── sdk.py # Webex SDK
├── webrtc/ # Generic WebRTC support
├── virtual_devices/ # Virtual camera/audio
│ ├── camera.py # v4l2loopback virtual camera
│ └── audio.py # PulseAudio routing
├── optimization/ # Performance tuning
│ ├── quality.py # Adaptive quality selection
│ └── large_meeting.py # Large meeting optimization
├── multiparticipant/ # Multi-participant handling
│ ├── quality.py # Per-participant quality tiers
│ ├── buffering.py # Frame buffering
│ ├── thumbnails.py # Thumbnail generation
│ └── audio_separation.py # Audio source separation
├── perception/ # Meeting perception
│ ├── body_language.py # Body language analysis
│ ├── emotion.py # Emotion tracking
│ └── behavior.py # Behavior analysis
├── meeting_intelligence/ # Analytics
│ ├── calendar.py # Calendar integration
│ ├── analytics.py # Meeting analytics
│ └── proactive_assistance/ # Proactive features
└── accessibility/ # Accessibility features
└── captioning.py # Real-time captions
Gateway Interface#
All platform adapters implement the unified MeetingAdapter interface:
python
from video_conferencing.gateway import MeetingAdapter
class MeetingAdapter(ABC):
@abstractmethod
async def join(self, meeting_url: str, **options) -> Session: ...
@abstractmethod
async def leave(self) -> None: ...
@abstractmethod
async def send_video(self, frame: np.ndarray) -> None: ...
@abstractmethod
async def send_audio(self, samples: np.ndarray) -> None: ...
@abstractmethod
async def receive_events(self) -> AsyncIterator[MeetingEvent]: ...
@abstractmethod
async def share_screen(self, enable: bool) -> None: ...
Virtual Device Injection#
Virtual Camera (v4l2loopback)#
python
from video_conferencing.virtual_devices import VirtualCamera
# Create virtual camera device
camera = VirtualCamera(device_path="/dev/video10")
await camera.initialize()
# Inject avatar frames
while True:
frame = await avatar_engine.render_frame()
await camera.inject_frame(frame)
Virtual Audio (PulseAudio)#
python
from video_conferencing.virtual_devices import VirtualAudio
# Create virtual audio device
audio = VirtualAudio()
await audio.initialize()
# Inject synthesized speech
while True:
samples = await voice_engine.synthesize()
await audio.inject_samples(samples)
Quick Start#
Installation#
bash
# Using Nx
nx install psyche-conferencing
# With all platform SDKs
nx install-all psyche-conferencing
# Platform-specific
nx install-zoom psyche-conferencing
nx install-teams psyche-conferencing
nx install-meet psyche-conferencing
# Or directly with Poetry
cd apps/psyche/conferencing
poetry install
poetry install --extras all
Environment Variables#
bash
# Zoom OAuth
ZOOM_CLIENT_ID=your-client-id
ZOOM_CLIENT_SECRET=your-client-secret
ZOOM_ACCOUNT_ID=your-account-id
# Microsoft Teams / Azure AD
AZURE_TENANT_ID=your-tenant-id
AZURE_CLIENT_ID=your-client-id
AZURE_CLIENT_SECRET=your-client-secret
# Google Meet
GOOGLE_CLIENT_ID=your-client-id
GOOGLE_CLIENT_SECRET=your-client-secret
GOOGLE_REFRESH_TOKEN=your-refresh-token
# Webex
WEBEX_CLIENT_ID=your-client-id
WEBEX_CLIENT_SECRET=your-client-secret
# Service Config
CONFERENCING_PORT=8006
CONFERENCING_HOST=0.0.0.0
LOG_LEVEL=INFO
# Virtual Devices
VIRTUAL_CAMERA_DEVICE=/dev/video10
PULSEAUDIO_SINK=virtual_speaker
Basic Usage#
python
from video_conferencing import gateway
# Select adapter based on meeting URL
adapter = gateway.get_adapter("https://zoom.us/j/123456789")
# Join meeting
session = await adapter.join(
meeting_url="https://zoom.us/j/123456789",
password="abc123",
display_name="AI Assistant",
)
# Inject avatar video
async for event in adapter.receive_events():
if event.type == "participant_speaking":
# React to speaker
frame = await avatar.render_reaction(event.participant_id)
await adapter.send_video(frame)
API Endpoints#
Gateway#
POST /meetings/join- Join a meetingPOST /meetings/leave- Leave current meetingGET /meetings/status- Get meeting statusGET /meetings/participants- List participants
Media#
POST /media/video- Send video framePOST /media/audio- Send audio samplesGET /media/stream- WebSocket media streamPOST /media/screen-share- Toggle screen share
Platform-Specific#
POST /zoom/schedule- Schedule Zoom meetingGET /zoom/recordings- Get Zoom recordingsPOST /teams/presence- Update Teams presenceGET /meet/calendar- Get Google Calendar events
Health#
GET /health- Health checkGET /ready- Readiness checkGET /metrics- Prometheus metrics
Development#
Running the Server#
bash
# Development mode
nx serve psyche-conferencing
# Production mode
nx serve-prod psyche-conferencing
Running Tests#
bash
nx test psyche-conferencing
nx test-unit psyche-conferencing
nx test-integration psyche-conferencing
nx test-cov psyche-conferencing
Docker#
bash
# Build image
nx docker-build psyche-conferencing
# Run with virtual device access
nx docker-run psyche-conferencing
Nx Integration#
bash
# Available targets
nx serve psyche-conferencing # Development server
nx serve-prod psyche-conferencing # Production server
nx build psyche-conferencing # Build package
nx install psyche-conferencing # Install dependencies
nx install-all psyche-conferencing # Install all platform SDKs
nx install-zoom psyche-conferencing # Install Zoom SDK
nx install-teams psyche-conferencing # Install Teams SDK
nx install-meet psyche-conferencing # Install Meet SDK
nx lint psyche-conferencing # Run linters
nx format psyche-conferencing # Format code
nx test psyche-conferencing # Run all tests
nx test-unit psyche-conferencing # Unit tests only
nx test-integration psyche-conferencing # Integration tests
nx test-cov psyche-conferencing # Tests with coverage
nx docker-build psyche-conferencing # Build Docker image
nx docker-run psyche-conferencing # Run container
Platform-Specific Features#
Zoom#
- Hybrid Adapter: Prefers SDK, falls back to browser automation
- Breakout Rooms: Create and manage breakout sessions
- Webinars: Full webinar support with panelist management
- Zoom Phone: PSTN integration
- Reactions: Emoji reactions and hand raise
- Recording: Cloud and local recording management
Microsoft Teams#
- Bot Framework: Teams app with calling capabilities
- Graph API: Full Microsoft Graph integration
- Application-Hosted Media: RTP/RTCP with SDP negotiation
- Presence: Real-time presence updates
- Calendar: Meeting scheduling via Graph
Google Meet#
- WebRTC: Native WebRTC Media API
- Browser Fallback: Playwright-based automation
- Screen Share: Full screen sharing support
- Chat: Real-time chat integration
Webex#
- SDK Integration: Native Webex SDK
- WebRTC Media: Standard WebRTC handling
Multi-Participant Optimization#
Adaptive Quality Selection#
python
from video_conferencing.multiparticipant import QualityManager
quality = QualityManager()
# Configure quality tiers
quality.set_tier("speaker", resolution=1080, fps=30)
quality.set_tier("active", resolution=720, fps=24)
quality.set_tier("passive", resolution=480, fps=15)
quality.set_tier("thumbnail", resolution=180, fps=5)
# Auto-assign based on activity
quality.update_participant_tier(participant_id, speaking=True)
Large Meeting Optimization#
- Participant quality rotation
- Bandwidth budgeting
- CPU throttling
- Roster sampling
- Cascading fallback
Performance#
Latency Targets#
| Operation | Target |
|---|---|
| Video injection | < 16ms (60 FPS capable) |
| Audio injection | < 10ms |
| Platform join | < 5s |
| Reconnection | < 2s |
Resource Usage#
- CPU: 2-4 cores per meeting
- Memory: 512MB - 2GB per meeting
- Network: 2-10 Mbps per meeting
Security#
- OAuth 2.0: Platform-specific OAuth flows
- Token Management: Secure credential storage
- Sandboxed Execution: Isolated per-meeting contexts
- Audit Logging: All meeting actions logged
License#
Proprietary - Oshun Platform