# Psyche Conferencing

Video conferencing platform integrations with virtual camera/audio injection.

Part of the Psyche AI Virtual Assistant Platform.

## Overview

The Conferencing service provides unified access to major video conferencing
platforms, enabling AI avatar injection into live meetings.

## Supported Platforms

| Platform | Integration Type          | Features                                             |
| -------- | ------------------------- | ---------------------------------------------------- |
| Zoom     | SDK + Browser Hybrid      | Audio, video, screen share, breakout rooms, webinars |
| Teams    | Graph API + Bot Framework | Audio, video, screen share, presence, calendar       |
| Meet     | WebRTC + Browser          | Audio, video, screen share, chat                     |
| Webex    | SDK                       | Audio, video, screen share                           |

## Architecture

```
video_conferencing/
├── gateway/                    # Unified meeting adapter interface
│   ├── interface.py            # Abstract MeetingAdapter base
│   └── session.py              # Session lifecycle management
├── zoom/                       # Zoom integration (29 files)
│   ├── sdk.py                  # Native Zoom SDK
│   ├── browser.py              # Browser-based fallback
│   ├── hybrid_adapter.py       # SDK-preferred hybrid approach
│   ├── api.py                  # REST API client
│   ├── auth.py                 # OAuth authentication
│   ├── phone.py                # Zoom Phone integration
│   ├── webinar.py              # Webinar support
│   └── ...
├── teams/                      # Microsoft Teams (16 files)
│   ├── adapter.py              # Teams adapter
│   ├── graph.py                # Graph API client
│   ├── auth.py                 # Azure AD authentication
│   ├── bot/                    # Bot framework
│   │   ├── manifest.py         # App manifest generation
│   │   └── registration.py     # Bot registration
│   └── media/                  # Application-hosted media
│       ├── streaming.py        # RTP/RTCP streaming
│       ├── sdp.py              # SDP negotiation
│       ├── codecs.py           # Opus, VP8, H.264
│       └── ...
├── meet/                       # Google Meet (10 files)
│   ├── adapter.py              # Meet adapter
│   ├── webrtc.py               # WebRTC Media API
│   ├── browser.py              # Browser automation
│   └── ...
├── webex/                      # Webex (5 files)
│   ├── adapter.py              # Webex adapter
│   └── sdk.py                  # Webex SDK
├── webrtc/                     # Generic WebRTC support
├── virtual_devices/            # Virtual camera/audio
│   ├── camera.py               # v4l2loopback virtual camera
│   └── audio.py                # PulseAudio routing
├── optimization/               # Performance tuning
│   ├── quality.py              # Adaptive quality selection
│   └── large_meeting.py        # Large meeting optimization
├── multiparticipant/           # Multi-participant handling
│   ├── quality.py              # Per-participant quality tiers
│   ├── buffering.py            # Frame buffering
│   ├── thumbnails.py           # Thumbnail generation
│   └── audio_separation.py     # Audio source separation
├── perception/                 # Meeting perception
│   ├── body_language.py        # Body language analysis
│   ├── emotion.py              # Emotion tracking
│   └── behavior.py             # Behavior analysis
├── meeting_intelligence/       # Analytics
│   ├── calendar.py             # Calendar integration
│   ├── analytics.py            # Meeting analytics
│   └── proactive_assistance/   # Proactive features
└── accessibility/              # Accessibility features
    └── captioning.py           # Real-time captions
```

## Gateway Interface

All platform adapters implement the unified `MeetingAdapter` interface:

```python
from video_conferencing.gateway import MeetingAdapter

class MeetingAdapter(ABC):
    @abstractmethod
    async def join(self, meeting_url: str, **options) -> Session: ...

    @abstractmethod
    async def leave(self) -> None: ...

    @abstractmethod
    async def send_video(self, frame: np.ndarray) -> None: ...

    @abstractmethod
    async def send_audio(self, samples: np.ndarray) -> None: ...

    @abstractmethod
    async def receive_events(self) -> AsyncIterator[MeetingEvent]: ...

    @abstractmethod
    async def share_screen(self, enable: bool) -> None: ...
```

## Virtual Device Injection

### Virtual Camera (v4l2loopback)

```python
from video_conferencing.virtual_devices import VirtualCamera

# Create virtual camera device
camera = VirtualCamera(device_path="/dev/video10")
await camera.initialize()

# Inject avatar frames
while True:
    frame = await avatar_engine.render_frame()
    await camera.inject_frame(frame)
```

### Virtual Audio (PulseAudio)

```python
from video_conferencing.virtual_devices import VirtualAudio

# Create virtual audio device
audio = VirtualAudio()
await audio.initialize()

# Inject synthesized speech
while True:
    samples = await voice_engine.synthesize()
    await audio.inject_samples(samples)
```

## Quick Start

### Installation

```bash
# Using Nx
nx install psyche-conferencing

# With all platform SDKs
nx install-all psyche-conferencing

# Platform-specific
nx install-zoom psyche-conferencing
nx install-teams psyche-conferencing
nx install-meet psyche-conferencing

# Or directly with Poetry
cd apps/psyche/conferencing
poetry install
poetry install --extras all
```

### Environment Variables

```bash
# Zoom OAuth
ZOOM_CLIENT_ID=your-client-id
ZOOM_CLIENT_SECRET=your-client-secret
ZOOM_ACCOUNT_ID=your-account-id

# Microsoft Teams / Azure AD
AZURE_TENANT_ID=your-tenant-id
AZURE_CLIENT_ID=your-client-id
AZURE_CLIENT_SECRET=your-client-secret

# Google Meet
GOOGLE_CLIENT_ID=your-client-id
GOOGLE_CLIENT_SECRET=your-client-secret
GOOGLE_REFRESH_TOKEN=your-refresh-token

# Webex
WEBEX_CLIENT_ID=your-client-id
WEBEX_CLIENT_SECRET=your-client-secret

# Service Config
CONFERENCING_PORT=8006
CONFERENCING_HOST=0.0.0.0
LOG_LEVEL=INFO

# Virtual Devices
VIRTUAL_CAMERA_DEVICE=/dev/video10
PULSEAUDIO_SINK=virtual_speaker
```

### Basic Usage

```python
from video_conferencing import gateway

# Select adapter based on meeting URL
adapter = gateway.get_adapter("https://zoom.us/j/123456789")

# Join meeting
session = await adapter.join(
    meeting_url="https://zoom.us/j/123456789",
    password="abc123",
    display_name="AI Assistant",
)

# Inject avatar video
async for event in adapter.receive_events():
    if event.type == "participant_speaking":
        # React to speaker
        frame = await avatar.render_reaction(event.participant_id)
        await adapter.send_video(frame)
```

## API Endpoints

### Gateway

- `POST /meetings/join` - Join a meeting
- `POST /meetings/leave` - Leave current meeting
- `GET /meetings/status` - Get meeting status
- `GET /meetings/participants` - List participants

### Media

- `POST /media/video` - Send video frame
- `POST /media/audio` - Send audio samples
- `GET /media/stream` - WebSocket media stream
- `POST /media/screen-share` - Toggle screen share

### Platform-Specific

- `POST /zoom/schedule` - Schedule Zoom meeting
- `GET /zoom/recordings` - Get Zoom recordings
- `POST /teams/presence` - Update Teams presence
- `GET /meet/calendar` - Get Google Calendar events

### Health

- `GET /health` - Health check
- `GET /ready` - Readiness check
- `GET /metrics` - Prometheus metrics

## Development

### Running the Server

```bash
# Development mode
nx serve psyche-conferencing

# Production mode
nx serve-prod psyche-conferencing
```

### Running Tests

```bash
nx test psyche-conferencing
nx test-unit psyche-conferencing
nx test-integration psyche-conferencing
nx test-cov psyche-conferencing
```

### Docker

```bash
# Build image
nx docker-build psyche-conferencing

# Run with virtual device access
nx docker-run psyche-conferencing
```

## Nx Integration

```bash
# Available targets
nx serve psyche-conferencing          # Development server
nx serve-prod psyche-conferencing     # Production server
nx build psyche-conferencing          # Build package
nx install psyche-conferencing        # Install dependencies
nx install-all psyche-conferencing    # Install all platform SDKs
nx install-zoom psyche-conferencing   # Install Zoom SDK
nx install-teams psyche-conferencing  # Install Teams SDK
nx install-meet psyche-conferencing   # Install Meet SDK
nx lint psyche-conferencing           # Run linters
nx format psyche-conferencing         # Format code
nx test psyche-conferencing           # Run all tests
nx test-unit psyche-conferencing      # Unit tests only
nx test-integration psyche-conferencing  # Integration tests
nx test-cov psyche-conferencing       # Tests with coverage
nx docker-build psyche-conferencing   # Build Docker image
nx docker-run psyche-conferencing     # Run container
```

## Platform-Specific Features

### Zoom

- **Hybrid Adapter**: Prefers SDK, falls back to browser automation
- **Breakout Rooms**: Create and manage breakout sessions
- **Webinars**: Full webinar support with panelist management
- **Zoom Phone**: PSTN integration
- **Reactions**: Emoji reactions and hand raise
- **Recording**: Cloud and local recording management

### Microsoft Teams

- **Bot Framework**: Teams app with calling capabilities
- **Graph API**: Full Microsoft Graph integration
- **Application-Hosted Media**: RTP/RTCP with SDP negotiation
- **Presence**: Real-time presence updates
- **Calendar**: Meeting scheduling via Graph

### Google Meet

- **WebRTC**: Native WebRTC Media API
- **Browser Fallback**: Playwright-based automation
- **Screen Share**: Full screen sharing support
- **Chat**: Real-time chat integration

### Webex

- **SDK Integration**: Native Webex SDK
- **WebRTC Media**: Standard WebRTC handling

## Multi-Participant Optimization

### Adaptive Quality Selection

```python
from video_conferencing.multiparticipant import QualityManager

quality = QualityManager()

# Configure quality tiers
quality.set_tier("speaker", resolution=1080, fps=30)
quality.set_tier("active", resolution=720, fps=24)
quality.set_tier("passive", resolution=480, fps=15)
quality.set_tier("thumbnail", resolution=180, fps=5)

# Auto-assign based on activity
quality.update_participant_tier(participant_id, speaking=True)
```

### Large Meeting Optimization

- Participant quality rotation
- Bandwidth budgeting
- CPU throttling
- Roster sampling
- Cascading fallback

## Performance

### Latency Targets

| Operation       | Target                  |
| --------------- | ----------------------- |
| Video injection | < 16ms (60 FPS capable) |
| Audio injection | < 10ms                  |
| Platform join   | < 5s                    |
| Reconnection    | < 2s                    |

### Resource Usage

- **CPU**: 2-4 cores per meeting
- **Memory**: 512MB - 2GB per meeting
- **Network**: 2-10 Mbps per meeting

## Security

- **OAuth 2.0**: Platform-specific OAuth flows
- **Token Management**: Secure credential storage
- **Sandboxed Execution**: Isolated per-meeting contexts
- **Audit Logging**: All meeting actions logged

## License

Proprietary - Oshun Platform