Technical documentation for Lilith's comprehensive multiplayer networking system covering real-time synchronization, voice chat, presence, and cross-platform connectivity.
Overview#
The Lilith Platform implements a multi-layer networking architecture supporting:
- Real-Time Multiplayer: Colyseus + Socket.io for state synchronization
- Voice Chat: WebRTC peer-to-peer with spatial audio
- Presence System: Online status and activity tracking
- Cross-Platform Sync: Seamless experience across devices
- Spatial Audio: 3D positional audio for immersion
Architecture#
┌─────────────────────────────────────────────────────────────────────────┐
│ Networking Architecture │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Transport Layer │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────────┐ │ │
│ │ │ Colyseus │ │ Socket.io │ │ WebRTC │ │ │
│ │ │ (Rooms) │──│ (Events) │──│ (Voice P2P) │ │ │
│ │ │ Path: /ws │ │ Path: /ws/* │ │ │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Synchronization Layer │ │
│ │ ┌───────────────┐ ┌───────────────┐ ┌─────────────────────┐ │ │
│ │ │ Multiplayer │ │ Presence │ │ Cross-Platform │ │ │
│ │ │ Sync │──│ Broadcaster │──│ Sync │ │ │
│ │ │ (30 Hz) │ │ │ │ (Multi-device) │ │ │
│ │ └───────────────┘ └───────────────┘ └─────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Optimization Layer │ │
│ │ ┌───────────────┐ ┌───────────────┐ ┌─────────────────────┐ │ │
│ │ │ Delta │ │ Message │ │ Spatial │ │ │
│ │ │ Compression │──│ Batching │──│ Partitioning │ │ │
│ │ │ │ │ (25ms) │ │ │ │ │
│ │ └───────────────┘ └───────────────┘ └─────────────────────┘ │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Transport Layer#
1. Colyseus Gateway#
Location: services/metaverse/src/multiplayer/colyseusGateway.ts
Game state management and room-based multiplayer.
Configuration#
interface ColyseusConfig {
transport: WebSocketTransport;
path: '/colyseus';
maxClients: 1000;
roomType: 'MetaverseRoom';
}
Features#
| Feature | Description |
|---|---|
| Room Management | Create, join, leave rooms |
| State Sync | Automatic diff-patch to clients |
| Schema Types | Type-safe state definitions |
| Patch Rate | 20 Hz (50ms intervals) |
| Max Clients | 1000 per room |
2. Socket.io Server#
Location:
services/metaverse/src/multiplayer/SocketIOPresenceBroadcaster.ts
Real-time event broadcasting and presence updates.
Configuration#
interface SocketIOConfig {
path: '/ws/presence';
cors: { origin: '*' };
transports: ['websocket', 'polling'];
}
Events Emitted#
| Event | Payload | Description |
|---|---|---|
multiplayer.presence |
PresencePayload |
Room population updates |
multiplayer.player |
PlayerEventPayload |
Join/leave events |
3. WebRTC (Voice Chat)#
Location: web/src/lib/metaverse-voice-chat.ts
Peer-to-peer voice communication with spatial audio.
ICE Servers#
const iceServers = [
{ urls: 'stun:stun.l.google.com:19302' },
{ urls: 'stun:stun1.l.google.com:19302' },
];
Signaling Messages#
type SignalingMessage =
| {
type: 'offer';
from: string;
to: string;
offer: RTCSessionDescriptionInit;
}
| {
type: 'answer';
from: string;
to: string;
answer: RTCSessionDescriptionInit;
}
| {
type: 'ice-candidate';
from: string;
to: string;
candidate: RTCIceCandidateInit;
};
Room Management#
MetaverseRoom#
Location: services/metaverse/src/multiplayer/MetaverseRoom.ts
Colyseus room implementation for metaverse state management.
Room State#
class MetaverseState extends Schema {
@type({ map: PlayerState })
public players = new MapSchema<PlayerState>();
@type('number')
public playerCount = 0;
}
class PlayerState extends Schema {
@type('string') id: string;
@type('string') sessionId: string;
@type('string') name: string;
@type('number') x: number;
@type('number') y: number;
@type('number') z: number;
@type('number') rotationX: number;
@type('number') rotationY: number;
@type('number') rotationZ: number;
@type('string') animation: string;
}
Room Lifecycle#
class MetaverseRoom extends Room<MetaverseState> {
onCreate(options: any) {
this.setState(new MetaverseState());
this.setSimulationInterval(() => this.update(), 33); // 30 Hz
this.setPatchRate(50); // 20 Hz patch rate
// Register message handlers
this.onMessage('move', this.handleMove.bind(this));
this.onMessage('latency', this.handleLatency.bind(this));
}
onJoin(client: Client, options: any) {
const player = new PlayerState();
player.id = client.id;
player.sessionId = client.sessionId;
player.name = options.name || 'Anonymous';
this.state.players.set(client.sessionId, player);
this.state.playerCount++;
// Broadcast presence
this.presenceBroadcaster.emit({
roomId: this.roomId,
playerCount: this.state.playerCount,
capacity: this.maxClients,
});
}
onLeave(client: Client, consented: boolean) {
this.state.players.delete(client.sessionId);
this.state.playerCount--;
}
}
Message Handlers#
| Message | Payload | Description |
|---|---|---|
move |
{ x, y, z, rx, ry, rz, animation } |
Position/rotation update |
latency |
{ latency: number } |
Client latency report |
State Synchronization#
MultiplayerSync#
Location: services/metaverse/src/multiplayer/MultiplayerSync.ts
Server-authoritative state synchronization with client prediction.
Configuration#
interface SyncConfig {
tickRate: number; // Default: 30 Hz (33ms)
maxPlayers: number; // Default: 1000
deltaCompression: boolean; // Default: true
interpolation: boolean; // Default: true
latencyCompensation: number; // Default: 100ms
}
Entity State#
interface EntityState {
id: string;
userId: string;
position: [number, number, number]; // [x, y, z]
rotation: [number, number, number]; // [rx, ry, rz]
velocity: [number, number, number]; // [vx, vy, vz]
timestamp: number;
sequence: number;
}
Synchronization Flow#
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Client A │ │ Server │ │ Client B │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
│ Input + Prediction │ │
│─────────────────────→ │ │
│ │ │
│ │ Validate & Update │
│ │──────────────────────→│
│ │ │
│ Authoritative State │ │
│←───────────────────── │ │
│ │ │
│ Reconciliation │ │
│ (if prediction error) │ │
│ │ │
Delta Compression#
Only changed fields are transmitted:
interface DeltaState {
position?: [number, number, number];
rotation?: [number, number, number];
velocity?: [number, number, number];
}
// Epsilon comparison for float equality
const EPSILON = 0.001;
function hasChanged(a: number, b: number): boolean {
return Math.abs(a - b) > EPSILON;
}
Bandwidth savings: ~70% for stationary players.
Client Prediction#
class PredictionBuffer {
private buffer: Map<number, EntityState> = new Map();
private maxSize = 100;
addPrediction(sequence: number, state: EntityState): void {
this.buffer.set(sequence, state);
if (this.buffer.size > this.maxSize) {
// Remove oldest entries
const oldest = Math.min(...this.buffer.keys());
this.buffer.delete(oldest);
}
}
reconcile(serverState: EntityState): void {
const predicted = this.buffer.get(serverState.sequence);
if (!predicted) return;
const error = calculatePositionError(predicted, serverState);
if (error > RECONCILIATION_THRESHOLD) {
// Correct to server state and replay inputs
this.correctAndReplay(serverState);
}
}
}
Reconciliation threshold: 0.1 units position error.
Server Tick Loop#
update(): void {
const now = Date.now();
const dt = (now - this.lastTick) / 1000;
this.lastTick = now;
// Process all entities
for (const [id, entity] of this.entities) {
// Apply physics (dead reckoning)
entity.position[0] += entity.velocity[0] * dt;
entity.position[1] += entity.velocity[1] * dt;
entity.position[2] += entity.velocity[2] * dt;
entity.sequence = this.tickNumber;
entity.timestamp = now;
}
// Broadcast to all clients
this.broadcastState();
this.tickNumber++;
}
Events#
| Event | Description |
|---|---|
started |
Sync server started |
stopped |
Sync server stopped |
tick |
Server tick completed |
connection_added |
New player connected |
connection_removed |
Player disconnected |
entity_updated |
Entity state changed |
entity_removed |
Entity deleted |
reconciliation |
Prediction corrected |
error |
Network error occurred |
Presence System#
PresencePayload#
interface PresencePayload {
roomId: string;
playerCount: number;
capacity: number;
timestamp: number;
}
interface PlayerEventPayload {
roomId: string;
playerId: string;
userId: string;
event: 'join' | 'leave';
timestamp: number;
}
Cross-Platform Presence#
Location: web/src/lib/cross-platform-sync.ts
interface PresenceData {
userId: string;
deviceId: string;
online: boolean;
currentActivity: string;
lastSeen: number;
}
Voice Chat#
MetaverseVoiceChat#
Location: web/src/lib/metaverse-voice-chat.ts
WebRTC-based voice communication with spatial audio.
Initialization#
const voiceChat = new MetaverseVoiceChat({
roomId: 'meditation-hall',
userId: 'user123',
signalingSocket: socket,
});
// Start voice chat
await voiceChat.start();
// Set listener position (for spatial audio)
voiceChat.setListenerPosition({ x: 0, y: 1.6, z: 0 });
voiceChat.setListenerOrientation({ x: 0, y: 0, z: -1 }, { x: 0, y: 1, z: 0 });
// Update participant positions
voiceChat.updateParticipantPosition('participant456', { x: 5, y: 1.6, z: 3 });
// Mute/unmute
voiceChat.setMuted(true);
// Stop voice chat
voiceChat.stop();
Audio Processing Pipeline#
Microphone → MediaStreamSource → GainNode → RTCPeerConnection
│
└→ AnalyserNode (level detection)
RTCPeerConnection → MediaStreamSource → PannerNode → GainNode → Destination
│
└→ AnalyserNode (level detection)
Audio Constraints#
const audioConstraints = {
audio: {
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
channelCount: 1,
sampleRate: 48000,
},
video: false,
};
Spatial Audio Configuration#
interface SpatialAudioConfig {
panningModel: 'HRTF' | 'equalpower';
distanceModel: 'inverse' | 'linear' | 'exponential';
refDistance: number; // Default: 1
maxDistance: number; // Default: 50
rolloffFactor: number; // Default: 1
coneInnerAngle: number; // Default: 360
coneOuterAngle: number; // Default: 360
coneOuterGain: number; // Default: 0
}
Signaling Flow#
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Client A │ │ Signaling │ │ Client B │
│ (Caller) │ │ Server │ │ (Callee) │
└──────┬───────┘ └──────┬───────┘ └──────┬───────┘
│ │ │
│ Create Offer │ │
│─────────────────────→ │ │
│ │ Forward Offer │
│ │──────────────────────→│
│ │ │
│ │ Create Answer │
│ │←──────────────────────│
│ Forward Answer │ │
│←───────────────────── │ │
│ │ │
│ ICE Candidates │ ICE Candidates │
│←─────────────────────→│←─────────────────────→│
│ │ │
╔═══════════════════════════════════════════════╗
║ Direct P2P Audio Connection ║
╚═══════════════════════════════════════════════╝
Spatial Audio (Environment)#
SpatialAudioManager#
Location: services/metaverse/src/audio/SpatialAudio.ts
Environment audio sources with 3D positioning.
Audio Source Configuration#
interface AudioSource {
id: string;
url: string;
position: [number, number, number];
volume: number; // 0-1
maxDistance: number; // Audio cutoff distance
refDistance: number; // Reference distance
rolloffFactor: number; // Attenuation rate
coneInnerAngle: number; // Directional cone inner
coneOuterAngle: number; // Directional cone outer
dopplerEnabled: boolean; // Doppler effect
occlusionEnabled: boolean; // Occlusion simulation
}
Audio Zones#
interface AudioZone {
id: string;
position: [number, number, number];
radius: number;
reverb: ReverbPreset;
sources: AudioSource[];
}
type ReverbPreset =
| 'none'
| 'small_room'
| 'large_room'
| 'hall'
| 'cathedral'
| 'cave'
| 'outdoor';
Usage#
const spatialAudio = new SpatialAudioManager();
// Add environment audio source
spatialAudio.addSource({
id: 'waterfall',
url: '/audio/waterfall.mp3',
position: [10, 2, -5],
volume: 0.7,
maxDistance: 30,
refDistance: 1,
rolloffFactor: 1,
});
// Create audio zone
spatialAudio.addZone({
id: 'cave',
position: [0, 0, 0],
radius: 20,
reverb: 'cave',
sources: [
/* sources */
],
});
// Update listener position
spatialAudio.setListenerPosition([0, 1.6, 0]);
spatialAudio.setListenerOrientation([0, 0, -1], [0, 1, 0]);
Message Optimization#
WebSocket Batcher#
Location: web/src/lib/websocket-batcher.ts
Batches multiple messages into single network packets.
Configuration#
interface BatcherConfig {
flushInterval: number; // Default: 25ms
maxBatchSize: number; // Default: 32 messages
deduplicate: boolean; // Default: true
}
Usage#
const batcher = new WebSocketSendBatcher(socket, {
flushInterval: 25,
maxBatchSize: 32,
deduplicate: true,
});
// Queue messages (automatically batched)
batcher.send('move', { x: 10, y: 0, z: 5 });
batcher.send('animation', { name: 'walk' });
// Messages sent together on next flush
Bandwidth reduction: ~95% fewer individual messages.
Position Update Batcher#
Location: web/src/lib/multiplayer-optimization.ts
Specialized batcher for position updates.
const positionBatcher = new PositionUpdateBatcher({
flushInterval: 50, // 20 Hz
});
// All position updates batched into single message
positionBatcher.queueUpdate('entity1', { x: 10, y: 0, z: 5 });
positionBatcher.queueUpdate('entity2', { x: -5, y: 0, z: 8 });
Spatial Partitioner#
Location: web/src/lib/multiplayer-optimization.ts
Cell-based spatial hashing for efficient proximity queries.
const partitioner = new SpatialPartitioner({
cellSize: 10, // 10 unit cells
});
// Register entities
partitioner.update('player1', { x: 0, y: 0, z: 0 });
partitioner.update('player2', { x: 5, y: 0, z: 3 });
// Query nearby entities
const nearby = partitioner.getNearby({ x: 0, y: 0, z: 0 }, 15);
// Returns: ['player1', 'player2']
Used for:
- Voice chat distance culling
- State update prioritization
- Efficient collision queries
Cross-Platform Sync#
CrossPlatformSync#
Location: web/src/lib/cross-platform-sync.ts
Seamless synchronization across web, mobile, and desktop.
Sync Data Types#
| Type | Description |
|---|---|
chat |
Chat messages (thread-based) |
audio |
Playback position, duration |
preferences |
Theme, language, quality |
progress |
Content completion percentage |
bookmark |
Saved positions with notes |
presence |
Online status, activity |
Message Format#
interface SyncMessage {
type: SyncDataType;
action: 'create' | 'update' | 'delete' | 'sync';
data: any;
timestamp: number;
deviceId: string;
userId: string;
version: number;
}
Usage#
const sync = CrossPlatformSync.getInstance();
// Initialize
await sync.initialize({
userId: 'user123',
platform: 'web',
});
// Sync progress
sync.syncProgress({
contentId: 'meditation-101',
progress: 0.75,
lastPosition: 450,
});
// Sync preferences
sync.syncPreferences({
theme: 'dark',
language: 'en',
audioQuality: 'high',
});
// Listen for updates from other devices
sync.on('progress', (data) => {
// Resume from last position
player.seek(data.lastPosition);
});
Offline Support#
// Messages queued during offline
sync.on('offline', () => {
console.log('Offline - queuing changes');
});
// Sync when back online
sync.on('online', () => {
console.log(`Syncing ${sync.pendingChanges} changes`);
});
Device Handoff#
Location: web/src/services/CrossDeviceHandoffManager.ts
Seamless audio transfer between devices.
Device Capabilities#
interface DeviceCapabilities {
audioPlayback: boolean;
audioEffects: boolean;
binauralBeats: boolean;
spatialAudio: boolean;
webAudioVersion: string;
offlineSync: boolean;
}
Handoff Process#
const handoff = new CrossDeviceHandoffManager();
// Discover available devices
const devices = await handoff.discoverDevices();
// Request handoff
await handoff.requestHandoff({
targetDeviceId: devices[0].id,
preserveAudioProfile: true,
preserveEffects: true,
seamlessTransition: true,
fadeOutDuration: 500,
fadeInDuration: 500,
});
Connection Management#
WebSocket Connection#
Location: web/src/lib/cross-platform-sync-connection.ts
Robust WebSocket client with auto-reconnection.
Features#
| Feature | Description |
|---|---|
| Auto-Reconnect | Exponential backoff (1s → 30s max) |
| Heartbeat | 30-second keep-alive |
| Visibility Detection | Pause when tab hidden |
| Offline Detection | Queue messages when offline |
Configuration#
interface ConnectionConfig {
url: string;
reconnectDelay: number; // Initial: 1000ms
maxReconnectDelay: number; // Max: 30000ms
heartbeatInterval: number; // Default: 30000ms
timeout: number; // Connection timeout
}
Usage#
const connection = new SyncConnection({
url: 'wss://api.lilith.app/ws/sync',
heartbeatInterval: 30000,
});
connection.on('connected', () => {
console.log('Connected');
});
connection.on('disconnected', () => {
console.log('Disconnected - reconnecting...');
});
connection.on('message', (data) => {
handleMessage(data);
});
connection.connect();
Server Authority#
Authority Model#
The server is authoritative for all game state:
- Position Validation: Server processes movement each tick
- Latency Compensation: Track per-connection latency (default 100ms)
- Entity Ownership: Players only control their own avatar
- Sequence Numbering: Prevents out-of-order processing
Reconciliation#
// Position error threshold
const RECONCILIATION_THRESHOLD = 0.1; // units
function reconcile(predicted: EntityState, server: EntityState): boolean {
const error = Math.sqrt(
Math.pow(predicted.position[0] - server.position[0], 2) +
Math.pow(predicted.position[1] - server.position[1], 2) +
Math.pow(predicted.position[2] - server.position[2], 2)
);
if (error > RECONCILIATION_THRESHOLD) {
// Correct to server state
applyCorrection(server);
// Replay pending inputs
replayInputs(server.sequence);
return true;
}
return false;
}
Security Considerations#
Connection Security#
| Measure | Description |
|---|---|
| TLS/WSS | Encrypted WebSocket connections |
| Connection Limits | Max 1000 clients per room |
| Heartbeat Timeout | Detect and clean stale connections |
| CORS Protection | Configurable origin whitelist |
State Validation#
| Measure | Description |
|---|---|
| Entity Ownership | Players can only modify own state |
| Server Authority | Server validates all state changes |
| Latency Limits | 100ms compensation window |
| Rate Limiting | Prevent message flooding |
Voice Chat Security#
| Measure | Description |
|---|---|
| STUN/TURN | NAT traversal without exposing IPs |
| Signaling Auth | Authenticated signaling channel |
| Audio Only | No video track to reduce attack surface |
Performance Metrics#
| Metric | Value |
|---|---|
| Bandwidth per Player | ~1-2 KB/s (with delta compression) |
| Tick Rate | 30 Hz (33ms) |
| Patch Rate | 20 Hz (50ms) |
| Latency Tolerance | ±100ms compensation |
| Position Error Threshold | 0.1 units |
| Max Concurrent Players | 1000 per room |
| Batch Flush Interval | 25ms |
| Heartbeat Interval | 30 seconds |
Integration Examples#
Complete Multiplayer Setup#
import { Client } from 'colyseus.js';
import { MetaverseVoiceChat } from '@/lib/metaverse-voice-chat';
import { SpatialPartitioner } from '@/lib/multiplayer-optimization';
// Connect to Colyseus
const client = new Client('wss://api.lilith.app');
const room = await client.joinOrCreate('MetaverseRoom', {
name: 'Player1',
});
// Setup voice chat
const voiceChat = new MetaverseVoiceChat({
roomId: room.id,
userId: room.sessionId,
signalingSocket: room.connection,
});
// Spatial partitioning for optimization
const partitioner = new SpatialPartitioner({ cellSize: 10 });
// Handle state changes
room.state.players.onAdd = (player, sessionId) => {
createPlayerAvatar(player);
partitioner.update(sessionId, player);
};
room.state.players.onChange = (player, sessionId) => {
updatePlayerAvatar(player);
partitioner.update(sessionId, player);
voiceChat.updateParticipantPosition(sessionId, player);
};
room.state.players.onRemove = (player, sessionId) => {
removePlayerAvatar(sessionId);
partitioner.remove(sessionId);
};
// Send position updates
function sendPosition(position: Vector3, rotation: Euler) {
room.send('move', {
x: position.x,
y: position.y,
z: position.z,
rx: rotation.x,
ry: rotation.y,
rz: rotation.z,
});
}
// Start voice chat
await voiceChat.start();
Cross-Platform Meditation Session#
import { CrossPlatformSync } from '@/lib/cross-platform-sync';
const sync = CrossPlatformSync.getInstance();
await sync.initialize({ userId: 'user123', platform: 'web' });
// Sync meditation progress
function onMeditationProgress(progress: number, position: number) {
sync.syncProgress({
contentId: 'daily-meditation',
progress,
lastPosition: position,
timestamp: Date.now(),
});
}
// Handle handoff from another device
sync.on('audio', (data) => {
if (data.action === 'handoff') {
// Resume from other device
player.load(data.contentId);
player.seek(data.lastPosition);
player.play();
}
});
Best Practices#
Do's#
- Use delta compression for frequent updates
- Batch messages to reduce packet count
- Implement client prediction for responsive feel
- Use spatial partitioning for large worlds
- Monitor latency and adjust compensation
- Clean up connections on disconnect
Don'ts#
- Don't trust client state without validation
- Don't send full state every tick (use deltas)
- Don't process audio for distant participants
- Don't skip heartbeats (causes stale connections)
- Don't exceed max client limits
Related Documentation#
Last Updated: January 2026