Common utilities library for the Psyche AI Virtual Assistant Platform.
Part of the Oshun Platform.
Overview#
This library provides shared utilities used across all Psyche services:
- GPU Optimization - Model quantization, batch inference, TensorRT, memory management
- Structured Logging - JSON logging with Loki/Promtail integration
- Success Metrics - Human likeness, conversation quality, performance, reliability, usage, cost tracking
- Performance - Streaming pipelines, parallel execution, latency optimization, caching
- Scalability - Horizontal scaling, load balancing, database optimization
Modules#
GPU Optimization (common.gpu)#
Utilities for GPU-accelerated inference:
python
from common.gpu import (
quantize_model,
BatchInferenceEngine,
optimize_for_tensorrt,
GPUMemoryManager,
)
# Quantize model to INT8
quantized = quantize_model(model, mode=QuantizationMode.INT8)
# Run batch inference
engine = BatchInferenceEngine(model)
results = await engine.run_batch(inputs)
# TensorRT optimization
optimized = optimize_for_tensorrt(model)
Structured Logging (common.logging)#
JSON-structured logging for observability:
python
from common.logging import get_logger, configure_logging
configure_logging(service_name="avatar-engine", level="INFO")
logger = get_logger(__name__)
logger.info("Processing frame", extra={
"frame_id": 123,
"duration_ms": 15.5,
"user_id": "user_abc",
})
Success Metrics (common.metrics)#
Comprehensive metrics collection:
python
from common.metrics import (
LatencyTracker,
ConversationQualityScorer,
UptimeTracker,
CostTracker,
)
# Track latency
tracker = LatencyTracker()
with tracker.measure("inference"):
result = await model.infer(input)
# Score conversation quality
scorer = ConversationQualityScorer()
score = scorer.score_conversation(conversation)
# Track costs
costs = CostTracker()
costs.record(category=CostCategory.API_CALL, amount=0.01)
Performance (common.performance)#
Performance optimization utilities:
python
from common.performance import (
StreamPipeline,
parallel_map,
LRUCache,
ConnectionPool,
)
# Streaming pipeline
pipeline = StreamPipeline()
pipeline.add_processor(transform_stage)
pipeline.add_processor(filter_stage)
await pipeline.process(input_stream)
# Parallel execution
results = await parallel_map(process_item, items, max_workers=4)
# Caching
cache = LRUCache(maxsize=1000)
result = cache.get_or_compute(key, compute_func)
Scalability (common.scalability)#
Horizontal scaling and database optimization:
python
from common.scalability import (
StatelessService,
LoadBalancer,
QueryOptimizer,
DatabaseConnectionPool,
)
# Stateless service pattern
service = StatelessService(config)
await service.start()
# Load balancing
balancer = LoadBalancer(strategy=LoadBalancingStrategy.ROUND_ROBIN)
endpoint = balancer.select_endpoint()
# Query optimization
optimizer = QueryOptimizer()
optimized_query = optimizer.optimize(query)
Project Structure#
text
src/common/
├── __init__.py # Package exports
├── gpu/ # GPU optimization
│ ├── quantization.py # Model quantization
│ ├── inference.py # Batch inference
│ ├── tensorrt.py # TensorRT integration
│ └── memory.py # Memory management
├── logging/ # Structured logging
│ ├── logger.py # Logger implementation
│ ├── formatters.py # JSON formatters
│ ├── handlers.py # Async/batch handlers
│ └── filters.py # Sensitive data filters
├── metrics/ # Success metrics
│ ├── human_likeness.py # A/B testing, detection
│ ├── conversation_quality.py
│ ├── performance.py # Latency, frame rate
│ ├── reliability.py # Uptime, errors, MTTR
│ ├── usage.py # Sessions, users
│ ├── cost.py # Cost tracking
│ └── dashboard.py # Operational dashboards
├── performance/ # Performance optimization
│ ├── streaming.py # Stream pipelines
│ ├── parallel.py # Parallel execution
│ ├── latency.py # Latency budgets
│ ├── caching.py # Cache strategies
│ └── network.py # Connection pooling
└── scalability/ # Scalability
├── horizontal.py # Stateless services
└── database.py # Query optimization
Development#
Using Nx#
bash
# Install dependencies
nx install psyche-common
# Install with GPU support
nx install-gpu psyche-common
# Install with all extras
nx install-all psyche-common
# Build
nx build psyche-common
# Linting
nx lint psyche-common
nx lint-fix psyche-common
# Formatting
nx format psyche-common
nx format-check psyche-common
# Type checking
nx typecheck psyche-common
# Testing
nx test psyche-common
nx test-cov psyche-common
Direct Commands#
bash
cd libs/psyche/common
# Install dependencies
poetry install
# With GPU support
poetry install --extras gpu
# Run tests
poetry run pytest
# Lint
poetry run ruff check src tests
Configuration#
Environment Variables#
| Variable | Description | Default |
|---|---|---|
LOG_LEVEL |
Logging level | INFO |
LOG_FORMAT |
Log format (json/text) | json |
METRICS_ENABLED |
Enable metrics collection | true |
CACHE_BACKEND |
Cache backend (memory/redis) | memory |
REDIS_URL |
Redis URL for caching | - |
Optional Dependencies#
Install extras for specific features:
bash
# GPU optimization (PyTorch, TensorRT)
poetry install --extras gpu
# Redis caching
poetry install --extras caching
# All extras
poetry install --extras all
License#
Proprietary - Oshun Platform