Eve library

Psyche Common

Other

Revision 1

On this page

Common utilities library for the Psyche AI Virtual Assistant Platform.

Part of the Oshun Platform.

Overview#

This library provides shared utilities used across all Psyche services:

  • GPU Optimization - Model quantization, batch inference, TensorRT, memory management
  • Structured Logging - JSON logging with Loki/Promtail integration
  • Success Metrics - Human likeness, conversation quality, performance, reliability, usage, cost tracking
  • Performance - Streaming pipelines, parallel execution, latency optimization, caching
  • Scalability - Horizontal scaling, load balancing, database optimization

Modules#

GPU Optimization (common.gpu)#

Utilities for GPU-accelerated inference:

python
from common.gpu import (
    quantize_model,
    BatchInferenceEngine,
    optimize_for_tensorrt,
    GPUMemoryManager,
)

# Quantize model to INT8
quantized = quantize_model(model, mode=QuantizationMode.INT8)

# Run batch inference
engine = BatchInferenceEngine(model)
results = await engine.run_batch(inputs)

# TensorRT optimization
optimized = optimize_for_tensorrt(model)

Structured Logging (common.logging)#

JSON-structured logging for observability:

python
from common.logging import get_logger, configure_logging

configure_logging(service_name="avatar-engine", level="INFO")
logger = get_logger(__name__)

logger.info("Processing frame", extra={
    "frame_id": 123,
    "duration_ms": 15.5,
    "user_id": "user_abc",
})

Success Metrics (common.metrics)#

Comprehensive metrics collection:

python
from common.metrics import (
    LatencyTracker,
    ConversationQualityScorer,
    UptimeTracker,
    CostTracker,
)

# Track latency
tracker = LatencyTracker()
with tracker.measure("inference"):
    result = await model.infer(input)

# Score conversation quality
scorer = ConversationQualityScorer()
score = scorer.score_conversation(conversation)

# Track costs
costs = CostTracker()
costs.record(category=CostCategory.API_CALL, amount=0.01)

Performance (common.performance)#

Performance optimization utilities:

python
from common.performance import (
    StreamPipeline,
    parallel_map,
    LRUCache,
    ConnectionPool,
)

# Streaming pipeline
pipeline = StreamPipeline()
pipeline.add_processor(transform_stage)
pipeline.add_processor(filter_stage)
await pipeline.process(input_stream)

# Parallel execution
results = await parallel_map(process_item, items, max_workers=4)

# Caching
cache = LRUCache(maxsize=1000)
result = cache.get_or_compute(key, compute_func)

Scalability (common.scalability)#

Horizontal scaling and database optimization:

python
from common.scalability import (
    StatelessService,
    LoadBalancer,
    QueryOptimizer,
    DatabaseConnectionPool,
)

# Stateless service pattern
service = StatelessService(config)
await service.start()

# Load balancing
balancer = LoadBalancer(strategy=LoadBalancingStrategy.ROUND_ROBIN)
endpoint = balancer.select_endpoint()

# Query optimization
optimizer = QueryOptimizer()
optimized_query = optimizer.optimize(query)

Project Structure#

text
src/common/
├── __init__.py           # Package exports
├── gpu/                  # GPU optimization
│   ├── quantization.py   # Model quantization
│   ├── inference.py      # Batch inference
│   ├── tensorrt.py       # TensorRT integration
│   └── memory.py         # Memory management
├── logging/              # Structured logging
│   ├── logger.py         # Logger implementation
│   ├── formatters.py     # JSON formatters
│   ├── handlers.py       # Async/batch handlers
│   └── filters.py        # Sensitive data filters
├── metrics/              # Success metrics
│   ├── human_likeness.py # A/B testing, detection
│   ├── conversation_quality.py
│   ├── performance.py    # Latency, frame rate
│   ├── reliability.py    # Uptime, errors, MTTR
│   ├── usage.py          # Sessions, users
│   ├── cost.py           # Cost tracking
│   └── dashboard.py      # Operational dashboards
├── performance/          # Performance optimization
│   ├── streaming.py      # Stream pipelines
│   ├── parallel.py       # Parallel execution
│   ├── latency.py        # Latency budgets
│   ├── caching.py        # Cache strategies
│   └── network.py        # Connection pooling
└── scalability/          # Scalability
    ├── horizontal.py     # Stateless services
    └── database.py       # Query optimization

Development#

Using Nx#

bash
# Install dependencies
nx install psyche-common

# Install with GPU support
nx install-gpu psyche-common

# Install with all extras
nx install-all psyche-common

# Build
nx build psyche-common

# Linting
nx lint psyche-common
nx lint-fix psyche-common

# Formatting
nx format psyche-common
nx format-check psyche-common

# Type checking
nx typecheck psyche-common

# Testing
nx test psyche-common
nx test-cov psyche-common

Direct Commands#

bash
cd libs/psyche/common

# Install dependencies
poetry install

# With GPU support
poetry install --extras gpu

# Run tests
poetry run pytest

# Lint
poetry run ruff check src tests

Configuration#

Environment Variables#

Variable Description Default
LOG_LEVEL Logging level INFO
LOG_FORMAT Log format (json/text) json
METRICS_ENABLED Enable metrics collection true
CACHE_BACKEND Cache backend (memory/redis) memory
REDIS_URL Redis URL for caching -

Optional Dependencies#

Install extras for specific features:

bash
# GPU optimization (PyTorch, TensorRT)
poetry install --extras gpu

# Redis caching
poetry install --extras caching

# All extras
poetry install --extras all

License#

Proprietary - Oshun Platform