Eve library

Psyche Logging

Other

Revision 1

On this page

Structured logging for Psyche AI Virtual Assistant.

Overview#

psyche-logging provides high-performance structured logging aligned with the @oshun/logging TypeScript patterns for cross-language compatibility. It includes context propagation, sampling, and PII redaction.

Features#

  • Structured Logging: JSON-formatted log entries with consistent schema
  • Context Propagation: Request-scoped context via contextvars
  • Log Sampling: Multiple sampling strategies to control log volume
  • PII Redaction: Automatic redaction of sensitive data
  • OpenTelemetry Integration: Optional trace context enrichment
  • Child Loggers: Context inheritance for scoped logging

Installation#

bash
# Basic installation
poetry add psyche-logging

# With OpenTelemetry support
poetry add psyche-logging[otel]

Quick Start#

Basic Logging#

python
from psyche_logging import create_logger, LogLevel

# Create a logger
logger = create_logger(
    service="my-service",
    version="1.0.0",
    level=LogLevel.INFO,
)

# Log messages
logger.info("User logged in", data={"user_id": "123"})
logger.warn("Rate limit approaching", data={"current": 90, "max": 100})
logger.error("Request failed", error=exc, data={"url": "/api/users"})

Global Logger#

python
from psyche_logging import configure_logger, get_logger, LoggerConfig, LogLevel

# Configure global logger once at startup
configure_logger(LoggerConfig(
    service="my-service",
    level=LogLevel.INFO,
    pretty=True,  # For development
))

# Use anywhere
logger = get_logger()
logger.info("Application started")

Request-Scoped Context#

python
from psyche_logging import get_logger, logging_context, async_request_context

logger = get_logger()

# All logs within context include request_id
with logging_context(request_id="req-123", user_id="user-456"):
    logger.info("Processing request")
    # ... do work
    logger.info("Request complete")

# Async version
async with async_request_context(request_id="req-123"):
    logger.info("Processing async request")

Child Loggers#

python
from psyche_logging import create_logger, LogContext

logger = create_logger(service="api")

# Create child logger with additional context
order_logger = logger.child(LogContext(extra={"module": "orders"}))
order_logger.info("Order created", data={"order_id": "ord-123"})

# Create request-scoped child
request_logger = logger.child(LogContext(
    request_id="req-123",
    user_id="user-456",
))
request_logger.info("Handling request")

Logging with Errors#

python
try:
    result = await process_request()
except Exception as e:
    logger.error(
        "Request processing failed",
        error=e,
        data={"request_id": request_id},
    )
    raise

Log Levels#

Level Value Description
TRACE 10 Very detailed debugging
DEBUG 20 Debugging information
INFO 30 General information
WARN 40 Warning conditions
ERROR 50 Error conditions
FATAL 60 Critical failures

Transports#

Console Transport#

python
from psyche_logging import ConsoleTransport, LoggerConfig, OshunLogger

# Pretty printing for development
config = LoggerConfig(
    transports=[
        ConsoleTransport(pretty=True, colors=True),
    ],
)
logger = OshunLogger(config)

File Transport#

python
from psyche_logging import FileTransport, LoggerConfig, OshunLogger

# JSON Lines file with rotation
config = LoggerConfig(
    transports=[
        FileTransport(
            filepath="/var/log/app/app.log",
            max_size_bytes=100 * 1024 * 1024,  # 100 MB
            max_files=5,
        ),
    ],
)
logger = OshunLogger(config)

Multiple Transports#

python
config = LoggerConfig(
    transports=[
        ConsoleTransport(pretty=True),
        FileTransport(filepath="/var/log/app.log"),
    ],
)

Sampling#

Fixed Rate Sampler#

python
from psyche_logging import FixedRateSampler, LoggerConfig, OshunLogger

sampler = FixedRateSampler(
    rate=0.1,  # Sample 10% of logs
    always_log_levels=[LogLevel.ERROR, LogLevel.FATAL],  # Always log errors
)

config = LoggerConfig(sampler=sampler)
logger = OshunLogger(config)

Adaptive Sampler#

python
from psyche_logging import AdaptiveSampler

sampler = AdaptiveSampler(
    target_logs_per_second=100.0,
    min_rate=0.01,
    max_rate=1.0,
)

Priority Sampler#

python
from psyche_logging import PrioritySampler

# Higher priority logs are sampled more frequently
sampler = PrioritySampler()

# Set priority in log data
logger.info("Important event", data={"priority": 8})

Consistent Hash Sampler#

python
from psyche_logging import ConsistentHashSampler

# Same user always gets same sampling decision
sampler = ConsistentHashSampler(
    rate=0.1,
    hash_fields=["userId", "requestId"],
)

PII Redaction#

Default Redaction#

python
from psyche_logging import create_redactor

redact = create_redactor()
data = {
    "username": "john",
    "password": "secret123",
    "token": "abc123",
}

result = redact(data)
# {"username": "john", "password": "[REDACTED]", "token": "[REDACTED]"}

Custom Redaction#

python
redact = create_redactor(
    paths=["credit_card", "ssn", "*.secret"],
    patterns=[r"[0-9]{4}-[0-9]{4}-[0-9]{4}-[0-9]{4}"],
    censor="***HIDDEN***",
)

Configure Logger Redaction#

python
config = LoggerConfig(
    redact_paths=["password", "api_key", "*.token"],
    redact_patterns=[r"sk-[A-Za-z0-9]+"],  # OpenAI API keys
)

Psyche-Specific Context#

python
from psyche_logging import session_context

# For Psyche session handling
async with session_context(
    session_id="sess-123",
    persona_id="persona-456",
    user_id="user-789",
    avatar_id="avatar-abc",
):
    logger.info("Session started")
    # All logs include session info

OpenTelemetry Integration#

python
config = LoggerConfig(
    enable_otel=True,  # Auto-extract trace context
)
logger = OshunLogger(config)

# Logs will include traceId and spanId when in an active span

Log Entry Format#

json
{
  "level": "info",
  "levelValue": 30,
  "message": "User logged in",
  "timestamp": "2024-01-15T10:30:00.000Z",
  "context": {
    "service": "api",
    "version": "1.0.0",
    "environment": "production",
    "requestId": "req-123",
    "userId": "user-456",
    "traceId": "abc123",
    "spanId": "def456"
  },
  "data": {
    "user_id": "user-456",
    "login_method": "oauth"
  }
}

Integration with @oshun/logging#

This library is designed to be compatible with the TypeScript @oshun/logging:

  • Same log entry structure
  • Same log level values
  • Same context fields (camelCase in JSON)
  • Same redaction patterns
  • Compatible sampling strategies

Logs from Python services can be processed alongside TypeScript service logs.

Dependencies#

  • pydantic >= 2.5.0 - Type validation
  • opentelemetry-api >= 1.20.0 (optional) - OpenTelemetry integration

License#

Proprietary - Oshun Platform