# Psyche Learning System

Learning & Skill Acquisition System with MemGPT/Letta-style persistent memory
architecture.

Part of the Psyche AI Virtual Assistant Platform.

## Overview

This service provides comprehensive learning and memory capabilities:

- **Persistent Memory Architecture**: Hierarchical memory tiers with automatic
  paging (MemGPT/Letta-style)
- **Self-Editing Memory Tools**: LLM-callable tools for memory manipulation
- **Cross-Session Persistence**: Database-backed storage with encryption
- **Memory Decay & Consolidation**: Cognitive-inspired memory dynamics
- **Learning from Demonstration**: Capture and replay user demonstrations
- **Skill Library System**: Voyager-style skill discovery and composition
- **Knowledge Graph Memory**: Temporal entity-relationship storage
- **Experience-Based Learning**: Approval learning and failure analysis

## Architecture

### Memory Tiers

The system implements a four-tier memory hierarchy:

| Tier               | Purpose                                | Capacity    | Speed   |
| ------------------ | -------------------------------------- | ----------- | ------- |
| **IN_CONTEXT**     | Active working memory (in LLM context) | 8K tokens   | Instant |
| **WORKING**        | Recently used, quickly accessible      | 32K tokens  | Fast    |
| **OUT_OF_CONTEXT** | Paged out but readily retrievable      | 128K tokens | Medium  |
| **ARCHIVAL**       | Long-term storage                      | 1M+ tokens  | Slower  |

### Memory Types

- **CORE**: Core identity/personality
- **PERSONA**: User preferences and assistant persona
- **CONVERSATION**: Conversation history
- **FACT**: Factual knowledge
- **SKILL**: Learned skills
- **PREFERENCE**: User preferences
- **CONTEXT**: Contextual information
- **SUMMARY**: Summarized content
- **REFLECTION**: Self-reflections

### Decay Models

The system supports multiple cognitive decay models:

| Model           | Description                                     |
| --------------- | ----------------------------------------------- |
| **Ebbinghaus**  | Classic forgetting curve with spaced repetition |
| **Power Law**   | Power-law decay based on rehearsal count        |
| **Exponential** | Simple exponential decay                        |

## Components

### Persistent Memory (`persistent_memory/`)

- **VirtualContextManager**: Hierarchical tier management with automatic paging
- **MemoryToolRegistry**: Self-editing tools (append, replace, search, insert)
- **MemoryPersistenceManager**: Database storage with optional encryption
- **DecayConsolidationManager**: Memory decay and consolidation cycles

### Learning from Demonstration (`learning_from_demonstration/`)

- **DemonstrationRecorder**: Capture user actions with verbal instructions
- **ActionCapture**: Mouse, keyboard, scroll, application state tracking
- **Recording Management**: Pause, resume, annotation support

### Knowledge Graph (`knowledge_graph/`)

- **TemporalKnowledgeGraphManager**: Entity-relationship graph with temporal
  validity
- **EntityExtractor**: Pattern-based and LLM-based entity extraction
- **RelationshipExtractor**: Relationship detection and classification
- **EntityResolver**: Merge duplicate entities with similarity matching

### Experience-Based Learning (`experience_based_learning/`)

- **ApprovalLearningManager**: Learn from explicit and implicit feedback
- **FailureAnalysisManager**: Root cause analysis and avoidance rules

## API Endpoints

### Health Check

```
GET /health                    # Service health
GET /ready                     # Readiness check
GET /metrics                   # Prometheus metrics
```

### Memory Management

```
POST /memory                   # Add a memory
GET /memory/{block_id}         # Get a memory
PUT /memory/{block_id}         # Update a memory
DELETE /memory/{block_id}      # Delete a memory
POST /memory/search            # Search memories
GET /memory/context            # Get context window content
```

### Memory Tools

```
GET /tools                     # List available tools
POST /tools/execute            # Execute a memory tool
```

Available tools:

- `core_memory_append`: Add content to active memory
- `core_memory_replace`: Update existing memory
- `archival_memory_insert`: Store in long-term memory
- `archival_memory_search`: Search long-term memory
- `conversation_search`: Search conversation history

### Session Management

```
POST /session/start            # Start a user session
POST /session/{id}/end         # End a session
GET /session/{user_id}/restore # Restore context from previous sessions
```

### Decay & Consolidation

```
GET /decay/status              # Get decay system status
GET /decay/memory/{block_id}   # Get memory strength/projection
POST /consolidate              # Trigger memory consolidation
POST /decay/run                # Run decay cycle manually
```

### Skills

```
GET /skills                    # List learned skills
GET /skills/{skill_id}         # Get skill details
POST /skills/search            # Semantic skill search
POST /skills/compose           # Compose skills for complex tasks
```

### Knowledge Graph

```
POST /graph/episode            # Add episode to graph
GET /graph/entity/{id}         # Get entity
POST /graph/query              # Query relationships
GET /graph/communities         # Get entity communities
```

## Configuration

### Environment Variables

| Variable                 | Description                    | Default                |
| ------------------------ | ------------------------------ | ---------------------- |
| `SERVICE_PORT`           | API port                       | 8010                   |
| `DATABASE_URL`           | PostgreSQL connection          | postgresql://...       |
| `REDIS_URL`              | Redis connection               | redis://localhost:6379 |
| `QDRANT_URL`             | Vector DB connection           | http://localhost:6333  |
| `ENABLE_ENCRYPTION`      | Encrypt memories at rest       | true                   |
| `ENCRYPTION_KEY`         | Fernet encryption key          | (generated)            |
| `IN_CONTEXT_MAX_TOKENS`  | Max in-context tokens          | 8000                   |
| `WORKING_MAX_TOKENS`     | Max working memory tokens      | 32000                  |
| `DECAY_MODEL`            | Decay model (ebbinghaus/power) | ebbinghaus             |
| `CONSOLIDATION_INTERVAL` | Consolidation cycle (seconds)  | 3600                   |

## Development

### Using Nx

```bash
# Install dependencies
nx install psyche-learning-system

# Run development server
nx serve psyche-learning-system

# Run production server
nx serve-prod psyche-learning-system

# Run tests
nx test psyche-learning-system
nx test-unit psyche-learning-system
nx test-integration psyche-learning-system

# Run tests with coverage
nx test-cov psyche-learning-system

# Lint and format
nx lint psyche-learning-system
nx format psyche-learning-system

# Database migrations
nx migrate psyche-learning-system
nx migrate-create psyche-learning-system -- "add_skills_table"
nx migrate-rollback psyche-learning-system

# Memory management tasks
nx decay-consolidation psyche-learning-system
nx skill-sync psyche-learning-system

# Docker
nx docker-build psyche-learning-system
nx docker-run psyche-learning-system
```

### Direct Poetry Commands

```bash
cd apps/psyche/learning-system
poetry install
poetry run pytest
poetry run uvicorn learning_system.main:app --reload --host 0.0.0.0 --port 8010
```

## Module Structure

```
src/learning_system/
├── __init__.py
├── main.py                        # FastAPI application
├── persistent_memory/             # MemGPT-style memory
│   ├── __init__.py
│   ├── context_management.py      # Hierarchical tiers, paging
│   ├── memory_tools.py            # Self-editing tools
│   ├── persistence.py             # Database storage, encryption
│   └── decay_consolidation.py     # Decay and consolidation
├── learning_from_demonstration/   # Demonstration capture
│   ├── __init__.py
│   ├── types.py                   # Action types, states
│   └── recorder.py                # Demonstration recorder
├── knowledge_graph/               # Temporal knowledge graph
│   ├── __init__.py
│   ├── types.py                   # Entity, relationship types
│   ├── extractors.py              # Entity/relationship extraction
│   ├── resolver.py                # Entity resolution
│   ├── storage.py                 # Graph storage
│   └── manager.py                 # Graph manager
├── experience_based_learning/     # Experience learning
│   ├── __init__.py
│   ├── approval_learning.py       # Feedback learning
│   └── failure_analysis.py        # Failure patterns
└── skill_library/                 # Voyager-style skills
    ├── __init__.py
    ├── discovery.py               # Skill discovery
    ├── composition.py             # Skill composition
    └── repository.py              # Skill storage
```

## Performance Targets

| Operation           | Target Latency |
| ------------------- | -------------- |
| Add memory          | < 10ms         |
| Get memory          | < 5ms          |
| Search (in-context) | < 20ms         |
| Search (archival)   | < 100ms        |
| Page in/out         | < 50ms         |
| Consolidation       | < 5s           |
| Skill search        | < 100ms        |
| Graph query         | < 200ms        |

## Security

- Memory encryption at rest using Fernet (AES-128-CBC)
- User-specific encryption keys derived from master key
- Memory isolation per user
- No PII in logs
- Secure key rotation support

## Research Background

This implementation is based on research from:

- **MemGPT**: Towards LLMs as Operating Systems (2023)
- **Letta**: Open-source MemGPT implementation
- **REMEMBRANCER**: A Continually Improving Chatbot
- **Voyager**: An Open-Ended Embodied Agent (skill library concepts)
- **Zep/Graphiti**: Temporal knowledge graph approaches
- **PMSA**: Procedural Memory for Self-Improving Agents

## License

Proprietary - Oshun Platform