# Psyche Storage

S3-compatible storage client for Psyche AI Virtual Assistant.

## Overview

`psyche-storage` provides a unified storage abstraction aligned with the
`@oshun/storage` TypeScript patterns for cross-language compatibility. It
includes specialized asset managers for avatar models, voice recordings, and
knowledge documents.

## Features

- **S3/MinIO Support**: Compatible with AWS S3 and MinIO backends
- **Async Operations**: All operations are async/await compatible
- **Multipart Uploads**: Automatic chunking for large files with progress
  tracking
- **Signed URLs**: Generate pre-signed URLs for direct upload/download
- **Asset Managers**: Specialized managers for Psyche-specific assets
- **Metadata Tracking**: Automatic metadata for asset categorization and
  tracking

## Installation

```bash
# Basic installation
poetry add psyche-storage

# For development
poetry add psyche-storage[dev]
```

## Quick Start

### Creating a Storage Client

```python
from psyche_storage import S3Config, StorageProvider, create_s3_client, create_minio_client

# AWS S3
config = S3Config(
    provider=StorageProvider.S3,
    region="us-east-1",
    access_key_id="your-access-key",
    secret_access_key="your-secret-key",
    bucket="psyche-assets",
)
client = create_s3_client(config)

# MinIO (local development)
client = create_minio_client(
    endpoint="localhost:9000",
    access_key_id="minioadmin",
    secret_access_key="minioadmin",
    bucket="psyche-assets",
    use_ssl=False,
)
```

### Basic Operations

```python
# Upload a file
result = await client.upload(
    key="path/to/file.txt",
    body=b"file content",
    options=UploadOptions(
        content_type="text/plain",
        metadata={"custom-key": "custom-value"},
    ),
)

# Download a file
result = await client.download("path/to/file.txt")
print(result.body)

# Check if file exists
exists = await client.exists("path/to/file.txt")

# Get file metadata
metadata = await client.get_metadata("path/to/file.txt")

# List files
result = await client.list_objects(
    ListObjectsOptions(prefix="path/to/", max_keys=100)
)
for obj in result.objects:
    print(f"{obj.key}: {obj.size} bytes")

# Delete a file
await client.delete("path/to/file.txt")
```

### Multipart Upload for Large Files

```python
from psyche_storage import MultipartUploadConfig, MultipartUploadProgress

def progress_callback(progress: MultipartUploadProgress):
    print(f"Upload progress: {progress.percentage}%")

result = await client.upload_multipart(
    key="large-file.zip",
    body=large_data,
    options=UploadOptions(content_type="application/zip"),
    multipart_config=MultipartUploadConfig(
        part_size=10 * 1024 * 1024,  # 10 MB parts
        concurrency=4,
        on_progress=progress_callback,
    ),
)
```

### Signed URLs

```python
from psyche_storage import SignedUrlOptions

# Generate download URL (expires in 1 hour)
result = await client.get_signed_download_url(
    "path/to/file.pdf",
    SignedUrlOptions(
        expires_in=3600,
        response_content_disposition='attachment; filename="document.pdf"',
    ),
)
print(f"Download URL: {result.url}")
print(f"Expires at: {result.expires_at}")

# Generate upload URL
result = await client.get_signed_upload_url(
    "uploads/new-file.jpg",
    SignedUrlOptions(
        expires_in=3600,
        content_type="image/jpeg",
    ),
)
```

## Asset Managers

### Avatar Storage

```python
from psyche_storage import AvatarStorageManager

avatar_manager = AvatarStorageManager(client)

# Upload avatar model
result = await avatar_manager.upload_model(
    avatar_id="avatar-123",
    data=model_data,
    filename="character.glb",
    version="v2",
    persona_id="persona-456",
)

# Upload texture
result = await avatar_manager.upload_texture(
    avatar_id="avatar-123",
    data=texture_data,
    filename="diffuse.png",
    texture_type="diffuse",
)

# Upload animation
result = await avatar_manager.upload_animation(
    avatar_id="avatar-123",
    data=animation_data,
    filename="wave.glb",
    animation_name="wave_greeting",
)

# Upload thumbnail
result = await avatar_manager.upload_thumbnail(
    avatar_id="avatar-123",
    data=thumbnail_data,
)

# List avatar assets
assets = await avatar_manager.list_avatar_assets(
    avatar_id="avatar-123",
    category="models",
)

# Delete all avatar assets
await avatar_manager.delete_avatar("avatar-123")
```

### Voice Storage

```python
from psyche_storage import VoiceStorageManager

voice_manager = VoiceStorageManager(client)

# Upload voice recording
result = await voice_manager.upload_recording(
    persona_id="persona-123",
    data=audio_data,
    filename="session_001.mp3",
    session_id="session-456",
    duration_ms=30000,
    transcript="Hello, how can I help you today?",
)

# Upload voice sample for cloning
result = await voice_manager.upload_sample(
    persona_id="persona-123",
    data=sample_data,
    filename="voice_sample.wav",
    sample_type="voice_clone",
)

# Upload voice model
result = await voice_manager.upload_voice_model(
    persona_id="persona-123",
    data=model_data,
    filename="tts_model.pt",
    model_type="tts",
    version="v1",
)

# List recordings
recordings = await voice_manager.list_recordings(
    persona_id="persona-123",
    session_id="session-456",
)
```

### Knowledge Document Storage

```python
from psyche_storage import KnowledgeStorageManager

knowledge_manager = KnowledgeStorageManager(client)

# Upload document
result = await knowledge_manager.upload_document(
    persona_id="persona-123",
    data=document_data,
    filename="company_info.pdf",
    title="Company Information",
    document_type="reference",
    tags=["company", "faq"],
)

# Upload pre-computed embeddings
result = await knowledge_manager.upload_embeddings(
    persona_id="persona-123",
    data=embeddings_data,
    document_key="knowledge/documents/persona-123/2024/01/01/doc.pdf",
    embedding_model="openai/text-embedding-3-small",
)

# Download document
doc = await knowledge_manager.download_document(
    "knowledge/documents/persona-123/2024/01/01/uuid-doc.pdf"
)

# List documents
docs = await knowledge_manager.list_documents(persona_id="persona-123")
```

### Session Storage

```python
from psyche_storage import SessionStorageManager

session_manager = SessionStorageManager(client)

# Upload session recording
result = await session_manager.upload_recording(
    session_id="session-123",
    data=recording_data,
    filename="full_session.webm",
    recording_type="full",
)

# Upload transcript
result = await session_manager.upload_transcript(
    session_id="session-123",
    transcript='{"turns": [...]}',
    format="json",
)
```

### Training Data Storage

```python
from psyche_storage import TrainingStorageManager

training_manager = TrainingStorageManager(client)

# Upload training data
result = await training_manager.upload_training_data(
    model_id="model-123",
    data=dataset_data,
    filename="training_data.jsonl",
    data_type="dataset",
)

# Upload checkpoint
result = await training_manager.upload_checkpoint(
    model_id="model-123",
    data=checkpoint_data,
    filename="checkpoint_epoch_10.pt",
    epoch=10,
    step=50000,
    metrics={"loss": 0.05, "accuracy": 0.95},
)

# List checkpoints
checkpoints = await training_manager.list_checkpoints(model_id="model-123")
```

## Utility Functions

```python
from psyche_storage import (
    format_bytes,
    parse_bytes,
    get_mime_type,
    sanitize_filename,
    compute_sha256,
    generate_file_key,
)

# Format bytes
print(format_bytes(1536))  # "1.50 KB"

# Parse size string
size = parse_bytes("1.5 MB")  # 1572864

# Get MIME type
mime = get_mime_type("image.jpg")  # "image/jpeg"

# Sanitize filename
safe = sanitize_filename("unsafe/name<>.txt")  # "unsafe_name.txt"

# Compute checksum
hash = compute_sha256(data)

# Generate unique key
key = generate_file_key(prefix="uploads", filename="image.jpg")
```

## Asset Categories

The library defines the following asset categories for organized storage:

| Category             | Path Prefix             |
| -------------------- | ----------------------- |
| Avatar Models        | `avatars/models/`       |
| Avatar Textures      | `avatars/textures/`     |
| Avatar Animations    | `avatars/animations/`   |
| Avatar Thumbnails    | `avatars/thumbnails/`   |
| Voice Recordings     | `voice/recordings/`     |
| Voice Samples        | `voice/samples/`        |
| Voice Models         | `voice/models/`         |
| Knowledge Documents  | `knowledge/documents/`  |
| Knowledge Embeddings | `knowledge/embeddings/` |
| Training Data        | `training/data/`        |
| Training Checkpoints | `training/checkpoints/` |
| Session Recordings   | `sessions/recordings/`  |
| Session Transcripts  | `sessions/transcripts/` |

## Environment Variables

```bash
# AWS S3
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1
S3_BUCKET=psyche-assets

# MinIO (local)
MINIO_ENDPOINT=localhost:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BUCKET=psyche-assets
```

## Integration with @oshun/storage

This library is designed to be compatible with the TypeScript `@oshun/storage`:

- Same storage class options
- Same signed URL patterns
- Compatible metadata structure
- Interoperable asset organization

Assets stored from Python can be accessed by TypeScript services and vice versa.

## Dependencies

- `pydantic` >= 2.5.0 - Type validation
- `boto3` >= 1.34.0 - AWS S3 SDK

## License

Proprietary - Oshun Platform