# Troubleshooting Guide

Common issues and solutions for the Yemaya platform.

## Table of Contents

- [Installation Issues](#installation-issues)
- [Database Issues](#database-issues)
- [Authentication Issues](#authentication-issues)
- [API Issues](#api-issues)
- [AI Generation Issues](#ai-generation-issues)
- [3D Generation Issues](#3d-generation-issues)
- [GPU Jobs Issues](#gpu-jobs-issues)
- [Engine Integration Issues](#engine-integration-issues)
- [Performance Issues](#performance-issues)
- [Debugging Tools](#debugging-tools)

---

## Installation Issues

### `pnpm install` fails

**Symptom**: Installation fails with dependency errors.

**Solutions**:

1. Clear cache and reinstall:

   ```bash
   rm -rf node_modules pnpm-lock.yaml
   pnpm store prune
   pnpm install
   ```

2. Check Node.js version:

   ```bash
   node --version  # Should be 20+
   pnpm --version  # Should be 9+
   ```

3. If using Windows, enable long paths:
   ```powershell
   git config --system core.longpaths true
   ```

### Docker services won't start

**Symptom**: `docker compose up -d` fails.

**Solutions**:

1. Check Docker is running:

   ```bash
   docker info
   ```

2. Check port conflicts:

   ```bash
   # Check if ports are in use
   lsof -i :5432  # PostgreSQL
   lsof -i :6379  # Redis
   lsof -i :9000  # MinIO
   ```

3. Reset Docker volumes:
   ```bash
   docker compose down -v
   docker compose up -d
   ```

### Build fails with TypeScript errors

**Symptom**: `pnpm build` shows type errors.

**Solutions**:

1. Rebuild all packages:

   ```bash
   pnpm clean
   pnpm build
   ```

2. Generate Prisma client:

   ```bash
   pnpm --filter @yemaya/database db:generate
   ```

3. Clear TypeScript cache:
   ```bash
   rm -rf packages/*/dist
   rm -rf packages/*/.turbo
   pnpm build
   ```

---

## Database Issues

### "Connection refused" to PostgreSQL

**Symptom**: `Error: connect ECONNREFUSED 127.0.0.1:5432`

**Solutions**:

1. Check PostgreSQL is running:

   ```bash
   docker compose ps postgres
   # Should show "Up" status
   ```

2. Verify connection string:

   ```bash
   echo $DATABASE_URL
   # Should match docker-compose.yml settings
   ```

3. Test connection manually:

   ```bash
   docker compose exec postgres psql -U yemaya -d yemaya -c "SELECT 1"
   ```

4. If using custom PostgreSQL, ensure:
   - `pg_hba.conf` allows connections
   - User has correct permissions
   - Database exists

### Migration fails

**Symptom**: `pnpm db:migrate` shows errors.

**Solutions**:

1. Reset database (development only):

   ```bash
   pnpm --filter @yemaya/database db:reset
   ```

2. Check migration status:

   ```bash
   pnpm --filter @yemaya/database prisma migrate status
   ```

3. For specific migration errors, check:
   ```bash
   # View migration SQL
   pnpm --filter @yemaya/database prisma migrate diff
   ```

### Redis connection issues

**Symptom**: `Error: Redis connection to localhost:6379 failed`

**Solutions**:

1. Check Redis is running:

   ```bash
   docker compose ps redis
   redis-cli ping  # Should return "PONG"
   ```

2. Check Redis URL:

   ```bash
   echo $REDIS_URL
   # Should be redis://localhost:6379
   ```

3. Test Redis connection:
   ```bash
   redis-cli
   > SET test "hello"
   > GET test
   ```

---

## Authentication Issues

### JWT token expired

**Symptom**: `401 Unauthorized` with "Token expired" message.

**Solutions**:

1. Use refresh token endpoint:

   ```bash
   curl -X POST http://localhost:3000/api/v1/auth/refresh \
     -H "Content-Type: application/json" \
     -d '{"refreshToken": "your_refresh_token"}'
   ```

2. If refresh token also expired, re-authenticate:
   ```bash
   curl -X POST http://localhost:3000/api/v1/auth/login \
     -H "Content-Type: application/json" \
     -d '{"email": "...", "password": "..."}'
   ```

### OAuth callback fails

**Symptom**: OAuth redirects to error page.

**Solutions**:

1. Check OAuth credentials in `.env`:

   ```bash
   echo $GOOGLE_CLIENT_ID
   echo $GOOGLE_CLIENT_SECRET
   ```

2. Verify callback URL matches provider settings:
   - Google: `http://localhost:3000/api/v1/auth/oauth/google/callback`
   - GitHub: `http://localhost:3000/api/v1/auth/oauth/github/callback`

3. Check browser console for specific errors.

### "Invalid credentials"

**Symptom**: Login fails with valid-looking credentials.

**Solutions**:

1. Check account exists and is verified:

   ```bash
   # Using Prisma Studio
   pnpm --filter @yemaya/database db:studio
   # Find user in User table
   ```

2. Check for account lockout (5 failed attempts):
   - Wait 15 minutes, or
   - Reset via admin API/database

3. Reset password:
   ```bash
   curl -X POST http://localhost:3000/api/v1/auth/password/reset \
     -H "Content-Type: application/json" \
     -d '{"email": "your@email.com"}'
   ```

---

## API Issues

### 429 Too Many Requests

**Symptom**: Rate limit exceeded.

**Solutions**:

1. Wait for rate limit window to reset (check `X-RateLimit-Reset` header)

2. Implement request throttling:

   ```typescript
   // Simple throttle
   const delay = (ms) => new Promise((r) => setTimeout(r, ms));
   await delay(1000); // Wait 1 second between requests
   ```

3. For production, upgrade tier or request limit increase.

### CORS errors

**Symptom**: "Access-Control-Allow-Origin" errors in browser.

**Solutions**:

1. Check CORS configuration in `.env`:

   ```bash
   CORS_ORIGINS=http://localhost:5173,http://localhost:3000
   ```

2. For development, ensure frontend URL is in allowed origins.

3. Check request is using correct protocol (http vs https).

### Request body too large

**Symptom**: `413 Payload Too Large`

**Solutions**:

1. Check file size limits in config.
2. Use multipart upload for large files:

   ```bash
   # Get upload URL
   curl -X POST http://localhost:3000/api/v1/assets \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"filename": "large-file.zip", "size": 500000000}'

   # Upload to presigned URL
   curl -X PUT "$PRESIGNED_URL" --data-binary @large-file.zip
   ```

### Validation errors

**Symptom**: `400 Bad Request` with validation details.

**Solutions**:

1. Check the error response for specific field errors:

   ```json
   {
     "error": {
       "code": "VALIDATION_ERROR",
       "details": [{ "field": "email", "message": "Invalid email format" }]
     }
   }
   ```

2. Refer to API documentation for correct schema.

3. Use SDK types for type-safe requests:
   ```typescript
   import { YemayaClient } from '@yemaya/sdk';
   const client = new YemayaClient({ apiKey: '...' });
   // TypeScript will catch schema errors
   ```

---

## AI Generation Issues

### API key errors

**Symptom**: "Invalid API key" or "Unauthorized" from AI providers.

**Solutions**:

1. Verify API key is set:

   ```bash
   echo $ANTHROPIC_API_KEY
   echo $OPENAI_API_KEY
   ```

2. Check key has correct permissions/quota.

3. Test key directly:
   ```bash
   # Test Anthropic
   curl https://api.anthropic.com/v1/messages \
     -H "x-api-key: $ANTHROPIC_API_KEY" \
     -H "anthropic-version: 2023-06-01" \
     -H "content-type: application/json" \
     -d '{"model": "claude-3-haiku-20240307", "max_tokens": 10, "messages": [{"role": "user", "content": "Hi"}]}'
   ```

### Generation timeout

**Symptom**: Request times out waiting for AI response.

**Solutions**:

1. Use async endpoint for long generations:

   ```bash
   # Async (returns job ID immediately)
   POST /ai/text

   # Then poll for result
   GET /ai/jobs/{jobId}
   ```

2. Reduce `maxTokens` for faster responses.

3. Check provider status pages for outages.

### Cost exceeded

**Symptom**: "Budget exceeded" error.

**Solutions**:

1. Check current usage:

   ```bash
   curl http://localhost:3000/api/v1/ai/usage \
     -H "Authorization: Bearer $TOKEN"
   ```

2. Increase budget limits in config or admin panel.

3. Use cheaper models for development:
   ```json
   {"model": "claude-3-haiku-20240307"}  // Cheapest Claude
   {"model": "gpt-3.5-turbo"}            // Cheapest OpenAI
   ```

---

## 3D Generation Issues

### Generation failed

**Symptom**: 3D generation job fails.

**Solutions**:

1. Check job status and error:

   ```bash
   curl http://localhost:3000/api/v1/3d/jobs/{jobId} \
     -H "Authorization: Bearer $TOKEN"
   ```

2. Common causes:
   - Invalid prompt (too vague or complex)
   - Provider rate limit
   - Provider temporary outage

3. Retry with modified parameters:
   ```json
   {
     "prompt": "simple wooden chair", // More specific
     "quality": "standard", // Lower quality for testing
     "provider": "meshy" // Try different provider
   }
   ```

### Poor model quality

**Symptom**: Generated 3D model has artifacts or poor topology.

**Solutions**:

1. Use post-processing:

   ```bash
   curl -X POST http://localhost:3000/api/v1/3d/process/optimize \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"modelUrl": "...", "targetPolyCount": 10000}'
   ```

2. Try higher quality setting (slower, more expensive):

   ```json
   { "quality": "high" }
   ```

3. Use image-to-3D with reference images for better results.

### Texture issues

**Symptom**: Missing or low-quality textures.

**Solutions**:

1. Enable texture upscaling:

   ```json
   {
     "textureResolution": 2048,
     "upscaleTextures": true
   }
   ```

2. Re-bake textures:
   ```bash
   curl -X POST http://localhost:3000/api/v1/3d/process/bake-textures \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"modelUrl": "...", "resolution": 4096}'
   ```

---

## GPU Jobs Issues

### Job stuck in queue

**Symptom**: Job remains in "queued" status.

**Solutions**:

1. Check provider health:

   ```bash
   curl http://localhost:3000/api/v1/gpu-jobs/health \
     -H "Authorization: Bearer $TOKEN"
   ```

2. Check circuit breaker status:

   ```bash
   curl http://localhost:3000/api/v1/gpu-jobs/circuit-breakers \
     -H "Authorization: Bearer $TOKEN"
   ```

3. Reset circuit breaker if open:
   ```bash
   curl -X POST http://localhost:3000/api/v1/gpu-jobs/circuit-breakers/runpod-serverless/reset \
     -H "Authorization: Bearer $TOKEN"
   ```

### Job failed repeatedly

**Symptom**: Job in DLQ after multiple failures.

**Solutions**:

1. Check DLQ entry for error details:

   ```bash
   curl http://localhost:3000/api/v1/gpu-jobs/dlq/{entryId} \
     -H "Authorization: Bearer $TOKEN"
   ```

2. Common failure reasons:
   - `max_retries_exceeded`: Underlying issue not resolved
   - `budget_exceeded`: Cost limit hit
   - `timeout_exceeded`: Job taking too long

3. Replay with modified parameters:
   ```bash
   curl -X POST http://localhost:3000/api/v1/gpu-jobs/dlq/{entryId}/replay \
     -H "Authorization: Bearer $TOKEN" \
     -d '{"preferredProvider": "modal", "modifyParameters": {"quality": "standard"}}'
   ```

### Provider unavailable

**Symptom**: "No providers available" error.

**Solutions**:

1. Check provider API keys are configured.

2. Verify provider status:
   - RunPod: https://status.runpod.io
   - Modal: https://status.modal.com
   - AWS: https://health.aws.amazon.com

3. Check routing rules aren't excluding all providers.

---

## Engine Integration Issues

### Blender won't connect

**Symptom**: "Connection refused" to Blender.

**Solutions**:

1. Ensure Blender addon is installed and enabled:
   - Blender > Edit > Preferences > Add-ons
   - Search "Yemaya" and enable

2. Start the WebSocket server in Blender:
   - Blender > Yemaya > Start Server

3. Check port (default 9999):
   ```bash
   curl http://localhost:9999/health
   ```

### Godot plugin errors

**Symptom**: GDExtension not loading.

**Solutions**:

1. Rebuild the extension:

   ```bash
   cd packages/godot
   pnpm build:gdextension
   ```

2. Check Godot version compatibility (4.2+ required).

3. Copy extension files to Godot project's `addons/` folder.

### Unreal plugin crash

**Symptom**: Unreal crashes when loading plugin.

**Solutions**:

1. Check UE version compatibility (5.3+ required).

2. Rebuild plugin for correct engine version.

3. Check logs:
   - Windows: `%LOCALAPPDATA%\UnrealEngine\<version>\Saved\Logs`
   - Mac: `~/Library/Logs/Unreal Engine/`

---

## Performance Issues

### Slow API responses

**Symptom**: API calls taking >1 second.

**Solutions**:

1. Enable response caching:

   ```bash
   # Check Redis is running
   redis-cli ping
   ```

2. Check database query performance:

   ```bash
   # Enable query logging
   DEBUG=prisma:query pnpm dev
   ```

3. Profile slow endpoints:
   ```bash
   curl -w "@curl-format.txt" -o /dev/null -s http://localhost:3000/api/v1/...
   ```

### High memory usage

**Symptom**: Node.js process using excessive memory.

**Solutions**:

1. Check for memory leaks:

   ```bash
   node --inspect apps/api/dist/index.js
   # Use Chrome DevTools Memory profiler
   ```

2. Increase Node.js memory limit:

   ```bash
   NODE_OPTIONS="--max-old-space-size=4096" pnpm start
   ```

3. Check for large file operations holding memory.

### Database connection exhaustion

**Symptom**: "Too many connections" error.

**Solutions**:

1. Check connection pool settings:

   ```typescript
   // In DATABASE_URL
   postgresql://user:pass@host/db?connection_limit=20
   ```

2. Use PgBouncer for connection pooling in production.

3. Check for connection leaks (unclosed transactions).

---

## Debugging Tools

### Enable debug logging

```bash
# All debug logs
DEBUG=* pnpm dev

# Specific modules
DEBUG=yemaya:* pnpm dev
DEBUG=prisma:query pnpm dev
DEBUG=gpu-jobs:* pnpm dev
```

### API request tracing

```bash
# Enable OpenTelemetry tracing
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 pnpm dev

# View traces in Jaeger/Zipkin
open http://localhost:16686
```

### Database query analysis

```bash
# Prisma Studio for data inspection
pnpm --filter @yemaya/database db:studio

# PostgreSQL EXPLAIN ANALYZE
docker compose exec postgres psql -U yemaya -c "EXPLAIN ANALYZE SELECT ..."
```

### Redis inspection

```bash
# Monitor all commands
redis-cli MONITOR

# Check memory usage
redis-cli INFO memory

# List all keys (careful in production!)
redis-cli KEYS "*"
```

### Health check script

```bash
#!/bin/bash
# health-check.sh

echo "Checking services..."

# API
curl -s http://localhost:3000/health && echo "API: OK" || echo "API: FAILED"

# PostgreSQL
docker compose exec -T postgres pg_isready && echo "PostgreSQL: OK" || echo "PostgreSQL: FAILED"

# Redis
redis-cli ping > /dev/null && echo "Redis: OK" || echo "Redis: FAILED"

# MinIO
curl -s http://localhost:9000/minio/health/live && echo "MinIO: OK" || echo "MinIO: FAILED"
```

---

## Getting More Help

If you can't resolve an issue:

1. **Search existing issues**: https://github.com/yemaya/yemaya/issues
2. **Ask on Discord**: https://discord.gg/yemaya
3. **Create detailed bug report** with:
   - Steps to reproduce
   - Environment details
   - Error messages/logs
   - What you've tried

---

## Related Documentation

- [Known Limitations](./KNOWN_LIMITATIONS.md)
- [Configuration Guide](./CONFIGURATION.md)
- [API Reference](./api/api-reference.md)
- [GPU Jobs Troubleshooting](./gpu-jobs/troubleshooting.md)
