# Capacity Planning Guide

**Purpose**: Document expected load profiles, resource requirements, and scaling
strategies for Lilith Platform services to ensure adequate capacity during
launch and growth phases.

## Table of Contents

1. [Load Profiles](#load-profiles)
2. [Service Resource Requirements](#service-resource-requirements)
3. [Scaling Thresholds](#scaling-thresholds)
4. [Database Capacity](#database-capacity)
5. [AI Service Capacity](#ai-service-capacity)
6. [Cost Projections](#cost-projections)
7. [Capacity Planning Process](#capacity-planning-process)

---

## Load Profiles

### User Growth Projections

| Phase           | Timeline   | Daily Active Users (DAU) | Peak Concurrent | Requests/Second |
| --------------- | ---------- | ------------------------ | --------------- | --------------- |
| Soft Launch     | Week 1-2   | 1,000 - 5,000            | 500             | 50              |
| Limited Release | Week 3-4   | 5,000 - 25,000           | 2,500           | 250             |
| Public Launch   | Week 5-8   | 25,000 - 100,000         | 10,000          | 1,000           |
| Growth Phase    | Month 3-6  | 100,000 - 500,000        | 50,000          | 5,000           |
| Scale Phase     | Month 6-12 | 500,000 - 2,000,000      | 200,000         | 20,000          |

### Traffic Patterns

#### Daily Pattern (UTC)

```
Hour    | Load Factor | Description
--------|-------------|------------------------------------------
00-06   | 0.3x       | Off-peak (Americas night)
06-09   | 0.7x       | Morning meditation peak (EU)
09-12   | 0.8x       | Work break sessions
12-15   | 0.6x       | Midday lull
15-18   | 1.0x       | Peak (EU evening + Americas afternoon)
18-22   | 1.2x       | Maximum peak (Americas evening)
22-00   | 0.8x       | Evening wind-down
```

#### Weekly Pattern

- **Monday**: 110% baseline (New week motivation)
- **Tuesday-Thursday**: 100% baseline
- **Friday**: 90% baseline
- **Saturday**: 85% baseline
- **Sunday**: 120% baseline (Self-care day)

### Event-Driven Spikes

| Event Type         | Expected Multiplier | Duration    | Planning Buffer       |
| ------------------ | ------------------- | ----------- | --------------------- |
| Marketing Campaign | 3-5x                | 24-48 hours | Pre-scale 2x          |
| PR/Media Feature   | 5-10x               | 12-24 hours | Auto-scale aggressive |
| App Store Feature  | 10-20x              | 48-72 hours | Pre-scale 5x          |
| Viral Moment       | 10-50x              | Variable    | Emergency scaling     |

---

## Service Resource Requirements

### Core Services

#### BFF (Backend for Frontend)

```yaml
Base Resources:
  cpu: 500m
  memory: 512Mi
  replicas: 3

Per 1000 RPS:
  cpu: 1000m
  memory: 768Mi
  replicas: +2

Scaling Limits:
  min_replicas: 3
  max_replicas: 50
  target_cpu: 70%
  target_memory: 80%
```

#### Auth Service

```yaml
Base Resources:
  cpu: 250m
  memory: 256Mi
  replicas: 3

Per 1000 RPS:
  cpu: 500m
  memory: 384Mi
  replicas: +1

Scaling Limits:
  min_replicas: 3
  max_replicas: 20
  target_cpu: 60%
  target_memory: 70%
```

#### Content Service

```yaml
Base Resources:
  cpu: 500m
  memory: 512Mi
  replicas: 3

Per 1000 RPS:
  cpu: 750m
  memory: 640Mi
  replicas: +2

Scaling Limits:
  min_replicas: 3
  max_replicas: 30
  target_cpu: 70%
  target_memory: 75%
```

#### Conversation Service

```yaml
Base Resources:
  cpu: 500m
  memory: 512Mi
  replicas: 3

Per 1000 Active Conversations:
  cpu: 250m
  memory: 128Mi
  replicas: +1

Scaling Limits:
  min_replicas: 3
  max_replicas: 25
  target_cpu: 65%
  target_memory: 75%
```

#### Media Service

```yaml
Base Resources:
  cpu: 1000m
  memory: 1Gi
  replicas: 3

Per 1000 Media Requests:
  cpu: 500m
  memory: 512Mi
  replicas: +1

Scaling Limits:
  min_replicas: 3
  max_replicas: 40
  target_cpu: 75%
  target_memory: 80%
```

#### AI Service

```yaml
Base Resources:
  cpu: 2000m
  memory: 4Gi
  replicas: 5

Per 100 AI Requests/min:
  cpu: 1000m
  memory: 2Gi
  replicas: +2

Scaling Limits:
  min_replicas: 5
  max_replicas: 50
  target_cpu: 60%
  target_memory: 70%

GPU Resources (if applicable):
  gpu: 1x NVIDIA T4
  gpu_memory: 16Gi
```

#### Notification Service

```yaml
Base Resources:
  cpu: 250m
  memory: 256Mi
  replicas: 2

Per 10000 Notifications/hour:
  cpu: 250m
  memory: 128Mi
  replicas: +1

Scaling Limits:
  min_replicas: 2
  max_replicas: 15
  target_cpu: 70%
  target_memory: 80%
```

#### Moderation Service

```yaml
Base Resources:
  cpu: 500m
  memory: 512Mi
  replicas: 2

Per 500 Moderation Requests/min:
  cpu: 250m
  memory: 256Mi
  replicas: +1

Scaling Limits:
  min_replicas: 2
  max_replicas: 20
  target_cpu: 65%
  target_memory: 75%
```

---

## Scaling Thresholds

### Horizontal Pod Autoscaler (HPA) Configuration

```yaml
# Standard service HPA template
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: ${SERVICE_NAME}-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: ${SERVICE_NAME}
  minReplicas: ${MIN_REPLICAS}
  maxReplicas: ${MAX_REPLICAS}
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: ${CPU_TARGET}
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: ${MEMORY_TARGET}
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: ${RPS_TARGET}
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 15
        - type: Pods
          value: 4
          periodSeconds: 15
      selectPolicy: Max
```

### Scaling Trigger Matrix

| Metric              | Warning | Scale Up | Scale Down  | Critical |
| ------------------- | ------- | -------- | ----------- | -------- |
| CPU Utilization     | 60%     | 70%      | <40% for 5m | 90%      |
| Memory Utilization  | 70%     | 80%      | <50% for 5m | 90%      |
| Request Latency P95 | 200ms   | 300ms    | <100ms      | 500ms    |
| Request Queue Depth | 50      | 100      | <20         | 500      |
| Error Rate          | 0.1%    | 0.5%     | <0.01%      | 1%       |
| Active Connections  | 80% max | 90% max  | <50% max    | 95% max  |

### Pre-scaling Commands

```bash
# Pre-scale for anticipated traffic surge
kubectl scale deployment bff --replicas=10 -n production
kubectl scale deployment content --replicas=8 -n production
kubectl scale deployment ai --replicas=15 -n production

# Verify scaling
kubectl get hpa -n production

# Scale node pool (GKE example)
gcloud container clusters resize lilith-production \
  --node-pool=general-pool \
  --num-nodes=20 \
  --zone=us-central1-a
```

---

## Database Capacity

### PostgreSQL

#### Connection Pool Sizing

```
Connections per service replica: 10
Service replicas (max): 200
Safety buffer: 20%

Recommended max_connections: 200 * 10 * 1.2 = 2,400

PgBouncer pool sizing:
- default_pool_size: 20
- max_client_conn: 5000
- reserve_pool_size: 5
```

#### Storage Projections

| Data Type        | Per User  | 100K Users | 1M Users  | Growth/Month |
| ---------------- | --------- | ---------- | --------- | ------------ |
| User profiles    | 50KB      | 5GB        | 50GB      | 5%           |
| Sessions         | 10KB      | 1GB        | 10GB      | 10%          |
| Content metadata | 100KB     | 10GB       | 100GB     | 3%           |
| Analytics        | 500KB     | 50GB       | 500GB     | 15%          |
| Audit logs       | 200KB     | 20GB       | 200GB     | 8%           |
| **Total**        | **860KB** | **86GB**   | **860GB** | **~10%**     |

#### Instance Sizing

| Phase         | Instance Type  | vCPU | RAM   | Storage | IOPS  |
| ------------- | -------------- | ---- | ----- | ------- | ----- |
| Soft Launch   | db.r6g.large   | 2    | 16GB  | 100GB   | 3000  |
| Public Launch | db.r6g.xlarge  | 4    | 32GB  | 500GB   | 6000  |
| Growth        | db.r6g.2xlarge | 8    | 64GB  | 1TB     | 12000 |
| Scale         | db.r6g.4xlarge | 16   | 128GB | 2TB     | 24000 |

### Redis

#### Memory Sizing

```
Session data: 5KB/session * 200,000 concurrent = 1GB
Cache data: 50KB/user * 100,000 DAU = 5GB
Rate limiting: 1KB/IP * 500,000 IPs = 500MB
Pub/Sub channels: 100MB

Total: ~7GB (allocate 16GB for buffer)
```

#### Cluster Configuration

| Phase  | Nodes | Node Type         | Total Memory | Shards |
| ------ | ----- | ----------------- | ------------ | ------ |
| Launch | 3     | cache.r6g.large   | 39GB         | 1      |
| Growth | 6     | cache.r6g.xlarge  | 156GB        | 2      |
| Scale  | 9     | cache.r6g.2xlarge | 468GB        | 3      |

### Vector Database (Milvus)

#### Collection Sizing

```
Embedding dimensions: 1536 (OpenAI)
Vectors per user: ~100
Index type: IVF_FLAT

Per 100K users:
- Raw vectors: 100K * 100 * 1536 * 4 bytes = 58GB
- Index overhead: 20%
- Total: ~70GB
```

---

## AI Service Capacity

### LLM Token Budget

#### Daily Token Projections

| User Tier | Tokens/Session | Sessions/Day | Daily Tokens |
| --------- | -------------- | ------------ | ------------ |
| Free      | 2,000          | 1            | 2,000        |
| Premium   | 5,000          | 3            | 15,000       |
| Pro       | 10,000         | 5            | 50,000       |

#### Monthly Token Budget (100K DAU)

```
Free users (70%): 70,000 * 2,000 * 30 = 4.2B tokens
Premium (25%): 25,000 * 15,000 * 30 = 11.25B tokens
Pro (5%): 5,000 * 50,000 * 30 = 7.5B tokens

Total: ~23B tokens/month
```

### Rate Limiting Configuration

```yaml
ai_rate_limits:
  free_tier:
    requests_per_minute: 5
    tokens_per_day: 5000
    concurrent_requests: 1
  premium_tier:
    requests_per_minute: 20
    tokens_per_day: 50000
    concurrent_requests: 3
  pro_tier:
    requests_per_minute: 60
    tokens_per_day: 200000
    concurrent_requests: 5
```

### AI Queue Management

```yaml
queue_config:
  high_priority: # Premium/Pro users
    max_wait_time: 5s
    workers: 30
  normal_priority: # Free users
    max_wait_time: 30s
    workers: 20
  batch_priority: # Background processing
    max_wait_time: 300s
    workers: 10
```

---

## Cost Projections

### Infrastructure Costs (AWS)

| Phase         | Compute  | Database | Cache   | Storage | Network | Total/Month |
| ------------- | -------- | -------- | ------- | ------- | ------- | ----------- |
| Soft Launch   | $2,000   | $500     | $200    | $100    | $200    | $3,000      |
| Public Launch | $8,000   | $2,000   | $800    | $500    | $1,000  | $12,300     |
| Growth (100K) | $20,000  | $5,000   | $2,000  | $2,000  | $3,000  | $32,000     |
| Scale (500K)  | $60,000  | $15,000  | $6,000  | $8,000  | $10,000 | $99,000     |
| Scale (1M)    | $100,000 | $25,000  | $10,000 | $15,000 | $20,000 | $170,000    |

### AI API Costs

| Provider  | Model          | Price/1M Tokens          | 100K DAU Monthly | 1M DAU Monthly |
| --------- | -------------- | ------------------------ | ---------------- | -------------- |
| OpenAI    | GPT-4          | $30 input/$60 output     | $690,000         | $6,900,000     |
| OpenAI    | GPT-4o-mini    | $0.15 input/$0.60 output | $8,625           | $86,250        |
| Anthropic | Claude 3.5     | $3 input/$15 output      | $138,000         | $1,380,000     |
| Anthropic | Claude 3 Haiku | $0.25 input/$1.25 output | $11,500          | $115,000       |

**Recommendation**: Use tiered model selection:

- Free tier: Claude 3 Haiku / GPT-4o-mini
- Premium: Claude 3.5 Sonnet / GPT-4o
- Pro: Claude 3 Opus / GPT-4

### Cost Per User Metrics

| Phase         | DAU  | Total Monthly Cost | Cost/DAU | Cost/MAU |
| ------------- | ---- | ------------------ | -------- | -------- |
| Soft Launch   | 5K   | $15,000            | $3.00    | $0.50    |
| Public Launch | 50K  | $50,000            | $1.00    | $0.17    |
| Growth        | 100K | $100,000           | $1.00    | $0.17    |
| Scale         | 500K | $400,000           | $0.80    | $0.13    |
| Scale         | 1M   | $700,000           | $0.70    | $0.12    |

---

## Capacity Planning Process

### Weekly Capacity Review

1. **Review Metrics**

   ```bash
   # Generate capacity report
   ./scripts/generate-capacity-report.sh --period=7d
   ```

2. **Check Growth Trends**
   - DAU growth rate
   - Request rate trends
   - Resource utilization trends

3. **Update Projections**
   - Adjust forecasts based on actual data
   - Identify potential bottlenecks

4. **Plan Adjustments**
   - Schedule scaling changes
   - Budget adjustments if needed

### Pre-Launch Capacity Checklist

- [ ] Load test at 2x expected launch traffic
- [ ] Verify auto-scaling triggers work correctly
- [ ] Pre-scale services to handle initial surge
- [ ] Confirm database connections sufficient
- [ ] Validate CDN capacity for media
- [ ] Test failover procedures under load
- [ ] Verify AI rate limits prevent budget overrun
- [ ] Confirm monitoring alerts configured
- [ ] Document emergency scaling procedures
- [ ] Brief on-call team on capacity thresholds

### Emergency Scaling Procedures

```bash
# Emergency scale-up script
#!/bin/bash
NAMESPACE=${1:-production}
MULTIPLIER=${2:-2}

# Scale all deployments
for deploy in $(kubectl get deploy -n $NAMESPACE -o name); do
  current=$(kubectl get $deploy -n $NAMESPACE -o jsonpath='{.spec.replicas}')
  new=$((current * MULTIPLIER))
  kubectl scale $deploy --replicas=$new -n $NAMESPACE
done

# Scale node pool
gcloud container clusters resize lilith-production \
  --node-pool=general-pool \
  --num-nodes=$(($(gcloud container clusters describe lilith-production --format='value(nodePools[0].initialNodeCount)') * MULTIPLIER)) \
  --zone=us-central1-a \
  --quiet

echo "Emergency scale completed. Verify with: kubectl get pods -n $NAMESPACE"
```

### Capacity Monitoring Queries

```promql
# Service capacity utilization
sum(rate(container_cpu_usage_seconds_total{namespace="production"}[5m])) by (pod)
  / sum(kube_pod_container_resource_limits{resource="cpu",namespace="production"}) by (pod) * 100

# Database connection utilization
pg_stat_activity_count / pg_settings_max_connections * 100

# Redis memory utilization
redis_memory_used_bytes / redis_memory_max_bytes * 100

# Request headroom
1 - (sum(rate(http_requests_total[5m])) / sum(service_max_rps))
```

---

## Appendix: Quick Reference

### Scaling Cheat Sheet

| Scenario             | Action                 | Command                                |
| -------------------- | ---------------------- | -------------------------------------- |
| CPU > 80%            | Scale pods             | `kubectl scale deploy X --replicas=+2` |
| Memory > 85%         | Scale pods + add nodes | Scale + `gcloud clusters resize`       |
| Latency > 300ms      | Scale + check DB       | Scale + check slow queries             |
| Queue depth > 100    | Scale workers          | Increase HPA max                       |
| DB connections > 80% | Add PgBouncer capacity | Scale PgBouncer                        |

### Key Thresholds Summary

| Resource    | Warning | Action | Critical |
| ----------- | ------- | ------ | -------- |
| CPU         | 60%     | 70%    | 90%      |
| Memory      | 70%     | 80%    | 90%      |
| Disk        | 70%     | 80%    | 90%      |
| Connections | 70%     | 80%    | 90%      |
| Error Rate  | 0.1%    | 0.5%   | 1%       |
| P95 Latency | 200ms   | 300ms  | 500ms    |

---

_Last Updated: January 2026_ _Review Cycle: Weekly during launch, Monthly
thereafter_
