# Environment Configuration Management

## Overview

This document describes the comprehensive environment configuration management
system for the Lilith platform, supporting multiple deployment environments
(development, staging, production) with environment-specific configurations,
security policies, and deployment automation.

## Architecture

### Environment Types

1. **Development (dev)**: Local development with debugging features enabled
2. **Staging**: Pre-production testing environment
3. **Production (prod)**: Live production environment with full security

### Directory Structure

```
config/
├── environments/
│   ├── development/
│   │   ├── .env                    # Environment variables
│   │   ├── services/               # Service-specific configs
│   │   │   ├── bff/
│   │   │   ├── ai/
│   │   │   └── ...
│   │   ├── vault/                 # Vault configurations
│   │   ├── monitoring/            # Monitoring configs
│   │   ├── networking/            # Network policies
│   │   └── security/              # Security policies
│   ├── staging/
│   │   └── [same structure as development]
│   └── production/
│       └── [same structure as development]
├── active -> environments/development    # Symlink to active environment
└── .env.active                             # Current environment file
```

## Configuration Components

### 1. Environment Variables (.env)

Each environment has its own `.env` file with:

#### Development Environment

```bash
NODE_ENV=development
ENVIRONMENT=development
DEBUG=true
HOT_RELOAD=true
LOG_LEVEL=debug

# URLs
API_BASE_URL=http://localhost:8080
WEB_BASE_URL=http://localhost:3000
VAULT_ADDR=http://localhost:8200

# Database
DB_HOST=localhost
DB_PORT=5433
DB_NAME=lilith_dev
DB_USER=lilith
DB_PASSWORD=dev_password

# Resources (generous)
CPU_LIMIT=2
MEMORY_LIMIT=4Gi
```

#### Production Environment

```bash
NODE_ENV=production
ENVIRONMENT=production
DEBUG=false
LOG_LEVEL=warn

# URLs
API_BASE_URL=https://api.lilith.app
WEB_BASE_URL=https://lilith.app
VAULT_ADDR=https://vault.lilith.app

# Database
DB_HOST=postgres-prod.internal
DB_PORT=5432
DB_NAME=lilith_prod
DB_USER=lilith_prod
DB_PASSWORD=

# Security
ENCRYPTION_ENABLED=true
AUDIT_LOGGING_ENABLED=true
RATE_LIMITING_ENABLED=true

# Resources (optimized)
CPU_LIMIT=1
MEMORY_LIMIT=1Gi
```

### 2. Docker Compose Overrides

Each environment gets its own Docker Compose override file:

#### Development (`docker-compose.development.yml`)

```yaml
services:
  bff:
    environment:
      - NODE_ENV=development
      - DEBUG=true
    volumes:
      - ./services/bff/src:/app/src
      - ./config/environments/development/.env:/app/.env:ro
    command: ['npm', 'run', 'dev']

  # Development tools
  nginx-proxy:
    image: nginx:alpine
    ports:
      - '8081:80'
```

#### Production (`docker-compose.production.yml`)

```yaml
services:
  bff:
    environment:
      - NODE_ENV=production
      - LOG_LEVEL=warn
    restart: unless-stopped
    deploy:
      replicas: 2
      resources:
        limits:
          memory: 1Gi
        reservations:
          memory: 512Mi
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: rollback
```

### 3. Vault Configuration

Environment-specific Vault configurations:

#### Development Vault (`config/environments/development/vault/vault.hcl`)

```hcl
ui = true
api_addr = "http://0.0.0.0:8200"
cluster_addr = "http://0.0.0.0:8201"

backend "file" {
  path = "/vault/data"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = true
  cluster_address = "0.0.0.0:8201"
}

log_level = "debug"
```

#### Production Vault (`config/environments/production/vault/vault.hcl`)

```hcl
ui = true
api_addr = "https://vault.lilith.app"
cluster_addr = "https://vault.lilith.app:8201"

backend "file" {
  path = "/vault/data"
}

listener "tcp" {
  address = "0.0.0.0:8200"
  tls_disable = false
  cluster_address = "0.0.0.0:8201"
}

log_level = "info"
```

### 4. Service Configurations

Each service gets environment-specific configuration:

#### Service Configuration JSON (`config/environments/production/services/bff/config.json`)

```json
{
  "service": "bff",
  "environment": "production",
  "version": "1.0.0",
  "features": {
    "healthChecks": true,
    "metrics": true,
    "tracing": true,
    "debug": false
  },
  "endpoints": {
    "health": "/health",
    "metrics": "/metrics",
    "readiness": "/ready"
  },
  "resources": {
    "cpu": {
      "request": "100m",
      "limit": "500m"
    },
    "memory": {
      "request": "256Mi",
      "limit": "1Gi"
    }
  },
  "environmentSpecific": {
    "debug": false,
    "logLevel": "warn",
    "performanceMonitoring": true,
    "errorReporting": true
  }
}
```

## Usage

### 1. Environment Setup

#### Generate All Configurations

```bash
# Generate configurations for all environments
./scripts/config-manager.sh all

# Generate for specific environment
./scripts/config-manager.sh generate development
./scripts/config-manager.sh generate staging
./scripts/config-manager.sh generate production
```

#### Validate Configuration

```bash
# Validate specific environment
./scripts/config-manager.sh validate production

# This checks:
# • Required files exist
# • JSON configurations are valid
# • Docker compose files are valid
# • No missing service configurations
```

### 2. Environment Switching

#### Quick Switch

```bash
# Switch to production environment
./scripts/config-manager.sh switch production

# Switch to development
./scripts/config-manager.sh switch dev
```

#### Using Environment Switch Script

```bash
# Switch environments
./scripts/switch-env.sh prod
./scripts/switch-env.sh staging
./scripts/switch-env.sh dev

# Show current environment
./scripts/switch-env.sh show
```

### 3. Deployment

#### Development Deployment

```bash
# Switch to development environment
./scripts/switch-env.sh dev

# Deploy development services
./scripts/deploy/development/deploy.sh
```

#### Staging Deployment

```bash
# Switch to staging environment
./scripts/switch-env.sh staging

# Deploy staging services
./scripts/deploy/staging/deploy.sh
```

#### Production Deployment

```bash
# Switch to production environment
./scripts/switch-env.sh production

# Deploy production services
./scripts/deploy/production/deploy.sh
```

### 4. Health Monitoring

#### Environment-Specific Health Checks

```bash
# Check current environment
./scripts/switch-env.sh show

# Run health check for current environment
./scripts/deploy/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/health-check.sh
```

## Configuration Management Workflow

### 1. Development Workflow

1. **Local Setup**

   ```bash
   ./scripts/config-manager.sh generate development
   ./scripts/switch-env.sh dev
   ```

2. **Development Services**

   ```bash
   docker compose -f docker-compose.yml -f docker-compose.development.yml up -d
   ```

3. **Feature Development**
   - Modify service code
   - Test locally
   - Validate configuration changes

### 2. Testing Workflow

1. **Staging Setup**

   ```bash
   ./scripts/config-manager.sh generate staging
   ./scripts/switch-env.sh staging
   ```

2. **Deploy to Staging**

   ```bash
   ./scripts/deploy/staging/deploy.sh
   ```

3. **Integration Testing**
   - Run automated tests
   - Validate service integration
   - Performance testing

### 3. Production Workflow

1. **Production Setup**

   ```bash
   ./scripts/config-manager.sh generate production
   ./scripts/switch-env.sh production
   ```

2. **Pre-deployment Validation**

   ```bash
   ./scripts/config-manager.sh validate production
   ```

3. **Production Deployment**

   ```bash
   ./scripts/deploy/production/deploy.sh
   ```

4. **Post-deployment Verification**
   ```bash
   ./scripts/deploy/production/health-check.sh
   ```

## Security Considerations

### 1. Configuration Security

#### Secrets Management

- **Never commit secrets** to version control
- **Use Vault** for all sensitive data
- **Environment-specific** secret paths in Vault
- **Automatic rotation** for critical secrets

#### Configuration Validation

- **JSON schema validation** for service configs
- **Docker compose validation** before deployment
- **Environment variable validation** for required settings

### 2. Network Security

#### Development Environment

- **Localhost access** only
- **Debug endpoints** enabled
- **Mock services** for external APIs

#### Staging Environment

- **Internal network** access only
- **Real services** (with staging data)
- **Security features** enabled

#### Production Environment

- **Full security** features enabled
- **Load balancer** with SSL termination
- **Network policies** and **service mesh**
- **External access** through controlled endpoints

### 3. Access Control

#### Environment Access

```bash
# Development: All developers
# Staging: Development and QA teams
# Production: DevOps and authorized personnel only
```

#### Service Configuration

```bash
# Production services:
# - No debug endpoints
# - Limited log levels
# - Error reporting only
# - Performance monitoring enabled
```

## Monitoring and Observability

### 1. Environment Monitoring

#### Service Health

```bash
# Health check endpoints for each service
GET /health
GET /ready
GET /metrics
```

#### Configuration Monitoring

```bash
# Track configuration changes
# Validate environment consistency
# Monitor service restarts
```

### 2. Alerting

#### Environment-Specific Alerts

- **Development**: Error notifications
- **Staging**: Performance and error alerts
- **Production**: Critical alerts with escalation

#### Configuration Drift Detection

```bash
# Compare configurations
./scripts/config-manager.sh compare development staging
./scripts/config-manager.sh compare staging production
```

## Best Practices

### 1. Configuration Management

#### Version Control

- **Configuration files** in version control
- **Secrets never** committed
- **Environment templates** for consistency

#### Change Management

- **Review process** for production changes
- **Staging validation** before production
- **Rollback plans** for critical changes

### 2. Environment Consistency

#### Standardization

- **Consistent structure** across environments
- **Standard naming conventions**
- **Required fields** validation

#### Testing

- **Configuration validation** in CI/CD
- **Environment testing** before deployment
- **Rollback testing** procedures

### 3. Operational Excellence

#### Automation

- **Automated deployment** pipelines
- **Configuration validation** scripts
- **Health monitoring** and alerting

#### Documentation

- **Comprehensive documentation** for each environment
- **Runbooks** for common operations
- **Troubleshooting guides** for issues

## Troubleshooting

### 1. Common Issues

#### Configuration Validation Failures

```bash
# Check configuration syntax
./scripts/config-manager.sh validate production

# Check JSON files
jq empty config/environments/production/services/*/config.json
```

#### Environment Switch Failures

```bash
# Check directory permissions
ls -la config/environments/
ls -la config/environments/active

# Check symlink
ls -la config/.env.active
```

#### Deployment Failures

```bash
# Check environment variables
cat config/environments/active/.env

# Check docker compose files
docker compose -f docker-compose.yml -f docker-compose.production.yml config
```

### 2. Debug Configuration

#### Enable Debug Mode

```bash
# Add debug logging
export DEBUG_CONFIG=true
export LOG_LEVEL=debug

# Check configuration loading
./scripts/deploy/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/deploy.sh --dry-run
```

#### Validate Service Configuration

```bash
# Check service config files
cat config/environments/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/services/bff/config.json

# Validate JSON
jq empty config/environments/$(echo $ENVIRONMENT | tr '[:upper:]' '[:lower:]')/services/bff/config.json
```

### 3. Health Monitoring

#### Service Health

```bash
# Check service health
curl -f http://localhost:8080/health

# Check all services
./scripts/health-monitor.sh all
```

#### Configuration Health

```bash
# Check current environment
./scripts/switch-env.sh show

# Validate configuration
./scripts/config-manager.sh validate $ENVIRONMENT
```

## Advanced Features

### 1. Configuration Templates

#### Template Generation

```bash
# Generate new environment template
./scripts/config-manager.sh generate custom-environment

# Customize based on existing environment
cp -r config/environments/production config/environments/custom-environment
# Modify as needed
```

### 2. Environment-Specific Features

#### Feature Flags

```json
{
  "features": {
    "experimentalFeatures": {
      "development": true,
      "staging": false,
      "production": false
    },
    "advancedMonitoring": {
      "development": false,
      "staging": true,
      "production": true
    }
  }
}
```

#### Resource Scaling

```json
{
  "resources": {
    "autoScaling": {
      "development": false,
      "staging": true,
      "production": true
    },
    "maxReplicas": {
      "development": 1,
      "staging": 2,
      "production": 5
    }
  }
}
```

### 3. Multi-Region Support

#### Region Configuration

```bash
# Add region-specific configurations
mkdir -p config/environments/production/us-east-1
mkdir -p config/environments/production/us-west-2

# Region-specific overrides
echo "REGION=us-east-1" >> config/environments/production/us-east-1/.env
echo "CLUSTER=lilith-prod-use1" >> config/environments/production/us-east-1/.env
```

## Integration with CI/CD

### 1. GitHub Actions

#### Environment Configuration

```yaml
name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Generate Production Config
        run: ./scripts/config-manager.sh generate production

      - name: Validate Configuration
        run: ./scripts/config-manager.sh validate production

      - name: Deploy to Production
        run: ./scripts/deploy/production/deploy.sh
```

### 2. Environment Variables in CI/CD

#### Secure Configuration Loading

```bash
# Load environment-specific configuration
source config/environments/$ENVIRONMENT/.env

# Use configuration in deployment
docker compose -f docker-compose.yml -f docker-compose.$ENVIRONMENT.yml up -d
```

### 3. Testing Pipeline

#### Multi-Environment Testing

```yaml
# Staging tests
- name: Run Staging Tests
  run: |
    ./scripts/switch-env.sh staging
    ./scripts/deploy/staging/deploy.sh
    ./scripts/deploy/staging/health-check.sh
    npm run test:e2e:staging

# Production validation
- name: Production Validation
  run: |
    ./scripts/config-manager.sh validate production
    ./scripts/deploy/production/deploy.sh --dry-run
```

## Migration Guide

### 1. Existing Environment Migration

#### Backup Current Configuration

```bash
# Create backup directory
mkdir -p backups/config-migration-$(date +%Y%m%d)

# Backup current configurations
cp -r docker-compose.* backups/config-migration-$(date +%Y%m%d)/
cp -r .env* backups/config-migration-$(date +%Y%m%d)/ 2>/dev/null || true
```

#### Migrate to New System

```bash
# Generate new configurations
./scripts/config-manager.sh all

# Validate new configurations
for env in development staging production; do
    ./scripts/config-manager.sh validate $env
done

# Test migration
./scripts/switch-env.sh development
# Test services work with new configuration
```

### 2. Service Migration

#### Update Service Configuration Loading

```javascript
// In service startup
import config from './config.json';

// Load environment-specific overrides
const env = process.env.ENVIRONMENT || 'development';
const envConfigPath = `config/environments/${env}/services/${process.env.SERVICE_NAME}/config.json`;

try {
  const envConfig = require(envConfigPath);
  config = { ...config, ...envConfig };
} catch (error) {
  console.warn(`No environment-specific config found, using defaults`);
}
```

#### Configuration Validation

```javascript
// Validate required configuration
const requiredFields = ['service', 'environment', 'endpoints'];
for (const field of requiredFields) {
  if (!config[field]) {
    throw new Error(`Missing required configuration: ${field}`);
  }
}
```

## References

- **Configuration Scripts**: `scripts/config-manager.sh`
- **Environment Switch**: `scripts/switch-env.sh`
- **Deployment Scripts**: `scripts/deploy/*/`
- **Health Monitoring**: `scripts/health-monitor.sh`
- **Vault Setup**: `docs/deploy/VAULT_SETUP.md`
- **Secret Rotation**: `docs/deploy/VAULT_MIGRATION.md`
