Eve library

Psyche AI Platform - Terraform Infrastructure

Other

Revision 1

On this page

This Terraform module deploys the complete infrastructure for the Psyche AI Virtual Assistant Platform on AWS, including GPU-accelerated EKS clusters, PostgreSQL with pgvector, Redis caching, S3 storage, and CloudFront CDN.

Architecture Overview#

text
                                    ┌─────────────────────────────────────────────────────────────────┐
                                    │                          AWS Cloud                               │
                                    │  ┌─────────────────────────────────────────────────────────────┐│
                                    │  │                           VPC                                ││
                                    │  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         ││
                                    │  │  │  Public     │  │  Public     │  │  Public     │         ││
┌──────────┐  ┌──────────────┐     │  │  │  Subnet     │  │  Subnet     │  │  Subnet     │         ││
│  Users   │─▶│  CloudFront  │─────┼──┼──│  (ALB/NLB)  │  │  (ALB/NLB)  │  │  (ALB/NLB)  │         ││
└──────────┘  └──────────────┘     │  │  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘         ││
                    │              │  │         │                │                │                ││
                    │              │  │  ┌──────▼──────┐  ┌──────▼──────┐  ┌──────▼──────┐         ││
                    │              │  │  │  Private    │  │  Private    │  │  Private    │         ││
                    │              │  │  │  Subnet     │  │  Subnet     │  │  Subnet     │         ││
                    │              │  │  │             │  │             │  │             │         ││
                    │              │  │  │ ┌─────────┐ │  │ ┌─────────┐ │  │ ┌─────────┐ │         ││
                    │              │  │  │ │EKS Node │ │  │ │EKS Node │ │  │ │EKS Node │ │         ││
                    │              │  │  │ │(General)│ │  │ │(General)│ │  │ │(General)│ │         ││
                    │              │  │  │ └─────────┘ │  │ └─────────┘ │  │ └─────────┘ │         ││
                    │              │  │  │ ┌─────────┐ │  │ ┌─────────┐ │  │             │         ││
                    │              │  │  │ │GPU Node │ │  │ │GPU Node │ │  │  Karpenter  │         ││
                    │              │  │  │ │(NVIDIA) │ │  │ │(NVIDIA) │ │  │  Auto-scale │         ││
                    │              │  │  │ └─────────┘ │  │ └─────────┘ │  │             │         ││
                    │              │  │  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘         ││
                    │              │  │         │                │                │                ││
                    │              │  │  ┌──────▼─────────────────▼────────────────▼──────┐         ││
                    │              │  │  │                 Database Subnet                │         ││
                    ▼              │  │  │  ┌────────────┐           ┌────────────┐       │         ││
             ┌──────────┐         │  │  │  │  RDS       │           │ElastiCache │       │         ││
             │    S3    │◀────────┼──┼──│  │ PostgreSQL │           │   Redis    │       │         ││
             │ (Assets) │         │  │  │  │ (pgvector) │           │            │       │         ││
             └──────────┘         │  │  │  └────────────┘           └────────────┘       │         ││
                                  │  │  └─────────────────────────────────────────────────┘         ││
                                  │  └─────────────────────────────────────────────────────────────┘│
                                  └─────────────────────────────────────────────────────────────────┘

Features#

Compute#

  • EKS Cluster: Kubernetes 1.29 with control plane encryption
  • General Purpose Nodes: m6i/m6a instances for standard workloads
  • GPU Nodes: g5/g4dn/p4d instances for AI inference
  • Karpenter: Just-in-time GPU node provisioning with spot support
  • NVIDIA Device Plugin: GPU resource management

Networking#

  • VPC: Multi-AZ with public, private, and database subnets
  • NAT Gateway: Single or per-AZ for cost optimization
  • VPC Endpoints: S3, ECR, STS, CloudWatch, Secrets Manager
  • Security Groups: Least-privilege network policies

Storage#

  • PostgreSQL RDS: pgvector extension for embeddings
  • ElastiCache Redis: Session state and caching
  • S3 Buckets: Models, assets, logs, backups
  • CloudFront CDN: Global distribution for avatar assets

Security#

  • KMS Encryption: All data encrypted at rest
  • Secrets Manager: Database and Redis credentials
  • IRSA: IAM roles for Kubernetes service accounts
  • VPC Flow Logs: Network traffic auditing

Observability#

  • CloudWatch Alarms: CPU, memory, storage alerts
  • Container Insights: EKS pod metrics
  • Performance Insights: RDS query analysis
  • DCGM Exporter: GPU metrics for Prometheus

Prerequisites#

  • Terraform >= 1.10.0
  • AWS CLI configured with appropriate credentials
  • kubectl for Kubernetes management
  • helm for chart installations

Usage#

Quick Start (Development)#

bash
cd environments/development
terraform init
terraform plan
terraform apply

Production Deployment#

bash
cd environments/production

# Create terraform.tfvars with production values
cat > terraform.tfvars <<EOF
aws_region                    = "us-east-1"
eks_public_access             = false
cloudfront_aliases            = ["cdn.psyche.oshun.dev"]
cloudfront_acm_certificate_arn = "arn:aws:acm:us-east-1:ACCOUNT:certificate/CERT_ID"
alarm_sns_topic_arns          = ["arn:aws:sns:us-east-1:ACCOUNT:alerts"]
enable_external_dns           = true
hosted_zone_arns              = ["arn:aws:route53:::hostedzone/ZONE_ID"]
EOF

terraform init
terraform plan -out=tfplan
terraform apply tfplan

Connect to EKS Cluster#

bash
# Get the kubectl config command from Terraform output
terraform output -raw kubectl_config | bash

# Verify connection
kubectl get nodes
kubectl get pods -A

Module Structure#

text
psyche/
├── versions.tf           # Terraform and provider versions
├── variables.tf          # Input variables
├── locals.tf             # Local values and data sources
├── vpc.tf                # VPC, subnets, NAT, endpoints
├── eks.tf                # EKS cluster, node groups, add-ons
├── karpenter.tf          # Karpenter for GPU auto-scaling
├── rds.tf                # PostgreSQL with pgvector
├── elasticache.tf        # Redis cluster
├── s3.tf                 # Storage buckets
├── cloudfront.tf         # CDN distribution
├── iam.tf                # IRSA roles
├── ecr.tf                # Container registries
├── helm.tf               # Kubernetes add-ons
├── outputs.tf            # Output values
├── README.md             # This file
└── environments/
    ├── development/      # Dev configuration
    ├── staging/          # Staging configuration
    └── production/       # Production configuration

Variables#

Required#

Variable Description
environment Environment name (dev, staging, prod)

Important Optional#

Variable Default Description
project_name psyche Project name for resource naming
aws_region us-east-1 AWS region
vpc_cidr 10.100.0.0/16 VPC CIDR block
az_count 3 Number of availability zones
eks_cluster_version 1.29 Kubernetes version
enable_gpu_nodes true Enable GPU node group
enable_karpenter true Enable Karpenter autoscaling

See variables.tf for the complete list.

Outputs#

Output Description
cluster_name EKS cluster name
cluster_endpoint EKS API endpoint
kubectl_config kubectl configuration command
rds_endpoint PostgreSQL endpoint
elasticache_primary_endpoint Redis endpoint
cloudfront_domain_name CDN domain
ecr_repository_urls ECR repository URLs
app_workload_role_arn IAM role for app workloads
gpu_workload_role_arn IAM role for GPU workloads

Environment Configurations#

Development#

  • Single NAT gateway
  • Spot instances for general nodes
  • Minimal GPU capacity (0-2 nodes)
  • Single-AZ RDS
  • No alarms

Staging#

  • Single NAT gateway
  • On-demand instances
  • Moderate GPU capacity (0-4 nodes)
  • Multi-AZ RDS
  • Basic alarms

Production#

  • NAT per AZ
  • On-demand instances only
  • Full GPU capacity (1-10 nodes)
  • Multi-AZ RDS with read replica
  • Full monitoring and alarms

GPU Instance Types#

Instance GPU VRAM Use Case
g4dn.xlarge T4 16GB Development, small models
g4dn.2xlarge T4 16GB Development, batch inference
g5.xlarge A10G 24GB Production, medium models
g5.2xlarge A10G 24GB Production, real-time inference
g5.4xlarge A10G 24GB Production, large batch
g5.8xlarge A10G 24GB High-throughput inference
g5.12xlarge 4x A10G 96GB Multi-model serving
p4d.24xlarge 8x A100 320GB Large model training

Cost Optimization#

  1. Karpenter Spot Instances: Configure spot capacity for non-critical GPU workloads
  2. Auto-scaling: Set appropriate min/max nodes
  3. Storage Lifecycle: Enable S3 lifecycle policies
  4. Single NAT Gateway: Use in dev/staging

Security Best Practices#

  1. Private EKS API: Disable public access in production
  2. IRSA: Use service account roles instead of node roles
  3. Secrets Manager: Never hardcode credentials
  4. Network Policies: Implement Kubernetes network policies
  5. Image Scanning: Enable ECR scan on push

Troubleshooting#

EKS Cluster Not Ready#

bash
# Check cluster status
aws eks describe-cluster --name psyche-dev --query 'cluster.status'

# Check node groups
aws eks describe-nodegroup --cluster-name psyche-dev --nodegroup-name psyche-dev-general

GPU Nodes Not Scheduling#

bash
# Check NVIDIA device plugin
kubectl get pods -n kube-system -l app.kubernetes.io/name=nvidia-device-plugin

# Check GPU resources
kubectl describe nodes -l nvidia.com/gpu.present=true | grep -A 10 "Allocatable"

Karpenter Not Provisioning#

bash
# Check Karpenter logs
kubectl logs -n karpenter -l app.kubernetes.io/name=karpenter

# Check NodePool status
kubectl get nodepools
kubectl get ec2nodeclasses

Maintenance#

Upgrade EKS Version#

  1. Update eks_cluster_version variable
  2. Run terraform plan to review changes
  3. Apply during maintenance window
  4. Update node groups sequentially

Scale GPU Capacity#

bash
# Modify Karpenter limits
kubectl edit nodepool gpu

# Or update Terraform and apply
terraform apply -target=kubectl_manifest.karpenter_node_pool_gpu

Integration with Oshun Platform#

This module is part of the Oshun AI Platform monorepo. Related infrastructure:

  • Kubernetes Manifests: infra/k8s/psyche/
  • Docker Configuration: docker/psyche/
  • Application Code: libs/psyche/

License#

Copyright (c) 2024 Oshun Platform. All rights reserved.