This Terraform module deploys the complete infrastructure for the Psyche AI Virtual Assistant Platform on AWS, including GPU-accelerated EKS clusters, PostgreSQL with pgvector, Redis caching, S3 storage, and CloudFront CDN.
Architecture Overview#
text
┌─────────────────────────────────────────────────────────────────┐
│ AWS Cloud │
│ ┌─────────────────────────────────────────────────────────────┐│
│ │ VPC ││
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ││
│ │ │ Public │ │ Public │ │ Public │ ││
┌──────────┐ ┌──────────────┐ │ │ │ Subnet │ │ Subnet │ │ Subnet │ ││
│ Users │─▶│ CloudFront │─────┼──┼──│ (ALB/NLB) │ │ (ALB/NLB) │ │ (ALB/NLB) │ ││
└──────────┘ └──────────────┘ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ ││
│ │ │ │ │ │ ││
│ │ │ ┌──────▼──────┐ ┌──────▼──────┐ ┌──────▼──────┐ ││
│ │ │ │ Private │ │ Private │ │ Private │ ││
│ │ │ │ Subnet │ │ Subnet │ │ Subnet │ ││
│ │ │ │ │ │ │ │ │ ││
│ │ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ ┌─────────┐ │ ││
│ │ │ │ │EKS Node │ │ │ │EKS Node │ │ │ │EKS Node │ │ ││
│ │ │ │ │(General)│ │ │ │(General)│ │ │ │(General)│ │ ││
│ │ │ │ └─────────┘ │ │ └─────────┘ │ │ └─────────┘ │ ││
│ │ │ │ ┌─────────┐ │ │ ┌─────────┐ │ │ │ ││
│ │ │ │ │GPU Node │ │ │ │GPU Node │ │ │ Karpenter │ ││
│ │ │ │ │(NVIDIA) │ │ │ │(NVIDIA) │ │ │ Auto-scale │ ││
│ │ │ │ └─────────┘ │ │ └─────────┘ │ │ │ ││
│ │ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ ││
│ │ │ │ │ │ ││
│ │ │ ┌──────▼─────────────────▼────────────────▼──────┐ ││
│ │ │ │ Database Subnet │ ││
▼ │ │ │ ┌────────────┐ ┌────────────┐ │ ││
┌──────────┐ │ │ │ │ RDS │ │ElastiCache │ │ ││
│ S3 │◀────────┼──┼──│ │ PostgreSQL │ │ Redis │ │ ││
│ (Assets) │ │ │ │ │ (pgvector) │ │ │ │ ││
└──────────┘ │ │ │ └────────────┘ └────────────┘ │ ││
│ │ └─────────────────────────────────────────────────┘ ││
│ └─────────────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────┘
Features#
Compute#
- EKS Cluster: Kubernetes 1.29 with control plane encryption
- General Purpose Nodes: m6i/m6a instances for standard workloads
- GPU Nodes: g5/g4dn/p4d instances for AI inference
- Karpenter: Just-in-time GPU node provisioning with spot support
- NVIDIA Device Plugin: GPU resource management
Networking#
- VPC: Multi-AZ with public, private, and database subnets
- NAT Gateway: Single or per-AZ for cost optimization
- VPC Endpoints: S3, ECR, STS, CloudWatch, Secrets Manager
- Security Groups: Least-privilege network policies
Storage#
- PostgreSQL RDS: pgvector extension for embeddings
- ElastiCache Redis: Session state and caching
- S3 Buckets: Models, assets, logs, backups
- CloudFront CDN: Global distribution for avatar assets
Security#
- KMS Encryption: All data encrypted at rest
- Secrets Manager: Database and Redis credentials
- IRSA: IAM roles for Kubernetes service accounts
- VPC Flow Logs: Network traffic auditing
Observability#
- CloudWatch Alarms: CPU, memory, storage alerts
- Container Insights: EKS pod metrics
- Performance Insights: RDS query analysis
- DCGM Exporter: GPU metrics for Prometheus
Prerequisites#
- Terraform >= 1.10.0
- AWS CLI configured with appropriate credentials
- kubectl for Kubernetes management
- helm for chart installations
Usage#
Quick Start (Development)#
bash
cd environments/development
terraform init
terraform plan
terraform apply
Production Deployment#
bash
cd environments/production
# Create terraform.tfvars with production values
cat > terraform.tfvars <<EOF
aws_region = "us-east-1"
eks_public_access = false
cloudfront_aliases = ["cdn.psyche.oshun.dev"]
cloudfront_acm_certificate_arn = "arn:aws:acm:us-east-1:ACCOUNT:certificate/CERT_ID"
alarm_sns_topic_arns = ["arn:aws:sns:us-east-1:ACCOUNT:alerts"]
enable_external_dns = true
hosted_zone_arns = ["arn:aws:route53:::hostedzone/ZONE_ID"]
EOF
terraform init
terraform plan -out=tfplan
terraform apply tfplan
Connect to EKS Cluster#
bash
# Get the kubectl config command from Terraform output
terraform output -raw kubectl_config | bash
# Verify connection
kubectl get nodes
kubectl get pods -A
Module Structure#
text
psyche/
├── versions.tf # Terraform and provider versions
├── variables.tf # Input variables
├── locals.tf # Local values and data sources
├── vpc.tf # VPC, subnets, NAT, endpoints
├── eks.tf # EKS cluster, node groups, add-ons
├── karpenter.tf # Karpenter for GPU auto-scaling
├── rds.tf # PostgreSQL with pgvector
├── elasticache.tf # Redis cluster
├── s3.tf # Storage buckets
├── cloudfront.tf # CDN distribution
├── iam.tf # IRSA roles
├── ecr.tf # Container registries
├── helm.tf # Kubernetes add-ons
├── outputs.tf # Output values
├── README.md # This file
└── environments/
├── development/ # Dev configuration
├── staging/ # Staging configuration
└── production/ # Production configuration
Variables#
Required#
| Variable | Description |
|---|---|
environment |
Environment name (dev, staging, prod) |
Important Optional#
| Variable | Default | Description |
|---|---|---|
project_name |
psyche |
Project name for resource naming |
aws_region |
us-east-1 |
AWS region |
vpc_cidr |
10.100.0.0/16 |
VPC CIDR block |
az_count |
3 |
Number of availability zones |
eks_cluster_version |
1.29 |
Kubernetes version |
enable_gpu_nodes |
true |
Enable GPU node group |
enable_karpenter |
true |
Enable Karpenter autoscaling |
See variables.tf for the complete list.
Outputs#
| Output | Description |
|---|---|
cluster_name |
EKS cluster name |
cluster_endpoint |
EKS API endpoint |
kubectl_config |
kubectl configuration command |
rds_endpoint |
PostgreSQL endpoint |
elasticache_primary_endpoint |
Redis endpoint |
cloudfront_domain_name |
CDN domain |
ecr_repository_urls |
ECR repository URLs |
app_workload_role_arn |
IAM role for app workloads |
gpu_workload_role_arn |
IAM role for GPU workloads |
Environment Configurations#
Development#
- Single NAT gateway
- Spot instances for general nodes
- Minimal GPU capacity (0-2 nodes)
- Single-AZ RDS
- No alarms
Staging#
- Single NAT gateway
- On-demand instances
- Moderate GPU capacity (0-4 nodes)
- Multi-AZ RDS
- Basic alarms
Production#
- NAT per AZ
- On-demand instances only
- Full GPU capacity (1-10 nodes)
- Multi-AZ RDS with read replica
- Full monitoring and alarms
GPU Instance Types#
| Instance | GPU | VRAM | Use Case |
|---|---|---|---|
| g4dn.xlarge | T4 | 16GB | Development, small models |
| g4dn.2xlarge | T4 | 16GB | Development, batch inference |
| g5.xlarge | A10G | 24GB | Production, medium models |
| g5.2xlarge | A10G | 24GB | Production, real-time inference |
| g5.4xlarge | A10G | 24GB | Production, large batch |
| g5.8xlarge | A10G | 24GB | High-throughput inference |
| g5.12xlarge | 4x A10G | 96GB | Multi-model serving |
| p4d.24xlarge | 8x A100 | 320GB | Large model training |
Cost Optimization#
- Karpenter Spot Instances: Configure spot capacity for non-critical GPU workloads
- Auto-scaling: Set appropriate min/max nodes
- Storage Lifecycle: Enable S3 lifecycle policies
- Single NAT Gateway: Use in dev/staging
Security Best Practices#
- Private EKS API: Disable public access in production
- IRSA: Use service account roles instead of node roles
- Secrets Manager: Never hardcode credentials
- Network Policies: Implement Kubernetes network policies
- Image Scanning: Enable ECR scan on push
Troubleshooting#
EKS Cluster Not Ready#
bash
# Check cluster status
aws eks describe-cluster --name psyche-dev --query 'cluster.status'
# Check node groups
aws eks describe-nodegroup --cluster-name psyche-dev --nodegroup-name psyche-dev-general
GPU Nodes Not Scheduling#
bash
# Check NVIDIA device plugin
kubectl get pods -n kube-system -l app.kubernetes.io/name=nvidia-device-plugin
# Check GPU resources
kubectl describe nodes -l nvidia.com/gpu.present=true | grep -A 10 "Allocatable"
Karpenter Not Provisioning#
bash
# Check Karpenter logs
kubectl logs -n karpenter -l app.kubernetes.io/name=karpenter
# Check NodePool status
kubectl get nodepools
kubectl get ec2nodeclasses
Maintenance#
Upgrade EKS Version#
- Update
eks_cluster_versionvariable - Run
terraform planto review changes - Apply during maintenance window
- Update node groups sequentially
Scale GPU Capacity#
bash
# Modify Karpenter limits
kubectl edit nodepool gpu
# Or update Terraform and apply
terraform apply -target=kubectl_manifest.karpenter_node_pool_gpu
Integration with Oshun Platform#
This module is part of the Oshun AI Platform monorepo. Related infrastructure:
- Kubernetes Manifests:
infra/k8s/psyche/ - Docker Configuration:
docker/psyche/ - Application Code:
libs/psyche/
License#
Copyright (c) 2024 Oshun Platform. All rights reserved.