Eve library

Networking Architecture

Guide

Revision 1

On this page

Network infrastructure design and configuration for the Lilith platform.

Table of Contents#

Overview#

The Lilith platform uses a multi-tier VPC architecture designed for:

  • Security: Network isolation between tiers
  • High Availability: Multi-AZ deployment
  • Scalability: Flexible CIDR allocation
  • Observability: VPC Flow Logs for all traffic

Network Diagram#

text
                                    Internet
                                       │
                              ┌────────▼────────┐
                              │  Internet GW    │
                              └────────┬────────┘
                                       │
              ┌────────────────────────┼────────────────────────┐
              │                        │                        │
      ┌───────▼───────┐       ┌───────▼───────┐       ┌───────▼───────┐
      │  Public Sub   │       │  Public Sub   │       │  Public Sub   │
      │  10.0.0.0/24  │       │  10.0.1.0/24  │       │  10.0.2.0/24  │
      │    (AZ-a)     │       │    (AZ-b)     │       │    (AZ-c)     │
      └───────┬───────┘       └───────┬───────┘       └───────┬───────┘
              │                       │                       │
      ┌───────▼───────┐       ┌───────▼───────┐       ┌───────▼───────┐
      │   NAT GW-a    │       │   NAT GW-b    │       │   NAT GW-c    │
      └───────┬───────┘       └───────┬───────┘       └───────┬───────┘
              │                       │                       │
      ┌───────▼───────┐       ┌───────▼───────┐       ┌───────▼───────┐
      │ Private Sub   │       │ Private Sub   │       │ Private Sub   │
      │ 10.0.10.0/24  │       │ 10.0.11.0/24  │       │ 10.0.12.0/24  │
      │ (EKS Nodes)   │       │ (EKS Nodes)   │       │ (EKS Nodes)   │
      └───────────────┘       └───────────────┘       └───────────────┘

              │                       │                       │
      ┌───────▼───────┐       ┌───────▼───────┐       ┌───────▼───────┐
      │ Database Sub  │       │ Database Sub  │       │ Database Sub  │
      │ 10.0.20.0/24  │       │ 10.0.21.0/24  │       │ 10.0.22.0/24  │
      │   (RDS/DB)    │       │   (RDS/DB)    │       │   (RDS/DB)    │
      └───────────────┘       └───────────────┘       └───────────────┘

VPC Architecture#

CIDR Planning#

Environment VPC CIDR Region
Development 10.0.0.0/16 us-east-1
Staging 10.1.0.0/16 us-east-1
Production (Primary) 10.2.0.0/16 us-east-1
Production (DR) 10.3.0.0/16 eu-west-1
Production (APAC) 10.4.0.0/16 ap-southeast-1

Terraform Configuration#

hcl
# modules/networking/main.tf

resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "${var.environment}-lilith-vpc"
    Environment = var.environment
    Project     = "lilith"
  }
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-lilith-igw"
    Environment = var.environment
  }
}

VPC Features Enabled#

Feature Status Purpose
DNS Hostnames Enabled Service discovery
DNS Support Enabled Internal DNS resolution
Flow Logs Enabled Network monitoring
IPv6 Disabled Not currently required

Subnet Design#

Subnet Allocation#

Subnet Type CIDR Offset Purpose Internet Access
Public /24 (0-2) Load balancers, NAT GW Direct (IGW)
Private /24 (10-12) EKS nodes, application pods Via NAT GW
Database /24 (20-22) RDS, ElastiCache None

Terraform Configuration#

hcl
# Public Subnets
resource "aws_subnet" "public" {
  count = length(var.availability_zones)

  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone       = var.availability_zones[count.index]
  map_public_ip_on_launch = true

  tags = {
    Name                     = "${var.environment}-lilith-public-${count.index + 1}"
    Environment              = var.environment
    Type                     = "public"
    "kubernetes.io/role/elb" = "1"  # For AWS LB Controller
  }
}

# Private Subnets
resource "aws_subnet" "private" {
  count = length(var.availability_zones)

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + 10)
  availability_zone = var.availability_zones[count.index]

  tags = {
    Name                              = "${var.environment}-lilith-private-${count.index + 1}"
    Environment                       = var.environment
    Type                              = "private"
    "kubernetes.io/role/internal-elb" = "1"  # For internal LBs
  }
}

# Database Subnets
resource "aws_subnet" "database" {
  count = length(var.availability_zones)

  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + 20)
  availability_zone = var.availability_zones[count.index]

  tags = {
    Name        = "${var.environment}-lilith-database-${count.index + 1}"
    Environment = var.environment
    Type        = "database"
  }
}

NAT Gateway Configuration#

hcl
# Elastic IPs for NAT Gateways
resource "aws_eip" "nat" {
  count = var.single_nat_gateway ? 1 : length(var.availability_zones)

  domain = "vpc"

  tags = {
    Name        = "${var.environment}-lilith-nat-eip-${count.index + 1}"
    Environment = var.environment
  }

  depends_on = [aws_internet_gateway.main]
}

# NAT Gateways
resource "aws_nat_gateway" "main" {
  count = var.single_nat_gateway ? 1 : length(var.availability_zones)

  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id

  tags = {
    Name        = "${var.environment}-lilith-nat-${count.index + 1}"
    Environment = var.environment
  }

  depends_on = [aws_internet_gateway.main]
}

Route Tables#

hcl
# Public Route Table
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = {
    Name        = "${var.environment}-lilith-public-rt"
    Environment = var.environment
  }
}

# Private Route Tables (one per AZ for HA NAT)
resource "aws_route_table" "private" {
  count = length(var.availability_zones)

  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = var.single_nat_gateway ? aws_nat_gateway.main[0].id : aws_nat_gateway.main[count.index].id
  }

  tags = {
    Name        = "${var.environment}-lilith-private-rt-${count.index + 1}"
    Environment = var.environment
  }
}

Security Groups#

EKS Cluster Security Group#

hcl
resource "aws_security_group" "eks_cluster" {
  name_prefix = "${var.environment}-lilith-eks-cluster"
  vpc_id      = aws_vpc.main.id

  # Allow inbound from VPC for API server
  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = [var.vpc_cidr]
    description = "Kubernetes API server"
  }

  # Allow all outbound
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name        = "${var.environment}-lilith-eks-cluster-sg"
    Environment = var.environment
  }
}

EKS Node Security Group#

hcl
resource "aws_security_group" "eks_nodes" {
  name_prefix = "${var.environment}-lilith-eks-nodes"
  vpc_id      = aws_vpc.main.id

  # Node-to-node communication
  ingress {
    from_port = 0
    to_port   = 65535
    protocol  = "tcp"
    self      = true
    description = "Node to node"
  }

  # Control plane to nodes
  ingress {
    from_port       = 1025
    to_port         = 65535
    protocol        = "tcp"
    security_groups = [aws_security_group.eks_cluster.id]
    description     = "Control plane to nodes"
  }

  # Kubelet API
  ingress {
    from_port       = 10250
    to_port         = 10250
    protocol        = "tcp"
    security_groups = [aws_security_group.eks_cluster.id]
    description     = "Kubelet API"
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name        = "${var.environment}-lilith-eks-nodes-sg"
    Environment = var.environment
  }
}

Database Security Group#

hcl
resource "aws_security_group" "database" {
  name_prefix = "${var.environment}-lilith-database"
  vpc_id      = aws_vpc.main.id

  # PostgreSQL from EKS nodes only
  ingress {
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.eks_nodes.id]
    description     = "PostgreSQL from EKS"
  }

  # Redis from EKS nodes only
  ingress {
    from_port       = 6379
    to_port         = 6379
    protocol        = "tcp"
    security_groups = [aws_security_group.eks_nodes.id]
    description     = "Redis from EKS"
  }

  tags = {
    Name        = "${var.environment}-lilith-database-sg"
    Environment = var.environment
  }
}

Network Policies#

Kubernetes Network Policies#

yaml
# Default deny all ingress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: lilith
spec:
  podSelector: {}
  policyTypes:
    - Ingress
---
# Allow ingress from same namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: lilith
spec:
  podSelector: {}
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector: {}
---
# Allow ingress from ingress controller
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress-controller
  namespace: lilith
spec:
  podSelector: {}
  policyTypes:
    - Ingress
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress

Service-Specific Policies#

yaml
# BFF can access all backend services
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: bff-egress
  namespace: lilith
spec:
  podSelector:
    matchLabels:
      app: bff
  policyTypes:
    - Egress
  egress:
    - to:
        - podSelector:
            matchLabels:
              tier: backend
---
# AI service can access external APIs
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ai-egress
  namespace: lilith
spec:
  podSelector:
    matchLabels:
      app: ai
  policyTypes:
    - Egress
  egress:
    - to: [] # Allow all egress for external AI APIs

DNS Configuration#

Route53 Configuration#

hcl
# Public hosted zone
resource "aws_route53_zone" "public" {
  name = var.domain_name

  tags = {
    Environment = var.environment
    Project     = "lilith"
  }
}

# Private hosted zone for internal services
resource "aws_route53_zone" "private" {
  name = "internal.${var.domain_name}"

  vpc {
    vpc_id = module.networking.vpc_id
  }

  tags = {
    Environment = var.environment
    Project     = "lilith"
  }
}

External DNS Configuration#

yaml
# External DNS deployment for automatic DNS management
apiVersion: v1
kind: ServiceAccount
metadata:
  name: external-dns
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/lilith-external-dns-role
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: external-dns
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: external-dns
  template:
    metadata:
      labels:
        app: external-dns
    spec:
      serviceAccountName: external-dns
      containers:
        - name: external-dns
          image: registry.k8s.io/external-dns/external-dns:v0.14.0
          args:
            - --source=service
            - --source=ingress
            - --domain-filter=lilith.app
            - --provider=aws
            - --policy=sync
            - --aws-zone-type=public
            - --registry=txt
            - --txt-owner-id=lilith-eks

Load Balancing#

AWS Load Balancer Controller#

yaml
# Ingress with ALB
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: lilith-ingress
  namespace: lilith
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:ACCOUNT_ID:certificate/CERT_ID
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/healthcheck-path: /health
spec:
  rules:
    - host: api.lilith.app
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: bff
                port:
                  number: 3000

Internal Load Balancer#

yaml
# Internal NLB for service-to-service
apiVersion: v1
kind: Service
metadata:
  name: internal-api
  namespace: lilith
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-internal: 'true'
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
spec:
  type: LoadBalancer
  ports:
    - port: 443
      targetPort: 3000
  selector:
    app: bff

VPN & Direct Connect#

Site-to-Site VPN#

hcl
# Customer Gateway
resource "aws_customer_gateway" "main" {
  bgp_asn    = 65000
  ip_address = var.vpn_peer_ip
  type       = "ipsec.1"

  tags = {
    Name        = "${var.environment}-lilith-cgw"
    Environment = var.environment
  }
}

# VPN Gateway
resource "aws_vpn_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-lilith-vgw"
    Environment = var.environment
  }
}

# VPN Connection
resource "aws_vpn_connection" "main" {
  vpn_gateway_id      = aws_vpn_gateway.main.id
  customer_gateway_id = aws_customer_gateway.main.id
  type                = "ipsec.1"
  static_routes_only  = true

  tags = {
    Name        = "${var.environment}-lilith-vpn"
    Environment = var.environment
  }
}

VPC Flow Logs#

hcl
resource "aws_flow_log" "main" {
  vpc_id          = aws_vpc.main.id
  traffic_type    = "ALL"
  iam_role_arn    = aws_iam_role.flow_log.arn
  log_destination = aws_cloudwatch_log_group.flow_log.arn

  tags = {
    Name        = "${var.environment}-lilith-flow-logs"
    Environment = var.environment
  }
}

resource "aws_cloudwatch_log_group" "flow_log" {
  name              = "/aws/vpc/flow-logs/${var.environment}-lilith"
  retention_in_days = 30
}

Monitoring & Troubleshooting#

VPC Reachability Analyzer#

bash
# Check connectivity between resources
aws ec2 create-network-insights-path \
  --source i-source-instance-id \
  --destination i-destination-instance-id \
  --protocol TCP \
  --destination-port 5432

aws ec2 start-network-insights-analysis \
  --network-insights-path-id nip-path-id

Common Issues#

Issue Symptom Solution
NAT Gateway limit Connection timeouts Scale to multiple NAT GWs
Security Group Connection refused Check ingress/egress rules
Route table No route to host Verify route associations
DNS resolution Unknown host Check DNS settings, VPC DHCP