Eve library

Oshun SQS Queues Module

Other

Revision 1

On this page

This Terraform module creates SQS queues for worker job dispatch across all Oshun domains, with dead-letter queues for failed message handling.

Features#

  • Standard Queues: High-throughput queues for general job dispatch
  • FIFO Queues: Ordered processing with exactly-once semantics
  • Dead-Letter Queues: Automatic DLQs for failed message investigation
  • Encryption: Server-side encryption with SQS-managed or KMS keys
  • Long Polling: Efficient message retrieval (20s default)
  • CloudWatch Alarms: Queue depth and DLQ monitoring
  • CloudWatch Dashboard: Visual monitoring for all queues

Usage#

hcl
module "sqs" {
  source = "../sqs"

  environment = "production"
  region      = "us-east-1"
  name_prefix = "oshun"

  # Optional: Custom KMS key for encryption
  # kms_key_id = module.kms.sqs_key_id

  # Override domain defaults if needed
  isis_queues = {
    isis-generation = {
      description                = "Generation job queue"
      visibility_timeout_seconds = 1800  # 30 minutes for long jobs
    }
  }

  # Add custom queues
  queues = {
    custom-processing = {
      domain                     = "custom"
      description                = "Custom processing queue"
      visibility_timeout_seconds = 600
    }
  }
}

Default Queues#

Isis Domain#

  • isis-generation: Generation job queue (15m visibility)
  • isis-workflow: Workflow orchestration queue
  • isis-output: Output processing queue

Sophia Domain#

  • sophia-ingestion: Document ingestion queue
  • sophia-embedding: Embedding generation queue
  • sophia-indexing: Search indexing queue

Hathor Domain#

  • hathor-simulation: World simulation queue (15m visibility)
  • hathor-npc: NPC update queue
  • hathor-generation: World generation queue

Bellona Domain#

  • bellona-build: Build job queue (30m visibility)
  • bellona-render: Render job queue (30m visibility)
  • bellona-asset: Asset processing queue

Yemaya Domain#

  • yemaya-project: Project processing queue
  • yemaya-export: Export job queue
  • yemaya-transcode: Media transcoding queue (20m visibility)

Lilith Domain#

  • lilith-ingestion: Content ingestion queue
  • lilith-notification: Notification delivery queue (1m visibility)
  • lilith-analytics: Analytics processing queue
  • lilith-meditation: Meditation generation queue

Queue Configuration Options#

hcl
queues = {
  my-queue = {
    # Required
    domain      = "isis"
    description = "My custom queue"

    # Optional (with defaults)
    enabled                    = true
    visibility_timeout_seconds = 300    # 5 minutes
    message_retention_seconds  = 1209600 # 14 days
    max_message_size          = 262144  # 256 KB
    delay_seconds             = 0
    receive_wait_time_seconds = 20      # Long polling

    # Dead letter queue
    enable_dlq            = true
    dlq_max_receive_count = 3
    dlq_retention_seconds = 1209600

    # FIFO queue (optional)
    fifo_queue                  = false
    content_based_deduplication = false

    # Tags
    tags = {
      Custom = "value"
    }
  }
}

Visibility Timeout Guidelines#

Set visibility timeout based on expected job processing time:

Job Type Recommended Timeout
Quick tasks (notifications, analytics) 60-300 seconds
Standard processing 300-600 seconds
Media processing 600-1200 seconds
Build/render jobs 1800-3600 seconds
Long-running generation 900-1800 seconds

Dead Letter Queue Behavior#

Messages are moved to DLQ after dlq_max_receive_count receive attempts:

  1. Worker receives message
  2. If processing fails, message becomes visible again after timeout
  3. After N failures, message moves to DLQ
  4. DLQ messages retained for investigation

Outputs#

Name Description
queue_arns Map of queue names to ARNs
queue_urls Map of queue names to URLs
queue_names Map of queue keys to AWS names
dlq_arns Map of DLQ names to ARNs
queues_by_domain Queues grouped by domain
ecs_queue_environment Environment variables for ECS
worker_policy_json IAM policy for worker access
domain_worker_policies Per-domain IAM policies
queues_summary Summary of deployed queues

Using with ECS Services#

The module outputs environment variables formatted for ECS:

hcl
module "ecs_services" {
  # ...

  services = {
    isis-worker = {
      environment = [
        for k, v in module.sqs.ecs_queue_environment : {
          name  = v.name
          value = v.value
        } if startswith(k, "ISIS")
      ]
    }
  }
}

IAM Policies#

Attach the worker policy to ECS task roles:

hcl
resource "aws_iam_role_policy" "worker_sqs" {
  name   = "sqs-access"
  role   = aws_iam_role.worker_task.id
  policy = module.sqs.domain_worker_policies["isis"]
}

CloudWatch Alarms#

The module creates two types of alarms:

  1. Queue Depth Alarms: Trigger when message backlog exceeds threshold
  2. DLQ Alarms: Trigger when any message lands in DLQ

Configure alarm actions in your monitoring setup:

hcl
resource "aws_sns_topic_subscription" "dlq_alerts" {
  for_each  = module.sqs.dlq_alarm_arns
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "email"
  endpoint  = "ops@example.com"
}

Best Practices#

  1. Visibility Timeout: Set to 6x expected processing time
  2. Long Polling: Keep at 20s to reduce empty responses
  3. DLQ Monitoring: Alert immediately on any DLQ messages
  4. Message Size: Keep under 256KB; use S3 for large payloads
  5. Idempotency: Design workers to handle duplicate messages
  6. Batch Processing: Use batch receive (up to 10 messages)