# Oshun SQS Queues Module

This Terraform module creates SQS queues for worker job dispatch across all
Oshun domains, with dead-letter queues for failed message handling.

## Features

- **Standard Queues**: High-throughput queues for general job dispatch
- **FIFO Queues**: Ordered processing with exactly-once semantics
- **Dead-Letter Queues**: Automatic DLQs for failed message investigation
- **Encryption**: Server-side encryption with SQS-managed or KMS keys
- **Long Polling**: Efficient message retrieval (20s default)
- **CloudWatch Alarms**: Queue depth and DLQ monitoring
- **CloudWatch Dashboard**: Visual monitoring for all queues

## Usage

```hcl
module "sqs" {
  source = "../sqs"

  environment = "production"
  region      = "us-east-1"
  name_prefix = "oshun"

  # Optional: Custom KMS key for encryption
  # kms_key_id = module.kms.sqs_key_id

  # Override domain defaults if needed
  isis_queues = {
    isis-generation = {
      description                = "Generation job queue"
      visibility_timeout_seconds = 1800  # 30 minutes for long jobs
    }
  }

  # Add custom queues
  queues = {
    custom-processing = {
      domain                     = "custom"
      description                = "Custom processing queue"
      visibility_timeout_seconds = 600
    }
  }
}
```

## Default Queues

### Isis Domain

- `isis-generation`: Generation job queue (15m visibility)
- `isis-workflow`: Workflow orchestration queue
- `isis-output`: Output processing queue

### Sophia Domain

- `sophia-ingestion`: Document ingestion queue
- `sophia-embedding`: Embedding generation queue
- `sophia-indexing`: Search indexing queue

### Hathor Domain

- `hathor-simulation`: World simulation queue (15m visibility)
- `hathor-npc`: NPC update queue
- `hathor-generation`: World generation queue

### Bellona Domain

- `bellona-build`: Build job queue (30m visibility)
- `bellona-render`: Render job queue (30m visibility)
- `bellona-asset`: Asset processing queue

### Yemaya Domain

- `yemaya-project`: Project processing queue
- `yemaya-export`: Export job queue
- `yemaya-transcode`: Media transcoding queue (20m visibility)

### Lilith Domain

- `lilith-ingestion`: Content ingestion queue
- `lilith-notification`: Notification delivery queue (1m visibility)
- `lilith-analytics`: Analytics processing queue
- `lilith-meditation`: Meditation generation queue

## Queue Configuration Options

```hcl
queues = {
  my-queue = {
    # Required
    domain      = "isis"
    description = "My custom queue"

    # Optional (with defaults)
    enabled                    = true
    visibility_timeout_seconds = 300    # 5 minutes
    message_retention_seconds  = 1209600 # 14 days
    max_message_size          = 262144  # 256 KB
    delay_seconds             = 0
    receive_wait_time_seconds = 20      # Long polling

    # Dead letter queue
    enable_dlq            = true
    dlq_max_receive_count = 3
    dlq_retention_seconds = 1209600

    # FIFO queue (optional)
    fifo_queue                  = false
    content_based_deduplication = false

    # Tags
    tags = {
      Custom = "value"
    }
  }
}
```

## Visibility Timeout Guidelines

Set visibility timeout based on expected job processing time:

| Job Type                               | Recommended Timeout |
| -------------------------------------- | ------------------- |
| Quick tasks (notifications, analytics) | 60-300 seconds      |
| Standard processing                    | 300-600 seconds     |
| Media processing                       | 600-1200 seconds    |
| Build/render jobs                      | 1800-3600 seconds   |
| Long-running generation                | 900-1800 seconds    |

## Dead Letter Queue Behavior

Messages are moved to DLQ after `dlq_max_receive_count` receive attempts:

1. Worker receives message
2. If processing fails, message becomes visible again after timeout
3. After N failures, message moves to DLQ
4. DLQ messages retained for investigation

## Outputs

| Name                   | Description                    |
| ---------------------- | ------------------------------ |
| queue_arns             | Map of queue names to ARNs     |
| queue_urls             | Map of queue names to URLs     |
| queue_names            | Map of queue keys to AWS names |
| dlq_arns               | Map of DLQ names to ARNs       |
| queues_by_domain       | Queues grouped by domain       |
| ecs_queue_environment  | Environment variables for ECS  |
| worker_policy_json     | IAM policy for worker access   |
| domain_worker_policies | Per-domain IAM policies        |
| queues_summary         | Summary of deployed queues     |

## Using with ECS Services

The module outputs environment variables formatted for ECS:

```hcl
module "ecs_services" {
  # ...

  services = {
    isis-worker = {
      environment = [
        for k, v in module.sqs.ecs_queue_environment : {
          name  = v.name
          value = v.value
        } if startswith(k, "ISIS")
      ]
    }
  }
}
```

## IAM Policies

Attach the worker policy to ECS task roles:

```hcl
resource "aws_iam_role_policy" "worker_sqs" {
  name   = "sqs-access"
  role   = aws_iam_role.worker_task.id
  policy = module.sqs.domain_worker_policies["isis"]
}
```

## CloudWatch Alarms

The module creates two types of alarms:

1. **Queue Depth Alarms**: Trigger when message backlog exceeds threshold
2. **DLQ Alarms**: Trigger when any message lands in DLQ

Configure alarm actions in your monitoring setup:

```hcl
resource "aws_sns_topic_subscription" "dlq_alerts" {
  for_each  = module.sqs.dlq_alarm_arns
  topic_arn = aws_sns_topic.alerts.arn
  protocol  = "email"
  endpoint  = "ops@example.com"
}
```

## Best Practices

1. **Visibility Timeout**: Set to 6x expected processing time
2. **Long Polling**: Keep at 20s to reduce empty responses
3. **DLQ Monitoring**: Alert immediately on any DLQ messages
4. **Message Size**: Keep under 256KB; use S3 for large payloads
5. **Idempotency**: Design workers to handle duplicate messages
6. **Batch Processing**: Use batch receive (up to 10 messages)
