Eve library

Oshun SNS Topics Module

Other

Revision 1

On this page

This Terraform module creates SNS topics for event fan-out across all Oshun domains, with subscriptions to SQS queues, Lambda functions, and HTTP endpoints.

Features#

  • Standard Topics: High-throughput topics for event distribution
  • FIFO Topics: Ordered message delivery with deduplication
  • Multiple Subscriptions: SQS, Lambda, HTTP/HTTPS endpoints
  • Filter Policies: Route messages based on attributes
  • Encryption: Optional KMS encryption for topics
  • CloudWatch Alarms: Failed delivery monitoring
  • CloudWatch Dashboard: Visual monitoring for all topics

Usage#

hcl
module "sns" {
  source = "../sns"

  environment = "production"
  region      = "us-east-1"
  name_prefix = "oshun"

  # Optional: KMS encryption
  # kms_key_id = module.kms.sns_key_id

  # Add SQS subscriptions
  sqs_subscriptions = {
    generation-to-output = {
      topic_key            = "isis-generation-completed"
      queue_arn            = module.sqs.queue_arns["isis-output"]
      raw_message_delivery = true
    }
  }

  # Add Lambda subscriptions
  lambda_subscriptions = {
    notify-on-build = {
      topic_key    = "bellona-build-completed"
      function_arn = module.lambda.notify_function_arn
    }
  }
}

Default Topics#

Isis Domain#

  • isis-generation-completed: Generation job completed
  • isis-generation-failed: Generation job failed
  • isis-workflow-state-changed: Workflow state change

Sophia Domain#

  • sophia-document-ingested: Document ingested
  • sophia-embedding-completed: Embeddings generated

Hathor Domain#

  • hathor-world-updated: World modification
  • hathor-simulation-tick: Simulation tick event

Bellona Domain#

  • bellona-build-completed: Build completed
  • bellona-build-failed: Build failed
  • bellona-render-completed: Rendering completed

Yemaya Domain#

  • yemaya-project-saved: Project saved
  • yemaya-export-completed: Export completed
  • yemaya-collaboration-event: Collaboration events

Lilith Domain#

  • lilith-user-activity: User activity events
  • lilith-content-updated: Content updates
  • lilith-meditation-completed: Meditation session completed
  • lilith-notification-request: Notification requests

Topic Configuration Options#

hcl
topics = {
  my-topic = {
    # Required
    domain      = "isis"
    description = "My custom topic"

    # Optional (with defaults)
    enabled                     = true
    fifo_topic                  = false
    content_based_deduplication = false  # For FIFO only

    # Encryption
    kms_key_id = "alias/my-key"

    # Tags
    tags = {
      Custom = "value"
    }
  }
}

Subscription Types#

SQS Subscriptions#

hcl
sqs_subscriptions = {
  my-subscription = {
    topic_key            = "isis-generation-completed"
    queue_arn            = "arn:aws:sqs:us-east-1:123456789:my-queue"
    raw_message_delivery = true  # Deliver message body directly

    # Optional: Filter messages
    filter_policy = jsonencode({
      status = ["success"]
    })
  }
}

Lambda Subscriptions#

hcl
lambda_subscriptions = {
  my-handler = {
    topic_key    = "bellona-build-completed"
    function_arn = "arn:aws:lambda:us-east-1:123456789:function:my-handler"

    # Optional: Filter messages
    filter_policy = jsonencode({
      buildType = ["production"]
    })
  }
}

HTTP/HTTPS Subscriptions#

hcl
http_subscriptions = {
  webhook = {
    topic_key            = "yemaya-export-completed"
    endpoint             = "https://api.example.com/webhooks/export"
    raw_message_delivery = false
  }
}

Filter Policies#

Route messages to specific subscribers based on message attributes:

hcl
sqs_subscriptions = {
  high-priority = {
    topic_key = "isis-generation-completed"
    queue_arn = module.sqs.queue_arns["priority-queue"]
    filter_policy = jsonencode({
      priority = ["high", "critical"]
    })
  }
  low-priority = {
    topic_key = "isis-generation-completed"
    queue_arn = module.sqs.queue_arns["standard-queue"]
    filter_policy = jsonencode({
      priority = ["low", "normal"]
    })
  }
}

Outputs#

Name Description
topic_arns Map of topic names to ARNs
topic_names Map of topic keys to AWS names
topics_by_domain Topics grouped by domain
ecs_topic_environment Environment variables for ECS
publisher_policy_json IAM policy for publishing
domain_publisher_policies Per-domain IAM policies
sqs_subscription_arns SQS subscription ARNs
lambda_subscription_arns Lambda subscription ARNs
topics_summary Summary of deployed topics

Using with ECS Services#

The module outputs environment variables formatted for ECS:

hcl
module "ecs_services" {
  # ...

  services = {
    isis-generation-api = {
      environment = [
        for k, v in module.sns.ecs_topic_environment : {
          name  = v.name
          value = v.value
        } if startswith(k, "ISIS")
      ]
    }
  }
}

IAM Policies#

Attach the publisher policy to ECS task roles:

hcl
resource "aws_iam_role_policy" "service_sns" {
  name   = "sns-publish"
  role   = aws_iam_role.service_task.id
  policy = module.sns.domain_publisher_policies["isis"]
}

Event-Driven Architecture#

Example fan-out pattern:

text
                    ┌─────────────────┐
                    │  ISIS API       │
                    │  (Publisher)    │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │  SNS Topic      │
                    │  generation-    │
                    │  completed      │
                    └────────┬────────┘
                             │
          ┌──────────────────┼──────────────────┐
          │                  │                  │
          ▼                  ▼                  ▼
   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐
   │  SQS Queue  │   │  Lambda     │   │  HTTP       │
   │  (Output    │   │  (Analytics │   │  (Webhook)  │
   │  Processing)│   │  Handler)   │   │             │
   └─────────────┘   └─────────────┘   └─────────────┘

Best Practices#

  1. Topic Per Event Type: Create separate topics for different event types
  2. Filter Policies: Use filters to route to appropriate subscribers
  3. Raw Delivery: Enable for SQS to avoid double-encoding
  4. Dead Letter Queues: Configure on SQS subscribers for failed messages
  5. Idempotency: Design subscribers to handle duplicate messages
  6. Message Attributes: Add attributes for filtering and routing