infra app

A queue-based GPU job worker (apps/infra/gpu-dispatcher, package gpu-dispatcher-worker, description "GPU job dispatcher worker for RunPod Serverless"). Its entry point src/index.ts loads a Zod-validated environment config (src/config.ts — Redis, queue, RunPod endpoints, storage, worker, and metrics sections), constructs the GpuWorker, wires its lifecycle events to structured logging, and installs SIGTERM/SIGINT graceful-shutdown handlers. The core is src/worker.ts: GpuWorker (an EventEmitter) drains jobs from a @oshun/queue createWorker, dispatches each to RunPod via createGpuDispatcher, polls for completion with dispatcher.waitForJob, converts the dispatcher result into the worker's GpuJobResult shape (output type/MIME inference, execution metrics, RunPod metadata in src/types.ts), emits progress at 10/30/90/100%, and posts an HMAC-able result callback via fetch. Failures flow through a dead-letter queue (createDeadLetterQueue, 30-day retention) with a getFailureReason classifier (timeout / cancelled / retry-exhausted / processing-error). This is real, domain-specific infrastructure code, not a scaffold, and it ships a multi-stage Dockerfile plus an ECS/Fargate deploy/ bundle (task-definition.json, service-definition.json, Terraform).

One honest caveat worth recording: src/callbacks.ts defines a complete, unit-tested CallbackHandler — an HTTP server for inbound RunPod webhooks (/webhook/runpod + /health) with timing-safe HMAC signature validation and a genuinely careful SSRF guard on tenant-supplied callback URLs (validateCallbackUrl blocks localhost, RFC1918, link-local, cloud-metadata, and IPv4-mapped-IPv6 addresses against a host-suffix allowlist). That handler is the surface the deploy/task-definition.json health check (curl http://localhost:8080/health) and WEBHOOK_SECRET expect — but the current src/index.ts entry point starts only the queue worker and does not instantiate CallbackHandler, so the webhook/HTTP path exists and is tested but is not yet wired into the running process. Similarly, the WorkerHealthStatus / EndpointHealthStatus types in src/types.ts are declared but not yet produced by the worker.

A RunPod-to-CloudWatch metrics collector (apps/infra/runpod-metrics, package runpod-metrics-collector, description "RunPod metrics collector for CloudWatch integration"). The whole implementation lives in src/index.ts: RunPodMetricsCollector polls each configured endpoint's health through @oshun/runpod-client's health(), computes derived metrics — worker utilisation, queue saturation (queued jobs vs. available capacity), failure rate, and an estimated hourly cost from running-worker count × a per-second GPU price (DEFAULT_GPU_COST_PER_SECOND = 0.00044, ≈ $1.58/hr) — and pushes both per-endpoint and cross-endpoint aggregate MetricDatum to AWS CloudWatch via @aws-sdk/client-cloudwatch's PutMetricDataCommand, batching at 500 metrics per request. It runs in two modes selected by RUN_MODE: continuous (setInterval on a configurable collection interval, default 60s, for an ECS task) or oneshot (collect once and exit, for a scheduled Lambda). loadConfig assembles the endpoint list from RUNPOD_ENDPOINT_IDS plus named feature-flag endpoints (RUNPOD_COMFYUI_ENDPOINT_ID, _SD_, _FLUX_) and throws if none are configured. This is real, working code with domain-specific metric formulas; cost/detailed metrics are gated behind enableCostTracking / detailedMetrics flags. It is an ESM package ("type": "module") and ships no Dockerfile or deploy/ of its own — only src/ plus tests (src/index.spec.ts).