# ADR-0005: Slim Worker Image and Volume-Backed Models for RunPod Serverless

**Status**: Accepted  
**Date**: 2026-09-11  
**Authors**: Isis Runtime Engineering  
**Supersedes**: the per-flavor baked-model images described in
`docker/runpod/comfyui/Dockerfile` (`MODEL_FLAVOR=sd15|sdxl|…|flagship-image`)
for every new endpoint; existing flavor entries are retained in
`infra/runpod/endpoints/inventory/` as records only.

## Context and Problem Statement

Isis needed real image generation on RunPod serverless with the full-precision
Chroma stack plus SAM 3, BiRefNet and RMBG-2.0, usable from the Isis dashboard
the same day and with spend bounded by configuration.

Findings on 2026-09-11:

- The image lineage every existing endpoint assumes, `oshunai/runpod-comfyui:*`
  on top of `oshunai/runpod-base`, has **never been published** (Docker Hub
  returns 404 for both repositories). Nothing in `desired-state.json` has ever
  run.
- That lineage bakes a per-flavor model set into the image. The
  `comfyui-flagship-image` inventory lists 47 baked artifacts. Every new model
  means a multi-hour, multi-hundred-GB image rebuild, and the build cannot run
  on the developer machine (16 GB RAM).
- RunPod's stock `runpod/worker-comfyui` image already implements the serverless
  handler contract (API-format graph in, base64/S3 images out), reads models
  from a network volume at `/runpod-volume/models/...`, and supports custom node
  packs via a derived Dockerfile.
- The user's constraints: no new third-party accounts, no public image, no
  quantized weights, cost strictly bounded.

## Decision

1. **Worker image = ComfyUI + node packs only.** `docker/runpod/comfyui-slim/`
   derives from `runpod/worker-comfyui:5.10.0-base` (pinned by tag and digest)
   and adds ComfyUI-RMBG at a pinned commit. No model file is ever copied into
   an image.
2. **Models live on a RunPod network volume**, seeded from a checked-in manifest
   (`infra/runpod/volumes/isis-chroma-models.manifest.json`: source, pinned
   revision or CivitAI file id, sha256, destination) by a stdlib-only loader
   running in a short-lived CPU pod. Adding a model is a manifest row plus a
   loader run. Only a new node pack rebuilds the image.
3. **The image is built by GitHub Actions and published to GHCR as a private
   package** (`ghcr.io/greychimp/oshun/runpod-comfyui-slim`), which the repo
   already uses for its other services. RunPod pulls it with a Container
   Registry Auth built from a read-only `read:packages` token.
4. **Every build runs an in-image smoke test before push**: ComfyUI boots on CPU
   and `/object_info` must list every node class the Isis Chroma catalog uses.
   The resulting `object_info.json` is the source of the generation API's
   pre-submit class check.
5. **Cost is bounded in desired state, not by attention**: `workersMax: 1`,
   `workersMin: 0`, `executionTimeout: 300 s`, `idleTimeout: 10 s`, plus the
   generation API's daily USD ledger (fails closed).
6. **The generation API renders catalog workflows to bare graphs itself**
   (`ISIS_RUNPOD_WORKER_KIND=worker-comfyui`), because the stock worker has no
   catalog renderer. The old handler contract remains selectable for the
   unpublished flavor images but is not the path forward.

## Alternatives Considered

- **Publish the baked flavor images.** Rejected: 50+ GB images, rebuild on every
  model change, cannot be built on the developer machine, and none of the
  flavors matched the Chroma stack anyway.
- **Docker Hub.** Rejected by the user (no new accounts). GHCR is already in
  use.
- **Public GHCR package** to avoid a pull credential. Rejected by the user; a
  read-only token is a one-time setup and keeps the package private.
- **RunPod's GitHub-integration build.** Would avoid a registry entirely, but
  requires connecting the GitHub account to RunPod and building the whole
  monorepo as context. Kept as a fallback; not needed.
- **Custom nodes on the volume instead of in the image.** Fragile (Python
  dependencies would need to be installed at cold start, on billed GPU seconds).
  Rejected.
- **Quantized Chroma (fp8/GGUF) to fit smaller GPUs.** Rejected by the user; the
  volume holds BF16/FP16 only and the manifest lint enforces it.

## Consequences

- Cold start now includes loading ~27 GB (checkpoint + T5) from the network
  volume, roughly 60–120 billed seconds per worker start. Warm images cost
  cents; idle costs nothing. The dashboard shows cold vs warm per job.
- A model referenced by a workflow but missing from the volume fails the
  pre-submit check (manifest cross-check), not at runtime.
- `groundingdino-py` is excluded from the ComfyUI-RMBG install (needs a compiler
  the runtime base lacks); the GroundingDINO-prompted `Segment` nodes are
  therefore unavailable. Text-prompted segmentation uses `SAM3Segment`.
- RMBG-2.0 carries a non-commercial license (BRIA). It is on the volume for
  evaluation; commercial use needs a BRIA agreement or switching to BiRefNet.
- Rotating the pull token is a documented three-step swap
  (`docker/runpod/comfyui-slim/README.md`).

## References

- Tracker: `ISIS_CHROMA_RUNPOD_MVP_TODOS_2026-09-11.md`
- Evidence: `docs/agents/isis-chroma-runpod-evidence.md`
- Provisioner: `scripts/isis/runpod-provision.mjs`
- Loader: `scripts/isis/runpod-volume-loader.py`
- ADR-0003 (serverless execution strategy), ADR-0004 (provisioning mechanism)
