Eve library

ADR-0005: Slim Worker Image and Volume-Backed Models for RunPod Serverless

ADR

Revision 1

On this page

Status: Accepted
Date: 2026-09-11
Authors: Isis Runtime Engineering
Supersedes: the per-flavor baked-model images described in docker/runpod/comfyui/Dockerfile (MODEL_FLAVOR=sd15|sdxl|…|flagship-image) for every new endpoint; existing flavor entries are retained in infra/runpod/endpoints/inventory/ as records only.

Context and Problem Statement#

Isis needed real image generation on RunPod serverless with the full-precision Chroma stack plus SAM 3, BiRefNet and RMBG-2.0, usable from the Isis dashboard the same day and with spend bounded by configuration.

Findings on 2026-09-11:

  • The image lineage every existing endpoint assumes, oshunai/runpod-comfyui:* on top of oshunai/runpod-base, has never been published (Docker Hub returns 404 for both repositories). Nothing in desired-state.json has ever run.
  • That lineage bakes a per-flavor model set into the image. The comfyui-flagship-image inventory lists 47 baked artifacts. Every new model means a multi-hour, multi-hundred-GB image rebuild, and the build cannot run on the developer machine (16 GB RAM).
  • RunPod's stock runpod/worker-comfyui image already implements the serverless handler contract (API-format graph in, base64/S3 images out), reads models from a network volume at /runpod-volume/models/..., and supports custom node packs via a derived Dockerfile.
  • The user's constraints: no new third-party accounts, no public image, no quantized weights, cost strictly bounded.

Decision#

  1. Worker image = ComfyUI + node packs only. docker/runpod/comfyui-slim/ derives from runpod/worker-comfyui:5.10.0-base (pinned by tag and digest) and adds ComfyUI-RMBG at a pinned commit. No model file is ever copied into an image.
  2. Models live on a RunPod network volume, seeded from a checked-in manifest (infra/runpod/volumes/isis-chroma-models.manifest.json: source, pinned revision or CivitAI file id, sha256, destination) by a stdlib-only loader running in a short-lived CPU pod. Adding a model is a manifest row plus a loader run. Only a new node pack rebuilds the image.
  3. The image is built by GitHub Actions and published to GHCR as a private package (ghcr.io/greychimp/oshun/runpod-comfyui-slim), which the repo already uses for its other services. RunPod pulls it with a Container Registry Auth built from a read-only read:packages token.
  4. Every build runs an in-image smoke test before push: ComfyUI boots on CPU and /object_info must list every node class the Isis Chroma catalog uses. The resulting object_info.json is the source of the generation API's pre-submit class check.
  5. Cost is bounded in desired state, not by attention: workersMax: 1, workersMin: 0, executionTimeout: 300 s, idleTimeout: 10 s, plus the generation API's daily USD ledger (fails closed).
  6. The generation API renders catalog workflows to bare graphs itself (ISIS_RUNPOD_WORKER_KIND=worker-comfyui), because the stock worker has no catalog renderer. The old handler contract remains selectable for the unpublished flavor images but is not the path forward.

Alternatives Considered#

  • Publish the baked flavor images. Rejected: 50+ GB images, rebuild on every model change, cannot be built on the developer machine, and none of the flavors matched the Chroma stack anyway.
  • Docker Hub. Rejected by the user (no new accounts). GHCR is already in use.
  • Public GHCR package to avoid a pull credential. Rejected by the user; a read-only token is a one-time setup and keeps the package private.
  • RunPod's GitHub-integration build. Would avoid a registry entirely, but requires connecting the GitHub account to RunPod and building the whole monorepo as context. Kept as a fallback; not needed.
  • Custom nodes on the volume instead of in the image. Fragile (Python dependencies would need to be installed at cold start, on billed GPU seconds). Rejected.
  • Quantized Chroma (fp8/GGUF) to fit smaller GPUs. Rejected by the user; the volume holds BF16/FP16 only and the manifest lint enforces it.

Consequences#

  • Cold start now includes loading ~27 GB (checkpoint + T5) from the network volume, roughly 60–120 billed seconds per worker start. Warm images cost cents; idle costs nothing. The dashboard shows cold vs warm per job.
  • A model referenced by a workflow but missing from the volume fails the pre-submit check (manifest cross-check), not at runtime.
  • groundingdino-py is excluded from the ComfyUI-RMBG install (needs a compiler the runtime base lacks); the GroundingDINO-prompted Segment nodes are therefore unavailable. Text-prompted segmentation uses SAM3Segment.
  • RMBG-2.0 carries a non-commercial license (BRIA). It is on the volume for evaluation; commercial use needs a BRIA agreement or switching to BiRefNet.
  • Rotating the pull token is a documented three-step swap (docker/runpod/comfyui-slim/README.md).

References#

  • Tracker: ISIS_CHROMA_RUNPOD_MVP_TODOS_2026-09-11.md
  • Evidence: docs/agents/isis-chroma-runpod-evidence.md
  • Provisioner: scripts/isis/runpod-provision.mjs
  • Loader: scripts/isis/runpod-volume-loader.py
  • ADR-0003 (serverless execution strategy), ADR-0004 (provisioning mechanism)