Status: Accepted
Date: 2026-09-11
Authors: Isis Runtime Engineering
Supersedes: the per-flavor baked-model images described in
docker/runpod/comfyui/Dockerfile (MODEL_FLAVOR=sd15|sdxl|…|flagship-image)
for every new endpoint; existing flavor entries are retained in
infra/runpod/endpoints/inventory/ as records only.
Context and Problem Statement#
Isis needed real image generation on RunPod serverless with the full-precision Chroma stack plus SAM 3, BiRefNet and RMBG-2.0, usable from the Isis dashboard the same day and with spend bounded by configuration.
Findings on 2026-09-11:
- The image lineage every existing endpoint assumes,
oshunai/runpod-comfyui:*on top ofoshunai/runpod-base, has never been published (Docker Hub returns 404 for both repositories). Nothing indesired-state.jsonhas ever run. - That lineage bakes a per-flavor model set into the image. The
comfyui-flagship-imageinventory lists 47 baked artifacts. Every new model means a multi-hour, multi-hundred-GB image rebuild, and the build cannot run on the developer machine (16 GB RAM). - RunPod's stock
runpod/worker-comfyuiimage already implements the serverless handler contract (API-format graph in, base64/S3 images out), reads models from a network volume at/runpod-volume/models/..., and supports custom node packs via a derived Dockerfile. - The user's constraints: no new third-party accounts, no public image, no quantized weights, cost strictly bounded.
Decision#
- Worker image = ComfyUI + node packs only.
docker/runpod/comfyui-slim/derives fromrunpod/worker-comfyui:5.10.0-base(pinned by tag and digest) and adds ComfyUI-RMBG at a pinned commit. No model file is ever copied into an image. - Models live on a RunPod network volume, seeded from a checked-in manifest
(
infra/runpod/volumes/isis-chroma-models.manifest.json: source, pinned revision or CivitAI file id, sha256, destination) by a stdlib-only loader running in a short-lived CPU pod. Adding a model is a manifest row plus a loader run. Only a new node pack rebuilds the image. - The image is built by GitHub Actions and published to GHCR as a private
package (
ghcr.io/greychimp/oshun/runpod-comfyui-slim), which the repo already uses for its other services. RunPod pulls it with a Container Registry Auth built from a read-onlyread:packagestoken. - Every build runs an in-image smoke test before push: ComfyUI boots on CPU
and
/object_infomust list every node class the Isis Chroma catalog uses. The resultingobject_info.jsonis the source of the generation API's pre-submit class check. - Cost is bounded in desired state, not by attention:
workersMax: 1,workersMin: 0,executionTimeout: 300 s,idleTimeout: 10 s, plus the generation API's daily USD ledger (fails closed). - The generation API renders catalog workflows to bare graphs itself
(
ISIS_RUNPOD_WORKER_KIND=worker-comfyui), because the stock worker has no catalog renderer. The old handler contract remains selectable for the unpublished flavor images but is not the path forward.
Alternatives Considered#
- Publish the baked flavor images. Rejected: 50+ GB images, rebuild on every model change, cannot be built on the developer machine, and none of the flavors matched the Chroma stack anyway.
- Docker Hub. Rejected by the user (no new accounts). GHCR is already in use.
- Public GHCR package to avoid a pull credential. Rejected by the user; a read-only token is a one-time setup and keeps the package private.
- RunPod's GitHub-integration build. Would avoid a registry entirely, but requires connecting the GitHub account to RunPod and building the whole monorepo as context. Kept as a fallback; not needed.
- Custom nodes on the volume instead of in the image. Fragile (Python dependencies would need to be installed at cold start, on billed GPU seconds). Rejected.
- Quantized Chroma (fp8/GGUF) to fit smaller GPUs. Rejected by the user; the volume holds BF16/FP16 only and the manifest lint enforces it.
Consequences#
- Cold start now includes loading ~27 GB (checkpoint + T5) from the network volume, roughly 60–120 billed seconds per worker start. Warm images cost cents; idle costs nothing. The dashboard shows cold vs warm per job.
- A model referenced by a workflow but missing from the volume fails the pre-submit check (manifest cross-check), not at runtime.
groundingdino-pyis excluded from the ComfyUI-RMBG install (needs a compiler the runtime base lacks); the GroundingDINO-promptedSegmentnodes are therefore unavailable. Text-prompted segmentation usesSAM3Segment.- RMBG-2.0 carries a non-commercial license (BRIA). It is on the volume for evaluation; commercial use needs a BRIA agreement or switching to BiRefNet.
- Rotating the pull token is a documented three-step swap
(
docker/runpod/comfyui-slim/README.md).
References#
- Tracker:
ISIS_CHROMA_RUNPOD_MVP_TODOS_2026-09-11.md - Evidence:
docs/agents/isis-chroma-runpod-evidence.md - Provisioner:
scripts/isis/runpod-provision.mjs - Loader:
scripts/isis/runpod-volume-loader.py - ADR-0003 (serverless execution strategy), ADR-0004 (provisioning mechanism)