The only worker image the Isis Chroma path uses. It is the stock
runpod/worker-comfyui base plus one node pack. No model is baked into it.
| Layer | Pin |
|---|---|
| Base image | runpod/worker-comfyui:5.10.0-base@sha256:e9d18d3db15839ebb10c10c824109fc2956f31ef7bf21916220c008855b0d538 |
| ComfyUI | 0.34.0 (from the base image), CUDA 12.8, Python 3.12 in /opt/venv |
| Node pack | ComfyUI-RMBG 58f1947a11567a9f8b707223185570850e773856 (pyproject 3.1.0 + post-release fixes) |
| Published to | ghcr.io/greychimp/oshun/runpod-comfyui-slim (private package) |
| Built by | .github/workflows/runpod-comfyui-slim.yml on GitHub runners — never on a laptop |
What is in, what is out#
A.01.09 (2026-09-14): the
PozzettiAndrea/ComfyUI-SAM3pack and ComfyUI-RMBG's vendored SAM 3 still node (py/AILab_SAM3Segment.py,models/sam3/) are removed from the image, with pycocotools, thesam3model folder, its symlink and the two SAM 3 manifest rows. Segmentation and video tracking run on ComfyUI core's SAM 3.1 nodes (SAM3_Detect,SAM3_VideoTrack,SAM3_TrackToMask) loadingcheckpoints/sam3.1_multiplex_fp16.safetensors. The smoke test fails the build if any removed class registers again. Passages below that describeSAM3Segmentor ComfyUI-SAM3 are kept as history.
- In: ComfyUI core (Chroma nodes:
UNETLoader,CLIPLoadertypechroma,T5TokenizerOptions,ModelSamplingAuraFlow,CFGGuider,BetaSamplingScheduler,SamplerCustomAdvanced, inpaint/latent nodes) and ComfyUI-RMBG classesRMBG,BiRefNetRMBG,SAM3Segment. - In:
pycocotools, which is in neither the pack'srequirements.txtnor ComfyUI's.SAM3Segmentneeds it: the SAM 3 source is vendored inside the pack (models/sam3/, not a PyPI package), and importingsam3.model_builderreaches a bareimport pycocotools.maskthroughsam3.model.sam3_video_inference->sam3.model.sam3_video_base->sam3.train.masks_ops. Without it the node silently vanished from/object_infofor a whole release (tracker C.12.18). - Out:
groundingdino-py(sdist needing a compiler the runtime base lacks). ComfyUI-RMBG's GroundingDINO-promptedSegment/SegmentV2nodes are therefore unavailable; text-prompted segmentation goes throughSAM3Segmentinstead, which does not import it. The pack loads each module in its owntry/except, so the missing optional dependency does not disable the rest —AILab_SegmentV2.pyis the one entry on the smoke test's allowlist of node files that are permitted to fail loading. - Out: any model file. See "Models" below.
Contract with the generation API#
Request (stock worker, POST https://api.runpod.ai/v2/<endpoint>/run):
{
"input": {
"workflow": { "<nodeId>": { "class_type": "...", "inputs": {} } },
"images": [{ "name": "input.png", "image": "data:image/png;base64,..." }]
}
}
Response (/status/<jobId> once COMPLETED):
{
"output": {
"images": [
{ "filename": "ComfyUI_00001_.png", "type": "base64", "data": "iVBOR..." }
],
"errors": []
},
"executionTime": 34210,
"delayTime": 91200
}
executionTime and delayTime (ms) are the billing truth; the Isis cost ledger
is computed from them, never from estimates. Keys the old Oshun handler
understood (inputs, workflowMetadata, workflow_id) are ignored by this
worker: the generation API renders the catalog JSON to a bare graph before
submitting (ISIS_RUNPOD_WORKER_KIND=worker-comfyui).
Worker environment variables that matter (set on the RunPod template):
| Variable | Purpose |
|---|---|
COMFY_POLLING_MAX_RETRIES |
Handler polls ComfyUI this many times before giving up (set 1500) |
COMFY_POLLING_INTERVAL_MS |
Poll interval (set 250) |
COMFY_LOG_LEVEL |
ComfyUI verbosity (INFO) |
BUCKET_ENDPOINT_URL family |
If set, outputs upload to S3 and type becomes s3_url (not used) |
WEBSOCKET |
Optional live-preview socket; not used by Isis |
Models#
Models are read from the network volume mounted at /runpod-volume, using
ComfyUI's folder layout under /runpod-volume/models/:
| Folder | Contents | Consumer |
|---|---|---|
diffusion_models/ |
Chroma1-HD, UnCanny v1.3 BF16, Photonic Fusion BF16 | UNETLoader |
text_encoders/ |
t5xxl_fp16.safetensors |
CLIPLoader (type chroma) |
vae/ |
ae.safetensors |
VAELoader |
RMBG/BiRefNet/ |
BiRefNet-portrait/-matting/-HR-matting.safetensors + config files |
BiRefNetRMBG |
loras/ |
Wan lightx2v + Animate relight; four Chroma LoRAs (C.12.01); LTX-2.3 distilled LoRA, IC-LoRA union control and motion-track (V.12.02/V.12.03) | LoraLoaderModelOnly, LTXICLoRALoaderModelOnly |
extra_model_paths.yaml in this directory maps every one of those folders;
RMBG/ is additionally symlinked from /comfyui/models/ at container start by
entry.sh (the image ships no dangling symlinks) because ComfyUI-RMBG
hard-codes <models_dir>/RMBG as its cache root. The authoritative inventory,
with sources, revisions and sha256, is
infra/runpod/volumes/isis-chroma-models.manifest.json; the loader that seeds
the volume is scripts/isis/runpod-volume-loader.py.
Rules:
- Models → volume copy + manifest row. Custom nodes → image rebuild.
- Pre-seed every model the catalog names. A worker that downloads at request time is spending billed GPU seconds on a network transfer.
- No quantized weights (GGUF / fp8 / int8 / DF11). The manifest lint rejects the filename patterns.
Smoke test#
smoke-test.sh runs inside the container on CPU (CI does this after every
build): boots ComfyUI, fetches /object_info, asserts every class the Isis
catalog needs is registered, fails on any custom-node import failure, and writes
object_info.json as a CI artifact. That artifact is what the generation API's
pre-submit class check is fed from.
It is fatal to the build, and it has to stay that way. It was briefly run
with set +e so that RunPod's console-only build log would not hide the reason
for a failure — and in that window the image shipped a release whose
REQUIRED_CLASSES listed SAM3Segment while the class was not registered at
all (C.12.18). A class list is only evidence if the check can fail the build.
The log problem is solved instead by dumping the entire smoke log to the build
output on failure and baking build-manifest/smoke.log + smoke.status into
the image, which entry.sh prints at container start where the worker log API
can read it.
Two failure modes it now catches that it did not before:
- ComfyUI-RMBG's
__init__.pyloads each node file inside atry/exceptthat printsError loading <file>: <err>and continues. ComfyUI does not count that as an import failure, so the class simply disappears. Every such line fails the smoke unless the file is onALLOWED_LOAD_FAILURES. Two entries today, each a dependency this image drops on purpose:AILab_SegmentV2.py(groundingdino-py) andAILab_Florence2.py(matplotlib, which the pack's ownrequirements.txtdoes not list either; Florence 2 is not in the Isis catalog). The Florence 2 failure was found by this check on its first fatal build and had been happening silently in every image before it. - The Dockerfile additionally performs
SAM3Segment's own module-level import (from sam3.model_builder import build_sam3_image_model) in its own layer, so a missing SAM 3 dependency fails with a real traceback rather than as an absent class.
Video contract (tracker V.01.04)#
Video comes back in images[]. Not a typo and not a bug to route around:
SaveVideo returns
PreviewVideo.as_dict() = {"images": [...], "animated": (True,)} (ComfyUI
0.34.0), and worker-comfyui's handler collects only node_output["images"]. So
mp4 / webm / mkv arrive in the same array as PNGs, as
{filename, type: 'base64' | 's3_url', data}. The adapter sniffs the bytes
rather than trusting the extension.
Audio is always muxed, never a second artifact. SaveAudio returns under an
audio key, which the handler does not collect — an audio-only save is
silently dropped. LTX-2.3's track therefore has to travel through
CreateVideo(images, audio) into SaveVideo. A graph that saves audio
separately produces nothing.
S3 output, for clips too large to base64. With BUCKET_ENDPOINT_URL,
BUCKET_ACCESS_KEY_ID and BUCKET_SECRET_ACCESS_KEY set on the endpoint, the
worker uploads the file and returns type: 's3_url' instead of the bytes. Those
three are attached to the endpoint from the environment by
runpod-provision.mjs and are listed in desired-state.json by name only —
a value there would be a committed credential.
Video input arrives through input/volume. entry.sh links
/comfyui/input/volume to /runpod-volume/inputs, so a clip PUT to the volume
at inputs/<jobId>/<name> is referenced by a LoadVideo node as
volume/<jobId>/<name> with no download-and-re-upload round trip. Small inputs
still ride input.images[], which ComfyUI writes to disk as-is under the given
name (it does no content check, so a small mp4 travels that path fine).
The packs, and why each one is here#
ComfyUI 0.34.0 already ships every Wan, VACE, Animate and LTX-2.3 generator node, so no pack is needed to make video. These six exist for what core does not do:
| Pack | Pin | Why |
|---|---|---|
Kosinkadink/ComfyUI-VideoHelperSuite |
4d907bee |
Container I/O beyond core's LoadVideo/SaveVideo (VHS_LoadVideo, VHS_VideoCombine) |
Fannovel16/comfyui_controlnet_aux |
59b1fc41 |
DWPose and Depth Anything V2 preprocessing for control-guided edits |
kijai/ComfyUI-WanAnimatePreprocess |
0e0b6a2a |
The pose/face detection Animate's driving video needs (OnnxDetectionModelLoader, PoseAndFaceDetection, DrawViTPose) |
kijai/ComfyUI-GIMM-VFI |
4c9a3123 |
Frame interpolation (DownloadAndLoadGIMMVFIModel, GIMMVFI_interpolate) |
lihaoyun6/ComfyUI-FlashVSR_Ultra_Fast |
4820b3f0 |
Video super-resolution (FlashVSRNode) |
PozzettiAndrea/ComfyUI-SAM3 |
de0ff5d2 |
SAM 3 video tracking (SAM3VideoSegmentation, SAM3Propagate, SAM3VideoOutput); ComfyUI-RMBG's SAM3Segment is stills only |
Three details that cost real time to establish and will cost it again if lost:
ComfyUI-SAM3cannot install itself here. Itsinstall.pyand root__init__.pyboth importcomfy_env, a pixi-based installer. At this SHA it ships norequirements.txtand noprestartup_script.py. The image deletesinstall.pyand replaces the root__init__.pywith a directfrom .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS— the pack's ownnodes/__init__.pyalready assembles both. Its real dependencies were read out ofnodes/comfy-env.tomlintorequirements-video.txt; the[cuda]extras (cc_torch,torch_generic_nms,flash-attn) are imported undertry/exceptupstream and are deliberately absent, sinceflash-attncompiles and the runtime base has no compiler.FlashVSR-v1.1is a folder name, not a label.nodes.py:140-141joinsmodels_dirwith the model combo value verbatim, and callssnapshot_downloadof the whole repo when the directory is missing.entry.shcreates the symlink under exactly that name.- Never
opencv-python. Four packs request it; its GUI build links libGL, which a runtime image does not carry, soimport cv2raiseslibGL.so.1: cannot open shared object file.requirements-video.txtinstallsopencv-python-headlessonce instead, and no pack's own requirements file is used directly.
Registry credential and rotation#
The package is private. RunPod pulls it with a Container Registry Auth made from
the GitHub username plus a personal access token with read:packages only
(env-master GHCR_PULL_TOKEN; registered by
scripts/isis/runpod-provision.mjs registry-auth create).
Rotation, in this order, so pulls never break:
- Create the new token on GitHub; add it to the env file as
GHCR_PULL_TOKEN. runpod-provision.mjs registry-auth create --yes(new auth id), thenendpoint update --registry-auth <newId> --yes.registry-auth delete <oldId> --yes, then revoke the old token on GitHub.
RunPod registry and endpoint landmines (2026-09-11)#
- Images built by RunPod's git integration live in
registry.runpod.netand can only be pulled by the endpoint/template that the build created. A plain template pointing at the same image name fails withFailed to get Hub registry auth. Is this a template you have access to? - Never change a git-built endpoint through the GraphQL
saveEndpointmutation (used once to set GPU pools): the endpoint stopped placing workers afterwards even with a queued job. Use the console or the RESTPATCH /v1/endpoints/{id}for in-place fields. - "Active workers" (workersMin) triggers a RunPod cost warning e-mail every time it is raised; boot a worker with a queued job instead.