Eve library

Oshun slim ComfyUI worker (RunPod serverless)

Other

Revision 1

On this page

The only worker image the Isis Chroma path uses. It is the stock runpod/worker-comfyui base plus one node pack. No model is baked into it.

Layer Pin
Base image runpod/worker-comfyui:5.10.0-base@sha256:e9d18d3db15839ebb10c10c824109fc2956f31ef7bf21916220c008855b0d538
ComfyUI 0.34.0 (from the base image), CUDA 12.8, Python 3.12 in /opt/venv
Node pack ComfyUI-RMBG 58f1947a11567a9f8b707223185570850e773856 (pyproject 3.1.0 + post-release fixes)
Published to ghcr.io/greychimp/oshun/runpod-comfyui-slim (private package)
Built by .github/workflows/runpod-comfyui-slim.yml on GitHub runners — never on a laptop

What is in, what is out#

A.01.09 (2026-09-14): the PozzettiAndrea/ComfyUI-SAM3 pack and ComfyUI-RMBG's vendored SAM 3 still node (py/AILab_SAM3Segment.py, models/sam3/) are removed from the image, with pycocotools, the sam3 model folder, its symlink and the two SAM 3 manifest rows. Segmentation and video tracking run on ComfyUI core's SAM 3.1 nodes (SAM3_Detect, SAM3_VideoTrack, SAM3_TrackToMask) loading checkpoints/sam3.1_multiplex_fp16.safetensors. The smoke test fails the build if any removed class registers again. Passages below that describe SAM3Segment or ComfyUI-SAM3 are kept as history.

  • In: ComfyUI core (Chroma nodes: UNETLoader, CLIPLoader type chroma, T5TokenizerOptions, ModelSamplingAuraFlow, CFGGuider, BetaSamplingScheduler, SamplerCustomAdvanced, inpaint/latent nodes) and ComfyUI-RMBG classes RMBG, BiRefNetRMBG, SAM3Segment.
  • In: pycocotools, which is in neither the pack's requirements.txt nor ComfyUI's. SAM3Segment needs it: the SAM 3 source is vendored inside the pack (models/sam3/, not a PyPI package), and importing sam3.model_builder reaches a bare import pycocotools.mask through sam3.model.sam3_video_inference -> sam3.model.sam3_video_base -> sam3.train.masks_ops. Without it the node silently vanished from /object_info for a whole release (tracker C.12.18).
  • Out: groundingdino-py (sdist needing a compiler the runtime base lacks). ComfyUI-RMBG's GroundingDINO-prompted Segment/SegmentV2 nodes are therefore unavailable; text-prompted segmentation goes through SAM3Segment instead, which does not import it. The pack loads each module in its own try/except, so the missing optional dependency does not disable the rest — AILab_SegmentV2.py is the one entry on the smoke test's allowlist of node files that are permitted to fail loading.
  • Out: any model file. See "Models" below.

Contract with the generation API#

Request (stock worker, POST https://api.runpod.ai/v2/<endpoint>/run):

json
{
  "input": {
    "workflow": { "<nodeId>": { "class_type": "...", "inputs": {} } },
    "images": [{ "name": "input.png", "image": "data:image/png;base64,..." }]
  }
}

Response (/status/<jobId> once COMPLETED):

json
{
  "output": {
    "images": [
      { "filename": "ComfyUI_00001_.png", "type": "base64", "data": "iVBOR..." }
    ],
    "errors": []
  },
  "executionTime": 34210,
  "delayTime": 91200
}

executionTime and delayTime (ms) are the billing truth; the Isis cost ledger is computed from them, never from estimates. Keys the old Oshun handler understood (inputs, workflowMetadata, workflow_id) are ignored by this worker: the generation API renders the catalog JSON to a bare graph before submitting (ISIS_RUNPOD_WORKER_KIND=worker-comfyui).

Worker environment variables that matter (set on the RunPod template):

Variable Purpose
COMFY_POLLING_MAX_RETRIES Handler polls ComfyUI this many times before giving up (set 1500)
COMFY_POLLING_INTERVAL_MS Poll interval (set 250)
COMFY_LOG_LEVEL ComfyUI verbosity (INFO)
BUCKET_ENDPOINT_URL family If set, outputs upload to S3 and type becomes s3_url (not used)
WEBSOCKET Optional live-preview socket; not used by Isis

Models#

Models are read from the network volume mounted at /runpod-volume, using ComfyUI's folder layout under /runpod-volume/models/:

Folder Contents Consumer
diffusion_models/ Chroma1-HD, UnCanny v1.3 BF16, Photonic Fusion BF16 UNETLoader
text_encoders/ t5xxl_fp16.safetensors CLIPLoader (type chroma)
vae/ ae.safetensors VAELoader
RMBG/BiRefNet/ BiRefNet-portrait/-matting/-HR-matting.safetensors + config files BiRefNetRMBG
loras/ Wan lightx2v + Animate relight; four Chroma LoRAs (C.12.01); LTX-2.3 distilled LoRA, IC-LoRA union control and motion-track (V.12.02/V.12.03) LoraLoaderModelOnly, LTXICLoRALoaderModelOnly

extra_model_paths.yaml in this directory maps every one of those folders; RMBG/ is additionally symlinked from /comfyui/models/ at container start by entry.sh (the image ships no dangling symlinks) because ComfyUI-RMBG hard-codes <models_dir>/RMBG as its cache root. The authoritative inventory, with sources, revisions and sha256, is infra/runpod/volumes/isis-chroma-models.manifest.json; the loader that seeds the volume is scripts/isis/runpod-volume-loader.py.

Rules:

  • Models → volume copy + manifest row. Custom nodes → image rebuild.
  • Pre-seed every model the catalog names. A worker that downloads at request time is spending billed GPU seconds on a network transfer.
  • No quantized weights (GGUF / fp8 / int8 / DF11). The manifest lint rejects the filename patterns.

Smoke test#

smoke-test.sh runs inside the container on CPU (CI does this after every build): boots ComfyUI, fetches /object_info, asserts every class the Isis catalog needs is registered, fails on any custom-node import failure, and writes object_info.json as a CI artifact. That artifact is what the generation API's pre-submit class check is fed from.

It is fatal to the build, and it has to stay that way. It was briefly run with set +e so that RunPod's console-only build log would not hide the reason for a failure — and in that window the image shipped a release whose REQUIRED_CLASSES listed SAM3Segment while the class was not registered at all (C.12.18). A class list is only evidence if the check can fail the build. The log problem is solved instead by dumping the entire smoke log to the build output on failure and baking build-manifest/smoke.log + smoke.status into the image, which entry.sh prints at container start where the worker log API can read it.

Two failure modes it now catches that it did not before:

  • ComfyUI-RMBG's __init__.py loads each node file inside a try/except that prints Error loading <file>: <err> and continues. ComfyUI does not count that as an import failure, so the class simply disappears. Every such line fails the smoke unless the file is on ALLOWED_LOAD_FAILURES. Two entries today, each a dependency this image drops on purpose: AILab_SegmentV2.py (groundingdino-py) and AILab_Florence2.py (matplotlib, which the pack's own requirements.txt does not list either; Florence 2 is not in the Isis catalog). The Florence 2 failure was found by this check on its first fatal build and had been happening silently in every image before it.
  • The Dockerfile additionally performs SAM3Segment's own module-level import (from sam3.model_builder import build_sam3_image_model) in its own layer, so a missing SAM 3 dependency fails with a real traceback rather than as an absent class.

Video contract (tracker V.01.04)#

Video comes back in images[]. Not a typo and not a bug to route around: SaveVideo returns PreviewVideo.as_dict() = {"images": [...], "animated": (True,)} (ComfyUI 0.34.0), and worker-comfyui's handler collects only node_output["images"]. So mp4 / webm / mkv arrive in the same array as PNGs, as {filename, type: 'base64' | 's3_url', data}. The adapter sniffs the bytes rather than trusting the extension.

Audio is always muxed, never a second artifact. SaveAudio returns under an audio key, which the handler does not collect — an audio-only save is silently dropped. LTX-2.3's track therefore has to travel through CreateVideo(images, audio) into SaveVideo. A graph that saves audio separately produces nothing.

S3 output, for clips too large to base64. With BUCKET_ENDPOINT_URL, BUCKET_ACCESS_KEY_ID and BUCKET_SECRET_ACCESS_KEY set on the endpoint, the worker uploads the file and returns type: 's3_url' instead of the bytes. Those three are attached to the endpoint from the environment by runpod-provision.mjs and are listed in desired-state.json by name only — a value there would be a committed credential.

Video input arrives through input/volume. entry.sh links /comfyui/input/volume to /runpod-volume/inputs, so a clip PUT to the volume at inputs/<jobId>/<name> is referenced by a LoadVideo node as volume/<jobId>/<name> with no download-and-re-upload round trip. Small inputs still ride input.images[], which ComfyUI writes to disk as-is under the given name (it does no content check, so a small mp4 travels that path fine).

The packs, and why each one is here#

ComfyUI 0.34.0 already ships every Wan, VACE, Animate and LTX-2.3 generator node, so no pack is needed to make video. These six exist for what core does not do:

Pack Pin Why
Kosinkadink/ComfyUI-VideoHelperSuite 4d907bee Container I/O beyond core's LoadVideo/SaveVideo (VHS_LoadVideo, VHS_VideoCombine)
Fannovel16/comfyui_controlnet_aux 59b1fc41 DWPose and Depth Anything V2 preprocessing for control-guided edits
kijai/ComfyUI-WanAnimatePreprocess 0e0b6a2a The pose/face detection Animate's driving video needs (OnnxDetectionModelLoader, PoseAndFaceDetection, DrawViTPose)
kijai/ComfyUI-GIMM-VFI 4c9a3123 Frame interpolation (DownloadAndLoadGIMMVFIModel, GIMMVFI_interpolate)
lihaoyun6/ComfyUI-FlashVSR_Ultra_Fast 4820b3f0 Video super-resolution (FlashVSRNode)
PozzettiAndrea/ComfyUI-SAM3 de0ff5d2 SAM 3 video tracking (SAM3VideoSegmentation, SAM3Propagate, SAM3VideoOutput); ComfyUI-RMBG's SAM3Segment is stills only

Three details that cost real time to establish and will cost it again if lost:

  • ComfyUI-SAM3 cannot install itself here. Its install.py and root __init__.py both import comfy_env, a pixi-based installer. At this SHA it ships no requirements.txt and no prestartup_script.py. The image deletes install.py and replaces the root __init__.py with a direct from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS — the pack's own nodes/__init__.py already assembles both. Its real dependencies were read out of nodes/comfy-env.toml into requirements-video.txt; the [cuda] extras (cc_torch, torch_generic_nms, flash-attn) are imported under try/except upstream and are deliberately absent, since flash-attn compiles and the runtime base has no compiler.
  • FlashVSR-v1.1 is a folder name, not a label. nodes.py:140-141 joins models_dir with the model combo value verbatim, and calls snapshot_download of the whole repo when the directory is missing. entry.sh creates the symlink under exactly that name.
  • Never opencv-python. Four packs request it; its GUI build links libGL, which a runtime image does not carry, so import cv2 raises libGL.so.1: cannot open shared object file. requirements-video.txt installs opencv-python-headless once instead, and no pack's own requirements file is used directly.

Registry credential and rotation#

The package is private. RunPod pulls it with a Container Registry Auth made from the GitHub username plus a personal access token with read:packages only (env-master GHCR_PULL_TOKEN; registered by scripts/isis/runpod-provision.mjs registry-auth create).

Rotation, in this order, so pulls never break:

  1. Create the new token on GitHub; add it to the env file as GHCR_PULL_TOKEN.
  2. runpod-provision.mjs registry-auth create --yes (new auth id), then endpoint update --registry-auth <newId> --yes.
  3. registry-auth delete <oldId> --yes, then revoke the old token on GitHub.

RunPod registry and endpoint landmines (2026-09-11)#

  • Images built by RunPod's git integration live in registry.runpod.net and can only be pulled by the endpoint/template that the build created. A plain template pointing at the same image name fails with Failed to get Hub registry auth. Is this a template you have access to?
  • Never change a git-built endpoint through the GraphQL saveEndpoint mutation (used once to set GPU pools): the endpoint stopped placing workers afterwards even with a queued job. Use the console or the REST PATCH /v1/endpoints/{id} for in-place fields.
  • "Active workers" (workersMin) triggers a RunPod cost warning e-mail every time it is raised; boot a worker with a queued job instead.