22Graph nodes
21Node classes
23Inputs
10Models
Tags: videoanimationanimatediffcontrolnetmotion-controlposedepth
Inputs (23)#
The typed parameter surface callers bind when they request this workflow. Enum options and numeric bounds are the values the workflow document declares.
control_modeenumdefault depthType of ControlNet guidance
options:
depthopenposecannylineartsoftedgescribblenormal_mapsemantic_segcontrol_sequenceimage_sequenceOptional sequence of control images (poses, depth maps, etc.)
control_strengthfloatdefault 0.75min 0.3max 1.0Strength of ControlNet guidance
animation_typeenumdefault character_motionType of animation being created
options:
character_motioncamera_movementscene_dynamicsabstract_flowmeditation_sequenceyoga_posesdance_movementmotion_styleenumdefault smooth_naturalStyle of motion
options:
smooth_naturalcinematicenergeticslow_motioncontemplativerhythmicfluidmotion_intensityenumdefault moderateIntensity of motion
options:
minimalsubtlemoderatedynamicintensetemporal_consistencyenumdefault highLevel of temporal consistency between frames
options:
lowmediumhighvery_highvisual_styleenumdefault artisticVisual style of the output
options:
realisticartisticpainterlyanimestylizedetherealmysticalsubject_descriptionstringdefault spiritual figure, flowing robes, serene expressionDescription of the animated subject
environment_descriptionstringdefault mystical background, soft lighting, peaceful atmosphereDescription of the environment/background
color_paletteenumdefault warm_spiritualColor palette for the video
options:
naturalwarm_spiritualcool_serenevibrantmuted_elegantgolden_hourcosmicadd_effectsenumdefault noneAdditional visual effects
options:
nonelight_particlesenergy_auraflowing_fabricparticle_trailambient_glowsacred_symbolsframe_countintegerdefault 48min 30max 128Total number of frames
fpsintegerdefault 8max 24Frames per second (AnimateDiff motion is trained near 8fps; use interpolation downstream for higher rates)
resolutionenumdefault 720pOutput resolution
options:
480p720p1080ploop_modeenumdefault noneLoop mode for the video
options:
noneforwardpingpongseamlessseedintegerdefault -1Random seed
stepsintegerdefault 25Sampling steps
cfg_scalefloatdefault 7.5CFG scale for prompt adherence
frame_upscalebooleandefault falsePer-frame 2x enhancement before encoding (RealESRGAN 4x then 0.5 resample); compute-heavy, off by default
checkpointenumdefault dreamshaper_8SD1.5 checkpoint: DreamShaper 8 finetune (default, baked VAE) or base 1.5
options:
dreamshaper_8sd15_basemotion_adapterbooleandefault trueAnimateDiff v3 domain-adapter LoRA (cleans motion-training artifacts)
smooth_motionbooleandefault falseRIFE 4.9 frame interpolation (2x): doubles effective fps before encoding
ComfyUI node graph (22)#
The executable ComfyUI prompt graph: 22 nodes across 21 distinct node classes, wired by 24 data dependencies. Nodes tinted green come from a custom node pack this workflow declares; the rest are ComfyUI core / baked-community classes.
flowchart TD
n0["1: CheckpointLoaderSimple"]
n1["2: ControlNetLoader"]
n2["3: IsisLoadImageList"]
n3["4: {{control_mode_configs[control_mode].preprocessor}}"]
n4["5: CLIPTextEncode"]
n5["6: CLIPTextEncode"]
n6["7: ControlNetApply"]
n7["8: EmptyLatentImage"]
n8["9: ADE_StandardUniformContextOptions"]
n9["10: ADE_MultivalDynamic"]
n10["11: ADE_AnimateDiffLoaderGen1"]
n11["12: KSampler"]
n12["13: VAEDecode"]
n13["14: TemporalConsistencyProcessor"]
n14["15: LoopProcessor"]
n15["16: VHS_VideoCombine"]
n16["17: SaveImage"]
n17["fups_loader: UpscaleModelLoader"]
n18["fups_apply: ImageUpscaleWithModel"]
n19["fups_scale: ImageScaleBy"]
n20["ad_adapter: LoraLoaderModelOnly"]
n21["rife: RIFE VFI"]
n2 -->|image| n3
n0 -->|clip| n4
n0 -->|clip| n5
n4 -->|conditioning| n6
n1 -->|control_net| n6
n3 -->|image| n6
n20 -->|model| n10
n8 -->|context_options| n10
n9 -->|scale_multival| n10
n10 -->|model| n11
n6 -->|positive| n11
n5 -->|negative| n11
n7 -->|latent_image| n11
n11 -->|samples| n12
n0 -->|vae| n12
n12 -->|images| n13
n13 -->|images| n14
n19 -->|images| n15
n3 -->|images| n16
n17 -->|upscale_model| n18
n21 -->|image| n18
n18 -->|image| n19
n0 -->|model| n20
n14 -->|frames| n21
classDef custom fill:#12b886,stroke:#0b6b4f,color:#04110c;
class n2,n8,n9,n10,n13,n14,n15,n21 custom;
Nodes (22)#
1CheckpointLoaderSimplecoreckpt_name = {{checkpoint_files[checkpoint]}} tmploutputs:
MODELCLIPVAE2ControlNetLoadercorecontrol_net_name = {{control_mode_configs[control_mode].model}}.safetensors tmploutputs:
CONTROL_NET3IsisLoadImageListcustom packfilenames = {{control_sequence}} tmplframe_count = {{frame_count}} tmploutputs:
IMAGE4{{control_mode_configs[control_mode].preprocessor}}coreimage = ◂ node 3 · out[0]resolution = {{resolution_map[resolution].width}} tmploutputs:
IMAGE5CLIPTextEncodecoretext = {{constructed_prompt}} tmplclip = ◂ node 1 · out[1]outputs:
CONDITIONING6CLIPTextEncodecoretext = ugly, blurry, low quality, distorted, deformed, mutated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed, missing limbs, out of frame, poorly drawn, jittery, flickering, inconsistentclip = ◂ node 1 · out[1]outputs:
CONDITIONING7ControlNetApplycoreconditioning = ◂ node 5 · out[0]control_net = ◂ node 2 · out[0]image = ◂ node 4 · out[0]strength = {{control_strength}} tmploutputs:
CONDITIONING8EmptyLatentImagecorewidth = {{resolution_map[resolution].width}} tmplheight = {{resolution_map[resolution].height}} tmplbatch_size = {{frame_count}} tmploutputs:
LATENT9ADE_StandardUniformContextOptionscustom packcontext_length = 16context_stride = 1context_overlap = 4fuse_method = pyramiduse_on_equal_length = falsestart_percent = 0.0guarantee_steps = 1outputs:
CONTEXT_OPTIONS10ADE_MultivalDynamiccustom packfloat_val = {{motion_intensity_configs[motion_intensity].motion_scale}} tmploutputs:
MULTIVAL11ADE_AnimateDiffLoaderGen1custom packmodel = ◂ node ad_adapter · out[0]model_name = v3_sd15_mm.ckptbeta_schedule = autoselectcontext_options = ◂ node 9 · out[0]scale_multival = ◂ node 10 · out[0]outputs:
MODEL12KSamplercoremodel = ◂ node 11 · out[0]positive = ◂ node 7 · out[0]negative = ◂ node 6 · out[0]latent_image = ◂ node 8 · out[0]seed = {{seed}} tmplsteps = {{steps}} tmplcfg = {{cfg_scale}} tmplsampler_name = dpmpp_2mscheduler = karrasdenoise = 1.0outputs:
LATENT13VAEDecodecoresamples = ◂ node 12 · out[0]vae = ◂ node 1 · out[2]outputs:
IMAGE14TemporalConsistencyProcessorcustom packimages = ◂ node 13 · out[0]consistency_strength = {{temporal_consistency_configs[temporal_consistency].strength}} tmplblend_frames = {{temporal_consistency_configs[temporal_consistency].blend_frames}} tmploutputs:
IMAGE15LoopProcessorcustom packimages = ◂ node 14 · out[0]loop_mode = {{loop_mode}} tmplblend_frames = 8outputs:
IMAGE16VHS_VideoCombinecustom packimages = ◂ node fups_scale · out[0]frame_rate = {{int(fps) * 2 if smooth_motion else int(fps)}} tmplloop_count = 0filename_prefix = animatediff_controlnetformat = video/h264-mp4save_output = trueoutputs:
VHS_FILENAMES17SaveImagecoreimages = ◂ node 4 · out[0]filename_prefix = control_sequence_previewfups_loaderUpscaleModelLoadercoremodel_name = RealESRGAN_x4plus.pthoutputs:
UPSCALE_MODELfups_applyImageUpscaleWithModelcoreupscale_model = ◂ node fups_loader · out[0]image = ◂ node rife · out[0]outputs:
IMAGEfups_scaleImageScaleBycoreimage = ◂ node fups_apply · out[0]upscale_method = lanczosscale_by = 0.5outputs:
IMAGEad_adapterLoraLoaderModelOnlycoremodel = ◂ node 1 · out[0]lora_name = v3_sd15_adapter.ckptstrength_model = 0.7outputs:
MODELrifeRIFE VFIcustom packckpt_name = rife49.pthframes = ◂ node 15 · out[0]clear_cache_after_n_frames = 10multiplier = 2fast_mode = trueensemble = truescale_factor = 1.0dtype = float32torch_compile = falsebatch_size = 1outputs:
IMAGEPrompt construction#
templateanimated video, {animation_prompt}, {motion_prompt}, {style_prompt}, {color_prompt}, {subject}, {environment}{effects_clause}, high quality animation, smooth motion, professional videoVariables (7)#
animation_prompt{{animation_type_prompts[animation_type]}} tmplmotion_prompt{{motion_style_prompts[motion_style]}} tmplstyle_prompt{{style_prompts[visual_style]}} tmplcolor_prompt{{color_prompts[color_palette]}} tmplsubject{{subject_description}} tmplenvironment{{environment_description}} tmpleffects_clause{{effects_prompts[add_effects]}} tmplParameter banks (11)#
The prompt / configuration lookup tables this workflow keys into from its inputs — the vocabulary that turns a style / palette / preset selection into graph parameters.
control_mode_configs (8)#
depth{"model": "control_v11f1p_sd15_depth_fp16", "preprocessor": "MiDaS-DepthMapPreprocessor", "description": "Depth-based structural guidance"}openpose{"model": "control_v11p_sd15_openpose_fp16", "preprocessor": "OpenposePreprocessor", "description": "Pose-based character guidance"}canny{"model": "control_v11p_sd15_canny_fp16", "preprocessor": "CannyEdgePreprocessor", "description": "Edge-based structural guidance"}lineart{"model": "control_v11p_sd15_lineart_fp16", "preprocessor": "LineArtPreprocessor", "description": "Line art based guidance"}softedge{"model": "control_v11p_sd15_softedge_fp16", "preprocessor": "CannyEdgePreprocessor", "description": "Soft edge guidance"}scribble{"model": "control_v11p_sd15_scribble_fp16", "preprocessor": "CannyEdgePreprocessor", "description": "Scribble-based loose guidance"}normal_map{"model": "control_v11p_sd15_normalbae_fp16", "preprocessor": "CannyEdgePreprocessor", "description": "Normal map based 3D guidance"}semantic_seg{"model": "control_v11p_sd15_seg_fp16", "preprocessor": "CannyEdgePreprocessor", "description": "Semantic segmentation guidance"}animation_type_prompts (7)#
character_motionanimated character, figure in motion, body movement, living pose sequencecamera_movementcamera movement animation, panning shot, zoom effect, cinematic motionscene_dynamicsdynamic scene, moving elements, environmental animation, active atmosphereabstract_flowabstract flowing animation, shape morphing, fluid transformation, visual flowmeditation_sequencemeditation sequence, contemplative movement, peaceful transitions, mindful animationyoga_posesyoga pose sequence, asana transitions, body awareness movement, flowing practicedance_movementdance movement animation, rhythmic motion, expressive body movement, fluid choreographymotion_style_prompts (7)#
smooth_naturalsmooth natural motion, organic movement, lifelike animation, realistic dynamicscinematiccinematic motion, film-quality movement, professional animation, movie-like dynamicsenergeticenergetic motion, dynamic movement, vibrant animation, lively dynamicsslow_motionslow motion, time-stretched movement, deliberate animation, contemplative pacecontemplativecontemplative motion, thoughtful movement, meditative animation, peaceful pacerhythmicrhythmic motion, pulse-like movement, beat-synchronized animation, musical dynamicsfluidfluid motion, liquid-like movement, graceful animation, flowing dynamicsstyle_prompts (7)#
realisticrealistic style, photorealistic, lifelike, detailed renderingartisticartistic style, creative interpretation, aesthetic quality, expressive renderingpainterlypainterly style, brush stroke texture, artistic rendering, painted lookanimeanime style, Japanese animation aesthetic, stylized characters, vibrant colorsstylizedstylized rendering, artistic interpretation, unique visual style, creative looketherealethereal style, dreamlike quality, spiritual aesthetic, otherworldly renderingmysticalmystical style, magical atmosphere, enchanted aesthetic, supernatural qualitycolor_prompts (7)#
naturalnatural color palette, realistic tones, organic colorswarm_spiritualwarm spiritual palette, golden amber tones, enlightened colorscool_serenecool serene palette, blue teal tones, peaceful colorsvibrantvibrant color palette, saturated tones, energetic colorsmuted_elegantmuted elegant palette, sophisticated tones, refined colorsgolden_hourgolden hour palette, sunset tones, warm amber colorscosmiccosmic color palette, space colors, nebula tones, stellar hueseffects_prompts (7)#
nonelight_particles, with floating light particles, ambient sparkles, magical motesenergy_aura, with energy aura, glowing outline, spiritual emanationflowing_fabric, with flowing fabric, billowing cloth, dynamic draperyparticle_trail, with particle trails, motion traces, energy pathsambient_glow, with ambient glow, soft luminosity, gentle radiancesacred_symbols, with floating sacred symbols, spiritual iconography, mystical signsmotion_intensity_configs (5)#
minimal{"motion_scale": 0.4, "context_overlap": 6}subtle{"motion_scale": 0.6, "context_overlap": 5}moderate{"motion_scale": 0.8, "context_overlap": 4}dynamic{"motion_scale": 1.0, "context_overlap": 3}intense{"motion_scale": 1.3, "context_overlap": 2}temporal_consistency_configs (4)#
low{"strength": 0.4, "blend_frames": 2}medium{"strength": 0.6, "blend_frames": 4}high{"strength": 0.8, "blend_frames": 6}very_high{"strength": 0.95, "blend_frames": 8}resolution_map (3)#
480p{"width": 512, "height": 288}720p{"width": 768, "height": 432}1080p{"width": 960, "height": 544}preset_configurations (5)#
meditation_guide{"control_mode": "openpose", "animation_type": "meditation_sequence", "motion_style": "contemplative", "motion_intensity": "subtle", "visual_style": "ethereal", "color_palette": "warm_spiritual", "add_effects": "ambient_glow", "description": "Guided meditation with pose sequences"}yoga_practice{"control_mode": "openpose", "animation_type": "yoga_poses", "motion_style": "smooth_natural", "motion_intensity": "moderate", "visual_style": "artistic", "color_palette": "natural", "description": "Yoga pose transition sequence"}spiritual_dance{"control_mode": "openpose", "animation_type": "dance_movement", "motion_style": "fluid", "motion_intensity": "dynamic", "visual_style": "mystical", "color_palette": "cosmic", "add_effects": "energy_aura", "description": "Mystical dance animation"}depth_animation{"control_mode": "depth", "animation_type": "scene_dynamics", "motion_style": "cinematic", "motion_intensity": "moderate", "visual_style": "realistic", "color_palette": "natural", "description": "Depth-guided scene animation"}abstract_transformation{"control_mode": "canny", "animation_type": "abstract_flow", "motion_style": "fluid", "motion_intensity": "dynamic", "visual_style": "stylized", "color_palette": "vibrant", "add_effects": "particle_trail", "description": "Abstract edge-guided transformation"}usage_notes (2)#
control_sequence_creationFor pose sequences: Use OpenPose to extract poses from reference videoFor depth sequences: Use MiDaS or ZoeDepth on reference framesFor edge sequences: Use Canny or LineArt on reference framesbest_practicesMatch control sequence frame count to desired output frame countUse higher control_strength for more precise followingLower motion_intensity if result is too chaoticIncrease temporal_consistency if flickering occursModels & dependencies#
Custom node packs (12)#
The non-core ComfyUI node classes this graph requires; the RunPod worker image the workflow runs on must bake or install a pack that provides every one of them.
ADE_AnimateDiffLoaderGen1ADE_MultivalDynamicADE_StandardUniformContextOptionsCannyEdgePreprocessorIsisLoadImageListLineArtPreprocessorLoopProcessorMiDaS-DepthMapPreprocessorOpenposePreprocessorRIFE VFITemporalConsistencyProcessorVHS_VideoCombineCheckpoint aliases (2)#
dreamshaper_8DreamShaper_8_pruned.safetensorssd15_basev1-5-pruned-emaonly.safetensorsModels required (10)#
DreamShaper_8_pruned.safetensorsRealESRGAN_x4plus.pthcontrol_v11f1p_sd15_depth_fp16.safetensorscontrol_v11p_sd15_canny_fp16.safetensorscontrol_v11p_sd15_lineart_fp16.safetensorscontrol_v11p_sd15_openpose_fp16.safetensorsrife49.pthv1-5-pruned-emaonly.safetensorsv3_sd15_adapter.ckptv3_sd15_mm.ckptOutput contract#
What a successful run of this workflow returns.
typevideoformatmp4codech264audiofalserender_passes{'pass': 'beauty', 'format': 'mp4', 'required': True, 'description': 'Final beauty output for editorial ingest'}{'pass': 'depth', 'format': 'exr', 'required': True, 'description': 'Depth pass for compositing and relighting'}{'pass': 'mask', 'format': 'png', 'required': False, 'description': 'Object/subject mask pass for selective grading'}{'pass': 'normals', 'format': 'exr', 'required': False, 'description': 'Surface normal pass for advanced post-effects'}editorial_package{"edl": {"artifact_id": "edl", "path_template": "editorial/{{job_id}}/timeline.edl", "format": "edl", "fps": 24, "drop_frame": false, "timeline_start_timecode": "01:00:00:00"}, "shot_manifest": {"artifact_id": "shot_manifest", "path_template": "editorial/{{job_id}}/shot-manifest.json", "format": "json", "manifest_version": "1.0.0", "include_render_pass_references": true}, "timecode_map": {"artifact_id": "timecode_map", "path_template": "editorial/{{job_id}}/timecode-map.csv", "format": "csv", "include_source_frame_ranges": true}}descriptionControlNet-guided animated videoTaxonomy & routing#
How the control plane classifies this workflow — from the committed workflow-taxonomy-registry.json. It drives the consistency / control surface the agentic director can exercise over the workflow.
assetFamilyfilm-previs-blocking-reeloutputPackageProfilevideo-master-profilecontrolModalitiescamera-lockcamera-path-guidecolor-script-lockcontrolnet-cannycontrolnet-depthcontrolnet-lineartcontrolnet-normalcontrolnet-posecontrolnet-scribblecontrolnet-segmentationcontrolnet-softedgedepth-constraintidentity-adapteridentity-lockkeyframe-anchor-locklatent-reuse +14 moreconsistencyDimensionsidentitymotionlightingcolor-scriptenvironmentnotesAuto-mapped from video defaults