Generated reference · ComfyUI workflow · Composition & editing

Controlled Composition

Structure-controlled SDXL generation: a reference image steers composition through the xinsir Union ControlNet (promax) as depth, pose, edge, or line art, with strength and scheduling controls for precise layout authority

Composition & editing15nodesGPU A6000out image~60sv1.0.0
workflow.json
On this page
15Graph nodes
13Node classes
14Inputs
6Models

Tags: compositioncontrolnetstructuredepthposecontrol2026-sota

Inputs (14)#

The typed parameter surface callers bind when they request this workflow. Enum options and numeric bounds are the values the workflow document declares.

promptstringrequired
What to generate inside the controlled structure
structure_imageimagerequired
Reference image whose structure (depth/pose/edges/lines) guides the composition
control_modeenumdefault depth
Which structural signal to extract and enforce
options: depthposeedgelineart
preprocessbooleandefault true
Extract the control map from the reference image; disable to feed an already-prepared control map directly
control_strengthfloatdefault 0.8min 0.0max 2.0
How strongly the structure constrains generation
control_endfloatdefault 0.8min 0.1max 1.0
Fraction of the denoise schedule under structural control; ending early frees fine detail
seedintegerdefault -1
Random seed (-1 for random)
widthintegerdefault 1024min 768max 1536
Output width
heightintegerdefault 1024min 768max 1536
Output height
stepsintegerdefault 32min 15max 60
Sampling steps
cfg_scalefloatdefault 7.0min 1.0max 15.0
Prompt adherence strength
hires_fixbooleandefault true
Two-pass hires refinement (1.5x latent upscale + low-denoise second pass)
freeubooleandefault true
FreeU v2 feature re-weighting for cleaner structure
checkpointenumdefault sdxl_base
SDXL checkpoint
options: realvis_xl_v5sdxl_base

ComfyUI node graph (15)#

The executable ComfyUI prompt graph: 15 nodes across 13 distinct node classes, wired by 21 data dependencies. Nodes tinted green come from a custom node pack this workflow declares; the rest are ComfyUI core / baked-community classes.

flowchart TD n0["1: CheckpointLoaderSimple"] n1["2: CLIPTextEncode"] n2["3: CLIPTextEncode"] n3["4: LoadImage"] n4["5: {{control_mode_configs[control_mode].preprocessor}}"] n5["6: ControlNetLoader"] n6["7: SetUnionControlNetType"] n7["8: ControlNetApplyAdvanced"] n8["9: EmptyLatentImage"] n9["10: KSampler"] n10["11: VAEDecode"] n11["12: SaveImage"] n12["freeu: FreeU_V2"] n13["hires_upscale: LatentUpscaleBy"] n14["hires_sampler: KSampler"] n0 -->|clip| n1 n0 -->|clip| n2 n3 -->|image| n4 n5 -->|control_net| n6 n1 -->|positive| n7 n2 -->|negative| n7 n6 -->|control_net| n7 n4 -->|image| n7 n12 -->|model| n9 n7 -->|positive| n9 n7 -->|negative| n9 n8 -->|latent_image| n9 n14 -->|samples| n10 n0 -->|vae| n10 n10 -->|images| n11 n0 -->|model| n12 n9 -->|samples| n13 n12 -->|model| n14 n7 -->|positive| n14 n7 -->|negative| n14 n13 -->|latent_image| n14
Data-flow DAG — scroll to zoom, drag to pan.

Nodes (15)#

1CheckpointLoaderSimplecore
ckpt_name = {{checkpoint_files[checkpoint]}} tmpl
outputs: MODELCLIPVAE
2CLIPTextEncodecore
text = {{prompt}} tmplclip = ◂ node 1 · out[1]
outputs: CONDITIONING
3CLIPTextEncodecore
text = ugly, deformed, blurry, low quality, watermark, text, signature, distorted anatomy, disfiguredclip = ◂ node 1 · out[1]
outputs: CONDITIONING
4LoadImagecore
image = {{structure_image}} tmpl
outputs: IMAGEMASK
5{{control_mode_configs[control_mode].preprocessor}}core
image = ◂ node 4 · out[0]resolution = 1024
outputs: IMAGE
6ControlNetLoadercore
control_net_name = controlnet_union_sdxl_promax.safetensors
outputs: CONTROL_NET
7SetUnionControlNetTypecore
control_net = ◂ node 6 · out[0]type = {{control_mode_configs[control_mode].union_type}} tmpl
outputs: CONTROL_NET
8ControlNetApplyAdvancedcore
positive = ◂ node 2 · out[0]negative = ◂ node 3 · out[0]control_net = ◂ node 7 · out[0]image = ◂ node 5 · out[0]strength = {{control_strength}} tmplstart_percent = 0.0end_percent = {{control_end}} tmpl
outputs: CONDITIONINGCONDITIONING
9EmptyLatentImagecore
width = {{width}} tmplheight = {{height}} tmplbatch_size = 1
outputs: LATENT
10KSamplercore
model = ◂ node freeu · out[0]positive = ◂ node 8 · out[0]negative = ◂ node 8 · out[1]latent_image = ◂ node 9 · out[0]seed = {{seed}} tmplsteps = {{steps}} tmplcfg = {{cfg_scale}} tmplsampler_name = dpmpp_2m_sdescheduler = karrasdenoise = 1.0
outputs: LATENT
11VAEDecodecore
samples = ◂ node hires_sampler · out[0]vae = ◂ node 1 · out[2]
outputs: IMAGE
12SaveImagecore
images = ◂ node 11 · out[0]filename_prefix = controlled_composition
freeuFreeU_V2core
model = ◂ node 1 · out[0]b1 = 1.1b2 = 1.2s1 = 0.6s2 = 0.4
outputs: MODEL
hires_upscaleLatentUpscaleBycore
samples = ◂ node 10 · out[0]upscale_method = bislerpscale_by = 1.5
outputs: LATENT
hires_samplerKSamplercore
model = ◂ node freeu · out[0]positive = ◂ node 8 · out[0]negative = ◂ node 8 · out[1]latent_image = ◂ node hires_upscale · out[0]seed = {{seed}} tmplsteps = 14cfg = {{cfg_scale}} tmplsampler_name = dpmpp_2mscheduler = karrasdenoise = 0.45
outputs: LATENT

Parameter banks (1)#

The prompt / configuration lookup tables this workflow keys into from its inputs — the vocabulary that turns a style / palette / preset selection into graph parameters.

control_mode_configs (4)#

depth
{"preprocessor": "DepthAnythingV2Preprocessor", "union_type": "depth", "description": "Depth-map structural guidance (Depth Anything V2)"}
pose
{"preprocessor": "DWPreprocessor", "union_type": "openpose", "description": "Whole-body pose guidance (DWPose: body + hands + face)"}
edge
{"preprocessor": "CannyEdgePreprocessor", "union_type": "canny/lineart/anime_lineart/mlsd", "description": "Hard edge guidance (Canny)"}
lineart
{"preprocessor": "LineArtPreprocessor", "union_type": "canny/lineart/anime_lineart/mlsd", "description": "Line art guidance"}

Models & dependencies#

Custom node packs (4)#

The non-core ComfyUI node classes this graph requires; the RunPod worker image the workflow runs on must bake or install a pack that provides every one of them.

CannyEdgePreprocessorDWPreprocessorDepthAnythingV2PreprocessorLineArtPreprocessor

Checkpoint aliases (2)#

realvis_xl_v5
RealVisXL_V5.0_fp16.safetensors
sdxl_base
sd_xl_base_1.0.safetensors

Models required (6)#

RealVisXL_V5.0_fp16.safetensors
controlnet_union_sdxl_promax.safetensors
depth_anything_v2_vits.pth
dw-ll_ucoco_384_bs5.torchscript.pt
sd_xl_base_1.0.safetensors
yolox_l.onnx

Output contract#

What a successful run of this workflow returns.

type
image
format
png
channels
4

Taxonomy & routing#

How the control plane classifies this workflow — from the committed workflow-taxonomy-registry.json. It drives the consistency / control surface the agentic director can exercise over the workflow.

assetFamily
structure-controlled-still
outputPackageProfile
image-single-profile
controlModalities
controlnet-depthcontrolnet-posecontrolnet-cannycontrolnet-lineartmodel-locksampler-scheduler-lockseed-lockprompt-template-lock
consistencyDimensions
environmentidentity
notes
SDXL structural control through the xinsir union controlnet (promax); one workflow covers depth/pose/edge/lineart via SetUnionControlNetType.