Model family compatibility matrix covering loaders

ComfyUI Model Compatibility Matrix

Stable Diffusion 1.5 (SD 1.5)

Overview

The original widely-adopted Stable Diffusion model. Huge ecosystem of fine-tunes, LoRAs, ControlNets, and embeddings. Still the most compatible and lightweight model family.

Configuration

Parameter	Value
Loader	CheckpointLoaderSimple
Native Resolution	512x512
Supported Resolutions	512x512, 512x768, 768x512, 768x768 (some fine-tunes)
VAE	Built-in or external (vae-ft-mse-840000-ema-pruned.safetensors)
CLIP	Single CLIP-L (output index 1 from checkpoint)
Text Encoder Node	CLIPTextEncode
CFG Range	7-12 (typical: 7.5)
Negative Prompt	Yes — very important for quality
Steps	20-30 (standard samplers)
Sampler	All standard samplers: euler, euler_ancestral, dpmpp_2m, dpmpp_sde, ddim
Scheduler	normal, karras
Denoise	1.0 (txt2img), 0.5-0.8 (img2img)
VRAM (FP16)	~2-3GB

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=512, height=512) → LATENT
KSampler (cfg=7.5, steps=20, sampler="euler", scheduler="normal") → LATENT
VAEDecode → IMAGE
SaveImage

VAE Notes

• Most SD 1.5 checkpoints have a built-in VAE, but it's often mediocre
• Recommended: Use external vae-ft-mse-840000-ema-pruned.safetensors for better color accuracy
• Load via VAELoader node and connect to VAEDecode
• FP16 VAE can produce NaN on some images. FP32 VAE is more stable

ControlNet Compatibility

SD 1.5 has the largest ControlNet ecosystem:

ControlNet	Model File Pattern	Notes
Canny	control_v11p_sd15_canny	Edge detection
Depth	control_v11f1p_sd15_depth	Depth map
OpenPose	control_v11p_sd15_openpose	Skeleton/pose
Scribble	control_v11p_sd15_scribble	Hand-drawn lines
Lineart	control_v11p_sd15_lineart	Clean lines
Softedge	control_v11p_sd15_softedge	Soft edges (HED)
Normal	control_v11p_sd15_normalbae	Normal maps
Seg	control_v11p_sd15_seg	Semantic segmentation
Tile	control_v11f1e_sd15_tile	Tile/upscale guidance
Inpaint	control_v11p_sd15_inpaint	Inpainting guidance
IP-Adapter	ip-adapter_sd15	Image prompt

LoRA Compatibility

• SD 1.5 LoRAs ONLY work with SD 1.5 base models
• Format: .safetensors in models/loras/
• Loader: LoraLoader node, which connects between checkpoint and CLIPTextEncode
• Strength range: 0.5-1.0 (higher can cause artifacts)

---

SDXL (Stable Diffusion XL)

Overview

Major upgrade from SD 1.5 with dual CLIP encoders, higher native resolution, and better prompt understanding. Includes Turbo and Lightning variants for fast generation.

Configuration — SDXL 1.0 (Base)

Parameter	Value
Loader	CheckpointLoaderSimple
Native Resolution	1024x1024
Supported Resolutions	1024x1024, 832x1216, 1216x832, 896x1152, 1152x896, 768x1344, 1344x768
VAE	Built-in (SDXL has good integrated VAE)
CLIP	Dual CLIP: CLIP-L + CLIP-G
Text Encoder Node	CLIPTextEncode (unified) or CLIPTextEncodeSDXL (separate G/L)
CFG Range	5-10 (typical: 7.0)
Negative Prompt	Yes — moderately important
Steps	20-40
Sampler	euler, euler_ancestral, dpmpp_2m, dpmpp_sde
Scheduler	normal, karras
Denoise	1.0 (txt2img), 0.5-0.8 (img2img)
VRAM (FP16)	~6-7GB

Configuration — SDXL Turbo

Parameter	Value
Loader	CheckpointLoaderSimple
Resolution	512x512 (optimized for lower res)
CFG	1.0-2.0
Steps	1-4
Sampler	euler_ancestral
Scheduler	normal
Negative Prompt	Minimal or empty
Denoise	1.0

Configuration — SDXL Lightning

Parameter	Value
Loader	CheckpointLoaderSimple + LoraLoader (Lightning LoRA)
Resolution	1024x1024
CFG	1.0-2.0
Steps	4-8 (match the Lightning variant: 2-step, 4-step, 8-step)
Sampler	euler
Scheduler	sgm_uniform
Negative Prompt	Empty or minimal
Special	Requires matching Lightning LoRA for the step count

SDXL Refiner

The optional SDXL refiner model does a second pass to improve fine details:

CheckpointLoaderSimple (base) → KSampler (steps=25, start=0, end=20)
CheckpointLoaderSimple (refiner) → KSampler (steps=25, start=20, end=25)

• The refiner uses KSamplerAdvanced with start_at_step and end_at_step
• Typically run the base for 80% of steps, refiner for the last 20%
• Refiner checkpoint: sd_xl_refiner_1.0.safetensors

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=1024, height=1024) → LATENT
KSampler (cfg=7.0, steps=25, sampler="dpmpp_2m", scheduler="karras") → LATENT
VAEDecode → IMAGE
SaveImage

ControlNet Compatibility

SDXL ControlNets are separate from SD 1.5 ControlNets:

ControlNet	Model File Pattern	Notes
Canny	control-lora-canny-rank256 or diffusers_xl_canny	Often LoRA-based
Depth	control-lora-depth-rank256 or diffusers_xl_depth
T2I-Adapter	t2i-adapter-*-sdxl	Lighter alternative to ControlNet
IP-Adapter	ip-adapter_sdxl	Image prompt adapter
InstantID	instantid-*	Face-specific

LoRA Compatibility

• SDXL LoRAs ONLY work with SDXL base models, NOT with SD 1.5
• Same LoraLoader node as SD 1.5
• Lightning LoRAs are SDXL LoRAs that enable few-step generation

---

Flux (Flux.1)

Overview

Black Forest Labs' model with a T5-XXL text encoder. Produces high-quality images without negative prompts. Available in schnell (fast) and dev (quality) variants.

Configuration — Flux Schnell

Parameter	Value
Loader	CheckpointLoaderSimple (single-file) or DualCLIPLoader + UNETLoader + VAELoader (split)
Native Resolution	1024x1024 (flexible aspect ratios)
Supported Resolutions	Flexible: 512x512 to 2048x2048, any aspect ratio
VAE	Separate Flux VAE (ae.safetensors) — NOT shared with SD models
CLIP	T5-XXL + CLIP-L via DualCLIPLoader
Text Encoder Node	CLIPTextEncode (single combined)
CFG	1.0 (MUST be 1.0 — higher values cause severe artifacts)
Negative Prompt	NONE — do not connect negative conditioning
Steps	4
Sampler	euler
Scheduler	simple or sgm_uniform
Denoise	1.0
VRAM (FP16)	~24GB (FP8: ~12GB)

Configuration — Flux Dev

Parameter	Value
Same as Schnell except:
Steps	20-50 (typical: 30)
Scheduler	sgm_uniform
VRAM (FP16)	~24GB (FP8: ~12GB)

Loading Methods

Method 1: Single Checkpoint (simplest)

CheckpointLoaderSimple (ckpt_name="flux1-schnell.safetensors")
  → MODEL(0), CLIP(1), VAE(2)

Method 2: Split Components (recommended for FP8)

UNETLoader (unet_name="flux1-schnell-fp8.safetensors") → MODEL
DualCLIPLoader (clip_name1="t5xxl_fp16.safetensors", clip_name2="clip_l.safetensors", type="flux") → CLIP
VAELoader (vae_name="ae.safetensors") → VAE

CRITICAL Rules

• CFG MUST be 1.0. Flux uses guidance embedded in the model, not classifier-free guidance
• No negative prompt. Empty string or don't connect the negative input at all
• Separate VAE required. Flux uses its own VAE (ae.safetensors), not SD VAEs
• FP8 strongly recommended for 24GB cards. FP16 Flux barely fits in 24GB VRAM
• T5-XXL encoder can be loaded in FP8 to save additional VRAM
• Kitchen quant column (this GPU): kitchen action:"status" reports gpu.fp8 (SM ≥ 8.9, Ada), gpu.nvfp4 and gpu.mxfp8 (SM ≥ 10.0, Blackwell). A UNETLoader on weight_dtype: default with an unquantized file and kitchen present is panel_kitchen action:"assess" rec fp8_unet_fast (set fp8_e4m3fn_fast). An NVFP4 sibling on disk is rec nvfp4_swap. MXFP8 is reported in status but not recommended until a loader path exposes it.

Workflow Pattern

UNETLoader (flux fp8) → MODEL
DualCLIPLoader (t5xxl + clip_l, type="flux") → CLIP
VAELoader (ae.safetensors) → VAE

CLIPTextEncode (positive prompt) → CONDITIONING
  (no negative CLIPTextEncode needed)

EmptyLatentImage (width=1024, height=1024) → LATENT

KSampler (cfg=1.0, steps=4, sampler="euler", scheduler="simple") → LATENT
VAEDecode (vae from VAELoader) → IMAGE
SaveImage

ControlNet Compatibility

Flux ControlNets are model-specific:

ControlNet	Notes
Flux ControlNet (Canny)	Specific Flux-compatible ControlNet
Flux ControlNet (Depth)	Specific Flux-compatible ControlNet
InstantX ControlNets	Community Flux ControlNets
Flux IP-Adapter	Image prompt for Flux

SD 1.5 and SDXL ControlNets do NOT work with Flux.

LoRA Compatibility

• Flux LoRAs ONLY work with Flux models
• Typically loaded via LoraLoader same as SD models
• Flux LoRA ecosystem is smaller than SD 1.5/SDXL but growing
• Some Flux LoRAs require specific trigger words

---

Stable Diffusion 3 / 3.5 (SD3)

Overview

Stability AI's next-generation model with triple CLIP architecture. Better prompt adherence and longer prompt support via T5-XXL.

Configuration

Parameter	Value
Loader	CheckpointLoaderSimple or triple-clip loader
Native Resolution	1024x1024
VAE	Built-in (integrated)
CLIP	Triple: CLIP-L + CLIP-G + T5-XXL
Text Encoder Node	CLIPTextEncode or CLIPTextEncodeSD3
CFG Range	4-7 (typical: 5.0)
Negative Prompt	Minimal — SD3 needs very little negative guidance
Steps	20-30
Sampler	euler, dpmpp_2m
Scheduler	sgm_uniform, normal
Denoise	1.0 (txt2img)
Shift	Some samplers support a shift parameter for SD3
VRAM (FP16)	~12GB (without T5-XXL: ~6GB)

Triple CLIP Loading

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)

Or for separate CLIP control:

DualCLIPLoader (clip_l + clip_g) → CLIP
CLIPLoader (t5xxl) → CLIP

Key Differences from SD 1.5/SDXL

• Much better text rendering capabilities
• Handles spatial relationships better ("cat on the left, dog on the right")
• T5-XXL enables long, detailed prompts (no 77-token limit concern)
• Lower CFG values (4-7 vs 7-12)
• Minimal negative prompting needed
• shift parameter in sampling affects noise schedule

ControlNet Compatibility

• SD3-specific ControlNets are limited
• Check for SD3-compatible community ControlNets
• SD 1.5 and SDXL ControlNets do NOT work with SD3

---

LTXV (Video Models)

Overview

Latent video diffusion models for text-to-video and image-to-video generation. VRAM-intensive.

Configuration

Parameter	Value
Loader	Special video checkpoint loader (varies by node pack)
Resolution	512x512 or 768x768 per frame (depends on model)
Frames	16-64 (depends on VRAM)
FPS	8-24
VRAM	20GB+ FP16, ~6-10GB FP8
Key Warning	Can OOM on 24GB VRAM — always use FP8 quantized models

VRAM Management

• Always use FP8 quantized models on 24GB cards
• Reduce frame count if OOM persists
• Lower resolution helps a lot
• Close other GPU-using applications
• Consider --lowvram flag for ComfyUI

---

Cross-Family Compatibility Rules

LoRA Compatibility

LoRAs are model-family specific and are NOT interchangeable:

LoRA Trained For	Works With	Does NOT Work With
SD 1.5	SD 1.5 and its fine-tunes	SDXL, Flux, SD3
SDXL	SDXL and its fine-tunes	SD 1.5, Flux, SD3
Flux	Flux models only	SD 1.5, SDXL, SD3
SD3	SD3/3.5 models only	SD 1.5, SDXL, Flux

Using a LoRA with the wrong base model will produce garbage images or errors.

ControlNet Compatibility

ControlNets are also model-family specific:

ControlNet Trained For	Works With	Does NOT Work With
SD 1.5 (v1.1 series)	SD 1.5 base + fine-tunes	SDXL, Flux, SD3
SDXL	SDXL base + fine-tunes	SD 1.5, Flux, SD3
Flux	Flux models only	SD 1.5, SDXL, SD3

VAE Compatibility

VAE	Compatible Models	Notes
vae-ft-mse-840000-ema-pruned	SD 1.5 family	Best external VAE for SD 1.5
SDXL built-in VAE	SDXL family	Good quality, no external needed
sdxl_vae.safetensors	SDXL family	External SDXL VAE option
ae.safetensors (Flux VAE)	Flux only	Required for Flux, incompatible with SD
SD3 built-in VAE	SD3 family	Integrated, no external needed

Rule: Never mix VAEs across model families. An SD 1.5 VAE decoding Flux latents will produce garbage.

Embedding/Textual Inversion Compatibility

Embedding Type	Compatible Models
SD 1.5 embeddings	SD 1.5 family only
SDXL embeddings	SDXL family only
Flux/SD3	Generally don't use traditional embeddings

Sampler/Scheduler Compatibility

Most samplers work across all models, but some combinations are optimal:

Model	Best Sampler	Best Scheduler	Notes
SD 1.5	euler_ancestral, dpmpp_2m	karras, normal	All standard samplers work
SDXL	dpmpp_2m, euler	karras, normal	Same as SD 1.5
SDXL Turbo	euler_ancestral	normal	Must use 1-4 steps
SDXL Lightning	euler	sgm_uniform	Must match step count to LoRA
Flux Schnell	euler	simple	4 steps only
Flux Dev	euler	sgm_uniform	20-50 steps
SD3	euler, dpmpp_2m	sgm_uniform, normal	Lower CFG needed

Quick Decision Guide

Choosing a Model

Use Case	Recommended Model	Why
Maximum ecosystem/community support	SD 1.5	Most LoRAs, ControlNets, embeddings
High quality, good prompt following	SDXL	Best balance of quality and ecosystem
Fastest generation	SDXL Turbo/Lightning	1-4 steps
Best prompt understanding	Flux Dev	T5-XXL encoder, natural language
Fast + good quality	Flux Schnell	4 steps, no negative needed
Text in images	SD3.5	Best text rendering
Low VRAM (<6GB)	SD 1.5	Smallest memory footprint
Video generation	LTXV / AnimateDiff	Only options for video

Choosing Resolution

Model	Minimum	Recommended	Maximum (before OOM on 24GB)
SD 1.5	256x256	512x512	768x768
SDXL	512x512	1024x1024	1536x1536
Flux (FP8)	512x512	1024x1024	2048x2048
Flux (FP16)	512x512	1024x1024	1024x1024 (tight)
SD3	512x512	1024x1024	1536x1536

Going below the recommended resolution produces blurry/low-quality results. Going above the maximum risks OOM errors or quality degradation (tiling artifacts).

Sources

• Official: none found as a vendor pairing matrix. Kitchen hardware gates: ComfyUI comfy/model_management.py (supports_fp8_compute, supports_nvfp4_compute, supports_mxfp8_compute); UNETLoader weight_dtype in nodes.py.
• Empirical: base-model / VAE / CLIP pairing rules from observed load failures.