StableCascade
StableCascade is an innovative AI model that marks a significant advancement in image generation technology. Built upon the Würstchen arch...
Last verified:
What is StableCascade?
StableCascade is an advanced text-to-image diffusion model developed by Stability AI, built upon the Würstchen architecture. Its key innovation is working in a much smaller latent space with a compression factor of 42, encoding a 1024x1024 image to just 24x24 while maintaining crisp reconstructions. This enables faster inference and cheaper training compared to models like Stable Diffusion which use a compression factor of 8.
The model consists of three stages: Stage A (a 20 million parameter VAE), Stage B (a diffusion model with 700 million or 1.5 billion parameter versions), and Stage C (the text-conditional diffusion model with 1 billion or 3.6 billion parameter versions). Stage A and B compress images, while Stage C generates the small 24x24 latents given a text prompt. This 3-stage cascade architecture is what gives the model its name.
StableCascade achieves impressive results both visually and in evaluation, performing best in prompt alignment and aesthetic quality in almost all comparisons against Playground v2, SDXL, SDXL Turbo, and Würstchen v2. The model supports all known extensions including finetuning, LoRA, ControlNet, IP-Adapter, and LCM. Multiple ControlNets are provided including Inpainting/Outpainting, Canny, and Super Resolution.
This tool is ideal for researchers working on generative models, artists creating artwork, designers incorporating AI into creative processes, and developers building educational or creative tools. It's particularly suited for usage scenarios where efficiency is important due to its highly compressed latent space. The codebase includes training and inference scripts, notebooks for text-to-image, image variation, image-to-image, ControlNet usage, LoRA training, and image reconstruction.
StableCascade pricing
Pricing model: Free
Stable Cascade is free to use. The code is under MIT LICENSE. The model weights from Hugging Face are under a STABILITY AI NON-COMMERCIAL RESEARCH COMMUNITY LICENSE, meaning they are free for non-commercial research purposes. Models can be downloaded from Hugging Face and there is a free demo site available. No paid plans are mentioned.
StableCascade pros
- 42x compression factor enables much faster inference than Stable Diffusion
- Cheaper training costs compared to previous architectures
- Best prompt alignment in human evaluations against competitors
- Best aesthetic quality in almost all comparisons
- Supports finetuning, LoRA, ControlNet, IP-Adapter, and LCM extensions
- Provides multiple ControlNets: Inpainting, Outpainting, Canny, Super Resolution
- 3.6 billion parameter Stage C version highly finetuned for best results
- 1.5 billion parameter Stage B excels at reconstructing fine details
- Image variation capability without requiring a prompt
- Image-to-image generation with controllable noising levels
- Available in diffusers library for easy integration
- Full and Lite model variations for different hardware requirements
- Combined pipeline for simplified single-step usage
- Model CPU offloading supported to reduce VRAM requirements
- LoRA training allows learning new tokens for custom subjects
StableCascade cons
- Faces and people may not be generated properly
- Autoencoding part of the model is lossy
- Codebase is in early development with potential unexpected errors
- Training and inference code not perfectly optimized
- Requires BF16 or FP32 precision (BF16 if GPU supported)
- Full model requires over 16GB VRAM without offloading enabled
- Lite versions have worse quality than full versions
- Model weights under non-commercial research license only
- torch.bfloat16 requires PyTorch 2.2.0 or higher
- No official releases published yet on GitHub
Frequently asked questions about StableCascade
What is Stable Cascade and how does it work?
Stable Cascade is a diffusion model trained to generate images from text prompts. It uses a 3-stage architecture: Stage A (VAE) and Stage B (diffusion model) compress images to a 24x24 latent space with 42x compression, while Stage C (text-conditional diffusion model) generates latents from text prompts. This highly compressed latent space enables faster inference and cheaper training.
What are the differences between Stage A, B, and C?
Stage A is a 20 million parameter VAE that is fixed. Stage B is a diffusion model with two versions: 700 million parameters or 1.5 billion parameters (the larger one excels at fine details). Stage C is the text-conditional model with two versions: 1 billion or 3.6 billion parameters (the 3.6B version is recommended as most work was put into its finetuning).
How does Stable Cascade compare to Stable Diffusion XL?
Stable Cascade achieves faster inference times despite having 1.4 billion more parameters than SDXL. It performs best in prompt alignment and aesthetic quality in human evaluations. The key difference is the compression factor: Stable Diffusion uses 8x compression while Stable Cascade achieves 42x compression, enabling much more efficient training and inference.
What ControlNets are available for Stable Cascade?
The release provides ControlNets for Inpainting/Outpainting, Canny, and Super Resolution. Face Identity ControlNet is mentioned but will be released at a later point. All can be used through the same ControlNet notebook by changing the config.
How do I reduce VRAM requirements when running Stable Cascade?
Enable model offloading in Settings -> Diffusers -> Model CPU offload. Without this, Stable Cascade will use more than 16GB of VRAM. You can also use the Lite versions of Stage B and Stage C which are smaller models, though with worse quality.
What precision should I use for Stable Cascade?
Set compute precision to BF16 if your GPU supports it, or FP32 if not. This is required in Settings -> Compute -> Precision. Note that torch.bfloat16 requires PyTorch 2.2.0 or higher for StableCascadeDecoderPipeline.
Can I train my own text-conditional model for Stable Cascade?
Yes, the codebase provides training code for training from scratch, finetuning, ControlNet, and LoRA. You can also use Stage A and B as an autoencoder to train your own text-conditional model with different architecture, benefiting from the 42x compression for faster training and running.
What sampler should I use for Stable Cascade?
The recommended sampler is 'Default'. In human evaluations, Stable Cascade was run with 30 inference steps and outperformed competitors including SDXL (50 steps) and Playground v2 (50 steps).
How do I use LoRA with Stable Cascade?
The repository provides LoRA implementation for training and using LoRAs with Stable Cascade to finetune Stage C. You can add and learn new tokens and add LoRA layers. A notebook shows how to use a trained LoRA, such as training on your dog with a custom token to generate images with that specific subject.
What license does Stable Cascade use?
The code is under MIT LICENSE. The model weights from Hugging Face are under STABILITY AI NON-COMMERCIAL RESEARCH COMMUNITY LICENSE, meaning they are free for non-commercial research only. The model is intended for research purposes, including generative model research, safe deployment研究, probing biases, artwork generation, and educational/creative tools.