Edge Dance
EDGE: Editable Dance Generation from Music is an AI tool that generates high-quality choreographies from music using music embeddings from ...
Last verified:
What is Edge Dance?
EDGE is a research method for generating editable, realistic dances from any input music using a transformer-based diffusion model combined with Jukebox for music feature extraction. It produces physically-plausible choreographies that align faithfully with the music's rhythm and style, generating 5-second clips that can be stitched into arbitrary-length sequences via temporal constraints for smooth transitions. The tool excels in editing capabilities like joint-wise conditioning, where specific body parts can be controlled independently, motion in-betweening for smooth interpolations between poses, and dance continuation to extend existing sequences seamlessly.
Key features include the innovative Contact Consistency Loss, which prevents unnatural foot sliding while preserving intentional slides during complex foot-ground contacts, ensuring high physical realism preferred by human raters over prior methods like Bailando and FACT. EDGE handles in-the-wild music samples effectively thanks to Jukebox embeddings, supporting diverse genres without retraining. Dances are output as joint positions compatible with SMPL models, enabling 3D rendering in tools like Blender via FBX conversion.
It is designed for researchers, animators, and developers in computer vision and graphics who need high-quality, customizable dance generation for projects in AI animation, virtual performances, or motion synthesis. Users with access to high-end GPUs can run demos, generate from custom music, or even train variants on datasets like AIST++. The open-source PyTorch implementation facilitates experimentation and integration into creative pipelines.
Edge Dance pricing
Pricing model: Free
Free and open-source under MIT license; no paid plans or tiers mentioned. Download model checkpoint from Google Drive, run locally with provided scripts. Requires own hardware resources like GPUs.
Edge Dance pros
- Realistic dances faithful to input music
- Physically-plausible motion generation
- Joint-wise conditioning for body part control
- Motion in-betweening for smooth transitions
- Dance continuation for extending sequences
- Handles arbitrary-length choreographies
- Contact Consistency Loss prevents foot sliding
- Uses powerful Jukebox music embeddings
- Supports in-the-wild music samples
- Transformer-based diffusion for high quality
- Human-preferred over Bailando and FACT
- Open-source PyTorch implementation
- Demo notebook for quick testing
- Custom music input via WAV files
- SMPL-compatible for 3D rendering
- FBX export for Blender integration
- Evaluated with Physical Foot Contact metric
Edge Dance cons
- Requires high-end NVIDIA GPUs (16GB+)
- Linux recommended, Windows unsupported
- Complex setup with multiple dependencies
- Jukebox feature extraction is time-intensive
- Dataset processing takes ~24 hours
- Training needs 6-24 hours on multi-GPUs
- Inference slow without cached features
- File names must be regularized
- No web demo, code-only access
- Research code, not regularly maintained
Frequently asked questions about Edge Dance
What is EDGE and how does it generate dances?
EDGE is a method using a transformer-based diffusion model with Jukebox embeddings to map input music to 5-second dance clips. Temporal constraints stitch clips into full videos, ensuring consistency and music alignment.
What editing features does EDGE offer?
It supports joint-wise conditioning to generate motion for specific body parts like lower body from upper, motion in-betweening between start/end poses, and dance continuation from existing sequences.
How does EDGE ensure physical realism?
The Contact Consistency Loss trains the model to handle foot-ground contacts accurately, avoiding unintentional sliding while keeping intentional slides, leading to physically plausible dances.
What are the hardware requirements?
At least 1-8 high-end NVIDIA GPUs with 16GB+ memory, 16GB RAM per GPU, CUDA 11.6, PyTorch 1.12.1, and Linux OS recommended for best performance.
How can I test EDGE on custom music?
Download the model checkpoint, place WAV files in a directory with regularized names, and run `python test.py --music_dir custom_music/` to generate dances.
Is there a demo available?
Yes, run `demo.ipynb` after downloading the model to explore the basic interface and generate sample dances interactively.
How to render EDGE outputs in 3D?
Generate motions with `--save_motions`, convert PKL files to FBX using `SMPL-to-FBX/Convert.py`, then import into Blender and retarget to rigs like Mixamo.
What dataset is used for training?
AIST++ dataset; download and process with `data/download_dataset.sh` and `create_dataset.py --extract-baseline --extract-jukebox`, taking ~24 hours and 50GB.
How to train a custom EDGE model?
After processing AIST++, run `accelerate launch train.py --batch_size 128 --epochs 2000 --feature_type jukebox --learning_rate 0.0002`, logging to wandb.
How is dance quality evaluated?
Uses Physical Foot Contact (PFC) score; generate samples with `--save_motions`, then run `eval/eval_pfc.py` for quantitative physical plausibility metrics.