Live-portrait
The fofr/live-portrait is an AI model that enables portrait animation by leveraging a driving video source. The main functionality is to ge...
Last verified:
What is Live-portrait?
The fofr/live-portrait model on Replicate is an efficient AI tool for portrait animation that takes a source portrait image and a driving video as inputs to generate a lifelike animated video. It leverages advanced stitching and retargeting techniques to transfer facial expressions, head poses, and movements from the driving video to the source portrait with high fidelity and minimal artifacts. This implementation is based on the LivePortrait research paper, utilizing ComfyUI custom nodes and safetensor weights for robust performance on Nvidia A40 GPUs.
Key features include customizable parameters such as crop input flag, relative input motion, pasteback options, maximum frames from driving video (default 128), and frame selection intervals. It supports both single image to video animation and video-to-video portrait editing, with options for auto-cropping driving videos and using pre-extracted motion templates (.pkl files) for privacy and speed. The model excels in self-reenactment and cross-reenactment scenarios, preserving identity while enabling precise control over output size, scaling, and positioning.
Designed for developers, content creators, and researchers, it is ideal for applications like avatar animation, video editing, and real-time demos via API integration. Runs are quick, with open-source roots allowing local Docker deployment, though Replicate provides seamless cloud access. It targets users needing high-quality, controllable portrait animation without diffusion model overhead, achieving speeds up to 12.8ms per frame on high-end GPUs.
Live-portrait pricing
Pricing model: Paid
Pay-per-use on Replicate at approximately $0.00/unit (billed by compute time); limited free daily generations for testing; no dedicated free tier or paid plans listed, scales with usage via credits.
Live-portrait pros
- Efficient implicit-keypoint framework
- Stitching module reduces artifacts
- Retargeting for eyes and lips control
- Supports image-to-video animation
- Handles video-to-video editing
- Auto-crops driving videos
- Motion templates for privacy
- Configurable frame limits
- Relative motion input option
- Pasteback for seamless blending
- Works across portrait styles
- Generalizes to animals with fine-tune
- API easy for app integration
- Open-source MIT licensed code
- Fast inference on A40 GPUs
- ComfyUI node compatibility
- Scalable training data used
Live-portrait cons
- No commercial use due to InsightFace
- Limited to portrait/head focus
- Relies on frontal neutral start frame
- Potential artifacts in complex poses
- Nvidia GPU hardware required
- Max 128 frames default limit
- Crop input may alter originals
- Slower on non-A40 hardware
- No audio driving support here
- Privacy risks without templates
Frequently asked questions about Live-portrait
What inputs does fofr/live-portrait require?
Requires a source face image or video and a driving video source; optional flags for cropping input, relative motion, pasteback, and parameters like max frames (default 128) and frame selection interval.
Is commercial use allowed?
No, cannot be used commercially because it uses InsightFace buffalo_l models; code and weights are MIT licensed but InsightFace restricts business applications.
What hardware does it run on?
Operates on Nvidia A40 (Large) GPUs via Replicate; local runs possible with Docker on compatible Nvidia hardware, achieving 12.8ms/frame on RTX 4090.
How to use example driving videos?
Use provided examples from https://github.com/KwaiVGI/LivePortrait/tree/main/assets/examples/driving; input URLs directly or download for custom animations.
What is the basis of this model?
Based on LivePortrait paper 'Efficient Portrait Animation with Stitching and Retargeting Control' (arXiv:2407.03168); uses ComfyUI-LivePortraitKJ node by Kijai and converted safetensor weights.
Can it animate animals?
Yes, generalizes to animals like cats, dogs, pandas via fine-tuned models; requires X-Pose dependency and tested on Linux/Windows Nvidia GPUs.
What output controls are available?
Controls include size, scaling, positioning, retargeting strength, eyes-open extent, lip-open extent scalars; stitching module enhances quality.
How to run locally?
Open-source; clone repo, install via Docker/Conda, download weights from HuggingFace, run inference.py with source and driving inputs.
What are common best practices for driving videos?
Crop to 1:1 aspect ratio (512x512), focus on head, minimize shoulders, start with frontal neutral expression; enable --flag_crop_driving_video for auto-handling.
Does it support video-to-video editing?
Yes, input source video with driving video or .pkl template; generates edited portrait video preserving original appearance with transferred motions.