Axolotl
Go ahead and axolotl questions
Last verified:
What is Axolotl?
Axolotl is a free and open-source LLM fine-tuning framework designed to streamline post-training and fine-tuning for the latest large language models. It supports a wide variety of models including LLaMA, Mistral, Mixtral, Qwen, Gemma, and many more available on Hugging Face Hub, as well as multimodal vision-language models like LLaVA, Qwen2-VL, Pixtral, and InternVL 3.5.
Key features include multiple training methods (full fine-tuning, LoRA, QLoRA, GPTQ, QAT, DPO, IPO, KTO, ORPO, GRPO, GDPO, and reward modeling), performance optimizations like Flash Attention 2/3/4, Xformers, Flex Attention, SageAttention, Liger Kernel, Sequence Parallelism, and multi-GPU/multi-node training with FSDP, DeepSpeed, and DDP. Axolotl uses a single YAML configuration file for the entire pipeline from dataset preprocessing to training, evaluation, quantization, and inference. It supports flexible dataset handling from local files, HuggingFace, and cloud storage (S3, Azure, GCP, OCI).
Axolotl is designed for AI researchers, machine learning engineers, developers fine-tuning LLMs, and anyone working on post-training large language models. It includes built-in documentation optimized for AI coding agents like Claude Code, Cursor, and Copilot. The tool is cloud-ready with Docker images and PyPI packages, and works with cloud providers like RunPod, Vast.ai, PRIME Intellect, Modal, Novita, and JarvisLabs.ai.
The framework is actively maintained with regular updates adding new model support (Mistral Medium 3.5, Gemma 4, Qwen3.5, Llama 4, Kimi-Linear, and more as of 2026) and new features like SonicMoSE fused LoRA, ScatterMoE LoRA, text diffusion training, and FP8 finetuning.
Axolotl pricing
Pricing model: Freemium
Axolotl is completely free and open-source under the Apache 2.0 License. There are no paid plans or subscription tiers. The tool can be installed via pip (uv pip install axolotl[deepspeed]) or Docker at no cost. All features including full fine-tuning, LoRA, QLoRA, multimodal training, distributed training, and all performance optimizations are available for free. Users only pay for their own hardware or cloud GPU costs when using cloud providers like RunPod, Vast.ai, PRIME Intellect, Modal, Novita, or JarvisLabs.ai.
Axolotl pros
- Free and open-source under Apache 2.0 License
- Supports 50+ model architectures including LLaMA, Mistral, Qwen, Gemma
- Multimodal training support for vision-language and audio models
- Multiple training methods: LoRA, QLoRA, FFT, DPO, GRPO, QAT
- Single YAML config for entire fine-tuning pipeline
- Flash Attention 2/3/4 and performance optimizations built-in
- Multi-GPU and multi-node distributed training support
- Works with cloud providers: RunPod, Vast.ai, Modal, Novita
- Docker images available for easy deployment
- Built-in AI agent documentation for Claude Code, Cursor, Copilot
- Supports cloud storage: S3, Azure, GCP, OCI datasets
- Gradio UI interface for inference testing
- LoRA weight merging into base model
- Dataset preprocessing CLI for large datasets
- Active community with Discord support and regular updates
- Opt-in telemetry for usage insights without personal data
- uv-first installation for fast package management
- Support for MoE expert quantization reducing VRAM usage
Axolotl cons
- Requires NVIDIA GPU (Ampere or newer) or AMD GPU
- Python 3.11+ required (3.12 recommended)
- PyTorch 2.9.1+ required which may be hard to install
- Steep learning curve for configuration YAML files
- ocu9s on command-line interface, limited GUI options
- Can run out of system RAM causing exitcode -9 errors
- NCCL communication issues in multi-GPU setups
- Some model architectures like LLaVA/Pixtral don't support mixed text+image datasets
Frequently asked questions about Axolotl
How do I start fine-tuning a model with Axolotl?
First install Axolotl using uv, then run 'axolotl fetch examples' to download example configs, and finally run 'axolotl train examples/llama-3/lora-1b.yml' to fine-tune a model using LoRA. You can create your own YAML config file specifying base_model, adapter type (lora/qlora/fft), datasets, and training parameters.
What training methods does Axolotl support?
Axolotl supports full fine-tuning (FFT), LoRA, QLoRA, GPTQ, Quantization Aware Training (QAT), Preference Tuning (DPO, IPO, KTO, ORPO, SimPO), Reinforcement Learning (GRPO, GDPO), and Reward Modelling/Process Reward Modelling (RM/PRM). You select the method by setting the adapter and related config options in your YAML file.
Can I fine-tune multimodal models with Axolotl?
Yes, Axolotl supports fine-tuning vision-language models (VLMs) including LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, SmolVLM2, GLM-4.6V, InternVL 3.5, Gemma 3n, and audio models like Voxtral with image, video, and audio support. Multimodal fine-tuning support was added in beta in March 2025.
What GPUs are required for Axolotl?
Axolotl requires an NVIDIA GPU with Ampere architecture or newer for bf16 and Flash Attention support, or an AMD GPU. For Blackwell GPUs, use the specific Docker tag axolotlai/axolotl-uv:main-py3.11-cu130-2.9.1. Single GPU training works with DDP, while multi-GPU requires FSDP or DeepSpeed.
How do I merge LoRA weights back into the base model?
Run 'axolotl merge-lora my_training.yml --lora-model-dir=./outputs/lora-out' to merge LoRA weights. The merged model will be saved in the {output_dir}/merged directory. Do not use your own scripts for merging as Axolotl handles vocabulary size expansion automatically.
What dataset formats does Axolotl support?
Axolotl supports multiple dataset formats including alpaca (instruction/input/output JSONL), conversation format for supervised fine-tuning, and stepwise supervised format for chain-of-thought reasoning datasets. Datasets can be loaded from local files, HuggingFace Hub, or cloud storage (S3, Azure, GCP, OCI).
How do I debug OOM or exitcode errors?
exitcode -9 usually means you ran out of system RAM. exitcode -7 with DeepSpeed suggests upgrading DeepSpeed. For GPU communication issues, check NCCL documentation. For stuck preprocessing, try setting CUDA_VISIBLE_DEVICES=0. Use the Debugging Guide and Discord community for additional help.
Can I use Axolotl with AI coding assistants?
Yes, Axolotl ships with built-in documentation optimized for AI coding agents like Claude Code, Cursor, and Copilot. Run 'axolotl agent-docs' for overview, or 'axolotl agent-docs sft' for topic-specific references on SFT, GRPO, preference_tuning, reward_modelling, or pretraining.
How do I disable telemetry in Axolotl?
Axolotl has opt-out telemetry enabled by default that collects basic system information, model types, and error rates (never personal data or file paths). To disable it, set the environment variable AXOLOTL_DO_NOT_TRACK=1 before running Axolotl commands.
What cloud providers work well with Axolotl?
Axolotl is cloud-ready and works well with RunPod, Vast.ai, PRIME Intellect, Modal, Novita, JarvisLabs.ai, and latitude.sh. Docker images are available at axolotlai/axolotl-uv:main-latest for easy deployment on these platforms. Cloud GPU rental is the main cost since Axolotl itself is free.