Llamafactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Last verified:
What is Llamafactory?
LLaMA Factory is an easy‑to‑use open‑source framework for training and fine‑tuning large language models locally, without writing code. It supports hundreds of pre‑trained models such as LLaMA, LLaVA, Mistral, Mixtral‑MoE, Qwen, Yi, Gemma, Baichuan, ChatGLM, and Phi, and lets users run full‑parameter, LoRA, and ultra‑low‑bit QLoRA fine‑tuning on consumer or data‑center GPUs. The platform provides a unified interface for multiple training paradigms, including instruction‑based supervised fine‑tuning, reward‑model training, PPO, DPO, KTO, and ORPO, plus optimized computation using methods like GaLore, BAdam, DoRA, LongLoRA, LLaMA Pro, Mixture‑of‑Depths, LoRA+, LoftQ, and PiSSA.
Key features include a web‑based UI (LlamaBoard) for training configuration and monitoring, built‑in dataset templates in Alpaca and ShareGPT formats, and support for common metrics tracking with tools such as TensorBoard, Wandb, MLflow, and SwanLab. Users can also switch between different inference engines (Transformers or vLLM) to speed up chat and batch inference, and can export or merge LoRA‑style adapters into standalone model weights. This makes LLamaFactory attractive both for researchers who want to experiment with cutting‑edge tuning methods and for practitioners who want to customize open‑source LLMs for domain‑specific tasks like coding assistants, chatbots, or domain‑specific QA.
The tool is aimed at developers, data scientists, and researchers who already have access to GPUs or cloud instances and want a streamlined, no‑code path into LLM fine‑tuning. It is especially useful for teams that need to iterate quickly on multiple model architectures or training strategies, because it centralizes YAML‑based configuration, experiment tracking, and model deployment workflows in one codebase. Since it runs locally or on private infrastructure, it also appeals to users who have data‑privacy or compliance requirements and prefer not to ship training data to external cloud services.
Llamafactory pricing
Pricing model: Freemium
LLaMA Factory is free and open‑source under the Apache‑2.0 license, with no paid tiers or subscription plans listed on the official documentation. Users pay only for the underlying compute (GPUs, cloud instances, or local hardware) and storage they choose to run the framework on, plus any optional services such as third‑party experiment trackers or cloud inference platforms. The project does not currently advertise any commercial or enterprise support packages, so all usage is self‑managed community‑supported.
Llamafactory pros
- Supports hundreds of open‑source LLMs and MLLMs out of the box
- Unifies supervised fine‑tuning, RLHF variants, and QLoRA in one codebase
- Runs locally without depending on a proprietary cloud service
- No‑code web UI (LlamaBoard) for configuration and monitoring
- Built‑in support for full‑parameter, LoRA, and multiple‑bit QLoRA
- Integrates advanced optimization methods like GaLore, BAdam, DoRA, LongLoRA
- Supports multiple precision backends (bf16, fp16, fp32, and mixed‑precision)
- Offers modular YAML configuration for easy experiment reuse
- Built‑in dataset templates in Alpaca and ShareGPT formats
- Flexible dataset plug‑in system via dataset_info.json
- Multi‑device training with DeepSpeed and other distributed backends
- Supports both Transformers and vLLM as inference engines
- Rich experiment tracking with LlamaBoard, TensorBoard, Wandb, MLflow, SwanLab
- Open‑source under Apache‑2.0 with active community
- Allows exporting and merging LoRA adapters into standalone models
- Enables batch inference and API‑style serving with minimal extra setup
Llamafactory cons
- No official hosted SaaS or web console; requires self‑hosting
- Steep initial setup curve for GPU drivers and CUDA / PyTorch
- Requires substantial GPU memory for full‑parameter fine‑tuning
- No built‑in managed dataset hosting or ingestion pipelines
- Limited out‑of‑the‑box MLOps or CI/CD automation
- Some advanced features need manual configuration or custom code
- Multi‑GPU and distributed training setup is not fully wizardized
- No official GUI bundle; advanced users must edit YAML or CLI flags
Frequently asked questions about Llamafactory
What models and model families can I fine‑tune with LLaMA Factory?
LLaMA Factory supports a wide range of LLM and MLLM families, including LLaMA, LLaVA, Mistral, Mixtral‑MoE, Qwen, Qwen‑VL, Yi, Gemma, Baichuan, ChatGLM, Phi, and others. The framework exposes them through a unified model loading and configuration system, so you typically only need to specify the model path and template in your YAML config to switch architectures.
Can I use LLaMA Factory without writing code?
Yes; LLaMA Factory is designed so that you can run most training, evaluation, and inference workflows without writing Python code. You can configure everything via the web UI (LlamaBoard) or through parameterized YAML files, and then invoke training or inference using the llamafactory-cli command‑line tool.
Which fine‑tuning methods are supported?
The framework supports multiple training paradigms per model, including incremental pre‑training, multimodal instruction‑based supervised fine‑tuning, reward‑model training, PPO, DPO, KTO, and ORPO. On the parameter‑efficiency side, you can choose full‑parameter fine‑tuning, frozen partial‑parameter tuning, LoRA, and various QLoRA bit‑widths (2/3/4/5/6/8‑bit) powered by backends such as AQLM, AWQ, GPTQ, LLM.int8, HQQ, and EETQ.
Does LLaMA Factory provide a web UI?
Yes; LLaMA Factory includes a web‑based UI called LlamaBoard that lets you configure training jobs, watch live loss curves, inspect datasets, and chat with models through a browser. The UI layers on top of the core training pipeline and exposes key parameters such as model, dataset, tuning method, and precision in a point‑and‑click interface.
How does dataset loading work in LLaMA Factory?
LLaMA Factory expects datasets in JSON format placed in the data directory, with definitions in data/dataset_info.json. It natively supports Alpaca‑style and ShareGPT‑style dialogue formats, mapping JSON fields such as instruction, input, output, system, and history or conversations to the internal training pipeline via configurable column mappings in the dataset info file.
Can I run LLaMA Factory on my own GPU or cloud instance?
Yes; the framework is designed to run on local machines or cloud VMs with compatible NVIDIA GPUs and CUDA. Installation instructions explicitly cover both Linux and Windows, and you can scale from single‑GPU toy experiments to multi‑node setups using DeepSpeed or other distributed backends for larger workloads.
How do I export or merge a LoRA‑tuned model?
LLaMA Factory provides a CLI command, such as llamafactory-cli export, to merge LoRA adapters back into the base model weights, producing a standalone model checkpoint. You specify the base model path, adapter path, template, and target export directory, and the tool rescales and merges the adapter weights according to the chosen precision and export format.
What inference engines and batching options are available?
By default, LLaMA Factory uses the Hugging Face Transformers engine for inference, but you can switch to vLLM for faster, higher‑throughput generation. The framework also supports batch inference over datasets and API‑style serving, where you define a YAML config with model, adapter, template, and backend, then launch an API process with llamafactory-cli api to accept JSON‑based requests.
What experiment tracking and logging tools integrate with LLaMA Factory?
LLaMA Factory natively integrates with several experiment‑monitoring tools, including LlamaBoard (its own web‑based monitor), TensorBoard, Wandb, MLflow, and SwanLab. You choose a logging backend in your YAML configuration, and the framework writes metrics, loss curves, and hyperparameters so you can compare runs and debug training behavior.
Is there a managed or hosted version of LLaMA Factory I can subscribe to?
No; the official documentation and repositories describe LLaMA Factory as a self‑hosted, open‑source framework rather than a paid SaaS product. Any hosted or managed variant you see would be operated by a third party on top of the open‑source code, and would not be an official pricing tier from the LLaMA Factory project itself.