Areal
The RL Bridge for LLM-based Agent Applications. Made Simple & Flexible.
Last verified:
What is Areal?
AReaL (Ant Reasoning RL) is an open-source, fully asynchronous reinforcement learning training system designed for large reasoning models (LRMs) and agentic AI applications. Developed at the RL Lab, Ant Research, and maintained by the inclusionAI community with contributions from Ant Group and academic partners like Tsinghua University, AReaL provides algorithm-system co-design to enable stable, high-throughput RL training that scales from a single node to thousands of GPUs.
The system completely decouples generation from training: rollout workers continuously generate new outputs without waiting, while training workers update the model whenever a batch of data is collected. This fully asynchronous architecture incorporates system-level optimizations for substantially higher GPU utilization and uses a staleness-enhanced PPO variant to handle outdated training samples. AReaL achieves up to 2.77x training speedup compared to synchronous systems on math and code reasoning benchmarks while matching or improving final performance.
Key features include the newly released AReaL-lite (a lightweight, algorithm-first codebase with 80% fewer lines of code), full SGLang support, reproducible training recipes, and multi-turn agentic RL experimental support. The project continuously releases all code, datasets, and training artifacts including SOTA models like AReaL-boba-RL-7B and training data like AReaL-boba-106k.
AReaL is designed for AI researchers, ML engineers, and organizations building large reasoning models for math, code, and agentic tasks. It is Apache-2.0 licensed with comprehensive documentation and examples for engineering integration.
Areal pricing
Pricing model: Freemium
AReaL is completely free and open-source under the Apache-2.0 license. There are no paid plans or tiers. All code, datasets, training recipes, and models are continuously released without any proprietary barriers. Users only need to provide their own computational resources (GPUs) to run training.
Areal pros
- Fully asynchronous RL architecture decouples generation from training
- Up to 2.77x training speedup compared to synchronous systems
- Scales seamlessly from 1 single node to 1000 GPUs
- Open-source with Apache-2.0 license
- All code, datasets, and training recipes continuously released
- Reproducible research artifacts with no hidden proprietary barriers
- High GPU utilization through system-level optimizations
- AReaL-lite provides lightweight 80% smaller codebase for rapid prototyping
- Full SGLang support with 1.5x speedup on 7B training
- SOTA 7B math reasoning model AReaL-boba-RL-7B released
- Staleness-enhanced PPO variant handles outdated training samples
- Multi-turn agentic RL experimental support included
- Comprehensive documentation with quickstart guides and code walkthroughs
- Active weekly minor releases and monthly major releases
- Community-driven with 31 contributors from Ant Research and Tsinghua
Areal cons
- Requires significant GPU resources for large-scale training
- Complex system architecture may be overwhelming for beginners
- Multi-turn agentic RL is still experimental not production-ready
- Primarily focused on math and code reasoning, other domains less mature
- Legacy AReaL has redundant code and unnecessary system abstractions
- AReaL-lite only maintains 90% of legacy performance
- Large-scale experiments still require legacy AReaL for stability
- Documentation primarily in English with limited Chinese resources
Frequently asked questions about Areal
What is AReaL?
AReaL (Ant Reasoning RL) is an open-source, fully asynchronous reinforcement learning training system for large reasoning models developed at the RL Lab, Ant Research. It completely decouples generation from training, allowing rollout workers to continuously generate outputs while training workers update the model whenever a batch is collected.
What makes AReaL different from other RL systems?
AReaL is fully asynchronous, unlike most existing large-scale RL systems that are synchronous and alternate generation and training in batches. This asynchronous approach achieves up to 2.77x training speedup while maintaining or improving final performance, and makes multi-turn agentic RL training extremely easy to set up.
What is AReaL-lite?
AReaL-lite is a lightweight version released in July 2025 designed for AI researchers and rapid prototyping. It features an algorithm-first API design with 80% fewer lines of code while maintaining 90% of AReaL's high performance and core functionality. It prioritizes ease of use and algorithm development with fully asynchronous agentic RL support.
What benchmarks does AReaL perform well on?
AReaL achieves SOTA performance on math and code reasoning benchmarks. The AReaL-boba-RL-7B model achieved 61.9 on AIME 2024, 48.3 on AIME 2025, and 47.6 on GPQA-Diamond, outperforming other 7B models. It also approaches QwQ-32B performance on AIME 2024 with just 200 data samples.
How scalable is AReaL?
AReaL seamlessly adapts to different computational resource settings, ranging from a single node to 1000 GPUs. Training convergence time varies by resource configuration: 7B model training takes ~252 hours on 32 GPUs, ~90 hours on 128 GPUs, and 1.5B models can train in ~27 hours on 128 GPUs.
Is AReaL free to use?
Yes, AReaL is completely free and open-source under the Apache-2.0 license. All code, datasets, and training recipes are continuously released without any hidden proprietary barriers. Users only need to provide their own GPU resources.
What models has AReaL released?
AReaL has released multiple models including AReaL-boba-RL-7B (SOTA 7B math reasoning), AReaL-boba-SFT-32B (approaching QwQ-32B performance), AReaL-1.5B-Preview models, and AReaL-boba-2-8B. They also released training datasets like AReaL-boba-106k and AReaL-boba-SFT-200.
Who maintains AReaL?
AReaL is maintained by the inclusionAI community with contributions from Ant Group (specifically the RL Lab at Ant Research) and academic partners including the Institute for Interdisciplinary Information Sciences at Tsinghua University. The project has 31 contributors and receives assistance from multiple groups including the SGLang team.
What is the future development plan for AReaL?
Future plans include system development (SGLang support, RL for coding problems, expert parallel for MOE, zero-bubble pipelining, RL for vision-language models, multi-turn agentic RL, function calling) and algorithm development (RL training recipes for 1.5B/7B/32B models, sample-efficient multi-task RL, stable MOE training). Releases are weekly for minor and monthly for major.
How do I get started with AReaL?
For AReaL-lite, run: python3 -m areal.launcher.local examples/gsm8k_grpo.py --config examples/configs/gsm8k_grpo.yaml. The training scripts automatically download datasets (openai/gsm8k) and models (Qwen/Qwen2-1.5B-Instruct). For cluster deployment, use python3 -m areal.launcher.ray with cluster parameters. Check the quickstart guide and documentation for detailed instructions.