Xturing
Xturing is an open-source AI personalization library dedicated for the customization of Large Language Models (LLMs). It provides a user-friendly interface designed to adapt LLMs to custom needs.
Last verified:
What is Xturing?
xTuring is an open-source AI personalization library that enables users to build, customize, and control their own Large Language Models (LLMs). It provides fast, efficient, and simple fine-tuning of popular LLMs such as LLaMA, GPT-J, GPT-2, BLOOM, Falcon, Galactica, Cerebras, DistilGPT-2, and OPT. The tool is designed to make AI personalization accessible to everyone, from beginners to experienced developers, by offering a simple interface that requires little to no Python knowledge.
Key features include memory-efficient fine-tuning techniques like LoRA that can reduce hardware costs by up to 90%, support for both single GPU and multi-GPU training, and the ability to load and fine-tune pre-trained models in less than 10 lines of code. xTuring supports multiple precision levels including LoRA, INT8, LoRA + INT8, and LoRA + INT4, allowing users to optimize for their specific hardware constraints. The library also includes a beautiful UI Playground for interactive model testing and chatting with fine-tuned models.
xTuring is ideal for developers building AI pipelines, businesses wanting to customize AI for their specific data or applications, researchers experimenting with LLM fine-tuning, and anyone looking to leverage AI power without extensive machine learning expertise. The tool handles everything from data pre-processing to fine-tuning, making it a complete solution for personalizing open-source LLMs. It is licensed under Apache 2.0 and offers community support through Discord and Twitter.
Xturing pricing
Pricing model: Free
xTuring is open-source software licensed under Apache 2.0, which means the software itself can be used without licensing fees. There is no publicly available detailed pricing information for paid plans or enterprise support on the website. The tool emphasizes hardware efficiency allowing faster fine-tuning with fewer GPUs, reducing costs. Users train models locally or on their own cloud infrastructure, so costs depend on their hardware/cloud setup rather than subscription fees. For exact pricing details or enterprise offerings, contacting Stochastic directly would be necessary.
Xturing pros
- Open-source software licensed under Apache 2.0
- Reduces hardware costs by up to 90% with LoRA
- Fine-tune models in less than 10 lines of code
- Supports single GPU and multi-GPU training
- Works with 11+ popular LLMs including LLaMA, GPT-J, GPT-2
- Memory-efficient techniques: LoRA, INT8, INT4 precision
- Simple interface requiring little to no Python knowledge
- Beautiful UI Playground for interactive model testing
- Complete solution from data pre-processing to fine-tuning
- Load models from local machine or xTuring Hub
- Generate custom datasets given HuggingFace engine
- Supports instruction datasets from .jsonl files
- Batch size configuration for faster processing
- Built-in logging for training progress
- Easy model saving with .save() method
- Active Discord community for support
- Works with HuggingFace Model Hub models
Xturing cons
- No publicly available detailed pricing information
- Requires GPU hardware for training
- Limited to specific pre-trained model families
- INT4 precision requires GenericLoraKbitModel class
- May need domain knowledge for complex fine-tuning
- No enterprise support plans mentioned
- Documentation primarily in Python API format
- Beginners may still need ML basics understanding
Frequently asked questions about Xturing
What is xTuring?
xTuring is an open-source AI personalization library that helps you build, customize, and control your own Large Language Models (LLMs). It provides a simple interface to fine-tune LLMs like LLaMA, GPT-J, GPT-2 for your specific needs, whether for personal data or applications.
What models does xTuring support?
xTuring supports BLOOM 1.1B, Cerebras 1.3B, DistilGPT-2, Falcon 7B, Galactica 6.7B, GPT-J 6B, GPT-2, LLaMA 7B, LLaMA2, and OPT 1.3B. All models support LoRA, INT8, LoRA + INT8, and LoRA + INT4 precision options.
How do I install xTuring?
You can install xTuring using pip: pip install xturing. To ensure you have the latest version, use: pip install xturing --upgrade. The tool is available on GitHub as stochasticai/xturing.
What is LoRA and how does it help?
LoRA (Low-Rank Adaptation) is a memory-efficient fine-tuning technique that can reduce your hardware costs by up to 90%. It allows you to train models in a fraction of the time by using less memory while maintaining model quality.
Can I use xTuring without GPU?
xTuring supports both single GPU and multi-GPU training for optimal performance. While the library can potentially work without GPU for inference, fine-tuning large language models significantly benefits from GPU acceleration for faster training times.
How do I launch the UI Playground?
You can launch the UI Playground in two ways: 1) Command-line: run 'xturing ui' in your terminal, or 2) Python script: use 'from xturing.ui.playground import Playground' followed by 'Playground().launch()'. You can also specify a model path when launching.
What dataset formats does xTuring support?
xTuring supports InstructionDataset from .jsonl files where each line is a json object with keys 'text', 'instruction', and 'target'. It also supports TextDataset and can generate custom datasets given a HuggingFace engine.
Is xTuring free to use?
Yes, xTuring is open-source software licensed under Apache 2.0, meaning there are no licensing fees to use the software. However, you'll need your own GPU hardware or cloud infrastructure for training, which incurs costs.
How do I save my fine-tuned model?
After fine-tuning, use the .save() method with a directory path: model.save('/path/to/a/directory/'). This saves your fine-tuned model weights to the specified directory for later use.
Where can I get help with xTuring?
You can join the Discord community to chat with other community members about ideas and get help. You can also follow @stochasticai on Twitter for updates, and participate in GitHub discussions at stochasticai/xturing.