ColossalChat
ColossalChat is a chatbot application that is powered by ColossalAI. As an AI-based tool, it uses machine learning algorithms for natural language processing an...
Last verified:
What is ColossalChat?
ColossalChat is an open-source AI chatbot framework that clones ChatGPT using a complete RLHF (Reinforcement Learning from Human Feedback) pipeline. Built on Meta's LLaMA pre-trained model, it provides an interactive demo that users can try online without registration or a waiting list. The platform delivers smooth, efficient conversations by utilizing advanced AI technology with strong bilingual proficiency in English and Chinese.
Key features include a complete three-stage RLHF training process: supervised instruct fine-tuning, reward model training, and reinforcement learning fine-tuning. The platform offers 4-bit quantized inference for 7 billion-parameter models requiring only 4GB GPU memory, making it accessible on consumer-grade GPUs like RTX 3060. ColossalChat includes an open-source 104K bilingual dataset of Chinese and English, complete RLHF training code for 7B and 13B models, and utilizes Colossal-AI's distributed techniques for maximized runtime performance.
ColossalChat is designed for researchers, developers, and AI enthusiasts who want to understand and replicate ChatGPT's technical solution without proprietary restrictions. It serves businesses looking to create AI-powered chatbots for customer service, organizations wanting to automate tasks like data entry and scheduling, and companies conducting market research. The platform is particularly valuable for those seeking to lower the high cost of building large AI models while protecting core data and IP from third-party API leaks.
The chatbot includes a simple safety filter to remove offensive content and operates under the LLaMA license for non-commercial use only. Users can report issues directly to contribute to continuous improvement, and the open-source nature allows full transparency into the training process and model architecture.
ColossalChat pricing
Pricing model: Free
ColossalChat is completely free and open-source. No account creation, subscription, or payment is required to access the interactive demo at chat.colossalai.org. The platform operates under the LLaMA license which permits only non-commercial use. All training code, datasets, and model weights are freely available on GitHub. Users can deploy the 7B model inference service on a single GPU with approximately 4GB memory at minimal hardware cost.
ColossalChat pros
- Fully open-source with complete RLHF training code
- No registration or waiting list required for demo access
- Strong bilingual proficiency in English and Chinese
- 4-bit quantized inference requiring only 4GB GPU memory
- Complete three-stage RLHF pipeline matching ChatGPT's technical route
- Open-source 104K bilingual dataset included
- Training speed 2x faster than FSDP used by Alpaca
- Works on consumer-grade GPUs like RTX 3060
- Low-cost fine-tuning using LoRA method
- 7B and 13B model training code available
- Zero+Gemini technology reduces memory redundancy
- GPTQ 4-bit quantization reduces memory by 75%
- No account creation or subscription required
- Immediate access without logging in
- Full transparency into training process and architecture
ColossalChat cons
- Non-commercial use only under LLaMA license
- Simple safety filter may accidentally flag normal content
- Limited computing power affects performance in some scenarios
- Dataset size limited compared to proprietary models
- May exhibit hallucination, toxicity, and bias like other LLMs
- No dedicated customer support
- Data source subject to OpenAI Terms restrictions
- Cannot develop competing models per OpenAI terms
- Reinforcement learning stage is most complex part
- Short sequence generation limited to 128-length
Frequently asked questions about ColossalChat
What is ColossalChat?
ColossalChat is an open-source AI solution for cloning ChatGPT with a complete RLHF pipeline. It is the first practical open-source project that includes supervised data collection, supervised fine-tuning, reward model training, and reinforcement learning fine-tuning based on the LLaMA pre-trained model, closely resembling the original ChatGPT technical solution.
Do I need to register to use the demo?
No registration or waiting list is required. The interactive demo at chat.colossalai.org can be tried online immediately without creating an account or logging in, providing a hassle-free experience.
What languages does ColossalChat support?
ColossalChat achieves strong bilingual proficiency in both English and Chinese. It uses an open-source 104K bilingual dataset containing approximately 100,000 Q&A pairs in both languages, collected from real-life question scenarios on social media platforms.
What is the RLHF pipeline in ColossalChat?
The RLHF pipeline involves three stages: Stage 1 performs supervised instruct fine-tuning using the bilingual dataset; Stage 2 trains a reward model by manually ranking different outputs for the same prompt; Stage 3 uses reinforcement learning (PPO algorithm) with experience generation and parameter update stages, plus PTX to maintain original language model performance.
What GPU memory is required for inference?
With 4-bit quantized inference using GPTQ, the 7 billion-parameter ColossalChat model requires only about 4GB of GPU memory for short sequence (128-length generation) inference, deployable on consumer-grade GPUs like RTX 3060 with one line of code.
Can I use ColossalChat commercially?
No. ColossalChat is licensed under LLaMA which is licensed for non-commercial use only. The instruction data derived from OpenAI's model API also prohibits developing competing models. It is intended solely for academic research purposes.
How fast is ColossalChat training compared to other methods?
ColossalChat's training speed can be improved by more than two times compared with FSDP (Fully Sharded Data Parallel) used by Alpaca, thanks to Colossal-AI's underlying infrastructure support and optimization technologies including Zero+Gemini and LoRA.
What model sizes are available?
ColossalChat includes open-source complete RLHF training code for 7B and 13B models. The 7B model only requires less than 10 billion parameters to attain bilingual proficiency while achieving results comparable to ChatGPT and GPT-3.5.
How does the safety filter work?
A simple safety filter is applied to remove offensive content and maintain respectful interactions. However, it might accidentally treat some normal content as offensive, and the team is continuously working to improve the accuracy of the safety filter.
Where can I get the training code and dataset?
The open-source address is https://github.com/hpcaitech/ColossalAI. The project includes complete RLHF training code, the 104K bilingual dataset (available through Google Drive sharing link in the readme), model weights, and inference code. Users can post issues or submit pull requests to contribute.