FreeWilly2
FreeWilly2 is an open-source AI model developed by Stability AI and fine-tuned with the Llama2 70B dataset. It is a language model that use...
Last verified:
What is FreeWilly2?
FreeWilly2 is an open-source large language model (LLM) developed by Stability AI and their Carper AI lab, based on Meta's Llama2 70B architecture. It is an auto-regressive language model fine-tuned using supervised fine-tuning (SFT) on an internal Orca-style dataset containing approximately 600,000 data points with synthetic data. The model generates text responses based on user prompts using a specific prompt format that includes system prompts, user prompts, and assistant responses.
Key features include exceptional performance in complex reasoning tasks, understanding linguistic subtleties, and solving problems in specialized domains such as law and mathematics. FreeWilly2 demonstrates performance comparable to GPT-3.5 on many tasks, achieving 86.4% accuracy on HellaSwag (outperforming GPT-3.5's 85.5%) and 68.8% on MMLU. The model was trained on only 10% of the original Orca dataset size, making it significantly more affordable and environmentally friendly with a smaller carbon footprint.
FreeWilly2 is intended for researchers, AI developers, students, and the open-source AI community who want to advance natural language understanding research. It is available under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC-4.0), which permits sharing and adapting the model for non-commercial purposes only with proper attribution. The model supports English language text generation and is available on Hugging Face for download and use with libraries like Transformers.
FreeWilly2 pricing
Pricing model: Free
FreeWilly2 is completely free to download and use under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC-4.0). There are no paid plans. The free tier includes full access to the unquantized fp16 model in pytorch format for GPU inference. However, usage is restricted to non-commercial purposes only - commercial use requires PRO Hugging Face Spaces or Inference Endpoints. The model is intended for research and open access promotion in the AI community.
FreeWilly2 pros
- Performance comparable to GPT-3.5 on multiple tasks
- Outperforms ChatGPT on HellaSwag benchmark (86.4% vs 85.5%)
- Exceptional complex reasoning capabilities
- Strong performance in specialized domains like law and mathematics
- Excellent at understanding linguistic subtleties
- Trained on only 10% of original Orca dataset size
- More affordable than original Orca model and leading LLMs
- Environmentally friendly with smaller carbon footprint
- Uses less energy during training
- Open-source with non-commercial license for research
- Based on powerful Llama2 70B architecture
- Promotes open access in AI community
- Advances natural language understanding research
- Enables complex task performance
- Available on Hugging Face for easy access
FreeWilly2 cons
- Non-commercial license prohibits profit-making or business use
- Requires 275GB memory to load (too large for automatic loading)
- Only supports English language
- Censored and SFW (safe for work) only
- Research experiment not intended for commercial deployment
- Potential biases and toxicity cannot be fully mitigated
- Underperforms GPT-3.5 on SAT Math arithmetic portion
- Cannot be used for enterprise or business objectives
Frequently asked questions about FreeWilly2
What is FreeWilly2 and what does it do?
FreeWilly2 is an auto-regressive language model developed by Stability AI based on Llama2 70B. It generates text responses to user prompts using supervised fine-tuning on an internal Orca-style dataset with 600,000 synthetic data points. The model excels in complex reasoning, linguistic subtlety recognition, and problem-solving in specialized fields like law and mathematics.
What license does FreeWilly2 use and can I use it commercially?
FreeWilly2 is released under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC-4.0). This license prohibits commercial use, profit-making, enterprise, or business objectives. You can share, adapt, and use it for non-commercial research purposes only, but must give appropriate credit to Stability AI. Commercial use requires PRO Hugging Face Spaces or Inference Endpoints.
How does FreeWilly2 perform compared to GPT-3.5?
FreeWilly2 performs on par with GPT-3.5 on many tasks. It achieved 86.4% accuracy on HellaSwag (outperforming GPT-3.5's 85.5%), 68.8% on MMLU (close to GPT-3.5's 70.0%), and outperforms GPT-3.5 on most AGIEval tasks except SAT Math arithmetic. Overall performance compares favorably with GPT-3.5 for various tasks.
What dataset was FreeWilly2 trained on?
FreeWilly2 was trained on an internal Orca-style dataset containing approximately 600,000 data points using synthetic data. This is only 10% of the original Orca dataset size. The dataset uses guidance from four datasets produced by Enrico Shippole, with prompts and language models chosen by the development team.
What are the hardware requirements for running FreeWilly2?
FreeWilly2 requires 275GB of memory to load, which is too large for automatic loading (exceeds 10GB limit). The model is intended for GPU inference and requires significant hardware resources. Users need GPUs with sufficient memory or must use quantized versions like TheBloke/FreeWilly2-GPTQ for more efficient inference.
What languages does FreeWilly2 support?
FreeWilly2 supports English language only. The model is trained specifically on English text data and generates English responses.
How was FreeWilly2 trained and what technique was used?
FreeWilly2 was trained using Supervised Fine-Tuning (SFT) in mixed-precision (BF16) with AdamW optimization. It uses the Orca Method from Microsoft's research, which teaches step-by-step reasoning processes rather than just mimicking output style. The model is fine-tuned on Llama2 70B base architecture.
Can I use FreeWilly2 for my business or startup?
No, FreeWilly2 cannot be used for商业 purposes. The CC BY-NC-4.0 license explicitly prohibits usage for profit-making, enterprise, or business objectives. The model is intended only for research and promoting open access in the AI community. For commercial use, you need to use PRO Hugging Face Spaces or Inference Endpoints.
Where can I download and access FreeWilly2?
FreeWilly2 is available on Hugging Face at stabilityai/FreeWilly2. You can download the original unquantized fp16 model in pytorch format for GPU inference. Quantized GPTQ versions for more efficient GPU inference are available at TheBloke/FreeWilly2-GPTQ. The model can be used with HuggingFace Transformers library.
What are the limitations and safety concerns of FreeWilly2?
FreeWilly2 has potential biases and toxicity that cannot be fully mitigated through fine-tuning despite using safer dataset distributions. The model is censored and SFW only. Stability AI conducted thorough risk assessment with an internal specialized team and encourages external feedback for safety improvements. Users should be mindful of potential issues in generated responses.