AnythingLLM
AnythingLLM is a comprehensive local software suite for Mac, Windows, and Linux that lets you install large language models and interact privately with your own files. Out of the box, the AI engine, data embedding, and indexing storage remain on your hardware, allowing for entirely disconnected usag
Last verified:
What is AnythingLLM?
AnythingLLM is the easiest-to-use, all-in-one AI application that enables RAG (Retrieval-Augmented Generation), AI Agents, and much more with no code or infrastructure headaches. It is built by Mintplex Labs, Inc and is open source under an MIT license. The application works locally and offline, letting users chat with documents, use AI Agents, and leverage powerful AI tooling without any setup.
Key features include support for any LLM (local or cloud-based like OpenAI, Azure, AWS, Anthropic, Google Gemini, Hugging Face, Mistral, Groq, and more), compatibility with every document type (PDFs, Word documents, CSVs, codebases, audio files, URLs), full privacy by default with everything stored locally, multi-modal support for text/images/audio, a built-in developer API, customizable agents and data connectors, a growing plugin ecosystem, multi-user management with admin controls, and white-labeling capabilities for businesses. The desktop version ships with a built-in LLM provider (Ollama) and vector database for zero-setup local AI.
AnythingLLM is designed for individuals who want private local AI without signup, developers who need an API for custom products, and teams or organizations that require multi-user access with permissioning. It works on MacOS, Windows, and Linux, and offers both a free Desktop version and hosted cloud/self-hosted options for teams.
AnythingLLM pricing
Pricing model: Free
AnythingLLM Desktop is free and open source (MIT licensed) with no account needed. For teams, AnythingLLM Hosted Cloud offers three plans: Basic at $50/month for independent use or teams under 5 users with <100 documents (includes private instance, custom subdomain, RAG and agents working out of the box, just bring an LLM API Key); Pro at $99/month for startups and large teams (includes private instance, RAG and agents, 72-hour support SLA); and Enterprise with white-glove premium service including on-premise installation, custom support SLA, custom domain, and custom integration support. Docker version can be run for free for self-hosting.
AnythingLLM pros
- Open source and MIT licensed, free to use
- Zero-setup desktop version with built-in LLM and vector database
- Fully private by default - everything stored locally
- No account or signup needed for Desktop version
- Supports any LLM - local (Ollama, LM Studio) or cloud (OpenAI, Anthropic, Google, Mistral)
- Works with all document types: PDFs, Word, CSV, codebases, audio, URLs
- AI Agents can scrape websites, search web, make charts, save files
- Multi-modal support for text, images, and audio in one interface
- Built-in Developer API for custom development integration
- Customizable with custom agents, data loaders, and data connectors
- Growing ecosystem of plugins and integrations
- Multi-user support with full tenant isolation for teams
- Fine-grained admin controls for user permissions
- White-labelable for company branding
- Compatible across MacOS, Windows, and Linux
- Community Hub for sharing agent skills, system prompts, and slash commands
- Chat logs and event logs for tracking interactions
- Model Router for managing multiple LLMs
- Memories feature for persistent context across conversations
- Embedded chat widgets for website integration
AnythingLLM cons
- Cloud hosted version has no built-in LLM due to CPU limitations
- Built-in embedder crashes on large documents (502 error) on cloud
- Accuracy Optimized search slows/unresponsive with many files on cloud
- No custom Agent support on hosted cloud due to security concerns
- No MCP (Model-Context Protocol) support on hosted cloud
- @agent command not fully supported yet, promised as future addition
- Agent functionalities feel rigid with some functions locked to defaults
- RAG retrieval quality can be disappointing in some tests
- Weak support for Chinese language
- Query mode overly conservative, fails to respond accurately even when content exists in document
Frequently asked questions about AnythingLLM
How much is AnythingLLM?
AnythingLLM Desktop is completely free and open source under MIT license with no account or signup required. For team use, AnythingLLM Hosted Cloud has three plans: Basic at $50/month for individuals or teams under 5 users with less than 100 documents, Pro at $99/month for startups and large teams, and Enterprise with custom on-premise support. You can also run AnythingLLM via Docker for free for self-hosting.
How can I use AnythingLLM?
You can use AnythingLLM in three ways: (1) Desktop - one-click install for individuals, local by default with built-in LLM provider, no account needed; (2) Self-Hosted via Docker for free, ideal for businesses wanting full control; (3) Hosted Cloud with paid plans (Basic $50/month, Pro $99/month, Enterprise) for teams needing multi-user access with admin controls and private instances.
Where are my documents saved?
In AnythingLLM Desktop, everything is stored locally on your desktop machine - models, documents, chats, and all data. It is designed to be local by default with full privacy. Nothing is shared unless you allow it. For cloud-hosted versions, documents are stored in your private instance but you should bring your own LLM API key.
Why use AnythingLLM over ChatGPT with a PDF plugin?
AnythingLLM provides full privacy with everything running locally, supports any LLM (not just one provider), works with all document types including codebases and audio, includes AI Agents that can scrape websites and search the web, has a built-in API for custom development, is open source and free, and offers multi-modal support. ChatGPT with PDF plugin is SaaS-based with less privacy and limited document/type support.
What LLMs does AnythingLLM support?
AnythingLLM supports any LLM locally through built-in Ollama, LM Studio, Local AI, and KobaldCPP. For cloud providers, it supports OpenAI, Azure OpenAI, Anthropic, Cohere, Google Gemini Pro, Hugging Face, Together AI, OpenRouter, Perplexity AI, Mistral API, Groq, and OpenAI generic. You can run local models or connect to enterprise cloud models.
What document types can I upload?
AnythingLLM supports PDFs, Word documents (.docx), CSV files, codebases, audio files, URLs, and many more formats. You can also import documents from online locations. The application turns any document, resource, or piece of content into context that any LLM can use as references during chatting.
What are AI Agents in AnythingLLM?
AI Agents are basically LLMs that have access to tools. Agents can scrape websites, list and summarize documents, search the web, make charts, and save files to desktop and their own memory. All agents share the same tools across workspaces but operate within the workspace they were invoked via @agent. You start an agent by typing @agent <your prompt> in any workspace.
Does AnythingLLM work offline?
Yes, AnythingLLM Desktop works fully locally and offline. It ships with a built-in LLM provider (Ollama) and vector database, so you can run AI locally with no setup or internet connection required. Everything from models to documents to chats is stored locally on your machine.
Can I use AnythingLLM for my team or organization?
Yes, AnythingLLM Hosted and self-hosted is designed for teams with multi-user access and privacy. It includes full tenant isolation between users, fine-grained admin controls to control what users can do and see, white-labeling for company branding, and custom subdomains. The Pro plan at $99/month is popular for large teams with lots of documents.
Is AnythingLLM customizable?
Yes, AnythingLLM is highly customizable. You can extend it with custom agents, data loaders, and data connectors. It supports appearance customization, system prompt variables, custom slash commands, and has a Community Hub for sharing agent skills, system prompts, and extensions. For businesses, it can be white-labeled with your own branding and customized for company identity.