Any Llm
Communicate with an LLM provider using a single interface
Last verified:
What is Any Llm?
Any Llm is a Python library that provides a single unified interface to access multiple Large Language Model (LLM) providers including OpenAI, Anthropic, Mistral, Google, AWS Bedrock, Azure OpenAI, Cohere, Together AI, and local models like llamafile and ollama. By changing only a single configuration parameter, developers can easily switch between providers and models without rewriting their code, making it ideal for adapting to changing needs, markets, or costs.
The library offers two main interfaces: Direct API Functions (completion, acompletion) recommended for simple use cases and scripts, and the AnyLLM Class recommended for production use cases with connection pooling and client state management. Key features include streaming support for real-time responses, full type hints for IDE support, standardized reasoning output across all models, List Models API to programmatically query supported models, Responses API support, and auto-updating provider compatibility matrix showing which features are supported by which providers.
Any Llm is designed for developers building agents, LLM-powered applications, or any project requiring flexibility to use different language models. It is framework-agnostic and can be used across different projects. The library leverages official provider SDKs when available to ensure compatibility, performance, and stability, and requires no proxy or gateway server setup. The Any Llm platform (in Beta) is built on top of the library and offers centralized API key management with client-side encryption, usage analytics, cost tracking, and a unified dashboard.
The library is actively maintained by Mozilla.ai, integrated with any-agent (one of their most community-engaged projects), and has reached production-ready v1.0 stability. It is used in Octonous, where every agent runs on Any Llm enabling seamless provider switching. The project has Apache-2.0 license, 1k GitHub stars, and 81 forks.
Any Llm pricing
Pricing model: Freemium
The any-llm Python library is open-source under Apache-2.0 license and completely free to use. The any-llm platform (built on top of the library) is currently in Beta and free to use. The platform allows secure API key storage with client-side encryption, usage analytics, and cost tracking across all LLM calls at no cost. In the future, Mozilla.ai plans to introduce paid tiers with advanced features and enterprise-grade capabilities, but they commit to keeping visibility into changes with no surprise paywalls. The library itself has no paid tiers - users only pay their chosen LLM providers directly for model usage.
Any Llm pros
- Single unified interface for all LLM providers
- Switch providers in seconds without rewriting code
- No proxy or gateway server required
- Uses official provider SDKs when available
- Full type hints for better IDE support
- Streaming support for real-time responses
- Actively maintained by Mozilla.ai team
- Framework-agnostic, works across projects
- Production-ready v1.0 stability
- Standardized reasoning output across all models
- List Models API to query supported models
- Auto-updating provider compatibility matrix
- Responses API support for OpenAI-style responses
- Async-first APIs for high-throughput use cases
- Client-side encryption for API key security in platform
- Free to use during Beta period
- Supports both cloud and local models
- Clear deprecation and experimental notices
- Reusable client connections for improved performance
Any Llm cons
- Python-only library (Go port is separate any-llm-go)
- Requires separate API keys for each provider
- Model IDs depend on provider documentation
- Platform currently in Beta only
- No budget alerts yet (coming soon)
- No hosted guardrails yet (coming soon)
- No smart routing yet (coming soon opt-in)
- No native batch completions yet (coming soon)
Frequently asked questions about Any Llm
What is any-llm and what does it do?
any-llm is a Python library providing a single unified interface to access multiple LLM providers including OpenAI, Anthropic, Mistral, Google, AWS Bedrock, Azure OpenAI, Cohere, Together AI, and local models. It normalizes outputs to return OpenAI ChatCompletion objects regardless of which provider is used under the hood, allowing developers to switch between providers by changing only a single configuration parameter without rewriting their code.
Which LLM providers does any-llm support?
any-llm supports OpenAI, Anthropic, Mistral, Google, AWS Bedrock, Azure OpenAI, Cohere, Together AI, and local models like llamafile and ollama. The library uses an auto-updating provider compatibility matrix showing which features are supported by which providers. Users can view the complete list of supported providers in the documentation.
How do I install any-llm?
Install any-llm using pip with the providers you plan to use: pip install 'any-llm-sdk[mistral,ollama]' or use the all option to install support for all providers: pip install any-llm-sdk[all]. You also need to set the appropriate API key environment variable for your provider (e.g., export MISTRAL_API_KEY='YOUR_KEY_HERE' or export OPENAI_API_KEY='YOUR_KEY_HERE'). Python 3.11 or newer is required.
What are the two main interfaces in any-llm?
any-llm offers two main interfaces: (1) Direct API Functions (completion, acompletion) recommended for simple use cases, scripts, and notebooks - creates a new provider client on each call with minimal code; (2) AnyLLM Class (AnyLLM.create) recommended for production use cases - reuses the same provider client across multiple calls with connection pooling and client state management for high-throughput and streaming use cases.
Do I need to set up a proxy or gateway server?
No, any-llm does not require any proxy or gateway server. You simply download the any-llm SDK and plug in your provider API keys to start building. This is different from proxy/gateway solutions like OpenRouter and Portkey that require setting up a hosted proxy server as an intermediary between your code and the LLM provider.
How does any-llm handle different API formats from providers?
any-llm solves API standardization issues by normalizing outputs to return OpenAI ChatCompletion objects regardless of which provider is used under the hood. The objects are returned as OpenAI Pydantic models, so you can access them just as you would if using the official OpenAI API SDK. This handles slight variations in parameter names, response formats, and feature sets across providers while maintaining a consistent interface.
Is any-llm actively maintained?
Yes, any-llm is actively maintained by Mozilla.ai. It is integrated with any-agent, one of Mozilla.ai's most community-engaged projects, which motivates continued support. The library reached version 1.0 in October 2025, marking production-ready stability. The project has 1k GitHub stars, 81 forks, 18 contributors, and 40 releases showing active development.
What is the any-llm platform?
The any-llm platform is built on top of the any-llm library and provides simplified management and analytics for all LLM providers. It allows you to securely store API keys with client-side encryption (keys encrypted client-side in browser, never pass through Mozilla's systems), gather usage insights across all LLM calls, track token counts per provider and user, see real-time spending in a unified dashboard, and get one API key configured to access multiple providers. The platform is currently in Beta and free to use.
How does any-llm handle streaming responses?
any-llm supports streaming for real-time responses from any model. The streaming support works across all providers, and the chat demo showcases real-time streaming with character-by-character display. The Async APIs (acompletion, aresponses) are designed for high-throughput and streaming use cases with reusable client connections. Collapsible thinking content display is available for supported models that produce reasoning output.
What's coming soon in any-llm?
Upcoming features include: native batch completions support, adding support for new providers, deeper integrations with any-suite libraries (any-guardrail, any-agent, mcpd), budget alerts for cost control, hosted guardrails for LLM safety and control, error and model latency tracking, smart routing to choose the best model (local or remote) for each request (opt-in), and model recommendations based on observed patterns (opt-in). The paid tiers with advanced features and enterprise-grade capabilities are also planned for the platform.