Openlit
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
Last verified:
What is Openlit?
OpenLIT is an open-source AI engineering platform that helps teams build, evaluate, and observe AI applications across the entire lifecycle from development to production. It is an OpenTelemetry-native LLM observability platform that enables production-ready AI application and agent monitoring without any code changes. The platform provides comprehensive tracing, evaluation, prompt management, experimentation, and real-time monitoring capabilities for LLM applications.
Key features include distributed tracing with OpenTelemetry-native SDKs for automatic instrumentation of LLMs, agents, vector databases, and GPUs; AI model evaluation with 11 built-in evaluator types (hallucination, bias, toxicity, relevance, coherence, faithfulness, safety, instruction following, completeness, conciseness, sensitivity); Prompt Hub for centrally managing, versioning, and deploying prompts; OpenGround for experimenting with prompts and models to find the most efficient ones; customizable dashboards with SQL queries and flexible widgets; and secure secrets management via Vault for storing LLM API keys.
OpenLIT is designed for AI engineers, ML practitioners, developers building LLM applications, DevOps teams managing AI infrastructure, and organizations running AI applications in production. It supports Python, TypeScript/JavaScript, and Go, and works with 60+ AI integrations including OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock, Ollama, LangChain, LlamaIndex, CrewAI, ChromaDB, Pinecone, and many more.
The platform is built for production workloads with minimal performance overhead, follows OpenTelemetry Semantic Conventions for AI observability standards, and is self-hostable under Apache 2.0 license. It provides zero-code observability through the OpenLIT Controller using eBPF for Kubernetes, Docker, and Linux deployments.
Openlit pricing
Pricing model: Freemium
OpenLIT is completely open source and free under the Apache 2.0 license. The platform is self-hosted with no vendor lock-in. Users deploy their own instance using Kubernetes (via Helm chart) or Docker Compose, which requires three components: OpenLIT platform, ClickHouse for storage, and OpenTelemetry Collector for telemetry collection. Existing ClickHouse or OpenTelemetry Collector infrastructure can be reused. There are no paid tiers or subscription plans - the entire feature set including distributed tracing, AI model evaluation, prompt management, dashboards, and secrets management is available free. Users can support the project through token supporter or context window hero contributions on GitHub, which provide recognition like logo placement on README and website.
Openlit pros
- Open source and always free under Apache 2.0 license
- Zero code changes required for AI observability
- OpenTelemetry-native for seamless integration with existing stacks
- Automatic instrumentation for 60+ AI integrations
- 11 built-in evaluation types including hallucination and bias detection
- Supports Python, TypeScript/JavaScript, and Go languages
- Self-hostable with privacy-first architecture
- Real-time cost tracking and token usage optimization
- Prompt Hub with version control and collaborative editing
- Customizable dashboards with SQL query support
- OpenLIT Controller provides zero-code Kubernetes observability via eBPF
- Online LLM-as-a-judge for automated quality monitoring
- Secure Vault for centrally storing LLM API keys
- Supports all major LLM providers out of the box
- Distributed tracing with complete end-to-end visibility
- GPU monitoring for NVIDIA and AMD hardware
- Programmatic evaluations for custom testing workflows
- Multi-deployment management from single dashboard
Openlit cons
- Requires self-hosting infrastructure (ClickHouse, OpenTelemetry Collector)
- Learning curve for OpenTelemetry concepts
- Documentation primarily focused on technical users
- Custom evaluation types require prompt engineering knowledge
- Dashboard customization requires SQL knowledge for advanced queries
- Limited managed cloud offering compared to commercial alternatives
- Community support instead of enterprise SLA
- Upgrade process requires --remove-orphans flag for Docker Compose
Frequently asked questions about Openlit
What is OpenLIT and what does it do?
OpenLIT is an open-source AI engineering platform that helps teams build, evaluate, and observe AI applications across the entire lifecycle from development to production. It is an OpenTelemetry-native LLM observability platform that enables production-ready AI application and agent monitoring without any code changes. OpenLIT provides distributed tracing, AI model evaluation with 11 built-in evaluators, prompt management via Prompt Hub, experimentation with OpenGround, customizable dashboards, and secure secrets management.
How do I install OpenLIT?
OpenLIT can be installed via pip with 'pip install openlit' and initialized with 'openlit.init()' for zero-code instrumentation. For full platform deployment, use Docker Compose by cloning the repository and running 'docker compose up -d', or deploy on Kubernetes using the OpenLIT Helm chart with 'helm install openlit openlit/openlit'. The Docker Compose setup deploys three components: OpenLIT platform, ClickHouse database, and OpenTelemetry Collector.
What LLM providers and frameworks does OpenLIT support?
OpenLIT supports 60+ AI integrations including LLM providers like OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock, Ollama, Groq, Cohere, Mistral, Deepseek, GPT4All, Amazon Bedrock, Vertex AI, and NVIDIA NIM. It also supports AI frameworks like LangChain, LlamaIndex, CrewAI, mem0, AG2, DSPy, Agno, Haystack, Phidata, and vector databases including ChromaDB, Pinecone, Qdrant, Milvus, and Weaviate. GPU monitoring is available for NVIDIA and AMD hardware.
What are the 11 built-in evaluation types in OpenLIT?
OpenLIT provides 11 evaluation types: Hallucination (identifies factual inaccuracies and fabricated information), Bias (monitors discriminatory patterns), Toxicity (screens for harmful language), Relevance (evaluates response addressing the prompt), Coherence (assesses logical flow and clarity), Faithfulness (measures alignment with context), Safety (detects jailbreak attempts), Instruction Following (evaluates constraint adherence), Completeness (assesses if all query parts are addressed), Conciseness (evaluates appropriate brevity), and Sensitivity (detects PII leakage and privacy concerns). Core evaluators (hallucination, bias, toxicity) are enabled by default.
How does the LLM-as-a-Judge feature work?
LLM-as-a-Judge is a technique to evaluate LLM application quality using powerful language models as evaluators. The LLM judge analyzes AI outputs and provides structured scores, classifications, and detailed reasoning about response quality, safety, and performance. It's scalable and cost-effective, evaluating thousands of outputs automatically. Users configure the provider and model via Settings, add API keys from Vault, and can enable auto-evaluation with cron schedules. Results show scores (0-1), classifications, explanations, and verdicts with detailed reasoning.
What is the Prompt Hub feature?
Prompt Hub is OpenLIT's central prompt management system that allows creating, editing, and tracking different versions of prompts. It supports major, minor, and patch versioning, dynamic variables using {{variableName}} placeholders that are replaced at runtime, and statistics viewing for download stats and version history. Prompts can be retrieved via SDK (Python or TypeScript) or API with authentication through API keys. The SDK allows fetching prompts by name or ID, compiling with variables, and accessing metadata.
How does OpenLIT handle zero-code observability?
OpenLIT provides zero-code observability through two main approaches. First, the OpenLIT SDK uses OpenTelemetry auto-instrumentation - simply run 'openlit-instrument python your_app.py' or call 'openlit.init()' to enable automatic instrumentation for AI apps and agents. Second, the OpenLIT Controller uses eBPF to discover and instrument LLM traffic across Kubernetes, Docker, and Linux without any code changes. Deploy the AutoInstrumentation CR and select workloads - the OpenLIT Operator automatically injects AI observability into Kubernetes workloads without rebuilds.
What programming languages does OpenLIT support?
OpenLIT supports three programming languages: Python with complete AI observability and automatic dependency detection for zero-code instrumentation; TypeScript/JavaScript with full LLM monitoring support, distributed tracing, metrics, and cost optimization; and Go with OpenTelemetry-native instrumentation that wraps OpenAI and Anthropic clients for automatic tracing, token tracking, and cost monitoring. Each language has dedicated SDKs and documentation.
How do OpenLIT dashboards work?
OpenLIT dashboards transform telemetry data into actionable insights through a streamlined process: AI telemetry flows through OpenTelemetry instrumentation into ClickHouse, widgets execute optimized SQL queries against the telemetry data, and results are displayed as charts, tables, statistics, and custom visualizations. Users can create and resize custom widgets with flexible configurations, write custom SQL queries to analyze AI telemetry, create interactive dashboards with filters and variables, and export/import dashboard configurations as JSON files. Dashboards support visualization from any OpenTelemetry-instrumented tool.
What is OpenGround and how does it help?
OpenGround is OpenLIT's experimentation playground for comparing prompts and models to find the most efficient ones. It allows users to experiment with different prompt variations and models to compare cost, duration, and response tokens across LLMs. This helps organizations choose the most efficient model for their specific use case by providing data-driven insights into performance metrics. OpenGround is part of the experimentation features that enable optimizing AI applications before production deployment.