Openllmetry
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
Last verified:
What is Openllmetry?
OpenLLMetry is an open-source observability platform specifically designed for GenAI and LLM applications, built on top of the industry-standard OpenTelemetry framework. It provides complete visibility into LLM applications by automatically instrumenting calls to LLM providers (OpenAI, Anthropic, Cohere, Bedrock, Gemini, and 20+ others), vector databases (Pinecone, Chroma, Weaviate, Qdrant, Milvus), and AI frameworks (LangChain, LlamaIndex, CrewAI, Haystack, LiteLLM). The tool captures prompts, responses, token usage, latency, model parameters, and error details without requiring manual code instrumentations beyond two lines of SDK initialization.
Key features include auto-instrumentation that hooks into popular AI libraries, support for Python/TypeScript/Go SDKs, workflow and task annotations to enrich traces, compatibility with 25+ observability backends (Datadog, Honeycomb, New Relic, Grafana Tempo, Dynatrace, Splunk, Traceloop, and more), and the three observability signals: traces, metrics, and logs. The semantic conventions developed for OpenLLMetry have been officially integrated into OpenTelemetry, making it a standardized approach to LLM observability. It includes a Prompt Playground, evaluation dashboards, CI/CD integrations, and prompt management capabilities when connected to Traceloop.
OpenLLMetry is designed for ML engineers, AI developers, DevOps teams, and anyone building or maintaining LLM-powered applications who needs to monitor performance, debug issues, track token costs, and optimize their AI implementations. It serves both teams new to observability (through the easy SDK) and experienced teams (through direct OpenTelemetry instrumentation). With over 6.5k GitHub stars and 881 dependent projects, it has become a widely adopted standard for LLM observability.
Openllmetry pricing
Pricing model: Freemium
OpenLLMetry itself is completely free and open-source under Apache 2.0 license with no costs. You can use it forever and connect it to any of the 25+ supported observability platforms. Traceloop (the hosted platform you can connect OpenLLMetry to) offers a Free Forever tier at $0/month including up to 50K spans/month, up to 5 seats, 24-hour data retention, monitoring dashboard, evaluation dashboard, CI/CD integration, prompt management, and all features open with no seat limit. Paid plans scale as you grow beyond the free tier limits.
Openllmetry pros
- Open-source and Apache 2.0 licensed - completely free to use
- Built on OpenTelemetry - works with 25+ existing observability platforms
- Auto-instrumentation requires only 2 lines of code to start
- Supports 20+ LLM providers including OpenAI, Anthropic, Cohere, Bedrock, Gemini
- Covers major vector DBs: Pinecone, Chroma, Weaviate, Qdrant, Milvus, Marqo, LanceDB
- Instrumentation for frameworks: LangChain, LlamaIndex, CrewAI, Haystack, LiteLLM
- Captures prompts, responses, token counts, latency, and model parameters automatically
- Supports traces, metrics, and logs - not just traces like many LLM tools
- SDK available for Python, TypeScript/JavaScript, and Go
- Vendor-neutral - no lock-in to a single observability platform
- Semantic conventions officially part of OpenTelemetry standard
- Works whether using supported frameworks or direct foundation model APIs
- Non-intrusive tracing that doesn't require code changes beyond SDK init
- Can disable batch sending for instant local trace visibility
- Anonymous telemetry is opt-out, not opt-in
Openllmetry cons
- Adds some overhead to LLM calls due to instrumentation
- Requires OpenTelemetry knowledge for advanced configuration
- Dashboard requires Traceloop cloud or self-hosted OpenTelemetry stack
- Some newer LLM providers may lag in instrumentation support
- Learning curve for teams new to OpenTelemetry concepts
- May be overkill for simple AI applications or proof-of-concepts
- SDK telemetry collects usage data (must be manually disabled)
- API keys are scoped to specific project and environment - must regenerate for new contexts
- 24-hour data retention on free tier limits historical analysis
- No built-in alerting without connecting to external observability platform
Frequently asked questions about Openllmetry
What is OpenLLMetry?
OpenLLMetry is an open-source project that allows you to easily start monitoring and debugging the execution of your LLM app. Tracing is done in a non-intrusive way, built on top of OpenTelemetry. You can choose to export the traces to Traceloop or to your existing observability stack like Datadog, Honeycomb, New Relic, or Grafana.
Can I use OpenLLMetry for free?
Yes! OpenLLMetry is Apache-2.0 licensed and you can use it for free, and connect it to any of the 25+ supported observability platforms including Traceloop. There are no costs for the OpenLLMetry instrumentation itself.
How do I start using OpenLLMetry?
The easiest way is to use the SDK. Install it with pip install traceloop-sdk for Python, then add just one line to your code: Traceloop.init(). That's it - you're now tracing your code with OpenLLMetry. For TypeScript, use npm install --save @traceloop/node-server-sdk and call traceloop.initialize().
What LLM providers does OpenLLMetry support?
OpenLLMetry supports OpenAI/Azure OpenAI, Anthropic, Cohere, Ollama, Mistral AI, HuggingFace, Bedrock (AWS), SageMaker (AWS), Replicate, Vertex AI (GCP), Google Generative AI (Gemini), IBM Watsonx AI, Together AI, Aleph Alpha, Groq, and WRITER - over 20 providers total.
What vector databases are instrumented?
OpenLLMetry instruments Chroma, Pinecone, Qdrant, Weaviate, Milvus, Marqo, and LanceDB, giving you full visibility into your vector database queries and results within your LLM application traces.
Can I use OpenLLMetry if I already have OpenTelemetry set up?
Yes! If you already have OpenTelemetry instrumented, you can just add any of the OpenLLMetry instrumentations directly without replacing your existing setup. The instrumentations integrate seamlessly with your existing OpenTelemetry configuration.
What frameworks does OpenLLMetry support?
OpenLLMetry supports LangChain, LangGraph, LlamaIndex, Haystack, LiteLLM, CrewAI, OpenAI Agents, Langflow, and the MCP protocol for comprehensive framework-level observability.
How do I connect OpenLLMetry to Traceloop?
You need an API key scoped to a specific project and environment. Generate one by going to Settings → Organization, clicking on your project (or creating a new one), selecting an environment, and clicking Generate API key. Set it as the TRACELOOP_API_KEY environment variable and you'll get instant visibility into everything happening with your LLM.
Does OpenLLMetry collect telemetry data?
The SDK contains a feature that collects anonymous usage information to detect exceptions within instrumentations. No personally identifiable information is collected. You can opt out by setting TRACELOOP_TELEMETRY environment variable to FALSE or passing telemetry_enabled=False to Traceloop.init(). Telemetry is only collected in the SDK, not when using instrumentations directly.
What observability destinations can I export to?
OpenLLMetry supports Traceloop, Axiom, Azure Application Insights, Braintrust, Dash0, Datadog, Dynatrace, Google Cloud, Grafana, Highlight, Honeycomb, HyperDX, IBM Instana, KloudMate, Laminar, New Relic, OpenTelemetry Collector, Oracle Cloud, Service Now Cloud Observability, SigNoz, Sentry, Splunk, and Tencent Cloud - over 25 destinations total.