Openinference
OpenTelemetry Instrumentation for AI Observability
Last verified:
What is Openinference?
OpenInference is an open-source semantic convention specification for AI application observability built on top of OpenTelemetry. It standardizes how LLM calls, agent reasoning steps, tool invocations, retrieval operations, embedding generations, and other AI-specific workloads are represented as distributed traces. Every OpenInference trace is a valid OTLP trace, with conventions that give attribute names their AI-specific meaning.
Key features include a comprehensive span-kind taxonomy with 10 span types (LLM, AGENT, CHAIN, TOOL, RETRIEVER, RERANKER, EMBEDDING, GUARDRAIL, EVALUATOR, PROMPT), detailed attribute schemas for inputs/outputs/token counts/model parameters, privacy controls via environment variables to hide sensitive data like prompts/embeddings/inputs/outputs, and multi-language support with instrumentations for Python and JavaScript. It supports popular frameworks like LangChain, LlamaIndex, Haystack, DSPy, and smolagents, and works with any OpenTelemetry-compatible backend including Arize Phoenix and Arize AX.
OpenInference is designed for AI/ML engineers, LLM application developers, DevOps teams building AI observability, and organizations deploying production AI agents. It helps teams debug LLM applications, monitor token economics, trace agentic control flow, reproduce nondeterministic LLM outputs, and maintain data privacy compliance while gaining visibility into their AI systems.
The project is open-source and free to use, with instrumentations available on PyPI and npm. While the instrumentation itself is free, using it with Arize's hosted AX platform offers free tiers (25k spans/month) and paid plans, but OpenInference traces can be sent to any OTLP-compatible backend of your choice.
Openinference pricing
Pricing model: Freemium
OpenInference itself is 100% free and open-source with no paid tiers. The instrumentation packages are available on PyPI and npm at no cost. When used with Arize Phoenix (self-hosted open-source), it is also free with user-managed trace spans, ingestion volume, projects, and retention. When used with Arize AX SaaS platform: AX Free tier includes 25k spans/month and 1GB data with 15-day retention; AX Pro is $50/month with 50k spans/month, 10GB data, and 30-day retention. Additional spans cost $10 per million and additional GB costs $3 per GB on paid plans.
Openinference pros
- Open-source and free to use with no licensing costs
- Built on OpenTelemetry for universal compatibility
- Every OpenInference trace is a valid OTLP trace
- Comprehensive span-kind taxonomy with 10 AI-specific types
- Standardizes LLM call representation across frameworks
- Supports token count tracking as first-class metrics
- Privacy controls via environment variables for data masking
- Multi-language support with Python and JavaScript instrumentations
- Works with any OpenTelemetry-compatible backend
- Native support for LangChain, LlamaIndex, Haystack, DSPy
- Captures multimodal content including images and audio
- Tracks agent reasoning loops and nested tool calls
- Environment variable configuration for quick setup
- Core utilities for session/user/metadata tracking
- Vendor and framework agnostic design
Openinference cons
- Requires OpenTelemetry SDK installation as dependency
- Steep learning curve for OpenTelemetry newcomers
- Configuration must run before any application code
- No built-in hosted UI - requires separate backend
- Manual instrumentation needed for custom operations
- Environment variables can be complex to manage at scale
- Base64 image length limited to 32000 characters by default
- Embedding vectors hidden requires explicit configuration
- No native support for non-OTLP backends
- Distributed tracing overhead on application performance
Frequently asked questions about Openinference
What is OpenInference?
OpenInference is a semantic convention specification for AI application observability built on OpenTelemetry. It standardizes how LLM calls, agent reasoning steps, tool invocations, retrieval operations, and other AI-specific workloads are represented as distributed traces. Every OpenInference trace is a valid OTLP trace, with conventions that give attribute names their AI-specific meaning.
Is OpenInference free to use?
Yes, OpenInference is completely free and open-source. The instrumentation packages are available on PyPI and npm at no cost, and there are no licensing fees. You can use it with self-hosted Phoenix (also free) or send traces to any OpenTelemetry-compatible backend.
What languages does OpenInference support?
OpenInference currently provides instrumentations for Python and JavaScript/Node.js. Both versions follow the same specification and semantic conventions, allowing consistent tracing across polyglot applications.
What span kinds does OpenInference define?
OpenInference defines 10 span kinds: LLM (language model API calls), AGENT (autonomous agent reasoning steps), CHAIN (deterministic operation sequences), TOOL (function/external API execution), RETRIEVER (vector store/search queries), RERANKER (relevance reordering), EMBEDDING (vector generation), GUARDRAIL (moderation checks), EVALUATOR (automated response evaluation), and PROMPT (named prompt template invocation).
Can I use OpenInference without Arize?
Yes, OpenInference works with any OpenTelemetry-compatible backend. While it is natively supported by Arize and Arize Phoenix, you can send OTLP traces to Jaeger, Zipkin, Honeycomb, Datadog, or any other OTLP-receiving backend.
How do I hide sensitive data in traces?
OpenInference provides environment variables for data privacy: OPENINFERENCE_HIDE_INPUTS, OPENINFERENCE_HIDE_OUTPUTS, OPENINFERENCE_HIDE_PROMPTS, OPENINFERENCE_HIDE_EMBEDDINGS_VECTORS, OPENINFERENCE_HIDE_LLM_INVOCATION_PARAMETERS, and more. These can also be configured in code using TraceConfig for finer control.
What frameworks does OpenInference support?
OpenInference provides out-of-the-box instrumentations for popular ML frameworks including LangChain, LlamaIndex, Haystack, DSPy, and smolagents. It supports LLM providers like OpenAI, Bedrock, MistralAI, VertexAI, and LiteLLM.
How do I install OpenInference Python?
Install the OpenTelemetry SDK first, then install the OpenInference instrumentation for your LLM framework. For example: pip install opentelemetry-sdk && pip install openinference-instrumentation-openai. You may also need @arizeai/openinference-semantic-conventions for manual instrumentation.
What attributes does OpenInference capture?
Attributes are typed key/value pairs using dot-separated namespaces like llm.input_messages, llm.token_count.prompt, llm.model_name, embedding.embeddings, tool.function.name. They carry prompts, responses, model names, retrieved documents, tool arguments, token counts, and all context needed to understand and reproduce executions.
How does OpenInference handle token economics?
Token counts are first-class metrics in OpenInference. It captures prompt tokens, completion tokens, and breakdowns for cached and reasoning tokens via attributes like llm.token_count.prompt, llm.token_count.completion, and llm.token_count.total. This enables operational monitoring of LLM costs.