Agentgateway
Next Generation Agentic Proxy for AI Agents and MCP servers
Last verified:
What is Agentgateway?
Agentgateway is an open source HTTP and gRPC gateway designed specifically for AI agent workloads, serving as a unified data plane for service, LLM, and MCP traffic. It routes, secures, observes, and governs services, LLM provider traffic, MCP tools, and agent-to-agent communication without requiring separate gateways. Built in Rust for high performance and memory safety, it handles stateful JSON-RPC sessions with long-lived connections that traditional API gateways cannot manage effectively.
Key features include native support for Model Context Protocol (MCP) and Agent-to-Agent (A2A) protocols, a unified OpenAI-compatible API for routing to OpenAI, Claude, Gemini, Bedrock, Vertex, and self-hosted models, and tool federation that aggregates multiple MCP servers behind a single endpoint. It provides enterprise-grade security with JWT authentication, RBAC authorization, mTLS, and protection against tool poisoning attacks. The gateway includes built-in OpenTelemetry observability with per-tool, per-agent, per-tenant metrics and distributed tracing, plus a self-service developer portal for discovering and securing agents and tools.
Agentgateway is designed for platform engineers, AI infrastructure teams, and organizations deploying agentic AI at scale. It works with any agent framework supporting MCP and A2A protocols including LangChain, CrewAI, LangGraph, AutoGen, kagent, Claude Desktop, and OpenAI SDK. The gateway can run standalone as a binary, in Docker containers, or on Kubernetes with Gateway API conformance, making it platform-agnostic for bare metal, VMs, containers, and cloud environments.
Agentgateway pricing
Pricing model: Freemium
Agentgateway is completely free and open source under the Apache 2.0 license, hosted by the Linux Foundation. There is no paid tier for the open source project itself. Enterprise software based on agentgateway is available through Solo.io (Gloo AI Gateway product) for organizations needing commercial support. The free tier includes all core features: LLM routing to OpenAI/Anthropic/Bedrock/Gemini/Vertex, MCP server connectivity, A2A protocol support, JWT authentication, RBAC, OpenTelemetry observability, Kubernetes deployment, and the self-service developer portal.
Agentgateway pros
- Open source under Apache 2.0 license hosted by Linux Foundation
- Built in Rust for high performance and memory safety
- Unified data plane for HTTP, gRPC, LLM, MCP, and A2A traffic
- Native MCP and A2A protocol support out of the box
- Unified OpenAI-compatible API for all major LLM providers
- Supports OpenAI, Anthropic, Bedrock, Gemini, Vertex, Cohere natively
- Intelligent inference routing based on GPU and KV cache utilization
- Tool federation aggregates multiple MCP servers at one endpoint
- Built-in JWT authentication and robust RBAC authorization system
- OpenTelemetry metrics and distributed tracing by default
- Self-service developer portal for agent and tool discovery
- Works with any agent framework (LangChain, CrewAI, LangGraph, AutoGen)
- Platform-agnostic: standalone binary, Docker, or Kubernetes deployment
- Dynamic configuration updates via xDS interface without downtime
- Automatic protocol upgrades and fallbacks for MCP/A2A evolution
- Protects against tool poisoning attacks with per-session authorization
- Kubernetes Gateway API conformant with HTTPRoute, GRPCRoute support
- Zero-config TLS with mTLS-rotation built in
- Semantic caching and prompt redaction for LLM traffic
- Per-team token budgets for cost management
Agentgateway cons
- Relatively new project with evolving feature set
- Requires learning YAML/JSON configuration format
- UI overwrites configuration file comments when saving
- Top-level config section cannot be dynamically updated
- Some LLM API features not supported (e.g., Vertex Responses API)
- Translation mode has limited support for new models or APIs
- Enterprise features available through Solo.io commercial product
- MCP auth spec compliance requires external OAuth provider setup
Frequently asked questions about Agentgateway
What are MCP and A2A?
Model Context Protocol (MCP) and Agent-to-Agent (A2A) are the leading protocols for enabling communication between agents and tools. MCP helps retrieve and exchange context with Large Language Models (LLMs) and connect LLMs to tools. A2A solves for long-running tasks and state management across multiple agents. Both are JSON-RPC protocols that define how an agent describes what it wants to do, how it calls tools, and how it hands off tasks to other agents.
What problem does agentgateway solve that MCP and A2A don't address?
While MCP and A2A define the RPC communication protocol, they don't address enterprise-level concerns: security (authentication, authorization, auditing), governance (policy enforcement across autonomous workflows, data residency, access control), observability (visibility into what agents are doing and why), and scalability/performance (low latency while handling retries, timeouts, and failures). Agentgateway provides built-in security, governance, and observability for all MCP and A2A communication.
What's the problem with traditional API and AI gateways?
Traditional gateways like Envoy were built for RESTful microservices with short-lived HTTP requests and no session context. MCP is a stateful JSON-RPC protocol requiring long-lived sessions where every request/response must tie to the same session context. MCP servers can also push messages asynchronously to clients. A single request like 'list available tools' may need to fan out across multiple MCP servers and aggregate responses. Traditional gateways cannot support the session and message awareness required for stateful, bidirectional communications without major re-architecture.
What is agentgateway and why should I use it?
Agentgateway is an open source gateway control plane and proxy data plane hosted by the Linux Foundation. It's a general-purpose HTTP and gRPC data plane with load balancing, timeouts, retries, TLS, rate limits, authorization, and traffic policies. You can front ordinary APIs and microservices with the same proxy used for LLM inference, MCP tool servers, and A2A agent traffic, eliminating the need for separate 'regular' and 'AI' gateways. It emphasizes enterprise-grade security, observability, resiliency, reliability, and multi-tenancy, and is built to be the most performant LLM/MCP gateway on the market.
How do I deploy agentgateway on Kubernetes?
Agentgateway includes a built-in Kubernetes controller that allows you to use the Kubernetes Gateway API to provision and configure agentgateway proxies. You can use HTTPRoute, GRPCRoute, TCPRoute, and TLSRoute resources. The Kubernetes controller conformantly implements the Gateway API project, allowing you to use agentgateway with any Gateway API implementation. Documentation is available at the Kubernetes docs section.
What's the difference between agentgateway and kagent?
Agentgateway governs agent-to-tool, agent-to-agent, and agent-to-LLM communication, ensuring these components can securely and reliably exchange data. It assumes MCP servers, tools, and agents already exist in your environment. Kagent, by contrast, is used to quickly develop, build, and run MCP servers and agents directly in Kubernetes. Kagent automates DevOps and platform engineering operations with out-of-the-box agents, tools, intelligent workflows, and built-in troubleshooting. Together they provide a complete production-ready agentic AI environment.
What license is agentgateway under?
The agentgateway project uses the Apache License 2.0, which is a permissive open source license allowing free use, modification, and distribution with minimal restrictions.
What LLM providers does agentgateway support?
Agentgateway supports OpenAI (native), Anthropic (native with translation for some APIs), Amazon Bedrock (translation), Azure (native), Google Gemini (native via compatibility endpoint), and Google Vertex AI (native via translation for some models). It also supports OpenAI-compatible providers like Cohere, Mistral, Groq, Together AI, Fireworks, and local models through Ollama, LM Studio, vLLM, and llama.cpp. The unified OpenAI-compatible API lets you switch providers without changing application code.
What agent frameworks are compatible with agentgateway?
Agentgateway is compatible with any agent framework supporting MCP and A2A protocols, including LangGraph, AutoGen, kagent, Claude Desktop, and OpenAI SDK. It also supports LangChain and CrewAI for agent-to-agent communication with identity, tracing, and replay capabilities. You can also use agentgateway to expose a REST API as an agent-native tool.
Is there enterprise software based on agentgateway?
Yes, Solo.io offers enterprise software based on agentgateway through their Gloo AI Gateway product. Solo.io donated the agentgateway project to the Linux Foundation, and they provide commercial enterprise solutions for organizations needing commercial support, SLAs, and additional enterprise features while building on the open source foundation.