Gateway
The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 200+ LLMs.
Last verified:
What is Gateway?
Adaline Gateway is a local, open‑source SDK that lets developers interact with more than 300 large language models through a single unified interface without routing requests through a proxy server. It runs directly in your own environment, so all traffic stays within your infrastructure, and it exposes consistent methods for chat completions, streaming, embeddings, and tool calls across providers like OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock, Groq, Together AI, Open Router, Vertex, xAI, and custom backends. The SDK is designed for teams building AI‑first applications that need tight control over latency, observability, security, and provider‑switching without rewriting core logic.
Gateway ships with features such as built‑in batching and retry queues with exponential backoff, OpenTelemetry integration for distributed tracing, and plug‑in points for custom caching, logging, and HTTP clients. It supports both Node.js and browsers (with a flag) and exposes a strongly typed message format that captures multi‑modal inputs across all supported models. This makes it easier to write once and run against many backends, and to swap models or providers during A/B testing or failover scenarios.
The tool targets AI engineers, product teams, and platform engineers who are shipping agentic applications and want a single API surface for multiple LLM providers, while still keeping their data inside their own network boundaries. It integrates neatly into existing TypeScript or JavaScript codebases and can be used alongside Adaline’s broader platform for prompt iteration, evaluation, deployment, and monitoring, or as a standalone super‑SDK if you only need the local routing layer.
Key use cases include: agentic applications that call multiple models or providers, multi‑vendor cost or latency testing, observability‑heavy environments that require OpenTelemetry traces, and security‑sensitive deployments where no LLM traffic should leave the local environment. Because the Gateway SDK is pluggable, advanced teams can drop in their own queue, cache, or logging backends to match their internal tooling while keeping the same outward API to the rest of the codebase.
Gateway pricing
Pricing model: Freemium
Adaline Gateway itself is published as an open‑source SDK and is free to use; there is no direct licensing fee for the Gateway package or its providers. However you remain responsible for the underlying LLM provider costs (for example OpenAI, Anthropic, Google, Bedrock, etc.) incurred when you call those services through the Gateway. The broader Adaline platform (which includes prompt iteration, evaluation, deployment, and monitoring) offers additional paid functionality, but the Gateway SDK can be used standalone without a paid Adaline plan. Detailed platform pricing and tiers are available on the Adaline pricing page and may vary by workspace, region, and usage limits.
Gateway pros
- Single SDK for 300+ LLMs
- Runs entirely locally with no proxy
- Eliminates data‑exfiltration risk via local execution
- Unified interface across OpenAI, Anthropic, Google, Azure, Bedrock, Groq, Together, Open Router, Vertex, xAI, and custom providers
- Native support for streaming responses via async generators
- Built‑in batching and retry queues with exponential backoff
- Pluggable cache, HTTP client, logger, and queue backends
- Native OpenTelemetry integration for tracing and metrics
- Strongly typed message and config types for multi‑modal inputs
- First‑class embeddings interface across providers
- Built‑in tool‑call execution support
- Isomorphic design that works in Node.js and browsers (with flag)
- Clear package split between core, types, and individual providers
- Type‑safe per‑model configuration with min/max validation
- Designed from the ground up for long‑running production services
- Supports custom provider configurations and non‑standard endpoints
- Helps reduce vendor‑lock‑in by abstracting provider‑specific APIs
Gateway cons
- Limited to JavaScript/TypeScript ecosystem for now
- Adds a dependency layer between your app and the LLM APIs
- Requires managers to coordinate one Gateway instance per application for queue/cache benefits
- Browser usage requires explicit opt‑in flag and carries different security assumptions
- Smaller community than established provider SDKs
- Complex configuration surface for advanced plug‑ins may be overkill for simple apps
- Documentation assumes familiarity with TypeScript and async patterns
- Does not by itself solve prompt‑engineering or evaluation workflows outside the wider Adaline platform
Frequently asked questions about Gateway
What is Adaline Gateway?
Adaline Gateway is a single, local SDK that lets you interact with more than 300 large language models through one unified API without using a proxy server. It runs entirely in your own environment, so all traffic stays within your infrastructure, and it exposes consistent methods for chat completions, streaming, embeddings, and tool calls across major providers like OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock, Groq, Together AI, Open Router, Vertex, xAI, and custom endpoints.
How do I install Gateway?
You install Gateway by adding the core package and one or more provider packages via npm, for example with npm install @adaline/gateway @adaline/types @adaline/openai and any additional provider packages you need such as @adaline/anthropic or @adaline/bedrock. Once installed, you import the Gateway class and your chosen provider, instantiate a Gateway object, configure a model with the provider, and then call methods such as completeChat or streamChat to invoke the LLM.
Does Gateway run in my browser?
Yes, Gateway is isomorphic and can run in both Node.js and browser environments, but in the browser you must explicitly opt‑in using the dangerouslyAllowBrowser flag. This flag signals that you accept the different security and performance implications of running the SDK client‑side, such as exposing provider configuration and API keys in the browser context.
Can I use my own LLM providers or self‑hosted endpoints?
Yes, Gateway supports custom providers and non‑standard endpoints. You can configure a custom provider by defining a name, API key, base URL, and the list of model names your endpoint exposes, then use that provider in the same way as built‑in providers. This allows you to connect Gateway to internal or self‑hosted models while still benefiting from the unified types and interfaces.
What does 'fully local' mean for Gateway?
Fully local means that Gateway does not route your requests through any intermediary proxy server operated by Adaline; instead it makes HTTP calls directly to the LLM providers from your own environment. Your data, prompts, and API keys never leave your infrastructure unless you explicitly send them to the provider, which helps reduce external data‑exfiltration risk and simplifies compliance and networking.
How does Gateway handle errors and retries?
Gateway wraps LLM calls in a configurable queue and adds automatic retry logic with exponential backoff to handle transient errors, rate limits, and network issues. The queue and retry behavior are pluggable, so teams can swap in their own queue or backoff strategies if they have stricter SLOs or internal reliability tooling.
Does Gateway support streaming responses?
Yes, Gateway provides first‑class support for streaming responses via async generator methods that emit chunks incrementally. This lets you build UIs and agents that render partial output as it arrives, without blocking on the full response, and still maintain the same unified interface across different providers that support streaming.
How does Gateway integrate with observability tools?
Gateway integrates natively with OpenTelemetry, producing spans and attributes for each LLM call, embedding operation, and tool execution. This allows you to plug into existing tracing, logging, and metrics systems such as Jaeger, OTLP, or internal observability platforms and treat LLM traffic as another first‑class service in your distributed tracing topology.
Can I customize the cache, logging, or HTTP client?
Yes, Gateway exposes plug‑in points for cache, HTTP client, logger, and queue backends, so you can replace the default implementations with your own. For example, you can plug in Redis or an in‑memory cache, a custom logger that matches your log format, or a specialized HTTP client that enforces your internal TLS policies and proxy settings.
Is there a pricing difference between using Gateway and the wider Adaline platform?
The Gateway SDK itself is free and open‑source; you pay only for the underlying LLM provider usage (for example OpenAI, Anthropic, Google, Bedrock, etc.) when you call those services through the SDK. The wider Adaline platform that includes prompt iteration, evaluation, deployment, and monitoring offers additional paid plans and usage‑based tiers, but you can use Gateway standalone without a paid Adaline workspace if you only need the local multi‑provider SDK.