Sie
Superlinked Inference Engine is an Open-source inference server and production cluster for embeddings, reranking, and extraction.
Last verified:
What is Sie?
SIE (Superlinked Inference Engine) is an open-source inference server for small AI models designed for search and document processing workloads. It runs encoders, rerankers, and entity extractors on your own infrastructure—from a laptop to a production Kubernetes cluster—without managing per-model deployments or paying per-token API costs. SIE exposes three core primitives: Encode (converts text or images to vectors for semantic search and RAG), Score (reranks query-document pairs for higher-precision retrieval), and Extract (pulls entities and structured data from unstructured text).
The tool supports 85+ models out of the box across encoding, scoring, and extraction tasks, including multi-modal and multi-vector models. The server automatically handles batching, GPU sharing, and model switching using LRU eviction, allowing many models to share one GPU. SIE is built by Superlinked, the team behind the Superlinked vector compute framework, and is released under Apache 2.0 license for AWS and GCP with Terraform and Helm included for easy setup.
SIE is designed for AI engineers building high-performance search, RAG, recommendations, and analytics applications that combine structured and unstructured data. It integrates with popular tools including Chroma, LanceDB, Qdrant, Weaviate, LangChain, LlamaIndex, CrewAI, DSPy, and Haystack. Users can deploy via Docker locally or use Helm charts for Kubernetes clusters, with SDKs available for Python and Node.js.
Sie pricing
Pricing model: Freemium
SIE is open source and free to use—the core inference server has no licensing cost. You pay only for your own GPU infrastructure on AWS or GCP. Cost comparison shows Your Cloud + SIE at $0.50 per billion tokens for bge-base on spot A10G ($0.38/hr), compared to OpenAI API at $20 per billion tokens and Modal + TEI at $1.30 per billion tokens. This represents up to 50x cost savings vs managed model APIs. Superlinked also offers managed cloud deployment for those who prefer not to self-host—contact sales to learn more.
Sie pros
- 50x cheaper vs managed model APIs like OpenAI
- 85+ SOTA models supported out of the box
- Open source under Apache 2.0 license
- Self-hosted so data stays in your AWS/GCP
- Multi-model GPU sharing with automatic LRU eviction
- Unified API for encode, score, and extract primitives
- Same Docker image works from laptop to production Kubernetes
- No per-token API costs—pay only for your GPUs
- Automatic batching, GPU sharing, and model switching
- Terraform and Helm charts included for easy deployment
- Python and Node.js SDKs with 5+ framework integrations
- Integrates with Chroma, LanceDB, Qdrant, Weaviate, LangChain, LlamaIndex
- SOC2 Type2 certified for enterprise security
- Lazy-loading allows all 85+ models available at query time
- CPU or GPU inference support for flexibility
Sie cons
- Requires your own GPU infrastructure and DevOps knowledge
- Only 2-3 standard models hot simultaneously on L4 GPU (24GB)
- Primarily optimized for small AI models, not large LLMs
- Managed cloud deployment requires contacting sales
- GPU sizing requires planning for production at scale
- Less suitable for single large model spread across many GPUs
- Newer than established servers like TEI or vLLM
- Model catalog limited to 85+ compared to thousands on HuggingFace
Frequently asked questions about Sie
What is SIE used for?
SIE is used to generate embeddings for semantic search and RAG pipelines, rerank search results to improve precision, and extract entities from unstructured text. All of this runs on your own infrastructure without paying per-token API costs. You can build applications like product search, compliance RAG, multimodal recommenders, and more using the SIE SDK.
Does SIE support GPU inference?
Yes. SIE runs on CPU or GPU. For production inference at scale, a GPU is strongly recommended. The tool supports various GPU types including A100-40GB and L4 spot instances on AWS and GCP. GPU sizing guidance is available in the Hardware and Capacity documentation.
How many models can SIE run at the same time?
SIE loads models on demand and evicts the least-recently-used models when GPU memory fills up. An L4 GPU (24GB) keeps 2 to 3 standard models hot simultaneously. All 85+ models are available at query time regardless of VRAM due to lazy-loading and LRU eviction.
Is SIE open source?
Yes. SIE is open source and available on GitHub under Apache 2.0 license. The core inference server is free to use. Superlinked also offers managed cloud deployment for enterprises that prefer not to self-host. Contact them to learn more about managed options.
How is SIE different from the Superlinked framework?
The Superlinked framework is a higher-level Python SDK for building multi-attribute search and recommendation systems. SIE is the inference layer underneath it that runs the actual model inference. You can use SIE standalone for encode/score/extract operations or as part of a full Superlinked stack.
How does SIE compare to TEI and OpenAI API?
SIE is self-hosted like TEI but supports multi-model on one GPU while TEI supports one model per server. SIE provides encode + score + extract primitives while TEI only does encode. Unlike OpenAI API, SIE is self-hosted, open source, has 85+ supported models, and has no per-token cost. SIE also validated correctness with quality and latency targets in CI.
What compute engine does SIE use under the hood?
SIE wraps PyTorch, SGLang, and Flash Attention behind three uniform primitives. The server picks the best engine per model automatically, providing a compute engine abstraction that optimizes performance without manual configuration.
Which cloud providers does SIE support?
SIE is released for AWS and GCP with Terraform modules and Helm charts included for easy setup. The same Docker image runs locally and in production Kubernetes clusters. CPU inference works on any platform including Apple Silicon.
What integrations does SIE have?
SIE has native integrations with Chroma, LanceDB, Qdrant, Weaviate, CrewAI, DSPy, Haystack, LangChain, and LlamaIndex. SDKs are available for Python (pip install sie-sdk) and Node.js (npm install @superlinked/sie-sdk), plus 5+ framework integrations.
What is the cost savings compared to managed APIs?
Users cut API costs by up to 50x compared to managed model APIs. The cost comparison shows OpenAI API at $20 per billion tokens, Modal + TEI at $1.30, and Your Cloud + SIE at $0.50 for bge-base on spot A10G. You pay for your own GPUs instead of per-token pricing.