Eino
The ultimate LLM/AI application development framework in Go.
Last verified:
What is Eino?
Eino is a Golang-based AI application development framework designed to be the ultimate tool for building LLM applications. It draws inspiration from LangChain and LlamaIndex while emphasizing simplicity, extensibility, reliability, and effectiveness that aligns with Go programming conventions. The framework provides component abstractions like ChatModel, Embedding, Retriever, Document Loader, PromptTemplate, and Tool, each with multiple out-of-the-box implementations.
Eino features a powerful Agent Development Kit (ADK) that provides high-level abstractions for building AI agents. This includes ChatModelAgent for ReAct-style agents, multi-agent orchestration with automatic context management, human-in-the-loop interrupt/resume mechanisms with checkpoint persistence, and pre-built agent patterns like Deep Agent for task orchestration, Supervisor for hierarchical coordination, and Sequential/Parallel/Loop agents. The ADK also includes an extensible middleware system for adding capabilities like filesystem tools and token reduction.
The framework offers three orchestration APIs: Chain for simple forward-only directed graphs, Graph for powerful cyclic or acirected graphs with runtime branching and state management, and Workflow for directed acyclic graphs with field-level data mapping. Eino handles type checking at compile time, complete stream processing (concatenation, merging, copying, conversion), concurrency management, aspect injection for callbacks/tracing/metrics, and option assignment at global/component/node levels.
Eino is designed for Go developers building production-grade AI applications and agents. It is suitable for teams needing reliable LLM applications with streaming support, complex agent orchestration, observability through callbacks, and human-in-the-loop workflows. The framework includes DevOps tools covering visual development, debugging, online tracing, and evaluation, with an Apache-2.0 open source license.
Eino pricing
Pricing model: Freemium
Eino is completely free and open source under the Apache-2.0 license. There are no paid plans, enterprise tiers, or usage fees. The framework is available on GitHub with 8.1k stars and includes all features: component abstractions, Agent Development Kit, orchestration (Chain/Graph/Workflow), callbacks, streaming, and DevOps tools. Component implementations are in the eino-ext repository, also free. CloudWeGo provides documentation and community support through GitHub Issues, Lark user groups, and community membership programs.
Eino pros
- Go-native framework with conventions matching Go programming style
- Strong type checking at compile time prevents runtime type errors
- Complete stream processing with automatic concatenation, merging, copying
- Agent Development Kit with pre-built patterns (Deep Agent, Supervisor, ReAct)
- Human-in-the-loop with interrupt/resume and checkpoint persistence
- Three orchestration options: Chain, Graph, and Workflow for different needs
- Multiple component implementations for ChatModel, Embedding, Retriever
- Extensible middleware system for agents without modifying core logic
- Callback system with 5 types: OnStart, OnEnd, OnError, OnStartWithStreamInput, OnEndWithStreamOutput
- Multi-agent support with automatic context management across agent boundaries
- Agent-as-tool capability allows one agent to call another
- Field-level data mapping in Workflow for precise input/output control
- Runtime branching allows dynamic path selection based on conditions
- State management with thread-safe StatePreHandler for shared state
- Option assignment at global, component type, or specific node level
- Apache-2.0 open source license with 8.1k GitHub stars
- Comprehensive DevOps tools for visual development and debugging
- Supports 4 streaming paradigms: Invoke, Stream, Collect, Transform
- Tool call defense mechanisms including JSON fixing and unknown tool handling
- Skill middleware for injecting knowledge/Instruction packs on demand
Eino cons
- Requires Go 1.18 or higher, limiting use with older Go versions
- No native batch processing support, requires custom graph building
- AgentTransfer/Supervisor pattern marked as not recommended due to context pollution
- Steep learning curve for complex graph orchestration concepts
- Stream handling requires careful Close() calls to avoid goroutine leaks
- Dependency on kin-openapi limited to v0.118.0 for Go 1.18 compatibility
- Sonic library incompatible with Go 1.24 without update to v1.13.2+
- Dynamic tool loading during Skill trigger may break Prompt Cache efficiency
- Multi-agent full context sharing leads to high token costs and latency
- Some model-specific issues require eino-ext package updates to fix
Frequently asked questions about Eino
What is Eino and what problem does it solve?
Eino is a Golang-based LLM application development framework that provides component abstractions, an Agent Development Kit, and powerful orchestration capabilities. It solves the challenges of building reliable AI applications by handling type checking, stream processing, concurrency management, aspect injection, and option assignment automatically. Eino draws inspiration from LangChain and LlamaIndex but emphasizes simplicity and effectiveness aligned with Go conventions.
How does Eino handle streaming responses from LLMs?
Eino provides complete stream processing with four capabilities: automatic concatenation of stream chunks for downstream nodes that only accept non-stream input, automatic conversion from non-stream to stream when needed, automatic merging of multiple streams converging to a single node, and automatic copying when streams fan out to multiple nodes or callback handlers. Compiled Graphs support 4 streaming paradigms: Invoke, Stream, Collect, and Transform.
What orchestration options does Eino provide?
Eino provides three orchestration APIs: Chain for simple forward-only directed graphs, Graph for powerful cyclic or acyclic directed graphs with runtime branching and state management, and Workflow for directed acyclic graphs with field-level data mapping. All three handle type checking, stream processing, concurrency management, aspect injection, and option assignment automatically.
What agent patterns are available in Eino ADK?
Eino ADK provides ChatModelAgent for ReAct-style agents with automatic tool calling, Deep Agent for complex task orchestration with sub-agent delegation and progress tracking, Supervisor pattern for hierarchical coordination (currently not recommended due to context issues), SequentialAgent for running agents in sequence, ParallelAgent for concurrent execution, and LoopAgent for iterative flows. Agents can transfer control to sub-agents with automatic context rewriting.
How does human-in-the-loop work in Eino?
Eino implements human-in-the-loop through Interrupt and Resume mechanisms with checkpoint persistence. Any agent can pause execution by calling adk.Interrupt() to wait for human approval or external input. Later, execution can resume precisely from where it was interrupted using runner.Resume(ctx, checkpointID). This enables interactive flows like approval workflows, missing-argument filling, and branch selection.
What components are available in Eino?
Eino provides component abstractions including ChatModel for LLM interaction, Embedding for text embeddings, Retriever for document retrieval, Document Loader for loading documents, PromptTemplate/ChatTemplate for prompt management, Tool for function calling, Indexer for indexing documents, and Lambda for custom functions. Each abstraction has multiple out-of-the-box implementations (e.g., OpenAI, Ark for ChatModel and Embedding).
How do I handle tool call errors in agents?
Eino provides multi-layered defense mechanisms: ToolArgumentsHandler for fixing invalid JSON arguments before tool execution, UnknownToolsHandler for handling calls to non-existent tools by returning prompts for self-correction, ToolAliases for handling name/parameter changes due to schema migration, and middleware for converting tool execution errors to string results so the model can auto-correct and retry instead of interrupting the flow.
Does Eino support structured model output?
Yes, Eino supports structured output through three methods: some models support direct configuration (like OpenAI's response format), obtaining structured data through tool call functionality, or writing prompts requiring structured output. After getting structured output from the model, you can use schema.NewMessageJSONParser to convert the message to the struct you need.
What dependencies does Eino require?
Eino requires Go 1.18 and above. Versions up to V0.6.0 depend on github.com/getkin/kin-openapi at version v0.118.0 for OpenAPI JSONSchema compatibility with Go 1.18. Versions after V0.6.0 no longer depend on kin-openapi. The sonic library requires version greater than v1.13.2 for compatibility with Go 1.24.
How do I visualize my graph topology?
Use the GraphCompileCallback mechanism to export the topology structure during graph.Compile. A code example for exporting as a mermaid diagram is available at https://github.com/cloudwego/eino-examples/tree/main/devops/visualize. This allows you to visualize the topology structure of graphs, chains, and workflows for debugging and documentation purposes.