Ten Framework
Open-source framework for conversational voice AI agents
Last verified:
What is Ten Framework?
Ten Framework is a real-time conversational AI agent powered by the TEN (Transformative Extensions Network) Framework, an open-source platform for building multimodal conversational AI. It delivers real-time capabilities to see, hear, and speak, supporting full-duplex conversation with natural interruption handling for seamless voice interactions. The agent integrates modern LLMs like Gemini 2.0 Live and OpenAI Realtime API, along with ASR (automatic speech recognition), TTS (text-to-speech), and RTC (real-time communication) components.
Key features include a modular architecture with self-contained extensions that can be built in Python, Node.js, Go, or C++, a visual Graph Designer (TMAN Designer) for drag-and-drop agent customization, a web-based playground UI at localhost:3000 for interacting with agents, and production-ready infrastructure for deploying conversational AI at scale. The framework supports edge-cloud integration, real-time state management, and is fully compatible with workflow platforms like Dify and Coze.
Ten Framework is designed for application developers building voice agents and conversational AI applications, extension developers creating custom integrations, and framework contributors. It's ideal for teams needing low-latency multimodal AI applications, real-time voice agents with natural conversation flow, and scalable deployment of conversational AI systems.
Ten Framework pricing
Pricing model: Freemium
TEN Framework is open-source and free to use. However, you need API keys from third-party services: Agora (free minutes every month), OpenAI (pay-per-use API key), Deepgram ASR (free credits available with signup), and ElevenLabs TTS (free credits available with signup). Hardware requirements are minimum 2 CPU cores and 4 GB RAM. Docker is required for development containers.
Ten Framework pros
- Open-source framework with active community support
- Full-duplex conversation with natural interruption handling
- Real-time multimodal AI with low latency
- Modular architecture with reusable extensions
- Multi-language support: Python, Node.js, Go, C++
- Visual Graph Designer for drag-and-drop customization
- Compatible with Gemini 2.0 Live and OpenAI Realtime API
- Works with Dify and Coze workflow platforms
- Edge-cloud integration for flexible deployment
- Production-ready infrastructure for scaling
- Real-time state management for responsive behavior
- Web-based playground UI for agent interaction
- Get started in under 10 minutes
- No glue code needed between extensions
- Runs on Windows, Mac, Linux, and mobile
Ten Framework cons
- Requires API keys from multiple services (Agora, OpenAI, Deepgram, ElevenLabs)
- Minimum 2 CPU cores and 4 GB RAM required
- Docker setup needed for development containers
- Learning curve for extension development
- Free tiers have limited minutes/credits from providers
- ARM Macs may have slower Docker build times
- Configuration via property.json can be complex
- Multiple third-party dependencies increase setup complexity
Frequently asked questions about Ten Framework
What is TEN Framework?
TEN stands for Transformative Extensions Network, an open-source framework for real-time multimodal conversational AI. It's the world's first truly real-time multimodal AI agent framework designed to establish a modular, flexible, and scalable platform that enhances development speed, maintainability, and scalability.
How do I get started with TEN Agent?
Get TEN Agent up and running in just three steps: 1) Clone the repository and set up environment variables with your API keys in .env file, 2) Start development containers, 3) Build and run the agent inside the container. The frontend will be available at http://localhost:3000. You can get started in under 10 minutes.
What API keys do I need?
You need accounts and API keys from: Agora (App ID and App Certificate - free minutes every month), OpenAI (API key or any compatible LLM), Deepgram (ASR key - free credits available with signup), and ElevenLabs (TTS key - free credits available with signup).
What are the hardware requirements?
Minimum hardware requirements are: CPU with 2 cores or more, and RAM of 4 GB or more. For macOS with Apple Silicon, you need to uncheck 'Use Rosetta for x86/amd64 emulation' in Docker settings.
Can I customize the agent behavior?
Yes, you can customize agents through the modular 'main' extension. Use the visual TMAN Designer by opening localhost:49483 in your browser, right-click to load a graph (e.g., Voice Assistant), configure API keys and preferences for each extension, then click 'Run App' to start your customized TEN Agent.
What programming languages are supported for extensions?
You can build extensions in Python, Node.js, Go, or C++. Python is ideal for AI/ML integrations (LLMs, TTS, ASR), Go for high-performance networking, and C++ for low-latency audio/video processing. You can mix and match languages in a single agent.
What is the project structure?
The project includes: property.json (main runtime configuration defining extension orchestration), extensions/ directory (modular extension modules), server/ (backend web server for HTTP/WebSocket requests), playground/ (web-based UI at localhost:3000), demo/ (production-ready deployment configurations), and agents/examples/ (ready-to-use configurations like voice-assistant, realtime-agent, story-teller).
Does TEN Agent support real-time voice?
Yes, TEN Agent supports full-duplex conversation with natural interruption handling, delivering seamless voice interactions. It provides native support for multimodal interactions with low latency, optimized for complex audio-visual AI applications.
Can I deploy TEN Agent in production?
Yes, TEN Agent is production-ready. It provides the infrastructure and patterns needed for deploying conversational AI at scale, with demo/ directory containing production-ready deployment configurations, example agent setups, deployment best practices, and integration patterns.
What LLMs can I use with TEN Agent?
TEN Agent integrates with modern LLMs like Gemini 2.0 Live and OpenAI Realtime API. You can use OpenAI API key or any compatible LLM. You can also build custom extensions for different LLM providers through the extension development framework.