Txtai
💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows
Last verified:
What is Txtai?
txtai is an all-in-one open-source AI framework for semantic search, LLM orchestration, and language model workflows. The key component is an embeddings database that combines vector indexes (sparse and dense), graph networks, and relational databases, enabling vector search and serving as a powerful knowledge source for large language model applications.
Key features include vector search with SQL, object storage, topic modeling, graph analysis, and multimodal indexing; embeddings creation for text, documents, audio, images, and video; pipelines powered by language models for LLM prompts, question-answering, labeling, transcription, translation, and summarization; workflows to join pipelines together; autonomous agents that connect embeddings, pipelines, and workflows; and Web and Model Context Protocol (MCP) APIs with bindings for JavaScript, Java, Rust, and Go.
txtai is built with Python 3.10+, Hugging Face Transformers, Sentence Transformers, and FastAPI. It is open-source under an Apache 2.0 license. The tool is designed for developers building autonomous agents, RAG processes, multi-model workflows, semantic search applications, and AI-powered microservices who want an integrated framework with batteries-included defaults.
Txtai pricing
Pricing model: Freemium
txtai is completely free and open-source under an Apache 2.0 license. There is no paid tier - the entire framework is available at no cost. NeuML (the company behind txtai) offers paid AI consulting services around their stack. A hosted solution called txtai.cloud is planned for future release which will offer an easy and secure way to run hosted txtai applications.
Txtai pros
- Open-source under permissive Apache 2.0 license
- All-in-one framework combining search, LLM orchestration, and workflows
- Embeddings for text, documents, audio, images, and video
- Vector search with SQL support
- Built-in FastAPI-backed full-featured API
- Language bindings for JavaScript, Java, Rust, and Go
- Supports autonomous agents built on smolagents framework
- RAG pipelines with answer and source citation capability
- YAML-configured applications with build once, run anywhere approach
- Works with micromodels to large language models
- Low footprint with optional dependencies
- Runs locally without shipping data to remote services
- Supports container orchestration (Kubernetes, Docker, serverless)
- Over 60 example notebooks and applications available
- Defaults included to get up and running in minutes
- Supports 100+ languages for translation with auto language detection
- OpenAI-compatible and MCP endpoint hosting support
- Graph networks and semantic graph analysis capabilities
Txtai cons
- Requires Python 3.10+ (older Python versions not supported)
- SQLite json_extract errors on older Python versions
- Segmentation faults on macOS requiring environment variable workarounds
- SQLite ANN errors on macOS needing special fixes
- ContextualVersionConflict issues on Google Colab requiring kernel restart
- Square brackets in pip install require escaping on MacOS zsh and Windows PowerShell
- Memory-bound systems need separate pipeline instances (best practice)
- Default shell escaping complexity for optional dependencies installation
Frequently asked questions about Txtai
What models are recommended for txtai?
txtai has a model guide with recommended models for each component: Embeddings uses all-MiniLM-L6-v2, Image Captions uses BLIP, Zero Shot Labels uses BART-Large-MNLI, Large Language Model uses Llama 3.1 Instruct, Summarization uses DistilBART, Text-to-Speech uses ESPnet JETS, Transcription uses Whisper, and Translation uses OPUS Model Series. Models can be loaded from Hugging Face Hub or local directories, and paths are optional with defaults loaded when not specified.
What is the best way to track progress of embeddings.index calls?
Wrap the list or generator passed to the index call with tqdm to track Progress. This allows you to monitor indexing progress for large datasets.
How can I analyze and debug a txtai process?
Enable observability in txtai processes as documented in the observability section. txtai also has a console application that provides debugging capabilities. The console application article has more detailed information on debugging txtai processes.
How can models be externally loaded and passed to embeddings and pipelines?
For embeddings, load the model and tokenizer externally using transformers AutoModel and AutoTokenizer, then pass them to the Embeddings constructor. For LLM pipelines, load the model and tokenizer externally (e.g., Phi 3.5-mini), then pass the tuple (model, tokenizer) to the LLM constructor. This gives you full control over model loading and configuration.
What programming languages are supported?
txtai is built with Python 3.10+ and has official language bindings for JavaScript, Java, Rust, and Go. All functionality can be accessed via the FastAPI-backed API, making it possible to use txtai from any programming language that can make HTTP requests.
How do I fix SQLite json_extract errors?
This error occurs when Python doesn't have SQLite support for json_extract. The solution is to upgrade to a newer Python version that includes proper SQLite support for this function.
How do I fix segmentation faults on macOS?
Set these environment parameters: export OMP_NUM_THREADS=1 to disable OpenMP threading (or it's handled internally), export PYTORCH_MPS_DISABLE=1 to disable PyTorch MPS device, and export LLAMA_NO_METAL=1 to disable llama.cpp metal. Refer to the GitHub issue for more details on macOS-specific workarounds.
How do I install optional dependencies on MacOS or Windows?
The default MacOS shell (zsh) and Windows PowerShell require escaping square brackets. Use pip install 'txtai[pipeline]' with single quotes around the package name to properly escape the brackets.
Can txtai run serverless?
Yes, txtai supports serverless compute on AWS Lambda, Google Cloud Functions, Azure Cloud Functions, and Kubernetes with Knative. The YAML-configured applications follow a build once, run anywhere approach, allowing API instances and workflows to run locally, on a server, on a cluster, or serverless.
What is the default API port for txtai?
The default port for the txtai API is 8000. This can be changed by configuring uvicorn. Documentation for a live running instance is available at the /docs URL (e.g., http://localhost:8000/docs).