Pixeltable
Declarative and Incremental Backend for Multimodal AI Applications
Last verified:
What is Pixeltable?
Pixeltable is the only open-source Python library providing declarative data infrastructure for building multimodal AI applications. It enables incremental storage, transformation, indexing, retrieval, and orchestration of images, video, audio, documents, and text in one unified system. Instead of stitching together 5-8 separate services (blob storage, vector databases, orchestrators, metadata stores), Pixeltable provides one table to rule them all with native multimodal column types.
Key features include: declarative computed columns that run automatically on new data (no DAG files or orchestrators needed), built-in embedding indexes for semantic search that self-maintain as data/models change, views and iterators for chunking documents/extracting video frames/splitting audio, version control with time travel and automatic snapshots, tool-calling agents with persistent memory, FastAPI serving to expose tables as HTTP endpoints, and 25+ AI provider integrations (OpenAI, Anthropic, Gemini, Hugging Face, CLIP, Whisper, YOLOX, and more). The system handles batching, retries, rate limits, and caching automatically.
Pixeltable is designed for ML engineers building multimodal AI applications, data scientists curating training datasets, developers building RAG systems and semantic search, teams building tool-calling agents with persistent memory, and anyone who needs to store, transform, index, serve, and version multimodal data without glue code. It replaces PostgreSQL/MySQL for schema, Pinecone/Weaviate/Qdrant for vector search, S3/boto3 for blob storage, Airflow/Prefect/Celery for orchestration, and LangChain/LlamaIndex for RAG patterns.
Pixeltable pricing
Pricing model: Freemium
Free and open source under Apache 2.0 license. No account required. Install with pip install pixeltable. Pixeltable OSS is completely free - run locally with same APIs that scale to production. Pixeltable Cloud offers data sharing available now (publish datasets publicly, replicate to local), with managed endpoints and live tables coming soon. No paid tiers mentioned - the core library is entirely free.
Pixeltable pros
- Open source with Apache 2.0 license - no account required
- Native multimodal types (Image, Video, Audio, Document) as first-class citizens
- Declarative computed columns run automatically on insert - no orchestrator needed
- Incremental updates - only affected cells recompute when model/UDF changes
- Built-in embedding indexes that self-maintain as data and models change
- Version control with time travel - query any historical version with table:N syntax
- Automatic snapshots for reproducible model training runs
- 25+ AI provider integrations out-of-the-box (OpenAI, Anthropic, Gemini, Hugging Face, CLIP, Whisper, YOLOX, etc.)
- Built-in batching, retries, rate limits, and caching for all AI calls
- Views and iterators for chunking documents, extracting video frames, splitting audio
- Tool-calling agents with persistent memory and reasoning traces
- FastAPI serving - tables and @pxt.query become HTTP endpoints in one line
- Full data lineage tracking - schema lineage, view lineage, computed column dependencies
- Test transformations on sample rows before committing with .head()
- Local-first architecture - 100% local with built-in versioning
- Python-native API - schema is Python, versioned automatically
- Replaces 8+ tools: PostgreSQL, Pinecone, S3, Airflow, LangChain, FastAPI, DVC, MLflow
- Multi-threaded API call support for improved throughput
- Custom UDFs with @pxt.udf and @pxt.query decorators
- Export to PyTorch, Parquet, CSV, JSON, COCO, LanceDB formats
- Import from CSV, JSON, Parquet, S3, Hugging Face datasets
- MCP server integration for interactive Pixeltable exploration
- Pixeltable Skill for AI coding assistants (Cursor, Claude Code, Codex, Windsurf)
- Primary key enforcement support
Pixeltable cons
- Requires Python 3.10 or higher
- Learning curve for declarative paradigm if coming from imperative pipelines
- Some integrations may require additional dependencies (torch, transformers, openai)
- Best suited for multimodal use cases - may be overkill for text-only applications
- Local-first means you manage your own storage infrastructure
- Cloud features (managed endpoints, live tables) still coming soon
- Some advanced video/audio UDFs may need GPU for reasonable performance
- Rate limiting still depends on external AI provider quotas
- Documentation primarily notebook-based may not suit all learning styles
- Startup overhead for small batch jobs may be higher than simple scripts
Frequently asked questions about Pixeltable
What is Pixeltable?
Pixeltable is the only open-source Python library providing declarative data infrastructure for building multimodal AI applications. It enables incremental storage, transformation, indexing, retrieval, and orchestration of data. With Pixeltable, you define your entire data processing and AI workflow declaratively using computed columns on tables, focusing on application logic rather than data plumbing.
How do computed columns work?
Computed columns are declarative transformations that run automatically on new data. When you add a computed column with add_computed_column(), Pixeltable orchestrates model execution and ensures results are stored, indexed, and accessible through the same query interface. When new data is inserted, Pixeltable incrementally runs all computed columns against only the new data - no orchestration code or DAG files needed. Change a model or UDF and only affected cells recompute.
What multimodal types does Pixeltable support?
Pixeltable supports native multimodal column types: Image, Video, Audio, Document, Json, Array, Binary, String, and Float. These are first-class citizens - not opaque blobs. Images, video, audio, and documents integrate seamlessly with structured data in one unified interface without requiring external blob storage.
How does version control work?
Every operation that modifies a table creates a new version automatically - no configuration required. You can view history with t.history(), query any historical version using table_name:version syntax (e.g., pxt.get_table('myapp.media:3')), revert changes with t.revert(), and create named persistent snapshots with pxt.create_snapshot(). Versioning is always on and tracks inserts, updates, deletes, and schema changes.
What AI providers does Pixeltable integrate with?
Pixeltable has built-in support for 25+ AI providers including OpenAI, Anthropic, Gemini, Bedrock, Mistral, Groq, DeepSeek, Hugging Face, CLIP, Whisper, Voyage, Jina, Together, Fireworks, Ollama, Replicate, fal.ai, RunwayML, BFL FLUX, Twelve Labs, and more. All integrations are available out-of-the-box with batching, retries, rate limits, and caching handled automatically.
How do embedding indexes work?
Embedding indexes are added with add_embedding_index() and provide vector search built in. Indexes self-maintain as data and models change - you don't need to manually rebuild them. You can search by similarity using .similarity() in queries, combining semantic search with metadata filtering in one expression. Multiple embedding indexes (CLIP, MiniLM, Twelve Labs) can coexist on the same table.
What are views and iterators?
Views are virtual derived tables that avoid storing redundant data and automatically keep derived data in sync with their source. Iterators explode rows: video to frames (FrameIterator), documents to chunks (document_splitter), audio to segments. You create views with pxt.create_view('name', source_table, iterator=...). Views are best for transforming data, filtering subsets, chaining multiple transformations, and saving storage by computing on demand.
How do I build agents with Pixeltable?
Pixeltable supports tool-calling agents with persistent memory. Use pxt.tools() to expose UDFs and @pxt.query functions as tools that LLMs can call. The LLM decides which tool to call and Pixeltable executes it. Agents get persistent memory through embedding indexes on conversation history, MCP server integration, and automatic reasoning traces. Tables themselves can act as agents with persistent memory.
How do I serve Pixeltable tables as HTTP endpoints?
Use FastAPIRouter or pxt serve to expose tables and @pxt.query as HTTP endpoints. Configure in pyproject.toml with [[tool.pixeltable.service]] specifying name, prefix, modules, and routes for insert/update/query operations. Run with pxt serve my-service. Background jobs are included. Tables and queries become endpoints in one line without writing FastAPI + Pydantic manually.
Can I use Pixeltable with PyTorch?
Yes. Pixeltable supports exporting to PyTorch with pxt.io.export_parquet() and can create PyTorch datasets directly with DataLoader(t.to_pytorch_dataset(), batch_size=32). This is ideal for curating, augmenting, and exporting training datasets. You can auto-annotate with AI, version everything, and export to Parquet or PyTorch for ML workflows.