Sieves
Plug-and-play document AI with zero-shot models.
Last verified:
What is Sieves?
Sieves is a Python library for zero-shot document AI with structured generation that enables rapid prototyping of document AI pipelines with validated output without requiring any training data. It bundles common NLP utilities, document parsing, and text chunking capabilities together with ready-to-use tasks like classification and information extraction, all organized in an observable pipeline architecture. The library is particularly valuable for rapid prototyping scenarios where structured output is needed but training data is scarce.
The library is built around three key components: Pipeline (the main orchestrator that runs NLP tasks sequentially), Task (pre-built or custom NLP operations including classification and extraction), and Doc (the fundamental data structure for document processing). It supports multiple model frameworks including DSPy, Outlines, Transformers, LangChain, and GLiNER, allowing users to work with both local models (via Ollama, vLLM, Hugging Face) and remote cloud providers (OpenAI, Anthropic, OpenRouter).
Sieves is designed for AI/ML engineers, data scientists, and developers who need to process documents with AI and require structured, validated output. It includes built-in observability tools for monitoring pipelines, tracking token consumption at per-chunk, per-task, and per-document levels, and inspecting raw model responses for debugging. The library also supports pipeline evaluation against ground-truth data with built-in metrics for deterministic tasks and model-based evaluation for generative tasks.
Sieves pricing
Pricing model: Freemium
Sieves is an open-source Python library available on PyPI (pip install sieves) and GitHub at no cost. The core package is free. Optional extras include ingestion libraries for document parsing (pip install "sieves[ingestion]") and distillation utilities for model fine-tuning (pip install "sieves[distill]"). All optional dependencies can be installed together with pip install "sieves[distill,ingestion]". There are no paid tiers or subscription plans as it is a free open-source library maintained by Mantis, an AI consultancy.
Sieves pros
- No training data required for zero-shot tasks
- Structured generation with validated output
- Rapid prototyping of document AI pipelines
- Supports multiple model frameworks (DSPy, Outlines, LangChain, Transformers, GLiNER)
- Works with both local and remote models
- Built-in observability and monitoring tools
- Automatic token usage tracking at multiple levels
- Raw model output capture for debugging
- Pre-built tasks for classification and information extraction
- Pipeline evaluation with ground-truth support
- Modular observable pipeline architecture
- Optional document ingestion libraries (PDF/DOCX parsing)
- Supports batch processing with configurable batch sizes
- Model-based evaluation for generative tasks
- Small footprint with minimal optional dependencies
- Open source with active GitHub repository
- Label descriptions improve classification accuracy
Sieves cons
- Document ingestion libraries are optional and not installed by default
- Requires Python programming knowledge to use
- No GUI - command-line/code interface only
- Local token counting is approximate for some frameworks
- DSPy caching can return None for token counts
- Learning curve for pipeline architecture concepts
- Requires external model setup (Ollama, vLLM, cloud APIs)
- Some frameworks lack remote model support (Transformers, GLiNER)
Frequently asked questions about Sieves
What is sieves and what does it do?
sieves is a library for zero-shot document AI with structured generation. It facilitates rapid prototyping of document AI pipelines with validated output without requiring training data. It bundles NLP utilities, document parsing, text chunking, and ready-to-use tasks like classification and information extraction in an observable pipeline architecture.
Do I need training data to use sieves?
No training data is required. sieves is designed for zero-shot and few-shot NLP tasks with structured generation, making it particularly valuable when structured output is needed but training data is scarce.
What are the three key components of sieves?
The three key components are: 1) Pipeline - the main orchestrator that runs NLP tasks sequentially, defined with Pipeline([...]) or chained with +; 2) Task - pre-built or custom NLP operations like classification and extraction; 3) Doc - the fundamental data structure for document processing.
Which model frameworks does sieves support?
sieves supports DSPy, Outlines, Transformers (zero-shot classification), LangChain, and GLiNER. Each framework offers different flexibility and efficiency trade-offs, with DSPy, Outlines, and LangChain supporting remote models via providers like OpenAI, Anthropic, and OpenRouter.
Can I use local models with sieves?
Yes, sieves supports local models via Ollama (install Ollama, pull a model like llama3) and vLLM (pip install vllM, start server). Outlines also supports local Hugging Face models with GPU acceleration using device_map='auto'.
How does observability work in sieves?
Every Doc object contains a meta dictionary populated with detailed execution traces when include_meta=True (default for predictive tasks). This includes raw model outputs stored in doc.meta[task_id]['raw'] for debugging, and token usage tracked at per-chunk, per-task, and per-document levels.
How do I install sieves?
Install the core package with pip install sieves. For document parsing (PDF/DOCX), install pip install "sieves[ingestion]". For distillation utilities, use pip install "sieves[distill]". For all features, use pip install "sieves[distill,ingestion]".
How do I evaluate my pipeline in sieves?
Provide expected results in the .gold field of Doc objects. Run inference, then call pipeline.evaluate(docs) to get a report. Deterministic tasks like Classification use metrics like accuracy and F1-score, while generative tasks like Summarization use model-based evaluation with a judge model.
Who maintains sieves?
sieves is maintained by Mantis, an AI consultancy that helps clients solve business problems related to natural human language and speech. They accept feedback, feature requests, and contributions via their GitHub issue tracker.
Can I create custom tasks for sieves?
Yes, you can create custom tasks. This requires understanding ModelWrapper (backend implementations powering tasks like outlines, dspy, langchain) and Bridge (connectors between Tasks and Model wrappers). The API reference contains detailed information about parameters, configurations, and best practices for custom task creation.