Sparrow
Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM
Last verified:
What is Sparrow?
Sparrow is an open-source AI tool for extracting structured data from documents using ML, LLM, and Vision LLM technologies. It handles invoices, receipts, bank statements, forms, tables, and other unstructured document sources, converting them into structured JSON output. The tool features a pluggable architecture that allows users to mix different pipelines including Sparrow Parse (Vision LLM), Sparrow Instructor (Text LLM), and Sparrow Agents for workflow orchestration.
Key features include support for multiple backends (MLX for Apple Silicon, Ollama, vLLM, Docker, Hugging Face Cloud GPU), multi-format support for images (PNG, JPG) and multi-page PDFs, JSON schema-based extraction with automatic validation, RESTful APIs for integration, instruction calling for text processing and decision making, and a built-in dashboard for visual monitoring. The web interface at sparrow.katanaml.io offers drag-and-drop document upload, real-time processing, and result annotation with bounding boxes.
Sparrow is designed for developers, data engineers, and organizations needing to automate document processing workflows. It suits teams working with financial reports, lab results, portfolio statements, medical prescriptions, and any scenario requiring extraction of structured data from scanned documents or images. The tool supports both open-source and commercial use with dual licensing options.
Sparrow pricing
Pricing model: Freemium
Sparrow is open-source under GPL 3.0 license, free for open-source projects and organizations with gross revenue below $5 million USD in the past 12 months. Commercial dual licensing is available for proprietary use, enterprise features, and dedicated support. Organizations exceeding the $5M revenue threshold or needing to integrate Sparrow into proprietary software without disclosing source code modifications must contact [email protected] for commercial licensing. The online demo at sparrow.katanaml.io runs on Mac Mini M4 Pro 64GB and is available for testing.
Sparrow pros
- Open-source with GPL 3.0 license for free use
- Pluggable architecture with multiple pipeline options
- Supports MLX backend for optimal Apple Silicon performance
- RESTful API-first design for easy integration
- Handles multi-page PDFs with per-page structured output
- JSON schema-based extraction with automatic validation
- Built-in dashboard for usage analytics and monitoring
- Supports local LLMs via Ollama and Apple MLX
- Table extraction mode optimized for large tables
- Drag-and-drop web interface for interactive processing
- Vision LLM support including QwenVL, Mistral Small, Gemma3
- Instruction calling beyond document extraction
- Bounding box annotation for result visualization
- Works with invoices, receipts, forms, bank statements
- Enterprise-ready with rate limiting and usage analytics
Sparrow cons
- Requires Python 3.12.10+ which may conflict with existing setups
- MLX backend only works on macOS with Apple Silicon
- Large Vision LLM models require significant GPU memory (35-50GB)
- CPU-only mode is significantly slower than GPU backends
- Requires poppler system dependency for PDF processing
- Complex setup with multiple virtual environments for different pipelines
- Memory errors possible with large PDFs without manual splitting
- Model loading can fail requiring cache clearing
- Documentation assumes technical expertise in ML/LLM
- Commercial licensing requires direct contact for pricing
Frequently asked questions about Sparrow
What types of documents can Sparrow process?
Sparrow handles invoices, receipts, bank statements, forms, tables, financial reports, lab results, portfolio statements, medical prescriptions, and other structured or unstructured documents. It supports image formats (PNG, JPG) and multi-page PDFs.
What backends does Sparrow support?
Sparrow supports multiple backends: MLX for Apple Silicon (macOS), Ollama for NVIDIA/AMD GPU, vLLM, Docker, and Hugging Face Cloud GPU. CPU-only mode is also available but significantly slower.
How do I install Sparrow?
Install pyenv and Python 3.12.10, create a virtual environment, clone the repository, install pipeline-specific dependencies (e.g., pip install -r requirements_sparrow_parse.txt), install poppler for PDF processing, and start the API server with python api.py.
What is the difference between Sparrow Parse and Sparrow Instructor?
Sparrow Parse uses Vision LLM for structured data extraction from documents with JSON schema validation. Sparrow Instructor uses Text LLM for instruction-based processing like summarization, analysis, and Q&A with free-form text output.
Can Sparrow extract data from tables?
Yes, Sparrow has a table processing mode optimized for large tables. Use --options tables_only to extract only tables, and the tool comes with separate template scripts to process OCR output into structured JSON with field mapping.
How do I use the API?
Start the server with python api.py (default port 8002), then use curl or HTTP clients to POST to /api/v1/sparrow-llm/inference for document extraction or /api/v1/sparrow-llm/instruction-inference for text instructions. Visit localhost:8002/api/v1/sparrow-llm/docs for Swagger documentation.
What models does Sparrow support?
Sparrow supports Vision LLMs including Qwen2.5-VL-72B-Instruct, Qwen2.5-VL-7B, Mistral-Small-3.2-24B, olmOCR, and Gemma3. Model selection depends on your backend: MLX for Apple Silicon, Ollama for GPU, or Hugging Face Cloud.
How do I handle large PDFs?
For large PDFs, use --crop-size to reduce image size, enable --options tables_only to process only tables, use --debug-dir to monitor processing, or manually split large PDFs into smaller files before processing.
Is commercial use allowed?
Yes, Sparrow is free for commercial use by organizations with gross revenue below $5M USD in the past 12 months. For larger organizations or proprietary integration without open-source requirements, contact [email protected] for dual licensing.
What is Sparrow Agent?
Sparrow Agent orchestrates complex multi-step document processing workflows with visual monitoring powered by Prefect. It supports chaining classification, extraction, and validation steps with robust error recovery and extensible custom agents.