Mellea
Mellea is a library for writing generative programs.
Last verified:
What is Mellea?
Mellea is an open-source Python library for writing generative programs—software systems that strategically integrate calls to Large Language Models (LLMs) in a reliable, predictable way. It replaces flaky agents and brittle prompts with structured, testable Python by wrapping LLM calls in readable, testable code with type-annotated outputs, verifiable requirements, and automatic retries. The library works across OpenAI, Ollama, vLLM, HuggingFace, IBM Watsonx, LiteLLM, and AWS Bedrock, making it backend-agnostic and flexible for diverse deployment scenarios.
Key features include the @generative decorator that turns typed function signatures into LLM specifications (docstrings become prompts, type hints become schemas), requirements-driven validation where you attach verifiable requirements to every LLM call, and built-in sampling strategies like rejection sampling, majority voting, and SOFAI for inference-time scaling. Mellea also offers constrained decoding for Ollama, vLLM, and HuggingFace that enforces valid output at the token level, MCP compatibility to expose programs as MCP tools, and the @mify decorator to make any Python class LLM-queryable without rewriting your data model.
Mellea is designed for Python developers, AI engineers, and teams building AI-powered pipelines, workflows, or scripts who need reliable LLM integration. It's particularly valuable for those frustrated with LangChain output parsers, untestable LLM outputs, and silent failures in AI systems. The library is open-source (204 GitHub stars), available on PyPI (version 0.2.4), and has been downloaded 7,466 times in the last month.
Mellea pricing
Pricing model: Freemium
Mellea is completely free and open-source. There is no paid tier or subscription—users can install it via 'uv pip install mellea' or 'pip install mellea' from PyPI (latest version 0.2.4). The default session runs IBM's Granite 4 (3B) model locally for free. Users can optionally use paid models from OpenAI, AWS Bedrock, Vertex AI, or other providers, but those costs are separate from Mellea itself. The library has 7,466 monthly PyPI downloads and is MIT-licensed on GitHub.
Mellea pros
- Open-source and free to use with no paid tiers
- Works with 10+ backends including OpenAI, Ollama, vLLM, HuggingFace, Watsonx, LiteLLM, and Bedrock
- Type-annotated outputs with @generative decorator—no templates or parsers needed
- Requirements-driven validation checks outputs before users see them
- Automatic retries and failure recovery built-in
- Constrained decoding enforces valid output at token level, not just retry-based
- Pluggable sampling strategies: rejection sampling, majority voting, SOFAI
- MCP compatible—expose any program as an MCP tool with validated output
- @mify decorator adds LLM query capabilities to existing Python classes without rewrites
- Explicit context threading with push/pop state for reproducible multi-turn workflows
- Async support with ainstruct(), aact(), and token-by-token streaming
- GuardianCheck detects harmful, off-topic, or hallucinated outputs
- Supports vision—pass images to instruct() and chat() with vision-capable backends
- Build RAG pipelines with vector search and grounded generation end-to-end
- Simple and composable prompting style—no large unwieldy prompts
- Pydantic models and Literal types for guaranteed structured output schemas
- Inference-time scaling with best-of-n, majority voting—swap strategies in one line
Mellea cons
- Relatively new library with smaller community compared to LangChain
- Requires Python knowledge—no GUI or low-code interface
- Intel Mac users may encounter torch/torchvision version errors requiring conda setup
- Documentation is tutorial-heavy but may lack comprehensive beginner guides
- Learning curve for generative programming concepts and patterns
- Constrained decoding only works with Ollama, vLLM, and HuggingFace, not OpenAI
- May need custom verifier writing for complex domain-specific validation
- Smaller ecosystem of pre-built components compared to established frameworks
Frequently asked questions about Mellea
What is Mellea and what problem does it solve?
Mellea is a Python library for writing generative programs—software systems that integrate LLM calls reliably. It solves the problem of unreliable LLM outputs by wrapping LLM calls in testable Python code with type-annotated outputs, verifiable requirements, and automatic retries. This replaces flaky agents and brittle prompts with structured, predictable AI.
What backends does Mellea support?
Mellea is backend-agnostic and supports OpenAI (GPT-4o, o3-mini), Ollama (local inference), vLLM, HuggingFace (Transformers), IBM Watsonx, AWS Bedrock, LiteLLM, and Google Vertex AI. The same program runs on any inference engine without code changes.
How does the @generative decorator work?
The @generative decorator turns typed Python function signatures into LLM-backed implementations. Docstrings become prompts, type hints become output schemas, and no DSL or templates are required. When you call the function, Mellea handles the LLM call and returns type-annotated, validated output.
What is requirements-driven validation in Mellea?
Requirements-driven validation lets you attach verifiable requirements to every LLM call using req(). Mellea checks every response before it leaves the session using LLM verifiers, programmatic checks, or domain-trained adapters. If a requirement fails, Mellea feeds the failure back and automatically retries.
What sampling strategies are available?
Mellea includes rejection sampling, majority voting, and SOFAI (inference-time scaling). You can swap between strategies with one parameter change—no rewrites needed. These strategies validate outputs and recover from failures automatically during inference.
How is Mellea different from LangChain?
Mellea is a predictable alternative to LangChain output parsers. While LangChain uses chains and complex abstractions, Mellea uses Python not prose—typed functions with docstrings as prompts. Mellea enforces constrained decoding at the token level (unlike Instructor and PydanticAI), has explicit context threading, and puts developers back in control rather than the LLM.
What is the @mify decorator used for?
The @mify decorator adds LLM query capabilities to any existing Python class with a single decorator—no rewrites or wrappers needed. It makes the class LLM-queryable and tool-accessible by creating an MObject, placing tools next to the data they operate over for better context engineering.
Does Mellea support async and streaming?
Yes, Mellea supports async with ainstruct(), aact(), and token-by-token streaming for production throughput and responsive UIs. This enables higher throughput in production environments and real-time streaming for user-facing applications.
Can I use Mellea with images and vision models?
Yes, you can pass images to instruct() and chat() with any vision-capable backend. This enables multimodal applications where LLMs analyze images alongside text inputs.
How do I get started with Mellea?
Install Mellea with 'uv pip install mellea' or 'pip install mellea'. By default, a new session runs IBM's Granite 4 (3B) model locally for free. Start with 'm = mellea.start_session()' and use 'm.instruct()' to generate from instructions. The documentation includes tutorials, code examples (RAG, agents, sampling, MObjects), and a full API reference.