Lmql
LMQL is a query language designed specifically for large language models (LLMs). It combines the natural language prompts with the expressi...
Last verified:
What is Lmql?
LMQL (Language Model Query Language) is an open-source programming language designed specifically for interacting with large language models (LLMs). It transforms natural language prompting into a robust, modular programming paradigm by combining typed prompting with control flow, constraints, and tool augmentation. Created by the SRI Lab at ETH Zurich and contributors, LMQL allows developers to write query programs as Python functions where top-level strings serve as prompts and template variables like [ANSWER] are automatically completed by the model.
Key features include declarative constraints enforced by the runtime (such as STOPS_AT, INT, REGEX, token/character length limits, and choice-from-set constraints), Python control flow for dynamic prompt construction (loops, conditions, function calls), nested queries for modularized local instructions, distribution clauses to obtain confidence probabilities, and automatic backend portability across llama.cpp, OpenAI API, and Hugging Face Transformers. The language uses eager token masking validation to ensure constraints are satisfied during generation rather than through post-parsing.
LMQL is ideal for developers, researchers, and engineers who want to build robust LLM applications requiring structured outputs, dynamic reasoning workflows, classification tasks, multi-part prompts, and tool-augmented queries. It's particularly valuable for applications needing guaranteed output formats (integers, JSON, specific choices), sentiment analysis, list generation, and any scenario where ad-hoc parsing of LLM output would be brittle.
Lmql pricing
Pricing model: Free
LMQL is free open-source software. It is available as a web-based Playground IDE for experimentation and as a Python package for local installation via pip or conda. There is no LMQL-specific pricing since costs depend entirely on the language model backend service used (e.g., OpenAI API charges per token, Hugging Face models may be free or paid, llama.cpp runs locally on your hardware). The Playground IDE is free to use for testing and development.
Lmql pros
- Open-source programming language with no licensing costs
- Python-based syntax familiar to developers
- Declarative constraints enforced during generation (not post-parsing)
- Support for control flow (loops, conditions, function calls)
- Type-safe generation with INT, REGEX, and set constraints
- Automatic backend portability across llama.cpp, OpenAI, Transformers
- Nested queries enable modular prompt components
- Distribution clause provides confidence probabilities
- Token masking enables eager constraint validation
- Web-based Playground IDE for experimentation without installation
- Supports multi-part and scripted prompts
- Cross-variable constraints prevent model contradiction
- Stopping phrase constraints (STOPS_AT, STOPS_BEFORE)
- Character and token length constraints available
- Compatible with Hugging Face Transformers, OpenAI API, and Langchain
- Single-line backend switching for easy model testing
- Results directly accessible as Python variables
Lmql cons
- Only supports integer constraints (no floats yet)
- REGEX constraints are preview feature (may not work perfectly)
- Requires local installation for self-hosted models
- Special escaping needed for [ ] { } literal characters
- Template variables cannot use reserved characters directly
- Distribution variables only work at end of program
- Greedy decoding terminates on constraint failure
- Learning curve for developers unfamiliar with constrained generation
- Token-level constraints depend on model's tokenization
- No official floating-point support yet
Frequently asked questions about Lmql
What is LMQL?
LMQL (Language Model Query Language) is an open-source programming language for large language models that combines typed prompting with control flow, constraints, and tools. It enables robust and modular LLM prompting using types, templates, constraints, and an optimizing runtime.
How do I install LMQL?
You can install LMQL locally using pip or conda, or use the web-based Playground IDE without installation. For self-hosted models via Hugging Face Transformers or llama.cpp, you must install LMQL locally.
What backends does LMQL support?
LMQL automatically makes your LLM code portable across llama.cpp, OpenAI API, and Hugging Face Transformers. You can switch between backends with a single line of code using the 'from' statement.
What are constraints in LMQL?
Constraints are logical expressions enforced by the runtime during text generation. Available constraints include STOPS_AT, STOPS_BEFORE, INT (integer type), REGEX (preview), character length (len), token length (len(TOKENS)), and choice-from-set (in []). Multiple constraints can be combined with 'and' and 'or'.
How does LMQL handle control flow?
LMQL supports Python control flow constructs including for loops, if/elif/else conditions, and function calls within prompt clauses. This enables dynamic prompt construction where programs can respond dynamically to model output.
What is the distribution clause?
The distribution clause obtains the full probability distribution over possible values for a variable. It requires explicit decoding algorithm specification (like argmax) and scores each individually provided value, normalizing into a probability distribution P(VARIABLE) showing model confidence.
Can I use LMQL with Langchain?
Yes, LMQL is compatible with Langchain. It also works with Hugging Face's Transformers and the OpenAI API, providing broad backend support for LLM integration.
What are nested queries?
Nested queries are a new LMQL feature that brings procedural programming to prompting. They enable modularized local instructions and re-use of prompt components by allowing query programs to be defined as functions with @lmql.query decorator.
How do constraints work technically?
LMQL constraints are evaluated eagerly on each generated token using token masking. The runtime generates token masks during generation to directly guide the model. If constraints cannot be satisfied, validation fails early, saving the cost of generating invalid output.
Is LMQL free to use?
Yes, LMQL is open-source software available for free. You can use the web-based Playground IDE or install locally. However, costs depend on your chosen LLM backend (OpenAI charges per token, while self-hosted models via llama.cpp or Transformers have no API fees).