Llm
Access large language models from the command-line
Last verified:
What is Llm?
LLM is a CLI tool and Python library for interacting with Large Language Models from providers like OpenAI, Anthropic's Claude, Google's Gemini, Meta's Llama, and dozens of other LLMs. It supports both remote APIs and models that can be installed and run locally on your own machine. The tool provides a unified command-line interface for prompting models, holding interactive chats, managing API keys, logging prompts/responses to SQLite, creating prompt templates, managing embeddings, and using tools that models can call.
Key features include: executing prompts via llm 'prompt', interactive chat mode with llm chat, support for 50+ remote API plugins (OpenAI, Claude, Gemini, Cohere, Grok, Hugging Face, together.ai, and more), local model support via plugins (Ollama, MLX, llama.cpp, GPT4All), multi-modal attachments for images/audio/video, system prompts, tool calling with --functions or --tool, JSON schema output support, text extraction from fenced code blocks, fragment management for reusable context, conversation continuation with -c, embedding generation and similarity search, prompt templates, and comprehensive logging to SQLite database that can be queried with llm logs.
LLM is designed for developers, data scientists, researchers, and anyone who wants to work with LLMs from the command line or integrate them into Python scripts. It's particularly useful for developers who want to pipe code for explanation, generate test cases, extract text from images, create reusable prompt templates, build RAG applications with embeddings, and experiment with different models without changing their workflow. The extensible plugin system allows adding new models, tools, embedding models, and custom functionality.
Llm pricing
Pricing model: Freemium
LLM itself is free and open-source with no subscription. However, using remote API models requires purchasing API credits from each provider separately. OpenAI, Anthropic, Google Gemini, and other providers charge per token. The default OpenAI gpt-4o-mini model is the fastest and least expensive. Local models run via plugins like llm-ollama, llm-mlx, or llm-gpt4all are free to use after downloading the model weights, requiring only hardware resources. API keys are stored locally in keys.json and can be set via llm keys set command or environment variables.
Llm pros
- Unified CLI for dozens of LLM providers in one tool
- Supports both remote API models and local models
- Extensible plugin architecture with 50+ available plugins
- Free and open-source with no subscription required
- Interactive chat mode saves reload time for local models
- Built-in SQLite logging of all prompts and responses
- Multi-modal support for images, audio, and video attachments
- Tool calling with custom Python functions via --functions
- JSON schema output for structured data extraction
- Prompt templates for reusable system prompts and workflows
- Fragment system for storing and reusing context snippets
- Embedding generation and cosine similarity search
- API key management with secure storage in keys.json
- Conversation continuation across multiple prompts with -c
- Text extraction from fenced code blocks with --extract
- Works with pip, pipx, Homebrew, uv, and uvx installation
- Python library for programmatic access in scripts
- Support for async prompt execution
- Customizable default model with llm models default
- Search models by query terms with -q option
Llm cons
- Requires separate API keys for each provider (not free models)
- Homebrew version has PyTorch compatibility issues for local models
- Streaming cannot be disabled for some models
- Token costs depend on external API provider pricing
- Documentation spread across multiple pages can be hard to navigate
- Plugin installation requires managing Python dependencies
- Local models require significant RAM and GPU resources
- Conversation continuation re-sends full history consuming tokens
Frequently asked questions about Llm
How do I install LLM?
Install LLM using pip with pip install llm, or use pipx with pipx install llm, Homebrew with brew install llm, or uv with uv tool install llm. You can also try it without installing using uvx llm 'your prompt' after setting your API key.
How do I set up my OpenAI API key?
Run llm keys set openai and paste your OpenAI API key when prompted. The key will be stored in keys.json at ~/Library/Application Support/io.datasette.llm/keys.json on macOS or ~/.config/io.datasette.llm/keys.json on Linux. You can also pass the key directly with --key option or use the OPENAI_API_KEY environment variable.
How do I use a different model?
Use the -m or --model option followed by the model ID, like llm 'prompt' -m gpt-4o. Run llm models to list all available models and their aliases. You can also search for models using -q options like llm models -q 4o -q mini. Set a default model with llm models default gpt-4o.
How do I start an interactive chat?
Run llm chat optionally with -m model_id like llm chat -m gpt-4o. Type 'exit' or 'quit' to end. Use !multi for multi-line input and !end to finish. Use !edit to open your editor and modify the prompt. Continue your last conversation with llm chat -c.
How do I install plugins for additional models?
Use llm install plugin-name to install plugins from PyPI into the LLM environment. For example, llm install llm-gemini for Google Gemini, llm install llm-anthropic for Claude, or llm install llm-ollama for local Ollama models. Run llm plugins to list installed plugins.
How do I work with images or multi-modal inputs?
Use the -a or --attachment option with a file path or URL: llm 'describe this image' -a image.jpg. You can attach multiple files: llm 'extract text' -a image1.jpg -a image2.jpg. Pipe an image with cat image.jpg | llm 'describe' -a -. Specify content type with --at if auto-detection fails.
How do I continue a previous conversation?
Use -c or --continue to continue your most recent conversation: llm 'more names' -c. To continue a specific conversation, use --cid with the conversation ID: llm 'continue' --cid 01h53zma5txeby33t1kbe3xk8q. Find conversation IDs with llm logs. The same model is automatically reused.
How do I create and use prompt templates?
Save a prompt with --save: llm -s 'write pytest tests' --save pytest. Then use it with -t: cat code.py | llm -t pytest. List templates with llm templates list, view with llm templates show name, and edit with llm templates edit name. Templates store system prompts and options for reuse.
How do I view my logged prompts and responses?
Run llm logs list to see recent prompts and responses. Use llm logs -n 10 for 10 entries or llm logs -n 0 for all. Filter by model with -m, search with -q, include token usage with -u, output JSON with --json, or view just the last response with -r. Check status with llm logs status and find the database path with llm logs path.
How do I use tools with LLM?
Define Python functions with --functions: llm --functions 'def multiply(x, y): return x * y' 'what is 3*4'. Install tool plugins like llm install llm-tools-simpleeval then use --tool simpleeval. Enable tool debugging with --td or approve each call interactively with --ta. List available tools with llm tools list.