Kiln
Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.
Last verified:
What is Kiln?
Kiln AI is the AI Workbench for Teams—a desktop application and open-source Python library that enables serious teams to build, evaluate, and optimize AI systems. It provides an integrated platform for evals, optimization, fine-tuning, RAG, skills, tools & MCP, sub-agents, structured JSON output, and reasoning capabilities. The app works on macOS, Windows, and Linux with a one-click install, requiring no terminal, coding, or prior data science experience.
Key features include: AI Eval Builder that translates intent into evals, LLM-as-Judge scoring, golden datasets, human ratings, Auto-Optimize that auto-tunes prompts to evals, fine-tuning across 60+ models on four providers (OpenAI, Google Gemini, Fireworks, Together AI), synthetic data generation with topic trees, serverless model deployment, distillation, and the Kiln Assistant—a chat-based AI agent that plans and runs experiments through conversation. Kiln also offers Git Auto-Sync for version control, shared prompts, structured ratings, and an AI-native issue tracker.
Kiln is built for the whole team: engineers deploy anywhere with the MIT-licensed Python library; data scientists try new models and dispatch experiments in seconds; PMs, QA, and subject matter experts contribute to quality without coding through feedback, ratings, evals, and data generation. The platform has over 10,000 developers, 4,500+ GitHub stars, supports 190+ models, and is trusted by AI engineers from major companies.
Kiln pricing
Pricing model: Freemium
Individual tier is Free with open-source Python library (MIT license), source-available desktop app, local datasets with git sync, Kiln Pro standard models (rate limited), and community support. Team tier requires request access and includes everything in Individual plus Kiln Pro enhanced models with higher limits, automatic agent optimization, priority access to new features, and email support. Enterprise tier is custom-priced and includes everything in Team plus SSO/SAML, annual contracts & procurement support, SLA & priority support, dedicated solutions engineer, and custom onboarding & training. Kiln Pro features (AI Assistant, Auto-Generated Evals, Kiln Optimizer) run on Kiln's servers and are charged separately—Individual gets standard/rate-limited access while Team & Enterprise get advanced with higher limits.
Kiln pros
- Free download with one-click install for Mac, Windows, and Linux
- Open-source MIT-licensed Python library for production deployment
- Source-available desktop app that can be audited and run yourself
- Git Auto-Sync version control without requiring terminal knowledge
- 190+ models tested and supported across multiple providers
- 60+ fine-tuneable models including Qwen, DeepSeek, Llama, GPT, Gemini
- Kiln Assistant AI agent plans and runs experiments through conversation
- Auto-Optimize automatically tunes prompts based on eval results
- Synthetic data generation builds 1,000+ diverse samples from seed examples
- Train on four providers (OpenAI, Google Vertex, Fireworks, Together) without GPU management
- Serverless model deployment that scales to zero with pay-per-token pricing
- Distillation feature reduces large models to smaller, faster, cheaper versions
- AI-native issue tracker captures structured AI data (input, output, model, hyperparameters)
- Shared prompt library with named variants that improves as team rates outputs
- No coding required for PMs, QA, and SMEs to contribute ratings and feedback
- Local-first design keeps datasets on your machine, synced through Git
- Supports structured JSON output with visual schema builder
- Fine-tuning for tool calling and reasoning with thinking traces
Kiln cons
- Kiln Assistant is currently in beta
- Kiln Optimizer only available on Team & Enterprise tiers, not Individual
- Standard models on Individual tier are rate limited
- AI Assistant on Individual tier is standard and rate limited vs advanced on Team
- Fine-tuning costs vary by provider and model (demo showed $1.47-$2.03)
- Training data sent to provider APIs when dispatching job (not fully local for fine-tuning)
- Team plan requires requesting access, not instantly available
- Enterprise tier needs custom pricing and contract negotiation
Frequently asked questions about Kiln
What is Kiln AI?
Kiln AI is the AI Workbench for Teams—a development platform for teams shipping AI products. It provides everything needed to build and optimize AI systems: evals, optimization, fine-tuning, RAG, skills, tools, sub-agents, and more. The desktop app works on macOS, Windows, and Linux, and there's an open-source MIT-licensed Python library for production deployment.
What is Kiln Assistant?
Kiln Assistant is a chat-based AI agent built into the Kiln desktop app. You describe what you want to improve (cost, quality, speed), and it analyzes your project, proposes a plan, and executes it. It can plan and run batches of experiments, write and iterate on prompts, kick off eval runs, read results, compare configurations, recommend models, and apply AI techniques like RAG, skills, and sub-agents. It has full access to your project data, eval results, and every action the app supports.
Do non-technical team members need to know Git?
No. Automatic Git Sync handles Git invisibly. Non-technical users connect via a git link and OAuth or token—changes sync within seconds. The app lets anyone build evals, run agents, and contribute to quality without terminal, coding, or prior data science experience.
Where does fine-tuning run?
You can choose your provider: OpenAI, Google Gemini (Vertex), Fireworks, or Together AI. Or export datasets to train any model on your own infrastructure using Unsloth, Axolotl, or Colab. Kiln handles provider formats, deployment, and configs without requiring GPU management.
How much does fine-tuning cost?
It varies by provider and model. In their demo, they trained 5 Llama models plus Mixtral on Fireworks for $1.47 and GPT-Mini for $2.03. Fine-tuned models deploy serverless with no recurring costs—just pay-per-token when the model is used.
Does my training data leave my machine?
Kiln is local-first. Data is sent to provider APIs only when you dispatch a fine-tuning job. Your datasets and evals live on your machine and sync through a git repo you control. You can export and train entirely on your own infrastructure if data residency requires it.
Can I fine-tune for tool calling or reasoning?
Yes to both. Reasoning distillation trains smaller models on thinking traces from large models like DeepSeek or Claude. Tool-use training teaches a model your tool set, with a built-in eval to measure improvement. Kiln trains for output, reasoning structure, and tool use.
How is Kiln Assistant different from ChatGPT or Claude?
General-purpose chatbots don't have access to your Kiln project, eval results, or the ability to take action in an AI development tool. Kiln Assistant is connected directly to your project—it reads your data, understands your setup, and acts on your behalf. It's a specialist for AI development, not a generalist chatbot.
What happens if two people edit at the same time?
Kiln's data model makes conflicts rare with unique IDs and append-only files. If a conflict occurs, Automatic Git Sync self-heals with zero data loss.
How are Kiln Issues different from JIRA?
Kiln Issues capture structured AI data—input, output, model, hyperparameters—not just descriptions. Each issue plugs into evals and synthetic data, so a fix is verified with evals, not just shipped. This enables regression detection for AI bugs.