VLM Run
Show HN: Visual Agents with Code Mode
Last verified:
What is VLM Run?
Orion 2 is VLM Run's most capable visual agent, now featuring code execution capabilities. It generates and executes computer-vision code on the fly, enabling it to see, reason over, and act on images, videos, and documents through a single unified API. Rather than calling tools sequentially one by one, Orion 2 creates a complete program and executes it end-to-end, resulting in fewer round-trips and significantly lower latency.
Key features include code-mode orchestration where the model writes one program upfront that runs in a sandboxed runtime with async-native parallelism. The visual DSL exposes all Orion 1 tools (detection, OCR, segmentation, cropping, image generation) as native primitives. Orion 2 supports self-correction by repairing and re-executing programs until completion, provides fully inspectable artifacts (programs, traces, outcomes), and handles compositional tasks like virtual try-on, robotics video analysis, multi-document workflows, and OpenCV image manipulations. It routes requests across multiple backbones including Gemma4-26B, Qwen3.6-35B, and Gemini 3.5 Flash automatically.
Orion 2 is built for AI teams, enterprises, and developers who need to process unstructured visual data at production scale. It serves customers running hundreds of thousands of requests spanning millions of tool-calls monthly. The tool is ideal for regulated industries needing inspectable, debuggable code; applications requiring object detection, segmentation, OCR, document parsing, video understanding, image generation/editing; and workflows needing parallel operations, loops, conditionals, and deterministic computation without numeric hallucinations.
The agent is available through chat.vlm.run for free testing and via the VLM Run Chat Completions API for production building. Every example in the blog is a live chat thread users can inspect and re-run, making it accessible for both experimentation and enterprise deployment.
VLM Run pricing
Pricing model: Freemium
Orion agents use credit-based pricing with 100 credits = $1. Three tiers: Auto (vlmrun-orion-1:auto, default) uses Fast tools by default and auto-selects Pro for complex tasks; Fast (vlmrun-orion-1:fast) uses lighter models for speed and cost-efficiency; Pro (vlmrun-orion-1:pro) uses powerful models for maximum quality but slower and more expensive. Image capabilities: Caption&Tag 1-2 credits/image, Detect/Point/Segment 1-2 credits/image, UI Parsing 1-2 credits/image, Generate&Edit 4-24 credits/image, Tools free. Document capabilities: OCR/Layout 0.10-0.40 credits/page, Parsing 1-2 credits/page, Tools 0 credits. Video capabilities: Caption/Summary/Tag 20-80 credits/hour input, Generate&Edit 15-40 credits/second output (standard ~6s = ~240 credits Pro = $2.40). Utility tools (image I/O, editing, video sampling/trimming, document navigation) are free within agent sessions but base session cost applies. Chat completion priced per token, free if under 1 credit.
VLM Run pros
- Generates and executes computer-vision code on the fly
- Significantly faster than Orion 1 with fewer round-trips
- Cheaper than sequential tool-calling approaches
- More reliable orchestration than Orion 1
- Code is reusable - scripts can be saved, versioned, re-run
- Code is inspectable - readable, debuggable, diffable, verifiable
- Code is composable - supports loops, conditionals, fan-out, parallel calls
- Deterministic computation eliminates numeric hallucinations
- Async-native parallelism dispatches independent operations concurrently
- Self-correction repairs and re-executes failed programs
- Full Orion 1 tool surface now programmable and deterministic
- Supports detection, OCR, segmentation, cropping, image generation
- Routes automatically to best backbone (Gemma4, Qwen3, Gemini 3.5)
- Open-weight backbones run on purpose-built inference runtime
- Pin fixed backbone for compliance or reproducibility requirements
- Inspectable artifacts include programs, traces, outcomes, crops, composites
- Model-agnostic harness works with any multimodal code-generation model
- Benchmarks expanded to 250+ multi-turn cases across all modalities
- Handles virtual try-on in single turn composing four visual operations
- Extracts robotics video frames and segments objects in parallel
- Classifies multi-page healthcare documents and extracts structured data
- Native OpenCV support for Gaussian blur, Canny edge, color pop
VLM Run cons
- Requires multimodal model with strong code generation capability
- Self-correction may add latency for complex failed programs
- Sandboxed runtime adds complexity over simple tool-calling
- Open-weight backbones may have lower quality than frontier models
- Pro tier takes longer and costs more than Fast tier
- Code generation can fail requiring multiple repair attempts
- Limited to tools exposed in visual DSL runtime
- May not support external or user-defined custom tools yet
- 10+ step workflows challenge long-term planning capabilities
- Calculation costs still apply even for free utility tools
- Base agent session cost applies even when tools are free
- Pro tier uses more powerful models that are slower
- Auto tier may select Pro tools for complex tasks increasing cost
- Video generation produces only ~6 seconds standard output
- Document parsing costs 1-2 credits per page depending on tier
- Image generation/editing costs 4-24 credits per image depending on tier
Frequently asked questions about VLM Run
What's the difference between Auto, Fast and Pro tiers?
Auto (vlmrun-orion-1:auto) and Fast (vlmrun-orion-1:fast) are optimized for speed and cost-efficiency using lighter models that process requests quickly. Pro (vlmrun-orion-1:pro) uses more powerful models producing higher quality results but taking longer and costing more. Choose Auto/Fast for prototyping and high-volume processing, and Pro for production use cases requiring maximum quality. Auto is the default tier and automatically selects Pro tools for complex tasks.
How is the total cost of agent execution priced?
The total cost is the sum of credits used for tools plus the cost of orchestration. For non-integer credits, they round up to the nearest credit. Each capability has a credit cost based on the processing tier selected (Fast or Pro), and pricing varies by category (image, document, video).
How is the cost of the chat portion priced (excluding tools)?
The chat completion cost is priced per token. This is a simple cost overhead for orchestration, and they do not charge for it if the cost of a single chat completion is less than 1 credit.
Are there any free tools?
Yes, utility tools like image I/O, basic editing operations, video sampling/trimming, and document navigation are included at no additional credit cost when used within an agent session. However, the base agent session cost still applies.
What is Orion 2's code-mode and how does it work?
Code-mode means Orion 2 generates and executes computer-vision code on the fly instead of calling tools sequentially. The model writes one program upfront in a visual DSL (reads like idiomatic Python), which runs in a sandboxed runtime with async-native parallelism. Independent operations dispatch concurrently via asyncio without per-step model round-trips. Execution results return to the harness for self-correction, repairing and re-executing until completion.
What backbones does Orion 2 support?
Orion 2 is model-agnostic and works with any multimodal model with strong code generation: open-weight VLMs like Gemma4-26B-A4B and Qwen3.6-35B-A3B, or frontier models like Gemini 3.5 Flash. The default vlmrun-orion-2:auto routes each request to the best backbone for the job automatically. Users can pin a fixed backbone through the gateway for compliance or reproducibility.
What capabilities does Orion 2 include from Orion 1?
The entire Orion 1 tool surface is now fully programmable and deterministic in Orion 2, including detection, OCR, segmentation, cropping, image generation, point-to-objects, UI parsing, and more. All are exposed as native primitives in the visual DSL that the model can compose in code.
How do I get started with Orion 2?
Try Orion 2 now at chat.vlm.run where every example is a live chat thread you can inspect and re-run. Bring your own images, documents, or videos. When ready to build, the same agent is available through the VLM Run Chat Completions API, and their team can help evaluate it on production workloads.
What benchmarks does Orion 2 use?
Orion 2 expands the benchmark to 250+ multi-turn cases across image, document, audio, video, and multi-file inputs, covering perception, counting, OCR, grounding, and cross-turn reasoning. The hard tier includes generative tool-use actions like cropping, masking, blurring, redaction, segmentation, and keyframe extraction. It achieves competitive performance on MMMU, MMBench, DocVQA, and MMLongBench.
What are the main use cases for Orion 2?
Use cases include virtual try-on (detecting person and garment, cropping, generating composite), robotics and Physical AI (frame extraction, parallel segmentation, 3D reconstruction), multi-document workflows (splitting healthcare PDFs, classifying documents, structured extraction), and image manipulation (OpenCV operations like Gaussian blur, Canny edge detection, color pop). It serves regulated industries needing inspectable code and enterprises processing unstructured visual data at scale.