Uzu
A high-performance inference engine for AI models
Last verified:
What is Uzu?
Mirai (with its core component uzu) is an on-device AI platform that enables developers to run high-performance AI models directly within iOS and macOS applications. The platform provides a Rust inference engine (uzu) optimized specifically for Apple Silicon, allowing AI to run with zero latency, full data privacy, and no inference costs. Unlike cloud-based AI solutions, Mirai's approach brings AI processing directly to the user's device, eliminating the need for network requests and enabling continuous, ambient AI interactions.
The platform offers multiple components including uzu (the Rust inference engine), uzu-swift (prebuilt bindings for Swift developers), uzu-ts (TypeScript bindings), lalamo (tools to optimize and convert models for on-device use), a CLI for chatting with models and serving them as local API, and a Platform console for managing organizations and API keys. The system supports tasks like text generation, information retrieval, classification, and summarization directly on the device.
Mirai is designed for developers building AI-native products for iOS and macOS who want to integrate high-performance AI without needing an ML team or weeks of setup. The platform claims any developer can get started with just 8 lines of code, making on-device AI accessible without deep systems expertise in memory management, kernel optimization, or hardware-aware execution. It's particularly suited for applications requiring real-time responses, data privacy, and offline functionality.
The platform recently raised $10M in seed funding led by Uncork Capital to make on-device inference accessible to developers. Mirai co-designs models and runtime specifically for Apple Silicon constraints, minimizing memory footprint while maximizing neural accelerator utilization. The system supports smart routing that automatically decides what runs locally versus in the cloud based on prompt type, latency constraints, and user context.
Uzu pricing
Pricing model: Freemium
The platform offers a free tier for the first 10K devices. Smart routing will be available soon in the SDK. The pricing includes full control and performance with the ability to run the most popular small LLMs. No specific paid plan pricing is mentioned on the website, but the free tier covers initial deployment for up to 10,000 devices.
Uzu pros
- Zero latency AI inference on device
- Full data privacy - all processing local
- No inference costs - runs locally
- Only 8 lines of code to integrate
- No ML team required
- Works offline without internet
- Optimized specifically for Apple Silicon
- Up to 37% faster generation vs competitors
- Up to 59% faster prefill vs competitors
- Supports Swift and TypeScript bindings
- Smart routing between device and cloud
- First 10K devices free
- Rust-based high-performance engine
- Prebuilt SDKs reduce setup time
- Runs popular small LLMs locally
Uzu cons
- Only supports Apple Silicon devices (iOS/macOS)
- Limited to small LLMs, not large models
- Smart routing feature not yet available in SDK
- Requires model conversion via lalamo
- Still in early development stage
- Limited model registry compared to cloud
- No Android or Windows support
- Requires Git dependency for Rust integration
Frequently asked questions about Uzu
Is this project open source?
The website indicates that Mirai has GitHub repositories available (trymirai/uzu, uzu-swift, lalamo), but the FAQ section does not explicitly state whether the project is fully open source. The repositories are publicly accessible on GitHub.
Which devices are supported?
Currently, only Apple Silicon (iOS/macOS) devices are supported. The inference engine is specifically optimized for Apple Silicon hardware.
Which models are supported?
The platform supports running the world's best LLMs or bringing your own. Popular small LLMs like Llama 3.2 1B are supported. Models need to be optimized and converted using lalamo for on-device use.
Do I need an ML team to use Mirai?
No, you don't need an ML team or weeks of setup. One developer can get Mirai running in minutes with just 8 lines of code, without touching low-level systems complexity.
What tasks can uzu perform?
uzu inference engine enables tasks like text generation, information retrieval, classification, and summarization. It makes it possible to run LLMs directly on your device.
How does smart routing work?
Smart routing automatically decides what runs local vs cloud based on prompt type, latency constraints, or user context. On-device is used when it's fast and private, cloud when it's heavy and contextual. You can define programmable policies based on prompt length, device capabilities, confidence thresholds, or user segments.
What is the integration process?
The full integration consists of 5 steps: 1) Get an API key from the Platform console, 2) Connect and configure the SDK, 3) Choose and download a model from the registry, 4) Choose an inference session configuration/preset, 5) Create an inference session and run a model.
How fast is Mirai compared to alternatives?
On certain model-device pairs, Mirai achieves up to 37% faster generation and up to 59% faster prefill compared to alternatives. It shows consistent gains over MLX and llama.cpp.
What is lalamo?
lalamo is a set of tools to optimize and convert models for on-device use. It handles the model conversion process needed before models can run on the uzu inference engine.
Can I use cloud models with Mirai?
Yes, existing cloud pipelines continue to work with Mirai, with optional hybrid routing between device and cloud. Smart routing allows you to automatically route to cloud models like GPT-4 or Claude when longer context or more accurate results are needed.