Picollm
On-device LLM Inference Powered by X-Bit Quantization
Last verified:
What is Picollm?
Picovoice picoLLM is an on-device LLM inference engine optimized for running compressed large language models locally without sending data to the cloud. It is a highly accurate, cross-platform SDK that runs on Linux (x86_64), macOS (arm64, x86_64), Windows (x86_64, arm64), Raspberry Pi 4 and 5, Android, iOS, and all major web browsers (Chrome, Safari, Edge, Firefox). The engine supports both CPU and GPU execution, making it ideal for developers building private AI experiences that deploy reliably to millions of devices.
Key features include proprietary picoLLM Compression that improves GPTQ accuracy by up to 98%, 100% local inference for complete privacy, support for open-weight models including Gemma, Llama-2, Llama-3, Llama-3.2, Mistral, Mixtral, Phi-2, Phi-3, and Phi-3.5, and the ability to generate text completions, embeddings, and even process images with vision-language models. The SDK ships as a self-contained package with no cloud API calls, no network latency, and no data leaving the device.
picoLLM is designed for developer-first teams building voice AI and LLM-powered products, including startups, enterprises, and individual developers working on privacy-sensitive applications. It is particularly suited for voice assistants, on-device chatbots, document QA with RAG, real-time transcription systems, and any application requiring low-latency, offline AI capabilities without variable cloud costs or connectivity dependencies.
Picollm pricing
Pricing model: Freemium
Picovoice offers three plans: Free ($0/year) for strictly non-commercial personal use with 1M picoLLM tokens/month, 100K Orca TTS characters/month, 250 minutes Cheetah/Leopard STT, 250 minutes Falcon/Eagle/Koala, 1 Porcupine/Rhino/Cobra monthly active user, and 10 GB/month model training downloads. Foundation Plan ($6,000/year) is for commercial use, available only to startups incorporated within 5 years with under 20 employees and under $50M raised, offering 100M tokens/month, 10M TTS characters/month, 25K minutes STT/diarization/noise suppression, 100 active users, and 1 TB/month model downloads with 6-hour email support. Enterprise Plan ($30,000/year) offers same usage as Foundation but with custom SLA, custom development, and white-glove support. All contracts are 12-month minimum with no monthly subscription option.
Picollm pros
- 100% on-device inference keeps data private and never leaves the device
- Cross-platform support covers Linux, macOS, Windows, Raspberry Pi, Android, iOS, and all major browsers
- Runs on both CPU and GPU for hardware flexibility
- picoLLM Compression improves GPTQ accuracy by up to 98%
- Free for open-weight models with no cost for model usage
- No cloud API calls required, eliminating network latency
- Supports 18+ popular open-weight models including Llama-3, Gemma, Mistral, and Phi families
- Can generate embeddings for vector search and RAG applications
- Supports vision-language models with image input capability
- Interrupt completion generation mid-stream for better user control
- Free tier includes 1M tokens/month for picoLLM inference
- Self-contained SDKs require no complex cloud infrastructure maintenance
- HIPAA and GDPR compliant due to offline processing
- Works on embedded devices including Raspberry Pi 4 and 5
- Open-source benchmarks validate performance claims publicly
- Picovoice Console provides browser-based model download and management
- Startup special Foundation plan available for new companies under $50M funding
Picollm cons
- Free plan strictly limited to non-commercial personal use only
- Requires internet connectivity for AccessKey validation despite offline inference
- Commercial pricing starts at $6,000/year which is expensive for small businesses
- Only supports open-weight models, no proprietary model support
- Model files must be downloaded separately from Picovoice Console
- 1M token/month limit on free tier may be insufficient for development testing
- Android deployment requires handling model files via expansion files or external hosting due to APK size limits
- No monthly subscription option, only 12-month minimum contracts for paid plans
- Limited to 10 GB/month model training download on free plan
Frequently asked questions about Picollm
What is picoLLM Inference Engine?
picoLLM Inference Engine is a highly accurate and cross-platform SDK optimized for running compressed large language models on-device. It enables 100% local LLM inference without sending data to the cloud, making it private and eliminating network latency. The engine supports both CPU and GPU execution across Linux, macOS, Windows, Raspberry Pi, Android, iOS, and web browsers.
What models does picoLLM support?
picoLLM supports 18+ open-weight models including Gemma (2b, 7b variants), Llama-2 (7b, 13b, 70b variants), Llama-3 (8b, 70b variants), Llama-3.2 (1b, 3b instruct), Mistral-7b (v0.1, v0.2 instruct), Mixtral-8x7b (base and instruct), Phi-2, Phi-3, and Phi-3.5. All model files (.pllm) are downloaded from Picovoice Console.
Is picoLLM free to use?
Yes, picoLLM is free for open-weight models. The Free Plan ($0/year) provides 1M tokens/month for non-commercial personal use with no credit card required. For commercial use, the Foundation Plan starts at $6,000/year offering 100M tokens/month, or Enterprise at $30,000/year with custom terms.
Does picoLLM require internet connectivity?
Yes, internet connectivity is required for AccessKey validation and plan limit checking, even though inference runs 100% offline. Picovoice uses AccessKey to authenticate usage and verify plan limits by calling home servers, but all audio/text processing happens locally on the device without sending data to remote servers.
How does picoLLM compression work?
picoLLM Compression employs dynamic bit allocation that surpasses traditional fixed-bit quantization. By leveraging task-specific cost functions, it automatically optimizes bit distribution across model weights while maintaining performance. This proprietary compression improves GPTQ accuracy by up to 98% and achieves only 1.2% MMLU drop at 4-bit quantization.
What platforms does picoLLM run on?
picoLLM is cross-platform running on Linux (x86_64), macOS (arm64 and x86_64), Windows (x86_64 and arm64), Raspberry Pi 4 and 5, Android (5.0+, API 21+), iOS, and web browsers (Chrome, Safari, Edge, Firefox). It works on both CPU and GPU across all supported platforms.
How do I get started with picoLLM?
Sign up for Picovoice Console (free, no credit card required), obtain your AccessKey from the console home page, download the appropriate SDK for your platform via GitHub or package managers (npm for Web, pip for Python, Maven for Android), download a .pllm model file from Picovoice Console, then create a picoLLM instance with your AccessKey and model path to generate completions.
What's the difference between Free Plan and Free Trial?
Free Plan is for individual developers working on personal non-commercial projects without commercial aspirations, with no time restriction as long as Terms of Use are complied with. Free Trial is for enterprise developers testing and evaluating Picovoice for integration into products that may be used by employers, clients, or customers. Free Trial grants Foundation Plan usage rights after approval, requires no credit card, and does not auto-renew.
How does Picovoice track usage?
Picovoice tracks usage either by data processed or number of users activated depending on the engine. picoLLM Inference reports usage based on tokens processed (measured in 1-token increments). You can check plan limits and usage on the Picovoice Console dashboard or profile page. Other engines track by audio duration (seconds), text characters, or monthly active users.
Which commercial plan should I choose?
Both Foundation and Enterprise plans grant access to all Picovoice engines at the same usage levels (100M tokens/month for picoLLM). Foundation Plan ($6,000/year) is available exclusively to new startup customers incorporated within 5 years, raised under $50M, with max 20 employees, offering self-service purchase with credit card and 6-hour email support. Enterprise Plan ($30,000/year) allows custom SLA terms, custom development, and white-glove support for enterprises requiring legal/process customization.