mochallama
Show HN: Run Llama.cpp In-Process from Java with Project Panama FFM
Last verified:
What is mochallama?
Mochallama is a local, tool-calling large language model (LLM) that runs inside your Java Virtual Machine (JVM). It is the only in-process, tool-calling local LLM built specifically for the JVM ecosystem, with a Spring-first architecture.
Key features include: in-process inference with no daemon or network hop (no Ollama-style sidecar needed), llama.cpp integration via Project Panama FFM without JNI dependencies, an OpenAI-compatible Spring Boot starter, and a Spring AI adapter for seamless integration. The model runs inside your application's own process, eliminating HTTP round-trips and idle resource drain. Inference is stateful and integrates with your app's lifecycle, Actuator health checks, and Micrometer metrics.
Mochallama is designed for Java developers, Spring Boot application developers, and enterprise teams who want to run local LLMs without external dependencies. It enables tool calling/function calling capabilities where the LLM can decide to invoke backend Java code when required, making it suitable for building AI agents and production-ready GenAI applications within Java ecosystems.
mochallama pricing
Pricing model: Freemium
Mochallama is open source and free to use. It is available as a Maven dependency (io.github.deemwario:mochallama-spring-ai) with no licensing fees. Users only need to provide their own GGUF model files from Hugging Face or other sources.
mochallama pros
- Runs entirely in-process within JVM - no daemon needed
- No network hop or HTTP round-trip latency
- No Ollama-style sidecar to install and supervise
- No idle resource drain from separate processes
- Uses llama.cpp via Project Panama FFM - no JNI required
- OpenAI-compatible API for familiar integration
- Spring Boot starter for easy setup
- Spring AI adapter for seamless Spring integration
- Stateful inference riding app's lifecycle
- Integrates with Actuator health checks
- Provides Micrometer metrics for monitoring
- Enables tool calling/function calling in Java
- No external runtime dependencies needed
- Runs models locally on your hardware
- Avoids sending sensitive data to external servers
- Perfect for enterprise Java applications
- Maven dependency - works with existing dependency management
mochallama cons
- Only works on JVM - not available for Python/Node.js
- Requires Java 21+ for modern features
- Limited to GGUF model format
- May have lower performance than optimized C++ solutions
- Spring-first design may limit non-Spring users
- Tool calling depends on model support (not all models work)
- Project Panama FFM is still preview in some Java versions
- Local inference requires sufficient RAM for models
Frequently asked questions about mochallama
What is Mochallama?
Mochallama is a local, tool-calling LLM that runs inside your JVM. It is the only in-process, tool-calling local LLM for the JVM with a Spring-first architecture, using llama.cpp via Project Panama FFM without requiring JNI.
How does Mochallama differ from Ollama?
Mochallama runs entirely in-process within your application's own process, while Ollama requires a separate sidecar daemon. Mochallama has no network hop, no HTTP round-trip, and no idle resource drain from a separate process.
What model format does Mochallama support?
Mochallama supports GGUF model format, which is the native format for llama.cpp. You can download GGUF models from Hugging Face or other sources.
Does Mochallama require JNI?
No, Mochallama uses Project Panama FFM (Foreign Function & Memory API) to interface with llama.cpp, eliminating the need for JNI (Java Native Interface) dependencies.
What Spring features does Mochallama integrate with?
Mochallama integrates with Spring Boot via an OpenAI-compatible starter, Spring AI via an adapter, Actuator for health checks, and Micrometer for metrics monitoring.
Can Mochallama do tool calling?
Yes, Mochallama supports tool calling/function calling where the LLM can decide when to invoke backend Java code. However, tool calling capability depends on the specific model used - models like llama3/llama3.1 support tools while others like gemma:2b do not.
What Java version is required?
Mochallama requires Java 21+ as it leverages modern JVM features including Project Panama FFM and potentially Vector API for performance optimizations.
Is Mochallama open source?
Yes, Mochallama is open source and available as a Maven dependency (io.github.deemwario:mochallama-spring-ai) with no licensing fees.
How do I install Mochallama?
Add the Maven dependency io.github.deemwario:mochallama-spring-ai to your Spring Boot project's pom.xml, then download a GGUF model from Hugging Face and configure it in your application.
What use cases is Mochallama best for?
Mochallama is best for enterprise Java applications, Spring Boot microservices, AI agents that need to call backend Java code, production-ready GenAI applications in Java ecosystems, and scenarios requiring local LLM execution without external API dependencies.