Introduction
Native LLM Inference and Embeddings for PHP
Last verified:
What is Introduction?
Introduction is a PHP 8.3+ extension written in Rust that loads GGUF models and runs LLM inference directly inside the PHP process via llama.cpp. This eliminates the need for HTTP servers, exec(), Python integrations, or external API calls when working with large language models in PHP applications.
Introduction pricing
Pricing model: Freemium
Free and open source. The extension is available as a pre-release GitHub project with no paid tiers or licensing costs.
Introduction pros
- Runs LLM inference directly inside PHP process
- No HTTP servers required
- No exec() calls needed
- No Python integration required
- No external API calls necessary
- Written in Rust for performance
- PHP-native fluent API
- Supports GGUF model format
- Works with llama.cpp backend
- PHP 8.3+ compatible
- Local inference - no cloud needed
- Supports semantic search and embeddings
- Compatible with Hugging Face GGUF models
- Supports multiple models like Qwen3
- Reduces infrastructure complexity
- Fast model loading with mmap support
- No additional microservices needed
- Direct model loading from path
- Lower latency than API-based solutions
- Data stays local - no向外 transmission
Introduction cons
- Pre-release status - not production-ready
- Requires PHP 8.3 or higher
- Must install Rust extension
- GGUF models can be large (GBs)
- Requires local GPU/CPU for inference
- Limited to llama.cpp compatible models
- No built-in model management
- Memory usage depends on model size
- Requires compiling the extension
- Less mature than Python alternatives
Frequently asked questions about Introduction
What is ext-infer?
ext-infer is a PHP 8.3+ extension written in Rust that loads GGUF models and runs LLM inference directly inside the PHP process via llama.cpp, eliminating the need for HTTP servers, exec(), Python, or external API calls.
What PHP version is required?
ext-infer requires PHP 8.3 or higher. It is not compatible with older PHP versions.
What model format does it support?
ext-infer supports GGUF (GGML Universal Format) models, which are the standard format for llama.cpp compatible large language models available on Hugging Face.
Does it need an external API or cloud service?
No, ext-infer runs local inference entirely within the PHP process. No external API calls, cloud services, or HTTP servers are required.
What can I use ext-infer for?
You can use it for LLM chats, embeddings, semantic search over documents, similar records search, zero-shot classification, semantic recommendations, and generating content from pre-trained models like Qwen3.
How do I install ext-infer?
The extension must be compiled from source using Rust/cargo. You need cargo installed, then run 'cargo build --release' to build the extension .so file for PHP.
Is ext-infer production-ready?
No, ext-infer is currently in pre-release status and is not yet considered production-ready for enterprise deployments.
What backends does it use for inference?
ext-infer uses llama.cpp as the inference backend, which supports various hardware accelerators including CUDA for NVIDIA GPUs, ROCm for AMD GPUs, and SYCL for Intel hardware.
Can I use models from Hugging Face?
Yes, ext-infer works with GGUF models from Hugging Face Hub. You can provide a path to the model file or use Hugging Face repo paths with llama.cpp.
What are the performance benefits?
ext-infer provides lower latency than API-based solutions since inference runs directly in the PHP process, eliminates network overhead, and supports mmap for fast model loading.