MLC LLM
A compiler-based engine that deploys the same model on phones, browsers, laptops and GPUs behind an OpenAI-compatible API.
MLC LLM is a machine-learning compiler and deployment engine for large language models. Its stated mission is to let everyone develop, optimize and deploy AI models natively on their own platforms. One engine, called MLCEngine, runs on AMD, NVIDIA, Apple and Intel GPUs through Vulkan, ROCm, CUDA and Metal, in web browsers through WebGPU and WebAssembly, on iPhones and iPads through Metal, and on Android phones through OpenCL.
MLCEngine offers an OpenAI-compatible API through a REST server and through Python, JavaScript, iOS and Android bindings, all backed by the same engine and compiler. A related project, WebLLM, covers in-browser use. Models are compiled for each target platform with MLC's own tools, so check the documentation to confirm that your model is supported before you plan around it.
The license is Apache 2.0. The repository publishes no GitHub releases, so follow the installation guide at llm.mlc.ai and pin the package or commit you test with. It is a developer tool for shipping models inside apps; if you only want to chat with a model on one computer, Ollama or LM Studio are simpler.
Good for
- Shipping an LLM inside an iOS, Android or browser app
- One engine across AMD, NVIDIA, Apple and Intel GPUs
- Apps that need an OpenAI-compatible local endpoint
Watch out for
- Models must be converted and compiled for the target platform, so confirm support for your model first
- There are no GitHub releases, so pin the package or commit you test with
- It is aimed at developers; for simple chatting on one computer a runner such as Ollama is easier