LocalAI
A self-hosted server that puts language, vision, voice, image and video models behind OpenAI-, Anthropic- and ElevenLabs-compatible APIs.
LocalAI describes itself as an open-source AI engine that runs language, vision, voice, image and video models on any hardware, including CPU-only machines. It is a small core with separate backends, such as llama.cpp, vLLM, whisper.cpp, stable-diffusion and MLX, and the README counts more than 60 of them. Each is pulled as its own image only when a model needs it. Its APIs are compatible with OpenAI, Anthropic and ElevenLabs, and it can also act as a drop-in Ollama API, so existing apps can often switch by changing the base URL.
You can start it from a macOS app or with a Docker command for CPU only, NVIDIA CUDA, AMD ROCm, Intel oneAPI or Vulkan. The command local-ai run loads a model from its gallery, from Hugging Face, from the Ollama registry or from a YAML file. Beyond a single user, the README lists API key authentication, user quotas, role-based access, saved chats with agents that use tools and MCP, a realtime speech API and a distributed cluster mode.
It is released under the MIT license and moves quickly. The newest release in the README, 4.11.0 of October 2026, added audio scenes with transcription and speaker labels, model failover chains and a Decisions API. It is a bigger system than Ollama or LM Studio, so it fits best when you want one self-hosted endpoint for several kinds of model.
Good for
- Replacing OpenAI, Anthropic or ElevenLabs API calls with an endpoint you host
- One server for chat, speech, images and video instead of several tools
- Small teams that need API keys, quotas and role-based access
- Running models on AMD, Intel, Vulkan or CPU-only hardware
Watch out for
- Backends download as separate images on first use, so plan for disk space and a first-run delay
- The macOS app is not signed by Apple, and the README gives a command to clear the quarantine flag
- It has many features and settings, so it is heavier than Ollama for simple local chat