AnythingLLM

An all-in-one app for chatting with your documents, building agents and running local models, as a desktop app or a multi-user Docker server.

  • Interfaces
  • MIT · Open source
  • macOS, Windows, Linux
  • Verified

AnythingLLM, from Mintplex Labs, bundles chat, document ingestion, agents and vector databases in one application. You connect a local or cloud model, upload documents such as PDF, TXT and DOCX files, and chat with them, with source citations. A desktop version exists for macOS, Windows and Linux, and the Docker version adds multi-user support with permissions, white-labeling and an embeddable chat widget.

It includes a built-in local model engine, powered by Lemonade and llama.cpp, that runs on CPU or GPU (NVIDIA through CUDA, AMD and Intel through Vulkan) with no extra setup. It can also connect to Ollama, LM Studio, LocalAI, KoboldCpp and many cloud providers. Other features in the README include a no-code agent builder, MCP compatibility, scheduled tasks, memories and a developer API.

The license is MIT. Because it is an application layer, answer quality on your documents depends on the chat model and the embedding model you plug in, and our embedding model list is a good place to choose an embedder.

Good for

  • Private question-answering over your own PDFs and notes
  • Teams that want a multi-user chat server with permissions
  • No-code agents that use tools and your documents
  • Connecting many local and cloud model providers from one interface

Watch out for

  • Multi-user support, white-labeling and the embeddable chat widget are available in the Docker version only
  • If you choose a cloud provider, it receives your prompts and the document excerpts sent with them
  • Answer quality depends on the embedding model and chat model you choose

Links