GGUF
GGUF is a single-file model format used by llama.cpp and the tools built on it, which bundles quantized weights with the information needed to run them.
A GGUF file holds the model's weights, usually quantized, together with its tokenizer and settings, so one file is enough to run it. Names such as Q4_K_M indicate the quantization level.
llama.cpp and tools built on it, such as LM Studio, read GGUF files, and Unsloth can export to the format. Many are published on Hugging Face by the model's developer or by community members. The license of the original model still applies to a GGUF made from it, so check it before you use the file commercially.
Related terms
- Quantization Quantization stores a model's weights with fewer bits, for example 4 instead of 16, to cut memory use and speed up inference at a small cost in quality.
- VRAM VRAM is the memory on a graphics card; it decides which models a GPU can hold, because the weights must fit in memory for the model to run quickly.