Glossary
Short definitions first, details after. Terms link to each other.
The basics
- Context window
- The context window is the maximum amount of text, measured in tokens, that a model can take into account at one time, including your prompt and its own answer.
- Hugging Face
- Hugging Face is the website where most open models are published: each model has a page with its files, license, model card and download count.
- Inference
- Inference is the act of running a trained model to get an output, such as answering a prompt, as opposed to training it.
- Machine translation
- Machine translation is software that translates text, speech or text inside images from one language to another. The open models of today usually start from a general model and are then trained only to translate.
- Open weights
- An open-weights model is an AI model whose trained parameters, called weights, are published so that anyone can download and run it, usually under a license that sets the terms of use.
- Parameters
- Parameters are the numbers inside a model that are adjusted during training; their count, such as 7B or 70B, is the usual measure of a model's size.
- Reinforcement learning (RL)
- Reinforcement learning trains a model by rewarding good outcomes, such as a correct answer or passing tests, instead of copying example text.
- Sovereign AI
- Sovereign AI means AI that a country or organization can build, host and govern under its own control, from the computing power and data to the model itself.
- Token
- A token is a small chunk of text, often a word or part of a word, that a language model reads and writes; models measure length and speed in tokens.
Models and architecture
- AI agent
- An AI agent is a system in which a model plans and carries out multi-step tasks by calling tools, such as running code, searching the web or editing files.
- Base model
- A base model is the raw result of pretraining, before it has been taught to follow instructions or chat; it is the usual starting point for fine-tuning.
- Decision model
- A decision model reads a situation and a list of questions, then returns a probability for each allowed answer instead of writing text.
- Dense model
- A dense model uses all of its parameters to process every token, unlike a mixture-of-experts model, which activates only a part of them.
- Diffusion model
- A diffusion model generates images or video by starting from random noise and removing the noise step by step until a picture emerges.
- Distillation
- Distillation trains a smaller student model to imitate a larger teacher model, keeping much of its ability at a fraction of the size.
- Efficient attention (sparse, linear and hybrid)
- Efficient attention designs let a model read very long inputs with far less memory and compute than standard attention, by looking at fewer tokens or compressing the past.
- Embedding model
- An embedding model turns text, images or other data into a list of numbers that captures its meaning, so that similar items end up close together.
- Looped transformer
- A looped transformer reruns some of its own layers several times for each token, so it gets more computation without storing more parameters.
- Matryoshka embeddings
- Matryoshka embeddings are trained so that the first part of the vector still works on its own, which lets you cut the vector shorter to save storage and search time.
- Mixture of experts (MoE)
- A mixture-of-experts model contains many specialist sub-networks and uses only a few of them for each token, so it can be very large yet fast.
- Multimodal model
- A multimodal model can handle more than one type of data, such as text, images, audio or video, as input, as output, or both.
- Music generation
- Music generation models turn a text description, and often lyrics, into audio: full songs with vocals, instrumental tracks, live streams of music or sound effects.
- OCR and document parsing
- OCR turns an image of text into text; document parsing goes further and rebuilds tables, formulas, charts and reading order, usually as Markdown.
- Reasoning model
- A reasoning model works through a problem step by step in a 'thinking' phase, visible or hidden, before it gives its final answer.
- Reranker
- A reranker re-scores a short list of search results by reading the question and each result together, which is slower but more accurate than embeddings alone.
- Speaker diarization (speaker labels)
- Diarization works out who spoke when in a recording and labels each stretch of speech with a speaker, such as Speaker 1 and Speaker 2.
- Speech recognition (ASR, speech-to-text)
- Speech recognition, also called ASR or speech-to-text, turns spoken audio into written text.
- Text encoder
- In an image or video generator, the text encoder is a separate language model that turns your prompt into numbers the generator can use. It adds to the download and to memory use.
- Text-to-speech (TTS)
- Text-to-speech models turn written text into spoken audio, and many can copy a voice from a short sample.
- Vision-language model (VLM)
- A vision-language model reads images, and often video, together with text, so you can ask questions about a picture, a screenshot or a document page.
Hardware and memory
- GGUF
- GGUF is a single-file model format used by llama.cpp and the tools built on it, which bundles quantized weights with the information needed to run them.
- KV cache
- The KV cache is the memory a model uses to remember the earlier tokens of a conversation, and it grows with the length of the context.
- Precision (BF16, FP8, INT4)
- Precision is how many bits are used to store each number in a model, such as 16-bit BF16, 8-bit FP8 or 4-bit INT4; fewer bits mean a smaller, faster model.
- Quantization
- Quantization stores a model's weights with fewer bits, for example 4 instead of 16, to cut memory use and speed up inference at a small cost in quality.
- Quantization-aware training (QAT)
- Quantization-aware training teaches a model to work with low-precision weights while it is still being trained, so it loses less quality at 4 bits than a model squeezed afterwards.
- Speculative decoding and multi-token prediction
- Speculative decoding speeds up text generation by letting a cheap draft guess several tokens ahead and having the full model check them all at once.
- Ternary weights (1.58-bit)
- Ternary weights limit every model weight to one of three values, -1, 0 or +1, so a model needs under 2 bits per weight instead of 16.
- Unified memory
- Unified memory is a design, used by Apple silicon and some other chips, in which the CPU and GPU share one pool of memory, so a large model can use most of it.
- VRAM
- VRAM is the memory on a graphics card; it decides which models a GPU can hold, because the weights must fit in memory for the model to run quickly.
Licensing
- Gated model
- A gated model is an open model whose download on Hugging Face requires you to log in and accept its terms, or request access, first.
- Non-commercial license
- A non-commercial license lets you use a model for research, study and personal projects, but not to make money from it.
- Open-source AI
- Open-source AI, as defined by the Open Source Initiative, is an AI system you may use, study, modify and share freely, with the weights, the code and detailed information about the training data all available.
- OpenRAIL (Responsible AI License)
- OpenRAIL is a family of licenses that let you use, modify and share a model, including commercially, but require every copy and derivative to carry a list of banned uses.
- Permissive license
- A permissive license, such as Apache 2.0 or MIT, lets you use, modify and sell a model with very few conditions, usually just keeping a notice.
- Revenue cap (revenue threshold)
- A revenue cap is a license condition that makes a model free below a certain income and requires a separate agreement above it.
- Territory restriction
- A territory restriction limits where a license is valid: using the model in the excluded countries or regions is not licensed.
Using models
- Agent memory
- Agent memory lets an AI agent keep facts, preferences and past experience between sessions instead of starting from a blank context.
- Agentic coding
- Agentic coding means a model works on software like a developer: it reads the codebase, edits files, runs commands and tests, and keeps going until the task is done.
- Benchmark
- A benchmark is a standardized test used to compare models, such as a set of maths, coding or knowledge questions; scores are useful but easy to over-read.
- Classifier-free guidance (CFG)
- Classifier-free guidance, usually called CFG, is a setting in diffusion models that controls how strictly the result follows your prompt, at the price of extra computation.
- Computer-use agent
- A computer-use agent looks at screenshots and operates a computer like a person, by clicking, typing and scrolling, and sometimes by running shell commands or code.
- Fine-tuning
- Fine-tuning continues a pretrained model's training on your own examples so that it behaves better for a specific task, style or domain.
- Hallucination
- A hallucination is a confident but false or invented statement produced by an AI model.
- LoRA
- LoRA, short for low-rank adaptation, fine-tunes a model by training a small set of added weights instead of changing all of its parameters.
- MTEB (embedding benchmark)
- MTEB, the Massive Text Embedding Benchmark, scores embedding models on many tasks such as search, classification and clustering, and its public leaderboard is a common way to compare them.
- OmniDocBench
- OmniDocBench is a benchmark for document parsing: it scores how well a model turns PDF pages of many types into text, tables and formulas.
- On-device AI
- On-device AI means the model runs on your own phone, laptop or small board instead of in the cloud, so data stays local and it works offline.
- OpenAI-compatible API
- An OpenAI-compatible API is a server that accepts the same requests as the OpenAI API, so apps written for it can use an open model instead.
- Retrieval-augmented generation (RAG)
- Retrieval-augmented generation lets a model answer from your own documents by retrieving the relevant passages and adding them to the prompt.
- Sampling steps
- Sampling steps are the number of passes a diffusion model makes to turn noise into an image or video. Fewer steps mean faster results, usually with some loss of detail or variety.
- Self-hosting
- Self-hosting means running a model on hardware you control, such as your own PC or a rented GPU server, instead of calling a vendor's API.
- Tool calling (function calling)
- Tool calling lets a model ask your program to run a function, such as a search or a calculator, and then use the result in its answer.
- Translation metrics
- Translation metrics are the scores that model cards use to rate a translation. Some count overlap with a human reference, others use a trained model or human reviewers, and their scales cannot be compared.
- Voice cloning
- Voice cloning makes a text-to-speech model speak in a particular person's voice, often from only a few seconds of recording.
- Word error rate (WER)
- Word error rate is the standard score for speech recognition: the share of words in a transcript that are wrong, missing or extra compared with a correct reference.