Parameters
Parameters are the numbers inside a model that are adjusted during training; their count, such as 7B or 70B, is the usual measure of a model's size.
A parameter is a single learned number, usually a weight. A model with 7 billion parameters, written 7B, stores seven billion of them. More parameters generally give a model more capacity to learn, but they also mean a bigger download and more memory to run.
The count can mislead for mixture-of-experts models, which list a total and a smaller active number used for each token. A model of '120B total, 5B active' needs memory for the total but computes with only the active part. Parameter count alone also does not tell you quality: training data, training method and architecture matter just as much.
Related terms
- Mixture of experts (MoE) A mixture-of-experts model contains many specialist sub-networks and uses only a few of them for each token, so it can be very large yet fast.
- VRAM VRAM is the memory on a graphics card; it decides which models a GPU can hold, because the weights must fit in memory for the model to run quickly.
- Quantization Quantization stores a model's weights with fewer bits, for example 4 instead of 16, to cut memory use and speed up inference at a small cost in quality.