Dense model
A dense model uses all of its parameters to process every token, unlike a mixture-of-experts model, which activates only a part of them.
Dense models are simpler to reason about. A 27B dense model needs memory for 27 billion parameters and does the work of 27 billion parameters on every token, so its speed and memory use scale together. That makes it easy to size for a given machine.
Dense models of roughly 20B to 35B parameters, such as Qwen3.8-27B and the 31B version of Gemma 4, are popular for running on a single gaming GPU or a well-equipped laptop at 4-bit precision.
Related terms
- Mixture of experts (MoE) A mixture-of-experts model contains many specialist sub-networks and uses only a few of them for each token, so it can be very large yet fast.
- Parameters Parameters are the numbers inside a model that are adjusted during training; their count, such as 7B or 70B, is the usual measure of a model's size.