Unified memory

Unified memory is a design, used by Apple silicon and some other chips, in which the CPU and GPU share one pool of memory, so a large model can use most of it.

  • Hardware and memory
  • Updated

On a typical PC the graphics card has its own VRAM, which is limited. On a Mac with Apple silicon, and on systems such as AMD Strix Halo and NVIDIA DGX Spark, the processor and graphics share a single pool of memory, so a machine with 128 GB can hold a model that no 24 GB graphics card could.

The trade-off is speed. Unified memory is usually slower than the memory on high-end GPUs, so large models run, but with fewer tokens per second. The 'Big-memory desktop' tier on this site refers to this class of machine.

Related terms

  • VRAM VRAM is the memory on a graphics card; it decides which models a GPU can hold, because the weights must fit in memory for the model to run quickly.
  • Quantization Quantization stores a model's weights with fewer bits, for example 4 instead of 16, to cut memory use and speed up inference at a small cost in quality.

Keep reading

All terms