Token

A token is a small chunk of text, often a word or part of a word, that a language model reads and writes; models measure length and speed in tokens.

  • The basics
  • Updated

Before a model reads text, a tokenizer splits it into tokens. For English, a token is on average about three quarters of a word, so 1,000 tokens is roughly 750 words. The exact split depends on the tokenizer, and other languages and code often need more tokens for the same content.

Limits and speeds are quoted in tokens. The context window is the maximum number of tokens a model can consider at once, and generation speed is reported in tokens per second.

Related terms

  • Context window The context window is the maximum amount of text, measured in tokens, that a model can take into account at one time, including your prompt and its own answer.
  • Inference Inference is the act of running a trained model to get an output, such as answering a prompt, as opposed to training it.

All terms