Retrieval-augmented generation (RAG)
Retrieval-augmented generation lets a model answer from your own documents by retrieving the relevant passages and adding them to the prompt.
A RAG system splits your documents into chunks and stores their embeddings. For each question it finds the closest chunks and hands them to a language model, which writes the answer. The model does not need to be retrained, and the answer can point back to the source passages.
RAG reduces hallucinations but does not remove them. The model can still misread the passages or ignore them, and results depend heavily on the quality of the embedding model and on how the documents are chunked.
Related terms
- Embedding model An embedding model turns text, images or other data into a list of numbers that captures its meaning, so that similar items end up close together.
- Hallucination A hallucination is a confident but false or invented statement produced by an AI model.
- Context window The context window is the maximum amount of text, measured in tokens, that a model can take into account at one time, including your prompt and its own answer.
- Agent memory Agent memory lets an AI agent keep facts, preferences and past experience between sessions instead of starting from a blank context.