Agent memory

Agent memory lets an AI agent keep facts, preferences and past experience between sessions instead of starting from a blank context.

  • Using models
  • Updated

Most chat models only see what fits in the current context window. Agent memory adds a store that outlives the conversation. The common design is an external memory framework such as Mem0 or MemOS. After each exchange a model decides what to write down, how precisely to word it and how new facts should update old ones, and a retrieval step brings the relevant entries back later. Tencent's Hy-Memory-30B-A3B is an open model trained only for that writing step.

A different approach, taken by MemTensor's MemT-1.0, builds the memory into the model. A fixed-size state sits beside the attention layers, is updated after each turn and can be saved to a file and loaded later, so the old conversation does not have to stay in the prompt. MemTensor says that at a 128K-token history this needs about 66 times less persistent storage than a KV cache, but it also says that strong text-based memory systems still lead overall.

Memory benchmarks such as LoCoMo and LongMemEval are young, and the published tables come from the developers' own runs, often on subsamples. Stored memories can hold personal information, so decide where they live, how users can inspect or delete them and what the agent is allowed to remember.

Related terms

  • AI agent An AI agent is a system in which a model plans and carries out multi-step tasks by calling tools, such as running code, searching the web or editing files.
  • Retrieval-augmented generation (RAG) Retrieval-augmented generation lets a model answer from your own documents by retrieving the relevant passages and adding them to the prompt.
  • Context window The context window is the maximum amount of text, measured in tokens, that a model can take into account at one time, including your prompt and its own answer.
  • KV cache The KV cache is the memory a model uses to remember the earlier tokens of a conversation, and it grows with the length of the context.

All terms