MemGPT: Towards LLMs as Operating Systems

Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, Joseph E. Gonzalez

UC Berkeley

Paper Discord GitHub

Teach LLMs to manage their own memory for unbounded context!

MemGPT system overview In MemGPT, a fixed-context LLM is augmented with a tiered memory system and a set of functions that allow it to manage its own memory. The LLM inputs text in it's main context (capped at the size of the standard LLM context window), and outputs text that is interpreted by a parser, resulting either in a yield or a function call. MemGPT uses functions to move data between main context and external context. When the LLM generates a function call, it can request immediate return of execution to chain together functions. In the case of a yield, the LLM will not be run again until the next external event trigger (e.g. a user message or scheduled interrupt).

Overview

  • LLMs are increasingly being used for perpetual chats.
  • Challenge: Limited context length makes perpetual chat challenging.
  • Inspired by memory management in OS, MemGPT uses RAM (in-context), and Disk (Archival) memory to effeciently manage the limited context length.
  • With MemGPT, we demonstrate that LLMs can be taught to manage their own memory!

Abstract

Large language models (LLMs) have revolutionized AI but are constrained by limited context windows, hindering their utility in tasks like extended conversa- tions and document analysis. Drawing inspiration from the hierarchical memory systems in traditional operating systems, we introduce MemGPT (Memory-GPT). Similar to how an operating system (OS) provides the illusion of large memory resources through data movement between fast and slow memory, MemGPT man- ages tiers of memory to effectively manage extended context within the language model's limited context window, and utilizes interrupts to manage control flow between itself and the user. We evaluate our OS-inspired design in two domains where the limited context windows of modern LLMs severely handicaps their per- formance: document analysis, where MemGPT is able to analyze large documents that far exceed the underlying LLM's context limit, and multi-session chat, where MemGPT enables conversational agents to remember, reflect, and evolve dynam- ically through long-term interactions with their users. Code and Data is available at https://memgpt.ai

Citation

@inproceedings{packer2023memgpt,
  title={{MemGPT}: Towards LLMs as Operating Systems},
  author={Packer, Charles and Fang, Vivian and Patil, Shishir G.
  and Lin, Kevin and Wooders, Sarah and Gonzalez, Joseph E.},
  year={2023}
}