302 lines
12 KiB
Markdown
302 lines
12 KiB
Markdown
<a href="#user-content-memgpt"><img src="https://memgpt.ai/assets/img/memgpt_logo_circle.png" alt="MemGPT logo" width="75" align="right"></a>
|
|
|
|
# [MemGPT](https://memgpt.ai)
|
|
|
|
<div align="center">
|
|
|
|
<strong>Try out our MemGPT chatbot on <a href="https://discord.gg/9GEQrxmVyE">Discord</a>!</strong>
|
|
|
|
[](https://discord.gg/9GEQrxmVyE)
|
|
[](https://arxiv.org/abs/2310.08560)
|
|
|
|
</div>
|
|
|
|
<details open>
|
|
<summary><h2>🤖 Create perpetual chatbots with self-editing memory!</h2></summary>
|
|
<div align="center">
|
|
<br>
|
|
<img src="https://memgpt.ai/assets/img/demo.gif" alt="MemGPT demo video" width="800">
|
|
</div>
|
|
</details>
|
|
|
|
<details>
|
|
<summary><h2>🗃️ Chat with your data - talk to your SQL database or your local files!</strong></h2></summary>
|
|
<strong>SQL Database</strong>
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/sql_demo.gif" alt="MemGPT demo video for sql search" width="800">
|
|
</div>
|
|
<strong>Local files</strong>
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/preload_archival_demo.gif" alt="MemGPT demo video for sql search" width="800">
|
|
</div>
|
|
</details>
|
|
|
|
<details>
|
|
<summary><h2>📄 You can also talk to docs - for example ask about <a href="memgpt/personas/examples/docqa">LlamaIndex</a>!</h1></summary>
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/docqa_demo.gif" alt="MemGPT demo video for llamaindex api docs search" width="800">
|
|
</div>
|
|
<details>
|
|
<summary><b>ChatGPT (GPT-4) when asked the same question:</b></summary>
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/llama_index_gpt4.png" alt="GPT-4 when asked about llamaindex api docs" width="800">
|
|
</div>
|
|
(Question from https://github.com/run-llama/llama_index/issues/7756)
|
|
</details>
|
|
</details>
|
|
|
|
## Quick setup
|
|
|
|
Join <a href="https://discord.gg/9GEQrxmVyE">Discord</a></strong> and message the MemGPT bot (in the `#memgpt` channel). Then run the following commands (messaged to "MemGPT Bot"):
|
|
* `/profile` (to create your profile)
|
|
* `/key` (to enter your OpenAI key)
|
|
* `/create` (to create a MemGPT chatbot)
|
|
|
|
Make sure your privacy settings on this server are open so that MemGPT Bot can DM you: \
|
|
MemGPT → Privacy Settings → Direct Messages set to ON
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/discord/dm_settings.png" alt="set DMs settings on MemGPT server to be open in MemGPT so that MemGPT Bot can message you" width="400">
|
|
</div>
|
|
|
|
You can see the full list of available commands when you enter `/` into the message box.
|
|
<div align="center">
|
|
<img src="https://memgpt.ai/assets/img/discord/slash_commands.png" alt="MemGPT Bot slash commands" width="400">
|
|
</div>
|
|
|
|
## What is MemGPT?
|
|
|
|
Memory-GPT (or MemGPT in short) is a system that intelligently manages different memory tiers in LLMs in order to effectively provide extended context within the LLM's limited context window. For example, MemGPT knows when to push critical information to a vector database and when to retrieve it later in the chat, enabling perpetual conversations. Learn more about MemGPT in our [paper](https://arxiv.org/abs/2310.08560).
|
|
|
|
## Running MemGPT locally
|
|
|
|
Install dependencies:
|
|
|
|
```sh
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
Extra step for Windows:
|
|
|
|
```sh
|
|
# only needed on Windows
|
|
pip install pyreadline3
|
|
```
|
|
|
|
Add your OpenAI API key to your environment:
|
|
|
|
```sh
|
|
# on Linux/Mac
|
|
export OPENAI_API_KEY=YOUR_API_KEY
|
|
```
|
|
|
|
```sh
|
|
# on Windows
|
|
set OPENAI_API_KEY=YOUR_API_KEY
|
|
```
|
|
|
|
To run MemGPT for as a conversation agent in CLI mode, simply run `main.py`:
|
|
|
|
```sh
|
|
python3 main.py
|
|
```
|
|
|
|
If you're using Azure OpenAI, set these variables instead:
|
|
|
|
```sh
|
|
# see https://github.com/openai/openai-python#microsoft-azure-endpoints
|
|
export AZURE_OPENAI_KEY = ...
|
|
export AZURE_OPENAI_ENDPOINT = ...
|
|
export AZURE_OPENAI_VERSION = ...
|
|
export AZURE_OPENAI_DEPLOYMENT = ...
|
|
|
|
# then use the --use_azure_openai flag
|
|
python main.py --use_azure_openai
|
|
```
|
|
|
|
To create a new starter user or starter persona (that MemGPT gets initialized with), create a new `.txt` file in [/memgpt/humans/examples](/memgpt/humans/examples) or [/memgpt/personas/examples](/memgpt/personas/examples), then use the `--persona` or `--human` flag when running `main.py`. For example:
|
|
|
|
```sh
|
|
# assuming you created a new file /memgpt/humans/examples/me.txt
|
|
python main.py --human me.txt
|
|
```
|
|
|
|
### GPT-3.5 support
|
|
You can run MemGPT with GPT-3.5 as the LLM instead of GPT-4:
|
|
```sh
|
|
python main.py --model gpt-3.5-turbo
|
|
```
|
|
|
|
**Note that this is experimental gpt-3.5-turbo support. It's quite buggy compared to gpt-4, but it should be runnable.**
|
|
|
|
Please report any bugs you encounter regarding MemGPT running on GPT-3.5 to https://github.com/cpacker/MemGPT/issues/59.
|
|
|
|
### Local LLM support
|
|
You can run MemGPT with local LLMs too. See [instructions here](/memgpt/local_llm) and report any bugs/improvements here https://github.com/cpacker/MemGPT/discussions/67.
|
|
|
|
### `main.py` flags
|
|
|
|
```text
|
|
--model
|
|
select which model to use ('gpt-4', 'gpt-3.5-turbo-0613', 'gpt-3.5-turbo')
|
|
--persona
|
|
load a specific persona file
|
|
--human
|
|
load a specific human file
|
|
--first
|
|
allows you to send the first message in the chat (by default, MemGPT will send the first message)
|
|
--debug
|
|
enables debugging output
|
|
--archival_storage_faiss_path=<ARCHIVAL_STORAGE_FAISS_PATH>
|
|
load in document database (backed by FAISS index)
|
|
--archival_storage_files="<ARCHIVAL_STORAGE_FILES_GLOB_PATTERN>"
|
|
pre-load files into archival memory
|
|
--archival_storage_files_compute_embeddings="<ARCHIVAL_STORAGE_FILES_GLOB_PATTERN>"
|
|
pre-load files into archival memory and also compute embeddings for embedding search
|
|
--archival_storage_sqldb=<SQLDB_PATH>
|
|
load in SQL database
|
|
```
|
|
|
|
### Interactive CLI commands
|
|
|
|
These are the commands for the CLI, **not the Discord bot**! The Discord bot has separate commands you can see in Discord by typing `/`.
|
|
|
|
While using MemGPT via the CLI (not Discord!) you can run various commands:
|
|
|
|
```text
|
|
//
|
|
enter multiline input mode (type // again when done)
|
|
/exit
|
|
exit the CLI
|
|
/save
|
|
save a checkpoint of the current agent/conversation state
|
|
/load
|
|
load a saved checkpoint
|
|
/dump
|
|
view the current message log (see the contents of main context)
|
|
/memory
|
|
print the current contents of agent memory
|
|
/pop
|
|
undo the last message in the conversation
|
|
/heartbeat
|
|
send a heartbeat system message to the agent
|
|
/memorywarning
|
|
send a memory warning system message to the agent
|
|
```
|
|
## Example applications
|
|
<details open>
|
|
<summary><h3>Use MemGPT to talk to your Database!</h3></summary>
|
|
|
|
MemGPT's archival memory let's you load your database and talk to it! To motivate this use-case, we have included a toy example.
|
|
|
|
Consider the `test.db` already included in the repository.
|
|
|
|
id | name | age
|
|
--- | --- | ---
|
|
1 | Alice | 30
|
|
2 | Bob | 25
|
|
3 | Charlie | 35
|
|
|
|
To talk to this database, run:
|
|
|
|
```sh
|
|
python main.py --archival_storage_sqldb=memgpt/personas/examples/sqldb/test.db
|
|
```
|
|
|
|
And then you can input the path to your database, and your query.
|
|
|
|
```python
|
|
Please enter the path to the database. test.db
|
|
...
|
|
Enter your message: How old is Bob?
|
|
...
|
|
🤖 Bob is 25 years old.
|
|
```
|
|
</details>
|
|
<details>
|
|
<summary><h3>Loading local files into archival memory</h3></summary>
|
|
MemGPT enables you to chat with your data locally -- this example gives the workflow for loading documents into MemGPT's archival memory.
|
|
|
|
To run our example where you can search over the SEC 10-K filings of Uber, Lyft, and Airbnb,
|
|
|
|
1. Download the .txt files from [Hugging Face](https://huggingface.co/datasets/MemGPT/example-sec-filings/tree/main) and place them in `memgpt/personas/examples/preload_archival`.
|
|
|
|
2. In the root `MemGPT` directory, run
|
|
```bash
|
|
python3 main.py --archival_storage_files="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic
|
|
```
|
|
|
|
If you would like to load your own local files into MemGPT's archival memory, run the command above but replace `--archival_storage_files="memgpt/personas/examples/preload_archival/*.txt"` with your own file glob expression (enclosed in quotes).
|
|
|
|
#### Enhance with embeddings search
|
|
In the root `MemGPT` directory, run
|
|
```bash
|
|
python3 main.py --archival_storage_files_compute_embeddings="<GLOB_PATTERN>" --persona=memgpt_doc --human=basic
|
|
```
|
|
|
|
This will generate embeddings, stick them into a FAISS index, and write the index to a directory, and then output:
|
|
```
|
|
To avoid computing embeddings next time, replace --archival_storage_files_compute_embeddings=<GLOB_PATTERN> with
|
|
--archival_storage_faiss_path=<DIRECTORY_WITH_EMBEDDINGS> (if your files haven't changed).
|
|
```
|
|
|
|
If you want to reuse these embeddings, run
|
|
```bash
|
|
python3 main.py --archival_storage_faiss_path="<DIRECTORY_WITH_EMBEDDINGS>" --persona=memgpt_doc --human=basic
|
|
```
|
|
|
|
|
|
</details>
|
|
<details>
|
|
<summary><h3>Talking to LlamaIndex API Docs</h3></summary>
|
|
|
|
MemGPT also enables you to chat with docs -- try running this example to talk to the LlamaIndex API docs!
|
|
|
|
1.
|
|
a. Download LlamaIndex API docs and FAISS index from [Hugging Face](https://huggingface.co/datasets/MemGPT/llamaindex-api-docs).
|
|
```bash
|
|
# Make sure you have git-lfs installed (https://git-lfs.com)
|
|
git lfs install
|
|
git clone https://huggingface.co/datasets/MemGPT/llamaindex-api-docs
|
|
mv llamaindex-api-docs
|
|
```
|
|
|
|
**-- OR --**
|
|
|
|
b. Build the index:
|
|
1. Build `llama_index` API docs with `make text`. Instructions [here](https://github.com/run-llama/llama_index/blob/main/docs/DOCS_README.md). Copy over the generated `_build/text` folder to `memgpt/personas/docqa`.
|
|
2. Generate embeddings and FAISS index.
|
|
```bash
|
|
cd memgpt/personas/docqa
|
|
python3 scrape_docs.py
|
|
python3 generate_embeddings_for_docs.py all_docs.jsonl
|
|
python3 build_index.py --embedding_files all_docs.embeddings.jsonl --output_index_file all_docs.index
|
|
|
|
3. In the root `MemGPT` directory, run
|
|
```bash
|
|
python3 main.py --archival_storage_faiss_path=<ARCHIVAL_STORAGE_FAISS_PATH> --persona=memgpt_doc --human=basic
|
|
```
|
|
where `ARCHIVAL_STORAGE_FAISS_PATH` is the directory where `all_docs.jsonl` and `all_docs.index` are located.
|
|
If you downloaded from Hugging Face, it will be `memgpt/personas/docqa/llamaindex-api-docs`.
|
|
If you built the index yourself, it will be `memgpt/personas/docqa`.
|
|
</details>
|
|
|
|
## Support
|
|
|
|
If you have any further questions, or have anything to share, we are excited to hear your feedback!
|
|
|
|
* By default MemGPT will use `gpt-4`, so your API key will require `gpt-4` API access
|
|
* For issues and feature requests, please [open a GitHub issue](https://github.com/cpacker/MemGPT/issues) or message us on our `#support` channel on [Discord](https://discord.gg/9GEQrxmVyE)
|
|
|
|
## Datasets
|
|
Datasets used in our [paper](https://arxiv.org/abs/2310.08560) can be downloaded at [Hugging Face](https://huggingface.co/MemGPT).
|
|
|
|
## 🚀 Project Roadmap
|
|
- [x] Release MemGPT Discord bot demo (perpetual chatbot)
|
|
- [x] Add additional workflows (load SQL/text into MemGPT external context)
|
|
- [x] Integration tests
|
|
- [x] Integrate with AutoGen ([discussion](https://github.com/cpacker/MemGPT/discussions/65))
|
|
- [x] Add official gpt-3.5-turbo support ([discussion](https://github.com/cpacker/MemGPT/discussions/66))
|
|
- [ ] Release MemGPT family of open models (eg finetuned Mistral) ([discussion](https://github.com/cpacker/MemGPT/discussions/67))
|
|
- [ ] CLI UI improvements ([issue](https://github.com/cpacker/MemGPT/issues/11))
|
|
- [ ] Add support for other LLM backends ([issue](https://github.com/cpacker/MemGPT/issues/18))
|