diff --git a/docs/.markdownlint.json b/docs/.markdownlint.json new file mode 100644 index 00000000..1928ee34 --- /dev/null +++ b/docs/.markdownlint.json @@ -0,0 +1,6 @@ +{ + "MD013": false, + "MD028": false, + "MD033": false, + "MD034": false +} \ No newline at end of file diff --git a/docs/adding_wrappers.md b/docs/adding_wrappers.md index eb4f2a1c..517ed67a 100644 --- a/docs/adding_wrappers.md +++ b/docs/adding_wrappers.md @@ -5,7 +5,7 @@ category: 6580da9a40bb410016b8b0c3 --- > ⚠️ MemGPT + local LLM failure cases -> + > When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls. ### What is a "wrapper"? @@ -42,10 +42,9 @@ class LLMChatCompletionWrapper(ABC): You can follow our example wrappers ([located here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_llm/llm_chat_completion_wrappers)). - ### Example with [Airoboros](https://huggingface.co/jondurbin/airoboros-l2-70b-2.1) (llama2 finetune) -To help you get started, we've implemented an example wrapper class for a popular llama2 model **finetuned on function calling** (Airoboros). We want MemGPT to run well on open models as much as you do, so we'll be actively updating this page with more examples. Additionally, we welcome contributions from the community! If you find an open LLM that works well with MemGPT, please open a PR with a model wrapper and we'll merge it ASAP. +To help you get started, we've implemented an example wrapper class for a popular llama2 model **fine-tuned on function calling** (Airoboros). We want MemGPT to run well on open models as much as you do, so we'll be actively updating this page with more examples. Additionally, we welcome contributions from the community! If you find an open LLM that works well with MemGPT, please open a PR with a model wrapper and we'll merge it ASAP. ```python class Airoboros21Wrapper(LLMChatCompletionWrapper): @@ -83,9 +82,9 @@ See full file [here](https://github.com/cpacker/MemGPT/tree/main/memgpt/local_ll MemGPT uses function calling to do memory management. With [OpenAI's ChatCompletion API](https://platform.openai.com/docs/api-reference/chat/), you can pass in a function schema in the `functions` keyword arg, and the API response will include a `function_call` field that includes the function name and the function arguments (generated JSON). How this works under the hood is your `functions` keyword is combined with the `messages` and `system` to form one big string input to the transformer, and the output of the transformer is parsed to extract the JSON function call. -In the future, more open LLMs and LLM servers (that can host OpenAI-compatable ChatCompletion endpoints) may start including parsing code to do this automatically as standard practice. However, in the meantime, when you see a model that says it supports “function calling”, like Airoboros, it doesn't mean that you can just load Airoboros into a ChatCompletion-compatable endpoint like WebUI, and then use the same OpenAI API call and it'll just work. +In the future, more open LLMs and LLM servers (that can host OpenAI-compatible ChatCompletion endpoints) may start including parsing code to do this automatically as standard practice. However, in the meantime, when you see a model that says it supports “function calling”, like Airoboros, it doesn't mean that you can just load Airoboros into a ChatCompletion-compatible endpoint like WebUI, and then use the same OpenAI API call and it'll just work. -1. When a model page says it supports function calling, they probably mean that the model was finetuned on some function call data (not that you can just use ChatCompletion with functions out-of-the-box). Remember, LLMs are just string-in-string-out, so there are many ways to format the function call data. E.g. Airoboros formats the function schema in YAML style (see https://huggingface.co/jondurbin/airoboros-l2-70b-3.1.2#agentfunction-calling) and the output is in JSON style. To get this to work behind a ChatCompletion API, you still have to do the parsing from `functions` keyword arg (containing the schema) to the model's expected schema style in the prompt (YAML for Airoboros), and you have to run some code to extract the function call (JSON for Airoboros) and package it cleanly as a `function_call` field in the response. +1. When a model page says it supports function calling, they probably mean that the model was fine-tuned on some function call data (not that you can just use ChatCompletion with functions out-of-the-box). Remember, LLMs are just string-in-string-out, so there are many ways to format the function call data. E.g. Airoboros formats the function schema in YAML style (see https://huggingface.co/jondurbin/airoboros-l2-70b-3.1.2#agentfunction-calling) and the output is in JSON style. To get this to work behind a ChatCompletion API, you still have to do the parsing from `functions` keyword arg (containing the schema) to the model's expected schema style in the prompt (YAML for Airoboros), and you have to run some code to extract the function call (JSON for Airoboros) and package it cleanly as a `function_call` field in the response. 2. Partly because of how complex it is to support function calling, most (all?) of the community projects that do OpenAI ChatCompletion endpoints for arbitrary open LLMs do not support function calling, because if they did, they would need to write model-specific parsing code for each one. diff --git a/docs/api.md b/docs/api.md index 3fb42042..16cdb29e 100644 --- a/docs/api.md +++ b/docs/api.md @@ -7,21 +7,21 @@ category: 658135e7f596b800715c1cee ![memgpt llama](https://raw.githubusercontent.com/cpacker/MemGPT/main/docs/assets/memgpt_server.webp) > ⚠️ API under active development -> +> > The MemGPT API is under **active development** and **breaking changes are being made frequently**. Do not expect any endpoints or API schema to persist until an official `v1.0` of the MemGPT API is released. -> +> > For support and to track ongoing developments, please visit [the MemGPT Discord server](https://discord.gg/9GEQrxmVyE) where you can chat with the MemGPT team and other developers about the API. > 📘 Check Discord for the latest development build -> +> > Make sure to check [Discord](https://discord.gg/9GEQrxmVyE) for updates on the latest development branch to use. The API reference viewable on this page may only apply to the latest dev branch, so if you plan to experiment with the API we recommend you [install MemGPT from source](https://memgpt.readme.io/docs/contributing#installing-from-source) for the time being. ## Starting a server process You can spawn a MemGPT server process using the following command: -```Text shell +```sh memgpt server ``` -Before attempting to launch a server process, make sure that you have already configured MemGPT (using `memgpt configure`) and are able to successfully create and message an agent using `memgpt run`. For more information, see [our quickstart guide](https://memgpt.readme.io/docs/quickstart). \ No newline at end of file +Before attempting to launch a server process, make sure that you have already configured MemGPT (using `memgpt configure`) and are able to successfully create and message an agent using `memgpt run`. For more information, see [our quickstart guide](https://memgpt.readme.io/docs/quickstart). diff --git a/docs/autogen.md b/docs/autogen.md index b48cd849..a9aa2abb 100644 --- a/docs/autogen.md +++ b/docs/autogen.md @@ -1,7 +1,7 @@ --- -title: MemGPT + AutoGen +title: MemGPT + AutoGen excerpt: Creating AutoGen agents powered by MemGPT -category: 6580dab16cade8003f996d17 +category: 6580dab16cade8003f996d17 --- > 📘 Need help? @@ -21,6 +21,7 @@ category: 6580dab16cade8003f996d17 MemGPT includes an AutoGen agent class ([MemGPTAgent](https://github.com/cpacker/MemGPT/blob/main/memgpt/autogen/memgpt_agent.py)) that mimics the interface of AutoGen's [ConversableAgent](https://microsoft.github.io/autogen/docs/reference/agentchat/conversable_agent#conversableagent-objects), allowing you to plug MemGPT into the AutoGen framework. To create a MemGPT AutoGen agent for use in an AutoGen script, you can use the `create_memgpt_autogen_agent_from_config` constructor: + ```python from memgpt.autogen.memgpt_agent import create_memgpt_autogen_agent_from_config @@ -56,6 +57,7 @@ memgpt_autogen_agent = create_memgpt_autogen_agent_from_config( ``` Now this `memgpt_autogen_agent` can be used in standard AutoGen scripts: + ```python import autogen @@ -97,6 +99,7 @@ Once you've confirmed that you're able to chat with a MemGPT agent using `memgpt > If you're using RunPod to run web UI, make sure that you set your endpoint to the RunPod IP address, **not the default localhost address**. > > For example, during `memgpt configure`: +> > ```text > ? Enter default endpoint: https://yourpodaddresshere-5000.proxy.runpod.net > ``` @@ -106,6 +109,7 @@ Once you've confirmed that you're able to chat with a MemGPT agent using `memgpt Now we're going to integrate MemGPT and AutoGen by creating a special "MemGPT AutoGen agent" that wraps MemGPT in an AutoGen-style agent interface. First, make sure you have AutoGen installed: + ```sh pip install pyautogen ``` @@ -117,7 +121,9 @@ In order to run this example on a local LLM, go to lines 46-66 in [examples/agen `config_list` is used by non-MemGPT AutoGen agents, which expect an OpenAI-compatible API. `config_list_memgpt` is used by MemGPT AutoGen agents, and requires additional settings specific to MemGPT (such as the `model_wrapper` and `context_window`. Depending on what LLM backend you want to use, you'll have to set up your `config_list` and `config_list_memgpt` differently: #### web UI example + For example, if you are using web UI, it will look something like this: + ```python # Non-MemGPT agents will still use local LLMs, but they will use the ChatCompletions endpoint config_list = [ @@ -132,7 +138,7 @@ config_list = [ config_list_memgpt = [ { "preset": DEFAULT_PRESET, - "model": None, # not required for web UI, only required for Ollama, see: https://memgpt.readme.io/docs/ollama + "model": None, # not required for web UI, only required for Ollama, see: https://memgpt.readme.io/docs/ollama "model_wrapper": "airoboros-l2-70b-2.1", # airoboros is the default wrapper and should work for most models "model_endpoint_type": "webui", "model_endpoint": "http://localhost:5000", # notice port 5000 for web UI @@ -142,7 +148,9 @@ config_list_memgpt = [ ``` #### LM Studio example + If you are using LM Studio, then you'll need to change the `api_base` in `config_list`, and `model_endpoint_type` + `model_endpoint` in `config_list_memgpt`: + ```python # Non-MemGPT agents will still use local LLMs, but they will use the ChatCompletions endpoint config_list = [ @@ -167,7 +175,9 @@ config_list_memgpt = [ ``` #### OpenAI example + If you are using the OpenAI API (e.g. using `gpt-4-turbo` via your own OpenAI API account), then the `config_list` for the AutoGen agent and `config_list_memgpt` for the MemGPT AutoGen agent will look different (a lot simpler): + ```python # This config is for autogen agents that are not powered by MemGPT config_list = [ @@ -192,7 +202,9 @@ config_list_memgpt = [ ``` #### Azure OpenAI example + Azure OpenAI API setup will be similar to OpenAI API, but requires additional config variables. First, make sure that you've set all the related Azure variables referenced in [our MemGPT Azure setup page](https://memgpt.readme.io/docs/endpoints#azure-openai) (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_VERSION`, `AZURE_OPENAI_ENDPOINT`, etc). If you have all the variables set correctly, you should be able to create configs by pulling from the env variables: + ```python # This config is for autogen agents that are not powered by MemGPT # See Auto @@ -219,17 +231,17 @@ config_list_memgpt = [ "azure_endpoint": os.getenv("AZURE_OPENAI_ENDPOINT"), "azure_version": os.getenv("AZURE_OPENAI_VERSION"), # if you are using Azure for embeddings too, include the following line: - "embedding_embedding_endpoint_type": "azure", + "embedding_embedding_endpoint_type": "azure", }, ] ``` - > 📘 Making internal monologue visible to AutoGen > > By default, MemGPT's inner monologue and function traces are hidden from other AutoGen agents. > > You can modify `interface_kwargs` to change the visibility of inner monologue and function calling: +> > ```python > interface_kwargs = { > "debug": False, # this is the equivalent of the --debug flag in the MemGPT CLI @@ -239,11 +251,13 @@ config_list_memgpt = [ > ``` The only parts of the `agent_groupchat.py` file you need to modify should be the `config_list` and `config_list_memgpt` (make sure to change `USE_OPENAI` to `True` or `False` depending on if you're trying to use a local LLM server like web UI, or OpenAI's API). Assuming you edited things correctly, you should now be able to run `agent_groupchat.py`: + ```sh python memgpt/autogen/examples/agent_groupchat.py ``` Your output should look something like this: + ```text User_proxy (to chat_manager): @@ -287,14 +301,14 @@ Remember, achieving one million dollars in revenue in such a short time frame wo -------------------------------------------------------------------------------- MemGPT_coder (to chat_manager): -Great goal! Generating a million dollars in one month with an app is ambitious, but definitely doable if you approach it the right way. Here are some tips and potential ideas that could help: +Great goal! Generating a million dollars in one month with an app is ambitious, but definitely doable if you approach it the right way. Here are some tips and potential ideas that could help: -1. Identify a niche market or trend (for example, AI-powered fitness apps or FinTech solutions). -2. Solve a significant problem for many people (such as time management or financial literacy). -3. Choose an effective monetization strategy like subscriptions, in-app purchases, or advertising. +1. Identify a niche market or trend (for example, AI-powered fitness apps or FinTech solutions). +2. Solve a significant problem for many people (such as time management or financial literacy). +3. Choose an effective monetization strategy like subscriptions, in-app purchases, or advertising. 4. Make sure your app is visually appealing and easy to use to keep users engaged. -Some ideas that might work: +Some ideas that might work: - AI-powered personal finance management app - A virtual assistant app that helps people manage their daily tasks - A social networking platform for job seekers or freelancers @@ -316,15 +330,18 @@ User_proxy (to chat_manager): First, follow the instructions in [Example - chat with your data - Creating an external data source](example_data/#creating-an-external-data-source): To download the MemGPT research paper we'll use `curl` (you can also just download the PDF from your browser): + ```sh # we're saving the file as "memgpt_research_paper.pdf" curl -L -o memgpt_research_paper.pdf https://arxiv.org/pdf/2310.08560.pdf ``` Now that we have the paper downloaded, we can create a MemGPT data source using `memgpt load`: + ```sh memgpt load directory --name memgpt_research_paper --input-files=memgpt_research_paper.pdf ``` + ```text loading data done loading data @@ -337,9 +354,11 @@ Generating embeddings: 100%|█████████████████ Note: you can ignore the "_LLM is explicitly disabled_" message. Now, you can run `agent_docs.py`, which asks `MemGPT_coder` what a virtual context is: + ```sh python memgpt/autogen/examples/agent_docs.py ``` + ```text Ingesting 65 passages into MemGPT_agent 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.47s/it] diff --git a/docs/cli_faq.md b/docs/cli_faq.md index f105e2fc..23d1caf2 100644 --- a/docs/cli_faq.md +++ b/docs/cli_faq.md @@ -1,11 +1,11 @@ --- title: Frequently asked questions (FAQ) excerpt: Check frequently asked questions -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d --- > 📘 Open / local LLM FAQ -> +> > Questions specific to running your own open / local LLMs with MemGPT can be found [here](local_llm_faq). ## MemGPT CLI diff --git a/docs/config.md b/docs/config.md index ece5cdc1..1e579de7 100644 --- a/docs/config.md +++ b/docs/config.md @@ -1,7 +1,7 @@ --- title: Configuration excerpt: Configuring your MemGPT agent -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d --- You can set agent defaults by running `memgpt configure`, which will store config information at `~/.memgpt/config` by default. @@ -18,7 +18,7 @@ The `memgpt run` command supports the following optional flags (if set, will ove * `--no-verify`: (bool) Bypass message verification (default=False) * `--yes`/`-y`: (bool) Skip confirmation prompt and use defaults (default=False) -You can override the parameters you set with `memgpt configure` with the following additional flags specific to local LLMs: +You can override the parameters you set with `memgpt configure` with the following additional flags specific to local LLMs: * `--model-wrapper`: (str) Model wrapper used by backend (e.g. `airoboros_xxx`) * `--model-endpoint-type`: (str) Model endpoint backend type (e.g. lmstudio, ollama) @@ -26,14 +26,17 @@ You can override the parameters you set with `memgpt configure` with the followi * `--context-window`: (int) Size of model context window (specific to model type) #### Updating the config location + You can override the location of the config path by setting the environment variable `MEMGPT_CONFIG_PATH`: -``` + +```sh export MEMGPT_CONFIG_PATH=/my/custom/path/config # make sure this is a file, not a directory ``` - ### Adding Custom Personas/Humans + You can add new human or persona definitions either by providing a file (using the `-f` flag) or text (using the `--text` flag). + ```sh # add a human memgpt add human [--name ] [-f ] [--text ] @@ -43,9 +46,11 @@ memgpt add persona [--name ] [-f ] [--text ] ``` You can view available persona and human files with the following command: + ```sh memgpt list [humans/personas] ``` ### Custom Presets + You can customize your MemGPT agent even further with [custom presets](presets) and [custom functions](functions). diff --git a/docs/contributing.md b/docs/contributing.md index ee03e6f5..809eb570 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -1,14 +1,14 @@ --- title: How to contribute excerpt: Learn how to contribute to the MemGPT project! -category: 6581eaa89a00e6001012822c +category: 6581eaa89a00e6001012822c --- ![memgpt llama](https://raw.githubusercontent.com/cpacker/MemGPT/main/docs/assets/memgpt_library.webp) MemGPT is an active [open source](https://en.wikipedia.org/wiki/Open_source) project and we welcome community contributions! There are many ways to contribute for both programmers and non-programmers alike. -> 📘 Discord contributor role +> 📘 Discord contributor role > > Contributing to the codebase gets you a **contributor role** on [Discord](https://discord.gg/9GEQrxmVyE). If you're a contributor and we forgot to assign you the role, message the MemGPT team [on Discord](https://discord.gg/9GEQrxmVyE)! @@ -22,4 +22,4 @@ We're always looking to improve our docs (like the page you're reading right now ## 🦙 Editing the MemGPT source code -If you're interested in editing the MemGPT source code, [check our guide on building and contributing from source](contributing_code). \ No newline at end of file +If you're interested in editing the MemGPT source code, [check our guide on building and contributing from source](contributing_code). diff --git a/docs/contributing_code.md b/docs/contributing_code.md index 19a694bb..839a877d 100644 --- a/docs/contributing_code.md +++ b/docs/contributing_code.md @@ -1,7 +1,7 @@ --- title: Contributing to the codebase excerpt: How to modify code and create pull requests -category: 6581eaa89a00e6001012822c +category: 6581eaa89a00e6001012822c --- If you plan on making big changes to the codebase, the easiest way to make contributions is to install MemGPT directly from the source code (instead of via `pypi`, which you do with `pip install ...`). @@ -9,16 +9,17 @@ If you plan on making big changes to the codebase, the easiest way to make contr Once you have a working copy of the source code, you should be able to modify the MemGPT codebase an immediately see any changes you make to the codebase change the way the `memgpt` command works! Then once you make a change you're happy with, you can open a pull request to get your changes merged into the official MemGPT package. > 📘 Instructions on installing from a fork and opening pull requests -> +> > If you plan on contributing your changes, you should create a fork of the MemGPT repo and install the source code from your fork. > -> Please see [our contributing guide](https://github.com/cpacker/MemGPT/blob/main/CONTRIBUTING.md) for instructions on how to install from a fork and open a PR. +> Please see [our contributing guide](https://github.com/cpacker/MemGPT/blob/main/CONTRIBUTING.md) for instructions on how to install from a fork and open a PR. ## Installing MemGPT from source **Reminder**: if you plan on opening a pull request to contribute your changes, follow our [contributing guide's install instructions](https://github.com/cpacker/MemGPT/blob/main/CONTRIBUTING.md) instead! To install MemGPT from source, start by cloning the repo: + ```sh git clone git@github.com:cpacker/MemGPT.git ``` @@ -28,39 +29,45 @@ git clone git@github.com:cpacker/MemGPT.git First, install Poetry using [the official instructions here](https://python-poetry.org/docs/#installation). Once Poetry is installed, navigate to the MemGPT directory and install the MemGPT project with Poetry: -```shell + +```sh cd MemGPT poetry shell -poetry install -E dev -E postgres -E local +poetry install -E dev -E postgres -E local ``` Now when you want to use `memgpt`, make sure you first activate the `poetry` environment using poetry shell: -```shell + +```sh $ poetry shell (pymemgpt-py3.10) $ memgpt run ``` Alternatively, you can use `poetry run` (which will activate the `poetry` environment for the `memgpt run` command only): -```shell + +```sh poetry run memgpt run ``` ### Installing dependencies with pip First you should set up a dedicated virtual environment. This is optional, but is highly recommended: -```shell + +```sh cd MemGPT python3 -m venv venv . venv/bin/activate ``` Once you've activated your virtual environment and are in the MemGPT project directory, you can install the dependencies with `pip`: -```shell + +```sh pip install -e '.[dev,postgres,local]' ``` Now, you should be able to run `memgpt` from the command-line using the downloaded source code (if you used a virtual environment, you have to activate the virtual environment to access `memgpt`): -```shell + +```sh $ . venv/bin/activate (venv) $ memgpt run ``` diff --git a/docs/contributing_docs.md b/docs/contributing_docs.md index b3c70c90..102909d5 100644 --- a/docs/contributing_docs.md +++ b/docs/contributing_docs.md @@ -1,14 +1,14 @@ --- -title: Contributing to the documentation -excerpt: How to add to the MemGPT documentation -category: 6581eaa89a00e6001012822c +title: Contributing to the documentation +excerpt: How to add to the MemGPT documentation +category: 6581eaa89a00e6001012822c --- -There are two ways to propose edits to the MemGPT documentation: editing the documentation files directly in the GitHub file editor (on the GitHub website), or cloning the source code and editing the documentation files (in your text/markdown editor of chocie). +There are two ways to propose edits to the MemGPT documentation: editing the documentation files directly in the GitHub file editor (on the GitHub website), or cloning the source code and editing the documentation files (in your text/markdown editor of choice). -# Editing directly via GitHub +## Editing directly via GitHub -> 📘 Requires a GitHub account +> 📘 Requires a GitHub account > > Before beginning, make sure you have an account on [github.com](https://github.com) and are logged in. @@ -23,6 +23,6 @@ The easiest way to edit the docs is directly via the GitHub website: 7. Add the necessary details describing the changes you've made, then click "Create pull request" 8. ✅ That's it! A MemGPT team member will then review your PR and if it looks good merge it into the main branch, at which point you'll see the changes updated on the docs page! -# Editing via the source code +## Editing via the source code -Editing documentation via the source code follows the same process as general source code editing - forking the repository, cloning your fork, editing a branch of your fork, and opening a PR from your fork to the main repo. See our [source code editing guide](contributing_code) for more details. \ No newline at end of file +Editing documentation via the source code follows the same process as general source code editing - forking the repository, cloning your fork, editing a branch of your fork, and opening a PR from your fork to the main repo. See our [source code editing guide](contributing_code) for more details. diff --git a/docs/data_sources.md b/docs/data_sources.md index b648c454..63208310 100644 --- a/docs/data_sources.md +++ b/docs/data_sources.md @@ -1,17 +1,20 @@ --- title: Attaching data sources excerpt: Connecting external data to your MemGPT agent -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d --- MemGPT supports pre-loading data into archival memory. In order to made data accessible to your agent, you must load data in with `memgpt load`, then attach the data source to your agent. You can configure where archival memory is stored by configuring the [storage backend](storage.md). ### Viewing available data sources + You can view available data sources with: -``` + +```sh memgpt list sources ``` -``` + +```sh +----------------+----------+----------+ | Name | Location | Agents | +----------------+----------+----------+ @@ -20,18 +23,22 @@ memgpt list sources | memgpt-docs | local | agent_1 | +----------------+----------+----------+ ``` + The `Agents` column indicates which agents have access to the data, while `Location` indicates what storage backend the data has been loaded into. ### Attaching data to agents + Attaching a data source to your agent loads the data into your agent's archival memory to access. You can attach data to your agent in two ways: *[Option 1]* From the CLI, run: -``` + +```sh memgpt attach --agent --data-source ``` *[Option 2]* While chatting with the agent, enter the `/attach` command and select the data source -``` + +```sh > Enter your message: /attach ? Select data source (Use arrow keys) » short-stories @@ -43,15 +50,18 @@ memgpt attach --agent --data-source > To encourage your agent to reference its archival memory, we recommend adding phrases like "_search your archival memory..._" for the best results. ### Loading a file or directory -You can load a file, list of files, or directry into MemGPT with the following command: + +You can load a file, list of files, or directly into MemGPT with the following command: + ```sh memgpt load directory --name \ [--input-dir ] [--input-files ...] [--recursive] ``` - ### Loading a database dump + You can load database into MemGPT, either from a database dump or a database connection, with the following command: + ```sh memgpt load database --name \ --query \ # Query to run on database to get data @@ -65,7 +75,9 @@ memgpt load database --name \ ``` ### Loading a vector database + If you already have a vector database containing passages and embeddings, you can load them into MemGPT by specifying the table name, database URI, and the columns containing the passage text and embeddings. + ```sh memgpt load vector-database --name \ --uri \ # Database URI @@ -73,15 +85,19 @@ memgpt load vector-database --name \ --text_column \ # Name of column containing text --embedding_column # Name of column containing embedding ``` + Since embeddings are already provided, MemGPT will not re-compute the embeddings. ### Loading a LlamaIndex dump + If you have a Llama Index `VectorIndex` which was saved to disk, you can load it into MemGPT by specifying the directory the index was saved to: + ```sh memgpt load index --name --dir ``` + Since Llama Index will have already computing embeddings, MemGPT will not re-compute embeddings. - ### Loading other types of data + We highly encourage contributions for new data sources, which can be added as a new [CLI data load command](https://github.com/cpacker/MemGPT/blob/main/memgpt/cli/cli_load.py). We recommend checking for [Llama Index connectors](https://gpt-index.readthedocs.io/en/v0.6.3/how_to/data_connectors.html) that may support ingesting the data you're interested in loading. diff --git a/docs/discord_bot.md b/docs/discord_bot.md index 861f95ee..b6bbffc5 100644 --- a/docs/discord_bot.md +++ b/docs/discord_bot.md @@ -1,23 +1,26 @@ --- -title: Chatting with MemGPT Bot +title: Chatting with MemGPT Bot excerpt: Get up and running with the MemGPT Discord Bot -category: 6580da8eb6feb700166e5016 +category: 6580da8eb6feb700166e5016 --- The fastest way to experience MemGPT is to chat with the MemGPT Discord Bot. Join Discord and message the MemGPT bot (in the `#memgpt` channel). Then run the following commands (messaged to "MemGPT Bot"): + * `/profile` (to create your profile) * `/key` (to enter your OpenAI key) * `/create` (to create a MemGPT chatbot) Make sure your privacy settings on this server are open so that MemGPT Bot can DM you: \ MemGPT → Privacy Settings → Direct Messages set to ON +
set DMs settings on MemGPT server to be open in MemGPT so that MemGPT Bot can message you
You can see the full list of available commands when you enter `/` into the message box. +
MemGPT Bot slash commands
diff --git a/docs/embedding_endpoints.md b/docs/embedding_endpoints.md index 52df427b..29b31f43 100644 --- a/docs/embedding_endpoints.md +++ b/docs/embedding_endpoints.md @@ -1,26 +1,32 @@ --- -title: Configuring embedding backends -excerpt: Connecting MemGPT to various endpoint backends -category: 6580d34ee5e4d00068bf2a1d +title: Configuring embedding backends +excerpt: Connecting MemGPT to various endpoint backends +category: 6580d34ee5e4d00068bf2a1d --- -MemGPT uses embedding models for retrieval search over archival memory. You can use embeddings provided by OpenAI, Azure, or any model on Hugging Face. +MemGPT uses embedding models for retrieval search over archival memory. You can use embeddings provided by OpenAI, Azure, or any model on Hugging Face. ## OpenAI -To use OpenAI, make sure your `OPENAI_API_KEY` enviornment variable is set. + +To use OpenAI, make sure your `OPENAI_API_KEY` environment variable is set. + ```sh export OPENAI_API_KEY=YOUR_API_KEY # on Linux/Mac ``` -Then, configure MemGPT and select `openai` as the embedding provider: -``` -> memgpt configure + +Then, configure MemGPT and select `openai` as the embedding provider: + +```text +> memgpt configure ... -? Select embedding provider: openai +? Select embedding provider: openai ... ``` ## Azure -To use Azure, set enviornment variables for Azure and an additional variable specifying your embedding deployment: + +To use Azure, set environment variables for Azure and an additional variable specifying your embedding deployment: + ```sh # see https://github.com/openai/openai-python#microsoft-azure-endpoints export AZURE_OPENAI_KEY = ... @@ -30,18 +36,22 @@ export AZURE_OPENAI_VERSION = ... # set the below if you are using deployment ids export AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT = ... ``` -Then, configure MemGPT and select `azure` as the embedding provider: -``` -> memgpt configure + +Then, configure MemGPT and select `azure` as the embedding provider: + +```text +> memgpt configure ... -? Select embedding provider: azure +? Select embedding provider: azure ... ``` ## Custom Endpoint -MemGPT supports running embeddings with any Hugging Face model using the [Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)(TEI) library. To get started, first make sure you follow TEI's [instructions](https://github.com/huggingface/text-embeddings-inference#get-started) for getting started. Once you have a running endpoint, you can configure MemGPT to use your endpoint: -``` -> memgpt configure + +MemGPT supports running embeddings with any Hugging Face model using the [Text Embeddings Inference](https://github.com/huggingface/text-embeddings-inference)(TEI) library. To get started, first make sure you follow TEI's [instructions](https://github.com/huggingface/text-embeddings-inference#get-started) for getting started. Once you have a running endpoint, you can configure MemGPT to use your endpoint: + +```text +> memgpt configure ... ? Select embedding provider: hugging-face ? Enter default endpoint: http://localhost:8080 @@ -50,24 +60,26 @@ MemGPT supports running embeddings with any Hugging Face model using the [Text E ... ``` -## Local Embeddings +## Local Embeddings -MemGPT can compute embeddings locally using a lightweight embedding model [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5). +MemGPT can compute embeddings locally using a lightweight embedding model [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5). > 🚧 Local LLM Performance -> -> The `BAAI/bge-small-en-v1.5` was chosen to be lightweight, so you may notice degraded performance with embedding-based retrieval when using this option. +> +> The `BAAI/bge-small-en-v1.5` was chosen to be lightweight, so you may notice degraded performance with embedding-based retrieval when using this option. -To compute embeddings locally, install dependencies with: -``` +To compute embeddings locally, install dependencies with: + +```sh pip install `pymemgpt[local]` ``` -Then, select the `local` option during configuration: -``` -> memgpt configure + +Then, select the `local` option during configuration: + +```text +memgpt configure + ... ? Select embedding provider: local ... ``` - - diff --git a/docs/endpoints.md b/docs/endpoints.md index f7e79e87..33649da3 100644 --- a/docs/endpoints.md +++ b/docs/endpoints.md @@ -1,13 +1,15 @@ --- -title: Configuring LLM backends -excerpt: Connecting MemGPT to various LLM backends -category: 6580d34ee5e4d00068bf2a1d +title: Configuring LLM backends +excerpt: Connecting MemGPT to various LLM backends +category: 6580d34ee5e4d00068bf2a1d --- You can use MemGPT with various LLM backends, including the OpenAI API, Azure OpenAI, and various local (or self-hosted) LLM backends. ## OpenAI + To use MemGPT with an OpenAI API key, simply set the `OPENAI_API_KEY` variable: + ```sh export OPENAI_API_KEY=YOUR_API_KEY # on Linux/Mac set OPENAI_API_KEY=YOUR_API_KEY # on Windows @@ -15,7 +17,8 @@ $Env:OPENAI_API_KEY = "YOUR_API_KEY" # on Windows (PowerShell) ``` When you run `memgpt configure`, make sure to select `openai` for both the LLM inference provider and embedding provider, for example: -``` + +```text $ memgpt configure ? Select LLM inference provider: openai ? Override default endpoint: https://api.openai.com/v1 @@ -27,11 +30,14 @@ $ memgpt configure ? Select storage backend for archival data: local ``` -#### OpenAI Proxies +### OpenAI Proxies + To use custom OpenAI endpoints, specify a proxy URL when running `memgpt configure` to set the custom endpoint as the default endpoint. ## Azure OpenAI + To use MemGPT with Azure, expore the following variables and then re-run `memgpt configure`: + ```sh # see https://github.com/openai/openai-python#microsoft-azure-endpoints export AZURE_OPENAI_KEY=... @@ -44,6 +50,7 @@ export AZURE_OPENAI_EMBEDDINGS_DEPLOYMENT=... ``` For example, if your endpoint is `customproject.openai.azure.com` (for both your GPT model and your embeddings model), you would set the following: + ```sh # change AZURE_OPENAI_VERSION to the latest version export AZURE_OPENAI_KEY="YOUR_AZURE_KEY" @@ -53,18 +60,20 @@ export AZURE_OPENAI_EMBEDDING_ENDPOINT="https://customproject.openai.azure.com" ``` If you named your deployments names other than their defaults, you would also set the following: + ```sh -# assume you called the gpt-4 (1106-Preview) deployment "personal-gpt-4-turbo" +# assume you called the gpt-4 (1106-Preview) deployment "personal-gpt-4-turbo" export AZURE_OPENAI_DEPLOYMENT="personal-gpt-4-turbo" -# assume you called the text-embedding-ada-002 deployment "personal-embeddings" +# assume you called the text-embedding-ada-002 deployment "personal-embeddings" export AZURE_OPENAI_EMBEDDING_DEPLOYMENT="personal-embeddings" ``` Replace `export` with `set` or `$Env:` if you are on Windows (see the OpenAI example). When you run `memgpt configure`, make sure to select `azure` for both the LLM inference provider and embedding provider, for example: -``` + +```text $ memgpt configure ? Select LLM inference provider: azure ? Select default model (recommended: gpt-4): gpt-4-1106-preview @@ -77,5 +86,6 @@ $ memgpt configure Note: **your Azure endpoint must support functions** or you will get an error. See [this GitHub issue](https://github.com/cpacker/MemGPT/issues/91) for more information. -## Local Models & Custom Endpoints +## Local Models & Custom Endpoints + MemGPT supports running open source models, both being run locally or as a hosted service. Setting up MemGPT to run with open models requires a bit more setup, follow [the instructions here](local_llm). diff --git a/docs/example_chat.md b/docs/example_chat.md index 1591a7a6..008d1940 100644 --- a/docs/example_chat.md +++ b/docs/example_chat.md @@ -1,11 +1,11 @@ --- -title: Example - perpetual chatbot +title: Example - perpetual chatbot excerpt: Using MemGPT to create a perpetual chatbot -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d --- > 📘 Confirm your installation -> +> > Before starting this example, make sure that you've [properly installed MemGPT](quickstart) In this example, we're going to use MemGPT to create a chatbot with a custom persona. MemGPT chatbots are "perpetual chatbots", meaning that they can be run indefinitely without any context length limitations. MemGPT chatbots are self-aware that they have a "fixed context window", and will manually manage their own memories to get around this problem by moving information in and out of their small memory window and larger external storage. @@ -15,6 +15,7 @@ MemGPT chatbots always keep a reserved space in their "core" memory window to st ### Creating a custom persona First, we'll create a text file with a short persona description. Let's make our chatbot a life coach named "Chaz". We'll also include a sentence at the top of the persona block to remind MemGPT that it should actively update its own persona over time. Open a text editor on your computer, and create a file called `chaz.txt`, and enter the following text: + ```text This is just the beginning of who I am. I should update my persona as I learn more about myself. @@ -27,15 +28,18 @@ I will help them achieve greatness! Huzzah! ``` Now that we've created a persona description inside `chaz.txt`, let's add this persona to MemGPT: + ```sh # --name specifies the profile name, -f specifies the file to load from memgpt add persona --name chaz -f chaz.txt ``` We can check that the persona is available: + ```sh memgpt list personas ``` + ```text ... | | | @@ -55,15 +59,18 @@ memgpt list personas Next, we'll create a custom user profile. To show you the different commands, we'll add the user profile by typing the text directly into the command line, instead of writing it into a file. Let's pretend I'm a software engineer named Bob Builder that works at a big tech company. Similar to the persona, we'll can register this user profile using `memgpt add human`, but this time, let's try registering the human profile directly with `--text`: + ```sh # Instead of using -f with a filename, we use --text and provide the text directly memgpt add human --name bob --text "Name: Bob Builder. Occupation: Software Engineer at a big tech company. Hobbies: running, hiking, rock climbing, craft beer, ultimate frisbee." ``` Now when we run `memgpt list human`, we should see "Bob Builder": + ```sh memgpt list humans ``` + ```text ... | | | @@ -79,6 +86,7 @@ Let's try out our new chatbot Chaz, combined with our new user profile Bob: # Alternatively we can run `memgpt configure`, then `memgpt run` without the --persona and --human flags memgpt run --persona chaz --human bob ``` + ```text 💭 First login detected. Prepare to introduce myself as Chaz, the AI life coach. Also, inquire about Bob's day and his expectations from our interaction. 🤖 Hello Bob! I'm Chaz, your AI life coach. I'm here to help you achieve your full potential! How was your day? And how may I assist you in becoming your best self? diff --git a/docs/example_data.md b/docs/example_data.md index 3121c7f6..71ea5527 100644 --- a/docs/example_data.md +++ b/docs/example_data.md @@ -1,11 +1,11 @@ --- title: Example - chat with your data excerpt: Using MemGPT to chat with your own data -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d --- > 📘 Confirm your installation -> +> > Before starting this example, make sure that you've [properly installed MemGPT](quickstart) In this example, we're going to use MemGPT to chat with a custom data source. Specifically, we'll try loading in the MemGPT research paper and ask MemGPT questions about it. @@ -15,15 +15,18 @@ In this example, we're going to use MemGPT to chat with a custom data source. Sp To feed external data into a MemGPT chatbot, we first need to create a data source. To download the MemGPT research paper we'll use `curl` (you can also just download the PDF from your browser): + ```sh # we're saving the file as "memgpt_research_paper.pdf" curl -L -o memgpt_research_paper.pdf https://arxiv.org/pdf/2310.08560.pdf ``` Now that we have the paper downloaded, we can create a MemGPT data source using `memgpt load`: + ```sh memgpt load directory --name memgpt_research_paper --input-files=memgpt_research_paper.pdf ``` + ```text loading data done loading data @@ -42,12 +45,14 @@ Note: you can ignore the "_LLM is explicitly disabled_" message. Now that we've created this data source, we can attach it to a MemGPT chatbot at any time. For the sake of this example, let's create a new chatbot using the `memgpt_doc` persona (but you can use any persona you want): + ```sh # reminder: `memgpt run --persona memgpt_doc` will create a new MemGPT agent using the `memgpt_doc` persona memgpt run --persona memgpt_doc ``` Once we're chatting with the agent, we can "attach" the data source to the agent's archival memory: + ```text Creating new agent... Created new agent agent_2. @@ -66,6 +71,7 @@ Attached data source memgpt_research_paper to agent agent_2, consisting of 130. ### Testing out our new chatbot Now that the data has been loaded into the chatbot's memory, we can start to ask questions about it: + ```text > Enter your message: The paper I loaded into your archival memory is called MemGPT. Can you tell me more about it? 💭 The search results show that the MemGPT paper explores operating-system-inspired techniques to enable large language models (LLMs) to manage memory and achieve unbounded context. The paper evaluates MemGPT in domains where LLMs are typically limited by finite context windows. This includes document analysis, allowing these models to process lengthy texts beyond their context limits, and conversational agents, where MemGPT allows for maintaining long-term memory, consistency, and evolution over extended dialogues. Time to relay this to Bob. diff --git a/docs/functions.md b/docs/functions.md index bb77a9d9..0c45a64b 100644 --- a/docs/functions.md +++ b/docs/functions.md @@ -1,12 +1,13 @@ --- title: Giving MemGPT more tools excerpt: Customize your MemGPT agents even further with your own functions -category: 6580daaa48aeca0038fc2297 +category: 6580daaa48aeca0038fc2297 --- If you would like to give MemGPT the ability to call new tools or functions, you can write a Python `.py` file with the functions you want to add, and place it inside of `~/.memgpt/functions`. You can see the example function sets provided [here](https://github.com/cpacker/MemGPT/tree/main/memgpt/functions/function_sets). As an example, we provide a preset called [`memgpt_extras`](https://github.com/cpacker/MemGPT/blob/main/memgpt/presets/examples/memgpt_extras.yaml) that includes additional functions to read and write from text files, as well as make HTTP requests: + ```yaml # this preset uses the same "memgpt_chat" system prompt, but has more functions enabled system_prompt: "memgpt_chat" @@ -37,18 +38,19 @@ There are three steps to adding more MemGPT functions: ### Simple example: giving MemGPT the ability to roll a D20 > ⚠️ Function requirements -> +> > The functions you write MUST have proper docstrings and type hints - this is because MemGPT will use these docstrings and types to automatically create a JSON schema that is used in the LLM prompt. Use the docstrings and types annotations from the [example functions](https://github.com/cpacker/MemGPT/blob/main/memgpt/functions/function_sets/base.py) for guidance. > ⚠️ Function output length > -> Your custom function should always return a string that is **capped in length**. If your string goes over the specified limit, it will be truncated internaly. This is to prevent potential context overflows caused by uncapped string returns (for example, a rogue HTTP request that returns a string larger than the LLM context window). +> Your custom function should always return a string that is **capped in length**. If your string goes over the specified limit, it will be truncated internally. This is to prevent potential context overflows caused by uncapped string returns (for example, a rogue HTTP request that returns a string larger than the LLM context window). > > If you return any type other than `str` (e.g. `dict``) in your custom functions, MemGPT will attempt to cast the result to a string (and truncate the result if it is too long). It is preferable to return strings - think of your function returning a natural language description of the outcome (see the D20 example below). In this simple example we'll give MemGPT the ability to roll a [D20 die](https://en.wikipedia.org/wiki/D20_System). First, let's create a python file `~/.memgpt/functions/d20.py`, and write some code that uses the `random` library to "roll a die": + ```python import random @@ -69,12 +71,13 @@ def roll_d20(self) -> str: """ dice_role_outcome = random.randint(1, 20) output_string = f"You rolled a {dice_role_outcome}" - return output_string + return output_string ``` Notice how we used [type hints](https://docs.python.org/3/library/typing.html) and [docstrings](https://peps.python.org/pep-0257/#multi-line-docstrings) to describe how the function works. **These are required**, if you do not include them MemGPT will not be able to "link" to your function. This is because MemGPT needs a JSON schema description of how your function works, which we automatically generate for you using the type hints and docstring (which you write yourself). Next, we'll create a custom preset that includes this new `roll_d20` function. Let's create a YAML file `~/.memgpt/presets/memgpt_d20.yaml`: + ```yaml system_prompt: "memgpt_chat" functions: @@ -86,7 +89,7 @@ functions: - "conversation_search_date" - "archival_memory_insert" - "archival_memory_search" - # roll a d20 + # roll a d20 - "roll_d20" ``` @@ -105,6 +108,7 @@ As we can see, MemGPT now has access to the `roll_d20` function! `roll_d20` is a _Example taken from [this pull request](https://github.com/cpacker/MemGPT/pull/282) by @cevatkerim_ As an example, if you wanted to give MemGPT the ability to make calls to Jira Cloud, you would write the function in Python (you would save this python file inside `~/.memgpt/functions/jira_cloud.py`): + ```python import os @@ -169,6 +173,7 @@ def run_jql(self, jql: str) -> dict: ``` Now we need to create a new preset file, let's create one called `~/.memgpt/presets/memgpt_jira.yaml`: + ```yaml # if we had created a new system prompt, we would replace "memgpt_chat" with the new prompt filename (no .txt) system_prompt: "memgpt_chat" @@ -187,9 +192,11 @@ functions: ``` Now when we run `memgpt configure`, we should see the option to use `memgpt_jira` as a preset: + ```sh memgpt configure ``` + ```text ... ? Select default preset: (Use arrow keys) diff --git a/docs/index.md b/docs/index.md index 985cd5d1..71644d0a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,7 @@ --- -title: Introduction +title: Introduction excerpt: Welcome to the MemGPT documentation! -category: 6580d34ee5e4d00068bf2a1d +category: 6580d34ee5e4d00068bf2a1d ---