feat: Migrate docs (#646)
* updated docs for readme * Update index.md * Update index.md * added header * broken link * sync heading sizes * fix various broken rel links * Update index.md * added webp * Update index.md * strip mkdocs/rtk files * replaced readthedocs references with readme
This commit is contained in:
Charles Packer
23
.github/workflows/rdme-docs.yml
vendored
Normal file
23
.github/workflows/rdme-docs.yml
vendored
Normal file
@@ -0,0 +1,23 @@
|
||||
# This GitHub Actions workflow was auto-generated by the `rdme` cli on 2023-12-18T23:15:45.852Z
|
||||
# You can view our full documentation here: https://docs.readme.com/docs/rdme
|
||||
name: ReadMe GitHub Action 🦉
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
# This workflow will run every time you push code to the following branch: `migrate-docs`
|
||||
# Check out GitHub's docs for more info on configuring this:
|
||||
# https://docs.github.com/actions/using-workflows/events-that-trigger-workflows
|
||||
- main
|
||||
|
||||
jobs:
|
||||
rdme-docs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out repo 📚
|
||||
uses: actions/checkout@v3
|
||||
|
||||
- name: Run `docs` command 🚀
|
||||
uses: readmeio/rdme@v8
|
||||
with:
|
||||
rdme: docs docs --key=${{ secrets.README_API_KEY }} --version=1.0
|
||||
@@ -1,19 +0,0 @@
|
||||
# Read the Docs configuration file for MkDocs projects
|
||||
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
|
||||
|
||||
# Required
|
||||
version: 2
|
||||
|
||||
# Set the version of Python and other tools you might need
|
||||
build:
|
||||
os: ubuntu-22.04
|
||||
tools:
|
||||
python: "3.12"
|
||||
|
||||
mkdocs:
|
||||
configuration: mkdocs.yml
|
||||
|
||||
# Optionally declare the Python requirements required to build your docs
|
||||
python:
|
||||
install:
|
||||
- requirements: docs/requirements.txt
|
||||
@@ -6,12 +6,12 @@
|
||||
|
||||
<strong>Try out our MemGPT chatbot on <a href="https://discord.gg/9GEQrxmVyE">Discord</a>!</strong>
|
||||
|
||||
<strong>⭐ NEW: You can now run MemGPT with <a href="https://memgpt.readthedocs.io/en/latest/local_llm/">open/local LLMs</a> and <a href="https://memgpt.readthedocs.io/en/latest/autogen/">AutoGen</a>! ⭐ </strong>
|
||||
<strong>⭐ NEW: You can now run MemGPT with <a href="https://memgpt.readme.io/docs/local_llm">open/local LLMs</a> and <a href="https://memgpt.readme.io/docs/autogen">AutoGen</a>! ⭐ </strong>
|
||||
|
||||
|
||||
[](https://discord.gg/9GEQrxmVyE)
|
||||
[](https://arxiv.org/abs/2310.08560)
|
||||
[](https://memgpt.readthedocs.io/en/latest/)
|
||||
[](https://memgpt.readme.io/docs)
|
||||
|
||||
</div>
|
||||
|
||||
@@ -96,7 +96,7 @@ You can run the following commands in the MemGPT CLI prompt:
|
||||
Once you exit the CLI with `/exit`, you can resume chatting with the same agent by specifying the agent name in `memgpt run --agent <NAME>`.
|
||||
|
||||
## Documentation
|
||||
See full documentation at: https://memgpt.readthedocs.io/
|
||||
See full documentation at: https://memgpt.readme.io
|
||||
|
||||
## Installing from source
|
||||
|
||||
|
||||
@@ -1,13 +0,0 @@
|
||||
# Building the docs
|
||||
|
||||
Run the following from the MemGPT directory.
|
||||
|
||||
1. Install requirements:
|
||||
```
|
||||
pip install -r docs/requirements.txt
|
||||
```
|
||||
|
||||
2. Serve docs:
|
||||
```
|
||||
mkdocs serve
|
||||
```
|
||||
@@ -1,6 +1,12 @@
|
||||
!!! warning "MemGPT + local LLM failure cases"
|
||||
---
|
||||
title: Adding support for new LLMs
|
||||
excerpt: Adding new LLMs via model wrappers
|
||||
category: 6580dabb585483000f0e6c7c
|
||||
---
|
||||
|
||||
When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
|
||||
> ⚠️ MemGPT + local LLM failure cases
|
||||
>
|
||||
> When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
|
||||
|
||||
### What is a "wrapper"?
|
||||
|
||||
|
||||
BIN
docs/assets/cozy_llama.webp
Normal file
BIN
docs/assets/cozy_llama.webp
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 191 KiB |
@@ -1,14 +1,20 @@
|
||||
!!! question "Need help?"
|
||||
---
|
||||
title: MemGPT + AutoGen
|
||||
excerpt: Creating AutoGen agents powered by MemGPT
|
||||
category: 6580dab16cade8003f996d17
|
||||
---
|
||||
|
||||
If you need help visit our [Discord server](https://discord.gg/9GEQrxmVyE) and post in the #support channel.
|
||||
|
||||
You can also check the [GitHub discussion page](https://github.com/cpacker/MemGPT/discussions/65), but the Discord server is the official support channel and is monitored more actively.
|
||||
> 📘 Need help?
|
||||
>
|
||||
> If you need help visit our [Discord server](https://discord.gg/9GEQrxmVyE) and post in the #support channel.
|
||||
>
|
||||
> You can also check the [GitHub discussion page](https://github.com/cpacker/MemGPT/discussions/65), but the Discord server is the official support channel and is monitored more actively.
|
||||
|
||||
!!! warning "Tested with `pyautogen` v0.2.0"
|
||||
|
||||
The MemGPT+AutoGen integration was last tested using AutoGen version v0.2.0.
|
||||
|
||||
If you are having issues, please first try installing the specific version of AutoGen using `pip install pyautogen==0.2.0` (or `poetry install -E autogen` if you are using Poetry).
|
||||
> ⚠️ Tested with `pyautogen` v0.2.0
|
||||
>
|
||||
> The MemGPT+AutoGen integration was last tested using AutoGen version v0.2.0.
|
||||
>
|
||||
> If you are having issues, please first try installing the specific version of AutoGen using `pip install pyautogen==0.2.0` (or `poetry install -E autogen` if you are using Poetry).
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -69,32 +75,31 @@ For the purposes of this example, we're going to serve (host) the LLMs using [oo
|
||||
|
||||
### Part 1: Get web UI working
|
||||
|
||||
Install web UI and get a model set up on a local web server. You can use [our instructions on setting up web UI](https://memgpt.readthedocs.io/en/latest/webui/).
|
||||
Install web UI and get a model set up on a local web server. You can use [our instructions on setting up web UI](webui).
|
||||
|
||||
!!! info "Choosing an LLM / model to use"
|
||||
|
||||
You'll need to decide on an LLM / model to use with web UI.
|
||||
|
||||
MemGPT requires an LLM that is good at function calling to work well - if the LLM is bad at function calling, **MemGPT will not work properly**.
|
||||
|
||||
Visit [our Discord server](https://discord.gg/9GEQrxmVyE) and check the #model-chat channel for an up-to-date list of recommended LLMs / models to use with MemGPT.
|
||||
> 📘 Choosing an LLM / model to use
|
||||
> You'll need to decide on an LLM / model to use with web UI.
|
||||
>
|
||||
> MemGPT requires an LLM that is good at function calling to work well - if the LLM is bad at function calling, **MemGPT will not work properly**.
|
||||
>
|
||||
> Visit [our Discord server](https://discord.gg/9GEQrxmVyE) and check the #model-chat channel for an up-to-date list of recommended LLMs / models to use with MemGPT.
|
||||
|
||||
### Part 2: Get MemGPT working
|
||||
|
||||
Before trying to integrate MemGPT with AutoGen, make sure that you can run MemGPT by itself with the web UI backend.
|
||||
|
||||
Try setting up MemGPT with your local web UI backend [using the instructions here](https://memgpt.readthedocs.io/en/latest/local_llm/#using-memgpt-with-local-llms).
|
||||
Try setting up MemGPT with your local web UI backend [using the instructions here](local_llm/#using-memgpt-with-local-llms).
|
||||
|
||||
Once you've confirmed that you're able to chat with a MemGPT agent using `memgpt configure` and `memgpt run`, you're ready to move on to the next step.
|
||||
|
||||
!!! info "Using RunPod as an LLM backend"
|
||||
|
||||
If you're using RunPod to run web UI, make sure that you set your endpoint to the RunPod IP address, **not the default localhost address**.
|
||||
|
||||
For example, during `memgpt configure`:
|
||||
```text
|
||||
? Enter default endpoint: https://yourpodaddresshere-5000.proxy.runpod.net
|
||||
```
|
||||
> 📘 Using RunPod as an LLM backend
|
||||
>
|
||||
> If you're using RunPod to run web UI, make sure that you set your endpoint to the RunPod IP address, **not the default localhost address**.
|
||||
>
|
||||
> For example, during `memgpt configure`:
|
||||
> ```text
|
||||
> ? Enter default endpoint: https://yourpodaddresshere-5000.proxy.runpod.net
|
||||
> ```
|
||||
|
||||
### Part 3: Creating a MemGPT AutoGen agent (groupchat example)
|
||||
|
||||
@@ -127,7 +132,7 @@ config_list = [
|
||||
config_list_memgpt = [
|
||||
{
|
||||
"preset": DEFAULT_PRESET,
|
||||
"model": None, # not required for web UI, only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
|
||||
"model": None, # not required for web UI, only required for Ollama, see: https://memgpt.readme.io/docs/ollama
|
||||
"model_wrapper": "airoboros-l2-70b-2.1", # airoboros is the default wrapper and should work for most models
|
||||
"model_endpoint_type": "webui",
|
||||
"model_endpoint": "http://localhost:5000", # notice port 5000 for web UI
|
||||
@@ -187,7 +192,7 @@ config_list_memgpt = [
|
||||
```
|
||||
|
||||
#### Azure OpenAI example
|
||||
Azure OpenAI API setup will be similar to OpenAI API, but requires additional config variables. First, make sure that you've set all the related Azure variables referenced in [our MemGPTAzure setup page](https://memgpt.readthedocs.io/en/latest/endpoints) (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_VERSION`, `AZURE_OPENAI_ENDPOINT`, etc). If you have all the variables set correctly, you should be able to create configs by pulling from the env variables:
|
||||
Azure OpenAI API setup will be similar to OpenAI API, but requires additional config variables. First, make sure that you've set all the related Azure variables referenced in [our MemGPT Azure setup page](https://memgpt.readme.io/docs/endpoints#azure-openai) (`AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_VERSION`, `AZURE_OPENAI_ENDPOINT`, etc). If you have all the variables set correctly, you should be able to create configs by pulling from the env variables:
|
||||
```python
|
||||
# This config is for autogen agents that are not powered by MemGPT
|
||||
# See Auto
|
||||
@@ -219,18 +224,19 @@ config_list_memgpt = [
|
||||
]
|
||||
```
|
||||
|
||||
!!! info "Making internal monologue visible to AutoGen"
|
||||
|
||||
By default, MemGPT's inner monologue and function traces are hidden from other AutoGen agents.
|
||||
|
||||
You can modify `interface_kwargs` to change the visibility of inner monologue and function calling:
|
||||
```python
|
||||
interface_kwargs = {
|
||||
"debug": False, # this is the equivalent of the --debug flag in the MemGPT CLI
|
||||
"show_inner_thoughts": True, # this controls if internal monlogue will show up in AutoGen MemGPT agent's outputs
|
||||
"show_function_outputs": True, # this controls if function traces will show up in AutoGen MemGPT agent's outputs
|
||||
}
|
||||
```
|
||||
> 📘 Making internal monologue visible to AutoGen
|
||||
>
|
||||
> By default, MemGPT's inner monologue and function traces are hidden from other AutoGen agents.
|
||||
>
|
||||
> You can modify `interface_kwargs` to change the visibility of inner monologue and function calling:
|
||||
> ```python
|
||||
> interface_kwargs = {
|
||||
> "debug": False, # this is the equivalent of the --debug flag in the MemGPT CLI
|
||||
> "show_inner_thoughts": True, # this controls if internal monlogue will show up in AutoGen MemGPT agent's outputs
|
||||
> "show_function_outputs": True, # this controls if function traces will show up in AutoGen MemGPT agent's outputs
|
||||
> }
|
||||
> ```
|
||||
|
||||
The only parts of the `agent_groupchat.py` file you need to modify should be the `config_list` and `config_list_memgpt` (make sure to change `USE_OPENAI` to `True` or `False` depending on if you're trying to use a local LLM server like web UI, or OpenAI's API). Assuming you edited things correctly, you should now be able to run `agent_groupchat.py`:
|
||||
```sh
|
||||
@@ -307,7 +313,7 @@ User_proxy (to chat_manager):
|
||||
|
||||
[examples/agent_docs.py](https://github.com/cpacker/MemGPT/blob/main/memgpt/autogen/examples/agent_docs.py) contains an example of a groupchat where the MemGPT autogen agent has access to documents.
|
||||
|
||||
First, follow the instructions in [Example - chat with your data - Creating an external data source](../example_data/#creating-an-external-data-source):
|
||||
First, follow the instructions in [Example - chat with your data - Creating an external data source](example_data/#creating-an-external-data-source):
|
||||
|
||||
To download the MemGPT research paper we'll use `curl` (you can also just download the PDF from your browser):
|
||||
```sh
|
||||
|
||||
@@ -1,14 +1,18 @@
|
||||
# Frequently asked questions
|
||||
---
|
||||
title: Frequently asked questions (FAQ)
|
||||
excerpt: Check frequently asked questions
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
!!! note "Open / local LLM FAQ"
|
||||
|
||||
Questions specific to running your own open / local LLMs with MemGPT can be found [here](../local_llm_faq).
|
||||
> 📘 Open / local LLM FAQ
|
||||
>
|
||||
> Questions specific to running your own open / local LLMs with MemGPT can be found [here](local_llm_faq).
|
||||
|
||||
## MemGPT CLI
|
||||
|
||||
### How can I use MemGPT to chat with my docs?
|
||||
|
||||
Check out our [chat with your docs example](../example_data) to get started.
|
||||
Check out our [chat with your docs example](example_data) to get started.
|
||||
|
||||
### How do I save a chat and continue it later?
|
||||
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
### Configuring the agent
|
||||
---
|
||||
title: Configuration
|
||||
excerpt: Configuring your MemGPT agent
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
You can set agent defaults by running `memgpt configure`, which will store config information at `~/.memgpt/config` by default.
|
||||
|
||||
The `memgpt run` command supports the following optional flags (if set, will override config defaults):
|
||||
@@ -43,3 +48,4 @@ memgpt list [humans/personas]
|
||||
```
|
||||
|
||||
### Custom Presets
|
||||
You can customize your MemGPT agent even further with [custom presets](presets) and [custom functions](functions).
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Contributing to the codebase
|
||||
excerpt: How to contribute to the MemGPT repo
|
||||
category: 6580dabb585483000f0e6c7c
|
||||
---
|
||||
|
||||
## Installing from source
|
||||
|
||||
To install MemGPT from source, start by cloning the repo:
|
||||
|
||||
@@ -1,4 +1,9 @@
|
||||
## Loading External Data
|
||||
---
|
||||
title: Attaching data sources
|
||||
excerpt: Connecting external data to your MemGPT agent
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
MemGPT supports pre-loading data into archival memory. In order to made data accessible to your agent, you must load data in with `memgpt load`, then attach the data source to your agent. You can configure where archival memory is stored by configuring the [storage backend](storage.md).
|
||||
|
||||
### Viewing available data sources
|
||||
@@ -34,11 +39,8 @@ memgpt attach --agent <AGENT-NAME> --data-source <DATA-SOURCE-NAME>
|
||||
memgpt-docs
|
||||
```
|
||||
|
||||
|
||||
!!! tip "Hint"
|
||||
To encourage your agent to reference its archival memory, we recommend adding phrases like "_search your archival memory..._" for the best results.
|
||||
|
||||
|
||||
> 👍 Hint
|
||||
> To encourage your agent to reference its archival memory, we recommend adding phrases like "_search your archival memory..._" for the best results.
|
||||
|
||||
### Loading a file or directory
|
||||
You can load a file, list of files, or directry into MemGPT with the following command:
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
## Chatting with the MemGPT Discord Bot
|
||||
---
|
||||
title: Chatting with MemGPT Bot
|
||||
excerpt: Get up and running with the MemGPT Discord Bot
|
||||
category: 6580da8eb6feb700166e5016
|
||||
---
|
||||
|
||||
The fastest way to experience MemGPT is to chat with the MemGPT Discord Bot.
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Configuring embedding backends
|
||||
excerpt: Connecting MemGPT to various endpoint backends
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
MemGPT uses embedding models for retrieval search over archival memory. You can use embeddings provided by OpenAI, Azure, or any model on Hugging Face.
|
||||
|
||||
## OpenAI
|
||||
@@ -47,11 +53,10 @@ MemGPT supports running embeddings with any Hugging Face model using the [Text E
|
||||
## Local Embeddings
|
||||
|
||||
MemGPT can compute embeddings locally using a lightweight embedding model [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5).
|
||||
!!! warning "Local LLM Performance"
|
||||
|
||||
The `BAAI/bge-small-en-v1.5` was chose to be lightweight, so you may notice degraded performance with embedding-based retrieval when using this option.
|
||||
|
||||
|
||||
> 🚧 Local LLM Performance
|
||||
>
|
||||
> The `BAAI/bge-small-en-v1.5` was chosen to be lightweight, so you may notice degraded performance with embedding-based retrieval when using this option.
|
||||
|
||||
To compute embeddings locally, install dependencies with:
|
||||
```
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Configuring LLM backends
|
||||
excerpt: Connecting MemGPT to various LLM backends
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
You can use MemGPT with various LLM backends, including the OpenAI API, Azure OpenAI, and various local (or self-hosted) LLM backends.
|
||||
|
||||
## OpenAI
|
||||
@@ -72,4 +78,4 @@ $ memgpt configure
|
||||
Note: **your Azure endpoint must support functions** or you will get an error. See [this GitHub issue](https://github.com/cpacker/MemGPT/issues/91) for more information.
|
||||
|
||||
## Local Models & Custom Endpoints
|
||||
MemGPT supports running open source models, both being run locally or as a hosted service. Setting up MemGPT to run with open models requires a bit more setup, follow [the instructions here](../local_llm).
|
||||
MemGPT supports running open source models, both being run locally or as a hosted service. Setting up MemGPT to run with open models requires a bit more setup, follow [the instructions here](local_llm).
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
!!! note "Note"
|
||||
---
|
||||
title: Example - perpetual chatbot
|
||||
excerpt: Using MemGPT to create a perpetual chatbot
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
Before starting this example, make sure that you've [properly installed MemGPT](../quickstart)
|
||||
> 📘 Confirm your installation
|
||||
>
|
||||
> Before starting this example, make sure that you've [properly installed MemGPT](quickstart)
|
||||
|
||||
## Using MemGPT to create a perpetual chatbot
|
||||
In this example, we're going to use MemGPT to create a chatbot with a custom persona. MemGPT chatbots are "perpetual chatbots", meaning that they can be run indefinitely without any context length limitations. MemGPT chatbots are self-aware that they have a "fixed context window", and will manually manage their own memories to get around this problem by moving information in and out of their small memory window and larger external storage.
|
||||
|
||||
MemGPT chatbots always keep a reserved space in their "core" memory window to store their `persona` information (describes the bot's personality + basic functionality), and `human` information (which describes the human that the bot is chatting with). The MemGPT chatbot will update the `persona` and `human` core memory blocks over time as it learns more about the user (and itself).
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
!!! note "Note"
|
||||
---
|
||||
title: Example - chat with your data
|
||||
excerpt: Using MemGPT to chat with your own data
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
Before starting this example, make sure that you've [properly installed MemGPT](../quickstart)
|
||||
> 📘 Confirm your installation
|
||||
>
|
||||
> Before starting this example, make sure that you've [properly installed MemGPT](quickstart)
|
||||
|
||||
## Using MemGPT to chat with your own data
|
||||
In this example, we're going to use MemGPT to chat with a custom data source. Specifically, we'll try loading in the MemGPT research paper and ask MemGPT questions about it.
|
||||
|
||||
### Creating an external data source
|
||||
@@ -69,4 +74,4 @@ Now that the data has been loaded into the chatbot's memory, we can start to ask
|
||||
|
||||
### Loading other data types
|
||||
|
||||
In this example, we loaded a single PDF into a chatbots external memory. However MemGPT supports various types of data, such as full directories of files and even databases - [see the full data sources list](../data_sources).
|
||||
In this example, we loaded a single PDF into a chatbots external memory. However MemGPT supports various types of data, such as full directories of files and even databases - [see the full data sources list](data_sources).
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Giving MemGPT more tools
|
||||
excerpt: Customize your MemGPT agents even further with your own functions
|
||||
category: 6580daaa48aeca0038fc2297
|
||||
---
|
||||
|
||||
If you would like to give MemGPT the ability to call new tools or functions, you can write a Python `.py` file with the functions you want to add, and place it inside of `~/.memgpt/functions`. You can see the example function sets provided [here](https://github.com/cpacker/MemGPT/tree/main/memgpt/functions/function_sets).
|
||||
|
||||
As an example, we provide a preset called [`memgpt_extras`](https://github.com/cpacker/MemGPT/blob/main/memgpt/presets/examples/memgpt_extras.yaml) that includes additional functions to read and write from text files, as well as make HTTP requests:
|
||||
@@ -30,15 +36,15 @@ There are three steps to adding more MemGPT functions:
|
||||
|
||||
### Simple example: giving MemGPT the ability to roll a D20
|
||||
|
||||
!!! warning "Function requirements"
|
||||
> ⚠️ Function requirements
|
||||
>
|
||||
> The functions you write MUST have proper docstrings and type hints - this is because MemGPT will use these docstrings and types to automatically create a JSON schema that is used in the LLM prompt. Use the docstrings and types annotations from the [example functions](https://github.com/cpacker/MemGPT/blob/main/memgpt/functions/function_sets/base.py) for guidance.
|
||||
|
||||
The functions you write MUST have proper docstrings and type hints - this is because MemGPT will use these docstrings and types to automatically create a JSON schema that is used in the LLM prompt. Use the docstrings and types annotations from the [example functions](https://github.com/cpacker/MemGPT/blob/main/memgpt/functions/function_sets/base.py) for guidance.
|
||||
|
||||
!!! warning "Function output length"
|
||||
|
||||
Your custom function should always return a string that is **capped in length**. If your string goes over the specified limit, it will be truncated internaly. This is to prevent potential context overflows caused by uncapped string returns (for example, a rogue HTTP request that returns a string larger than the LLM context window).
|
||||
|
||||
If you return any type other than `str` (e.g. `dict``) in your custom functions, MemGPT will attempt to cast the result to a string (and truncate the result if it is too long). It is preferable to return strings - think of your function returning a natural language description of the outcome (see the D20 example below).
|
||||
> ⚠️ Function output length
|
||||
>
|
||||
> Your custom function should always return a string that is **capped in length**. If your string goes over the specified limit, it will be truncated internaly. This is to prevent potential context overflows caused by uncapped string returns (for example, a rogue HTTP request that returns a string larger than the LLM context window).
|
||||
>
|
||||
> If you return any type other than `str` (e.g. `dict``) in your custom functions, MemGPT will attempt to cast the result to a string (and truncate the result if it is too long). It is preferable to return strings - think of your function returning a natural language description of the outcome (see the D20 example below).
|
||||
|
||||
In this simple example we'll give MemGPT the ability to roll a [D20 die](https://en.wikipedia.org/wiki/D20_System).
|
||||
|
||||
|
||||
@@ -1,4 +1,12 @@
|
||||
# 📚🦙 Welcome to MemGPT
|
||||
---
|
||||
title: Introduction
|
||||
excerpt: Welcome to the MemGPT documentation!
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
|
||||
|
||||
## What is MemGPT?
|
||||
|
||||
MemGPT enables LLMs to manage their own memory and overcome limited context windows!
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: koboldcpp
|
||||
excerpt: Setting up MemGPT with koboldcpp
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
1. Download + install [koboldcpp](https://github.com/LostRuins/koboldcpp/) and the model you want to test with
|
||||
2. In your terminal, run `./koboldcpp.py <MODEL> -contextsize <CONTEXT_LENGTH>`
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: llama.cpp
|
||||
excerpt: Setting up MemGPT with llama.cpp
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
1. Download + install [llama.cpp](https://github.com/ggerganov/llama.cpp) and the model you want to test with
|
||||
2. In your terminal, run `./server -m <MODEL> -c <CONTEXT_LENGTH>`
|
||||
|
||||
|
||||
@@ -1,16 +1,22 @@
|
||||
!!! warning "Important LM Studio settings"
|
||||
---
|
||||
title: LM Studio
|
||||
excerpt: Setting up MemGPT with LM Studio
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
**Context length**: Make sure that "context length" (`n_ctx`) is set (in "Model initialization" on the right hand side "Server Model Settings" panel) to the max context length of the model you're using (e.g. 8000 for Mistral 7B variants).
|
||||
> 📘 Update your LM Studio
|
||||
>
|
||||
> The current `lmstudio` backend will only work if your LM Studio is version 0.2.9 or newer.
|
||||
>
|
||||
> If you are on a version of LM Studio older than 0.2.9 (<= 0.2.8), select `lmstudio-legacy` as your backend type.
|
||||
|
||||
**Automatic Prompt Formatting = OFF**: If you see "Automatic Prompt Formatting" inside LM Studio's "Server Options" panel (on the left side), turn it **OFF**. Leaving it **ON** will break MemGPT.
|
||||
|
||||
**Context Overflow Policy = Stop at limit**: If you see "Context Overflow Policy" inside LM Studio's "Tools" panel on the right side (below "Server Model Settings"), set it to **Stop at limit**. The default setting "Keep the system prompt ... truncate middle" will break MemGPT.
|
||||
|
||||
!!! note "Update your LM Studio"
|
||||
|
||||
The current `lmstudio` backend will only work if your LM Studio is version 0.2.9 or newer.
|
||||
|
||||
If you are on a version of LM Studio older than 0.2.9 (<= 0.2.8), select `lmstudio-legacy` as your backend type.
|
||||
> ⚠️ Important LM Studio settings
|
||||
>
|
||||
> **Context length**: Make sure that "context length" (`n_ctx`) is set (in "Model initialization" on the right hand side "Server Model Settings" panel) to the max context length of the model you're using (e.g. 8000 for Mistral 7B variants).
|
||||
>
|
||||
> **Automatic Prompt Formatting = OFF**: If you see "Automatic Prompt Formatting" inside LM Studio's "Server Options" panel (on the left side), turn it **OFF**. Leaving it **ON** will break MemGPT.
|
||||
>
|
||||
> **Context Overflow Policy = Stop at limit**: If you see "Context Overflow Policy" inside LM Studio's "Tools" panel on the right side (below "Server Model Settings"), set it to **Stop at limit**. The default setting "Keep the system prompt ... truncate middle" will break MemGPT.
|
||||
|
||||
<img width="911" alt="image" src="https://github.com/cpacker/MemGPT/assets/5475622/d499e82e-348c-4468-9ea6-fd15a13eb7fa">
|
||||
|
||||
|
||||
@@ -1,14 +1,20 @@
|
||||
!!! question "Need help?"
|
||||
---
|
||||
title: MemGPT + open models
|
||||
excerpt: Set up MemGPT to run with open LLMs
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
If you need help visit our [Discord server](https://discord.gg/9GEQrxmVyE) and post in the #support channel.
|
||||
|
||||
You can also check the [GitHub discussion page](https://github.com/cpacker/MemGPT/discussions/67), but the Discord server is the official support channel and is monitored more actively.
|
||||
> 📘 Need help?
|
||||
>
|
||||
> If you need help visit our [Discord server](https://discord.gg/9GEQrxmVyE) and post in the #support channel.
|
||||
>
|
||||
> You can also check the [GitHub discussion page](https://github.com/cpacker/MemGPT/discussions/67), but the Discord server is the official support channel and is monitored more actively.
|
||||
|
||||
!!! warning "MemGPT + local LLM failure cases"
|
||||
|
||||
When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
|
||||
|
||||
Make sure to check the [local LLM troubleshooting page](../local_llm_faq) to see common issues before raising a new issue or posting on Discord.
|
||||
> ⚠️ MemGPT + open LLM failure cases
|
||||
>
|
||||
> When using open LLMs with MemGPT, **the main failure case will be your LLM outputting a string that cannot be understood by MemGPT**. MemGPT uses function calling to manage memory (eg `edit_core_memory(...)` and interact with the user (`send_message(...)`), so your LLM needs generate outputs that can be parsed into MemGPT function calls.
|
||||
>
|
||||
> Make sure to check the [local LLM troubleshooting page](local_llm_faq) to see common issues before raising a new issue or posting on Discord.
|
||||
|
||||
### Installing dependencies
|
||||
To install dependencies required for running local models, run:
|
||||
@@ -95,19 +101,19 @@ Note that even though grammar-based sampling can reduce the mistakes your LLM ma
|
||||
|
||||
Currently, MemGPT supports the following backends:
|
||||
|
||||
* [oobabooga web UI](../webui) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [LM Studio](../lmstudio) (Mac, Windows) (❌ does not support grammars)
|
||||
* [koboldcpp](../koboldcpp) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [llama.cpp](../llamacpp) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [vllm](../vllm) (Mac, Windows, Linux) (❌ does not support grammars)
|
||||
* [oobabooga web UI](webui) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [LM Studio](lmstudio) (Mac, Windows) (❌ does not support grammars)
|
||||
* [koboldcpp](koboldcpp) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [llama.cpp](llamacpp) (Mac, Windows, Linux) (✔️ supports grammars)
|
||||
* [vllm](vllm) (Mac, Windows, Linux) (❌ does not support grammars)
|
||||
|
||||
If you would like us to support a new backend, feel free to open an issue or pull request on [the MemGPT GitHub page](https://github.com/cpacker/MemGPT)!
|
||||
|
||||
### Which model should I use?
|
||||
|
||||
!!! info "Recommended LLMs / models"
|
||||
|
||||
To see a list of recommended LLMs to use with MemGPT, visit our [Discord server](https://discord.gg/9GEQrxmVyE) and check the #model-chat channel.
|
||||
> 📘 Recommended LLMs / models
|
||||
>
|
||||
> To see a list of recommended LLMs to use with MemGPT, visit our [Discord server](https://discord.gg/9GEQrxmVyE) and check the #model-chat channel.
|
||||
|
||||
If you are experimenting with MemGPT and local LLMs for the first time, we recommend you try the Dolphin Mistral finetune (e.g. [ehartford/dolphin-2.2.1-mistral-7b](https://huggingface.co/ehartford/dolphin-2.2.1-mistral-7b) or a quantized variant such as [dolphin-2.2.1-mistral-7b.Q6_K.gguf](https://huggingface.co/TheBloke/dolphin-2.2.1-mistral-7B-GGUF)), and use the default `airoboros` wrapper.
|
||||
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
### Problems getting MemGPT + local LLMs set up
|
||||
---
|
||||
title: Troubleshooting
|
||||
excerpt: FAQ for MemGPT + custom LLM backends
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
#### "Unable to connect to host ...", "API call got non-200 response code"
|
||||
## Problems getting MemGPT + local LLMs set up
|
||||
|
||||
### "Unable to connect to host ...", "API call got non-200 response code"
|
||||
|
||||
This error happens when MemGPT tries to run the LLM on the remote server you specified, but the server isn't working as expected.
|
||||
|
||||
@@ -11,23 +17,23 @@ Exception: API call got non-200 response code (code=400, msg={"error": {"message
|
||||
|
||||
Correcting the endpoint from `http://localhost:5001/v1` to `http://localhost:5001` (no `/v1` suffix) fixes the example error.
|
||||
|
||||
### Common errors while running MemGPT with local LLMs
|
||||
## Common errors while running MemGPT with local LLMs
|
||||
|
||||
#### "Warning: no wrapper specified for local LLM, using the default wrapper"
|
||||
### "Warning: no wrapper specified for local LLM, using the default wrapper"
|
||||
|
||||
**You can ignore this warning.**
|
||||
|
||||
This warning means that you did not specify a specific wrapper using the `--model-wrapper` flag, so MemGPT is using the default wrapper. If you would like to silence this warning, specify a wrapper with `--model-wrapper` or during `memgpt configure`.
|
||||
|
||||
#### "Failed to parse JSON from local LLM response"
|
||||
### "Failed to parse JSON from local LLM response"
|
||||
|
||||
This error occurs when the LLM you're using outputs a string that cannot be parsed into a MemGPT function call. This is basically an LLM output error - the LLM was not able to properly follow MemGPT instructions and generate a MemGPT-compatible function call string.
|
||||
|
||||
**You can reduce the frequency of these errors by using better models, and wrappers with grammar-based sampling**. For example, moving from a 2B model to a 70B model, or moving from a quantized model to the full unquantized version of the same model.
|
||||
|
||||
**If you use really small models (< 7B) or heavily quantized models (< Q5), you are likely to run into many MemGPT LLM output errors.** Try using the [recommended models first](../local_llm) before experimenting with your own custom models.
|
||||
**If you use really small models (< 7B) or heavily quantized models (< Q5), you are likely to run into many MemGPT LLM output errors.** Try using the [recommended models first](local_llm) before experimenting with your own custom models.
|
||||
|
||||
Many JSON-related output errors can be fixed by using a wrapper that uses grammars (required a grammar-enabled backend). See instructions about [grammars here](../local_llm).
|
||||
Many JSON-related output errors can be fixed by using a wrapper that uses grammars (required a grammar-enabled backend). See instructions about [grammars here](local_llm).
|
||||
|
||||
For example, let's look at the following error:
|
||||
```text
|
||||
@@ -62,6 +68,6 @@ This string is not correct JSON - it is missing closing brackets and has a stray
|
||||
}
|
||||
```
|
||||
|
||||
#### "Got back an empty response string from ..."
|
||||
### "Got back an empty response string from ..."
|
||||
|
||||
MemGPT asked the server to run the LLM, but got back an empty response. Double-check that your server is running properly and has context length set correctly (it should be set to 8k if using Mistral 7B models).
|
||||
|
||||
@@ -1,8 +1,14 @@
|
||||
!!! warning "Make sure to use tags when downloading Ollama models!"
|
||||
|
||||
Don't do **`ollama run dolphin2.2-mistral`**, instead do **`ollama run dolphin2.2-mistral:7b-q6_K`**.
|
||||
---
|
||||
title: Ollama
|
||||
excerpt: Setting up MemGPT with Ollama
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
If you don't specify a tag, Ollama may default to using a highly compressed model variant (e.g. Q4). We highly recommend **NOT** using a compression level below Q5 when using GGUF (stick to Q6 or Q8 if possible). In our testing, certain models start to become extremely unstable (when used with MemGPT) below Q6.
|
||||
> ⚠️ Make sure to use tags when downloading Ollama models!
|
||||
>
|
||||
> Don't do **`ollama run dolphin2.2-mistral`**, instead do **`ollama run dolphin2.2-mistral:7b-q6_K`**.
|
||||
>
|
||||
> If you don't specify a tag, Ollama may default to using a highly compressed model variant (e.g. Q4). We highly recommend **NOT** using a compression level below Q5 when using GGUF (stick to Q6 or Q8 if possible). In our testing, certain models start to become extremely unstable (when used with MemGPT) below Q6.
|
||||
|
||||
1. Download + install [Ollama](https://github.com/jmorganca/ollama) and the model you want to test with
|
||||
2. Download a model to test with by running `ollama run <MODEL_NAME>` in the terminal (check the [Ollama model library](https://ollama.ai/library) for available models)
|
||||
@@ -32,7 +38,7 @@ In your terminal where you're running MemGPT, run `memgpt configure` to set the
|
||||
? Select LLM inference provider: local
|
||||
? Select LLM backend (select 'openai' if you have an OpenAI compatible proxy): ollama
|
||||
? Enter default endpoint: http://localhost:11434
|
||||
? Enter default model name (required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama): dolphin2.2-mistral:7b-q6_K
|
||||
? Enter default model name (required for Ollama, see: https://memgpt.readme.io/docs/ollama): dolphin2.2-mistral:7b-q6_K
|
||||
...
|
||||
```
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Creating new MemGPT presets
|
||||
excerpt: Presets allow you to customize agent functionality
|
||||
category: 6580daaa48aeca0038fc2297
|
||||
---
|
||||
|
||||
MemGPT **presets** are a combination default settings including a system prompt and a function set. For example, the `memgpt_docs` preset uses a system prompt that is tuned for document analysis, while the default `memgpt_chat` is tuned for general chatting purposes.
|
||||
|
||||
You can create your own presets by creating a `.yaml` file in the `~/.memgpt/presets` directory. If you want to use a new custom system prompt in your preset, you can create a `.txt` file in the `~/.memgpt/system_prompts` directory.
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: Quickstart
|
||||
excerpt: Get up and running with MemGPT
|
||||
category: 6580d34ee5e4d00068bf2a1d
|
||||
---
|
||||
|
||||
### Installation
|
||||
|
||||
To install MemGPT, make sure you have Python installed on your computer, then run:
|
||||
@@ -9,7 +15,7 @@ pip install pymemgpt
|
||||
If you already have MemGPT installed, you can update to the latest version with:
|
||||
|
||||
```sh
|
||||
pip install pymemgpt -U --pre
|
||||
pip install pymemgpt -U
|
||||
```
|
||||
|
||||
### Running MemGPT using the OpenAI API
|
||||
@@ -32,8 +38,8 @@ memgpt run
|
||||
|
||||
In this example we use the OpenAI API, but you can run MemGPT with other backends! See:
|
||||
|
||||
* [Running MemGPT on OpenAI Azure and custom OpenAI endpoints](endpoints.md)
|
||||
* [Running MemGPT with your own LLMs (Llama 2, Mistral 7B, etc.)](local_llm.md)
|
||||
* [Running MemGPT on OpenAI Azure and custom OpenAI endpoints](endpoints)
|
||||
* [Running MemGPT with your own LLMs (Llama 2, Mistral 7B, etc.)](local_llm)
|
||||
|
||||
### Command-line arguments
|
||||
|
||||
@@ -73,5 +79,5 @@ Once you exit the CLI with `/exit`, you can resume chatting with the same agent
|
||||
|
||||
Check out the following tutorials on how to set up custom chatbots and chatbots for talking to your data:
|
||||
|
||||
* [Using MemGPT to create a perpetual chatbot](example_chat.md)
|
||||
* [Using MemGPT to chat with your own data](example_data.md)
|
||||
* [Using MemGPT to create a perpetual chatbot](example_chat)
|
||||
* [Using MemGPT to chat with your own data](example_data)
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# Configuring Storage Backends
|
||||
---
|
||||
title: Configuring storage backends
|
||||
excerpt: Customizing the MemGPT storage backend
|
||||
category: 6580dabb585483000f0e6c7c
|
||||
---
|
||||
|
||||
!!! warning "Switching storage backends"
|
||||
|
||||
MemGPT can only use one storage backend at a time. If you switch from local to database storage, you will need to re-load data and start agents from scratch. We currently do not support migrating between storage backends.
|
||||
> ⚠️ Switching storage backends
|
||||
>
|
||||
> MemGPT can only use one storage backend at a time. If you switch from local to database storage, you will need to re-load data and start agents from scratch. We currently do not support migrating between storage backends.
|
||||
|
||||
MemGPT supports both local and database storage for archival memory. You can configure which storage backend to use via `memgpt configure`. For larger datasets, we recommend using a database backend.
|
||||
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
---
|
||||
title: vLLM
|
||||
excerpt: Setting up MemGPT with vLLM
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
1. Download + install [vLLM](https://docs.vllm.ai/en/latest/getting_started/installation.html)
|
||||
2. Launch a vLLM **OpenAI-compatible** API server using [the official vLLM documentation](https://docs.vllm.ai/en/latest/getting_started/quickstart.html)
|
||||
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
!!! question "web UI troubleshooting"
|
||||
---
|
||||
title: oobobooga web UI
|
||||
excerpt: Setting up MemGPT with web UI
|
||||
category: 6580da9a40bb410016b8b0c3
|
||||
---
|
||||
|
||||
If you have problems getting web UI set up, please use the [official web UI repo for support](https://github.com/oobabooga/text-generation-webui)! There will be more answered questions about web UI there vs here on the MemGPT repo.
|
||||
> 📘 web UI troubleshooting
|
||||
>
|
||||
> If you have problems getting web UI set up, please use the [official web UI repo for support](https://github.com/oobabooga/text-generation-webui)! There will be more answered questions about web UI there vs here on the MemGPT repo.
|
||||
|
||||
To get MemGPT to work with a local LLM, you need to have the LLM running on a server that takes API requests.
|
||||
|
||||
|
||||
2
main.py
2
main.py
@@ -2,6 +2,6 @@ from memgpt.main import app
|
||||
import typer
|
||||
|
||||
typer.secho(
|
||||
"Command `python main.py` no longer supported. Please run `memgpt run`. See https://memgpt.readthedocs.io/en/latest/quickstart/.",
|
||||
"Command `python main.py` no longer supported. Please run `memgpt run`. See https://memgpt.readme.io/docs/quickstart.",
|
||||
fg=typer.colors.YELLOW,
|
||||
)
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# MemGPT + Autogen integration
|
||||
|
||||
See [https://memgpt.readthedocs.io/en/latest/autogen](https://memgpt.readthedocs.io/en/latest/autogen/) for documentation on integrating MemGPT with AutoGen.
|
||||
See [https://memgpt.readme.io/docs/autogen](https://memgpt.readme.io/docs/autogen) for documentation on integrating MemGPT with AutoGen.
|
||||
|
||||
@@ -58,7 +58,7 @@ elif LLM_BACKEND == "azure":
|
||||
azure_openai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
|
||||
assert (
|
||||
azure_openai_api_key is not None and azure_openai_version is not None and azure_openai_endpoint is not None
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readthedocs.io/en/latest/endpoints/#azure)"
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readme.io/docs/endpoints#azure-openai)"
|
||||
|
||||
# This config is for AutoGen agents that are not powered by MemGPT
|
||||
config_list = [
|
||||
@@ -108,7 +108,7 @@ elif LLM_BACKEND == "local":
|
||||
config_list_memgpt = [
|
||||
{
|
||||
"preset": DEFAULT_PRESET,
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readme.io/docs/ollama
|
||||
"context_window": 8192, # the context window of your model (for Mistral 7B-based models, it's likely 8192)
|
||||
"model_wrapper": "airoboros-l2-70b-2.1", # airoboros is the default wrapper and should work for most models
|
||||
"model_endpoint_type": "lmstudio", # can use webui, ollama, llamacpp, etc.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Example of how to add MemGPT into an AutoGen groupchat and chat with docs.
|
||||
|
||||
See https://memgpt.readthedocs.io/en/latest/autogen/#loading-documents
|
||||
See https://memgpt.readme.io/docs/autogen#part-4-attaching-documents-to-memgpt-autogen-agents
|
||||
|
||||
Based on the official AutoGen example here: https://github.com/microsoft/autogen/blob/main/notebook/agentchat_groupchat.ipynb
|
||||
|
||||
@@ -61,7 +61,7 @@ elif LLM_BACKEND == "azure":
|
||||
azure_openai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
|
||||
assert (
|
||||
azure_openai_api_key is not None and azure_openai_version is not None and azure_openai_endpoint is not None
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readthedocs.io/en/latest/endpoints/#azure)"
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readme.io/docs/endpoints#azure-openai)"
|
||||
|
||||
# This config is for AutoGen agents that are not powered by MemGPT
|
||||
config_list = [
|
||||
@@ -111,7 +111,7 @@ elif LLM_BACKEND == "local":
|
||||
config_list_memgpt = [
|
||||
{
|
||||
"preset": DEFAULT_PRESET,
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readme.io/docs/ollama
|
||||
"context_window": 8192, # the context window of your model (for Mistral 7B-based models, it's likely 8192)
|
||||
"model_wrapper": "airoboros-l2-70b-2.1", # airoboros is the default wrapper and should work for most models
|
||||
"model_endpoint_type": "lmstudio", # can use webui, ollama, llamacpp, etc.
|
||||
@@ -153,7 +153,7 @@ memgpt_agent = create_memgpt_autogen_agent_from_config(
|
||||
default_auto_reply="...", # Set a default auto-reply message here (non-empty auto-reply is required for LM Studio)
|
||||
skip_verify=False, # NOTE: you should set this to True if you expect your MemGPT AutoGen agent to call a function other than send_message on the first turn
|
||||
)
|
||||
# NOTE: you need to follow steps to load document first: see https://memgpt.readthedocs.io/en/latest/autogen/#loading-documents
|
||||
# NOTE: you need to follow steps to load document first: see https://memgpt.readme.io/docs/autogen#part-4-attaching-documents-to-memgpt-autogen-agents
|
||||
memgpt_agent.load_and_attach("memgpt_research_paper", "directory")
|
||||
|
||||
# Initialize the group chat between the agents
|
||||
|
||||
@@ -59,7 +59,7 @@ elif LLM_BACKEND == "azure":
|
||||
azure_openai_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
|
||||
assert (
|
||||
azure_openai_api_key is not None and azure_openai_version is not None and azure_openai_endpoint is not None
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readthedocs.io/en/latest/endpoints/#azure)"
|
||||
), "Set all the required OpenAI Azure variables (see: https://memgpt.readme.io/docs/endpoints#azure-openai)"
|
||||
|
||||
# This config is for AutoGen agents that are not powered by MemGPT
|
||||
config_list = [
|
||||
@@ -109,7 +109,7 @@ elif LLM_BACKEND == "local":
|
||||
config_list_memgpt = [
|
||||
{
|
||||
"preset": DEFAULT_PRESET,
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama/
|
||||
"model": None, # only required for Ollama, see: https://memgpt.readme.io/docs/ollama
|
||||
"context_window": 8192, # the context window of your model (for Mistral 7B-based models, it's likely 8192)
|
||||
"model_wrapper": "airoboros-l2-70b-2.1", # airoboros is the default wrapper and should work for most models
|
||||
"model_endpoint_type": "lmstudio", # can use webui, ollama, llamacpp, etc.
|
||||
|
||||
@@ -111,7 +111,7 @@ def configure_model(config: MemGPTConfig, model_endpoint_type: str):
|
||||
if model_endpoint_type == "ollama":
|
||||
default_model = config.model if config.model and config.model_endpoint_type == "ollama" else DEFAULT_OLLAMA_MODEL
|
||||
model = questionary.text(
|
||||
"Enter default model name (required for Ollama, see: https://memgpt.readthedocs.io/en/latest/ollama):",
|
||||
"Enter default model name (required for Ollama, see: https://memgpt.readme.io/docs/ollama):",
|
||||
default=default_model,
|
||||
).ask()
|
||||
model = None if len(model) == 0 else model
|
||||
@@ -310,12 +310,12 @@ def configure():
|
||||
print(f"Using deployment id {azure_creds['azure_deployment']}")
|
||||
else:
|
||||
raise ValueError(
|
||||
"Missing environment variables for Azure (see https://memgpt.readthedocs.io/en/latest/endpoints/#azure). Please set then run `memgpt configure` again."
|
||||
"Missing environment variables for Azure (see https://memgpt.readme.io/docs/endpoints#azure-openai). Please set then run `memgpt configure` again."
|
||||
)
|
||||
if model_endpoint_type == "openai" or embedding_endpoint_type == "openai":
|
||||
if not openai_key:
|
||||
raise ValueError(
|
||||
"Missing environment variables for OpenAI (see https://memgpt.readthedocs.io/en/latest/endpoints/#openai). Please set them and run `memgpt configure` again."
|
||||
"Missing environment variables for OpenAI (see https://memgpt.readme.io/docs/endpoints#azure-openai). Please set them and run `memgpt configure` again."
|
||||
)
|
||||
|
||||
config = MemGPTConfig(
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# MemGPT + local LLMs
|
||||
|
||||
See [https://memgpt.readthedocs.io/en/latest/local_llm](https://memgpt.readthedocs.io/en/latest/local_llm/) for documentation on running MemGPT with custom LLM backends.
|
||||
See [https://memgpt.readme.io/docs/local_llm](https://memgpt.readme.io/docs/local_llm) for documentation on running MemGPT with custom LLM backends.
|
||||
|
||||
100
mkdocs.yml
100
mkdocs.yml
@@ -1,100 +0,0 @@
|
||||
site_name: MemGPT
|
||||
site_url: https://memgpt.ai/
|
||||
repo_name: cpacker/MemGPT
|
||||
repo_url: https://github.com/cpacker/MemGPT
|
||||
# https://github.com/mkdocs/mkdocs/issues/2416
|
||||
edit_uri: ./edit/main/docs
|
||||
site_description: MemGPT documentation
|
||||
nav:
|
||||
- Home: index.md
|
||||
- 'User Guide':
|
||||
- 'Quickstart': quickstart.md
|
||||
- 'Example - perpetual chatbot': example_chat.md
|
||||
- 'Example - chat with your data': example_data.md
|
||||
- 'Configuration': config.md
|
||||
- 'External data sources': data_sources.md
|
||||
- 'Configuring LLMs': endpoints.md
|
||||
- 'Configuring embeddings': embedding_endpoints.md
|
||||
- 'FAQ': cli_faq.md
|
||||
- 'Discord Bot':
|
||||
- 'Chatting with MemGPT Bot': discord_bot.md
|
||||
- 'LLM Backends':
|
||||
- 'MemGPT + open models': local_llm.md
|
||||
- 'oobabooga web UI': webui.md
|
||||
# - 'oobabooga web UI (on RunPod)': webui_runpod.md
|
||||
- 'LM Studio': lmstudio.md
|
||||
- 'llama.cpp': llamacpp.md
|
||||
- 'KoboldCpp': koboldcpp.md
|
||||
- 'Ollama': ollama.md
|
||||
- 'vLLM': vllm.md
|
||||
- 'Troubleshooting': local_llm_faq.md
|
||||
- 'Customizing MemGPT':
|
||||
- 'Creating new MemGPT presets': presets.md
|
||||
- 'Giving MemGPT more tools': functions.md
|
||||
- 'Integrations':
|
||||
- 'MemGPT + AutoGen': autogen.md
|
||||
- 'Advanced':
|
||||
- 'Configuring storage backends': storage.md
|
||||
- 'Adding support for new LLMs': adding_wrappers.md
|
||||
- 'Contributing to the codebase': contributing.md
|
||||
theme:
|
||||
name: material
|
||||
features:
|
||||
- announce.dismiss
|
||||
- content.action.edit
|
||||
- content.action.view
|
||||
- content.code.annotate
|
||||
- content.code.copy
|
||||
# - content.code.select
|
||||
# - content.tabs.link
|
||||
- content.tooltips
|
||||
# - header.autohide
|
||||
# - navigation.expand
|
||||
- navigation.footer
|
||||
- navigation.indexes
|
||||
# - navigation.instant
|
||||
# - navigation.instant.prefetch
|
||||
# - navigation.instant.progress
|
||||
# - navigation.prune
|
||||
- navigation.sections
|
||||
# - navigation.path
|
||||
# - navigation.expansion
|
||||
# - navigation.tabs
|
||||
# - navigation.tabs.sticky
|
||||
# - navigation.top
|
||||
- navigation.tracking
|
||||
- search.highlight
|
||||
# - search.share
|
||||
# - search.suggest
|
||||
# - toc.follow
|
||||
- toc.integrate
|
||||
palette:
|
||||
- scheme: default
|
||||
primary: indigo
|
||||
accent: indigo
|
||||
toggle:
|
||||
icon: material/brightness-7
|
||||
name: Switch to dark mode
|
||||
- scheme: slate
|
||||
primary: black
|
||||
accent: indigo
|
||||
toggle:
|
||||
icon: material/brightness-4
|
||||
name: Switch to light mode
|
||||
font:
|
||||
text: Roboto
|
||||
code: Roboto Mono
|
||||
favicon: assets/favicon.ico
|
||||
logo: assets/memgpt_logo_circle.png
|
||||
# icon:
|
||||
markdown_extensions:
|
||||
- admonition
|
||||
- pymdownx.highlight:
|
||||
anchor_linenums: true
|
||||
line_spans: __span
|
||||
pygments_lang_class: true
|
||||
- pymdownx.inlinehilite
|
||||
- pymdownx.snippets
|
||||
- pymdownx.superfences
|
||||
plugins:
|
||||
- search
|
||||
Reference in New Issue
Block a user