Commit Graph

497 Commits

Author SHA1 Message Date
Charles Packer
e5add4e430 Configurable presets to support easy extension of MemGPT's function set (#420)
* partial

* working schema builder, tested that it matches the hand-written schemas

* correct another schema diff

* refactor

* basic working test

* refactored preset creation to use yaml files

* added docstring-parser

* add code for dynamic function linking in agent loading

* pretty schema diff printer

* support pulling from ~/.memgpt/functions/*.py

* clean

* allow looking for system prompts in ~/.memgpt/system_prompts

* create ~/.memgpt/system_prompts if it doesn't exist

* pull presets from ~/.memgpt/presets in addition to examples folder

* add support for loading agent configs that have additional keys

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2023-11-13 10:43:28 -08:00
Sarah Wooders
dd5819893a fix config (#438) 2023-11-13 08:47:53 -08:00
Charles Packer
46a3fd9290 [version] bump release to 0.2.2 (#436) 2023-11-13 07:13:00 -08:00
Charles Packer
624650c13d patch #428 (#433) 2023-11-12 22:59:53 -08:00
Charles Packer
d6335f81cc patch (#435) 2023-11-12 22:59:41 -08:00
Sarah Wooders
ecdefc661b [fix] remove asserts for OPENAI_API_BASE (#432)
* mark depricated API section

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* CLI bug fixes for azure

* check azure before running

* Update README.md

* Update README.md

* bug fix with persona loading

* remove print

* make errors for cli flags more clear

* format

* fix imports

* fix imports

* add prints

* update lock

* remove asserts
2023-11-12 14:34:00 -08:00
Anjalee Sudasinghe
9a9d1e7937 fix memgptagent attach docs error (#427)
Co-authored-by: Anjalee Sudasinghe <anjalee@codegen.net>
2023-11-12 13:49:11 -08:00
Sarah Wooders
1837665625 [version] bump version to 0.2.1 (#417) 2023-11-10 13:14:58 -08:00
Sarah Wooders
c55835f507 add errors to make sure envs set correctly (#418) 2023-11-10 12:45:32 -08:00
Sarah Wooders
551952083b Fix model configuration for when config.model == "local" previously (#415)
* fix agent load

* fix model config
2023-11-10 12:16:33 -08:00
Charles Packer
7f950b05e8 Patch local LLMs with context_window (#416)
* patch

* patch ollama

* patch lmstudio

* patch kobold
2023-11-10 12:06:41 -08:00
Sarah Wooders
5647f8b63b fix agent load (#412) 2023-11-10 10:34:37 -08:00
Vivian Fang
05aeaf1927 Hotfix openai create all with context_window kwarg (#413) 2023-11-10 10:30:11 -08:00
Vivian Fang
f5ab162852 Fix main.yml to not rely on requirements.txt (#411) 2023-11-10 09:24:40 -08:00
Sarah Wooders
d6115d65d4 [version] bump version to 0.2.0 (#410) 2023-11-10 08:54:50 -08:00
Charles Packer
dab47001a9 Fix max tokens constant (#374)
* stripped LLM_MAX_TOKENS constant, instead it's a dictionary, and context_window is set via the config (defaults to 8k)

* pass context window in the calls to local llm APIs

* safety check

* remove dead imports

* context_length -> context_window

* add default for agent.load

* in configure, ask for the model context window if not specified via dictionary

* fix default, also make message about OPENAI_API_BASE missing more informative

* make openai default embedding if openai is default llm

* make openai on top of list

* typo

* also make local the default for embeddings if you're using localllm instead of the locallm endpoint

* provide --context_window flag to memgpt run

* fix runtime error

* stray comments

* stray comment
2023-11-09 17:59:03 -08:00
Hans Raaf
12f9bf29fd I added some json repairs that helped me with malformed messages (#341)
* I added some json repairs that helped me with malformed messages

There are two of them: The first will remove hard line feeds that appear
in the message part because the model added those instead of escaped
line feeds. This happens a lot in my experiments and that actually fixes
them.

The second one is less tested and should handle the case that the model
answers with multiple blocks of strings in quotes or even uses unescaped
quotes. It should grab everything betwenn the message: " and the ending
curly braces, escape them and makes it propper json that way.

Disclaimer: Both function were written with the help of ChatGPT-4 (I
can't write much Python). I think the first one is quite solid but doubt
that the second one is fully working. Maybe somebody with more Python
skills than me (or with more time) has a better idea for this type of
malformed replies.

* Moved the repair output behind the debug flag and removed the "clean" one

* Added even more fixes (out of what I just encountered while testing)

It seems that cut of json can be corrected and sometimes the model is to
lazy to add not just one curly brace but two. I think it does not "cost"
a lot to try them all out. But the expeptions get massive that way :)

* black

* for the final hail mary with extract_first_json, might as well add a double end bracket instead of single

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-09 17:05:42 -08:00
Vivian Fang
11326ec24e Remove AsyncAgent and async from cli (#400)
* Remove AsyncAgent and async from cli

Refactor agent.py memory.py

Refactor interface.py

Refactor main.py

Refactor openai_tools.py

Refactor cli/cli.py

stray asyncs

save

make legacy embeddings not use async

Refactor presets

Remove deleted function from import

* remove stray prints

* typo

* another stray print

* patch test

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-09 14:51:12 -08:00
Sarah Wooders
01adfaa4be Return empty list if archival memory search over empty local index (#402) 2023-11-09 14:49:23 -08:00
Bob Kerns
b17a8e89fb Dockerfile for running postgres locally (#393) 2023-11-09 14:26:53 -08:00
Sarah Wooders
ecad9a45ad Use ~/.memgpt/config to set questionary defaults in memgpt configure + update tests to use specific config path (#389) 2023-11-09 14:01:11 -08:00
Charles Packer
354bd520e0 softpass test when keys are missing (#369)
* softpass test when keys are missing

* update to use local model

* both openai and local

* typo

* fix

* Specify model inference and embedding endpoint separately  (#286)

* Fix config tests (#343)

Co-authored-by: Vivian Fang <hi@vivi.sh>

* Avoid throwing error for older `~/.memgpt/config` files due to missing section `archival_storage` (#344)

* avoid error if has old config type

* Dependency management  (#337)

* Divides dependencies into `pip install pymemgpt[legacy,local,postgres,dev]`. 
* Update docs

* Relax verify_first_message_correctness to accept any function call (#340)

* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access

* Update `poetry.lock` (#346)

* mark depricated API section

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* CLI bug fixes for azure

* check azure before running

* Update README.md

* Update README.md

* bug fix with persona loading

* remove print

* make errors for cli flags more clear

* format

* fix imports

* fix imports

* add prints

* update lock

* Add autogen example that lets you chat with docs (#342)

* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access

* Support attach in memgpt autogen agent

* Add docs example

* Add documentation, cleanup

* add gpt-4-turbo (#349)

* add gpt-4-turbo

* add in another place

* change to 3.5 16k

* Revert relaxing verify_first_message_correctness, still add archival_memory_search as an exception (#350)

* Revert "Relax verify_first_message_correctness to accept any function call (#340)"

This reverts commit 30e911057d755f5946d7bc2ba54619b5f2e08dc3.

* add archival_memory_search as an exception for verify

* Bump version to 0.1.18 (#351)

* Remove `requirements.txt` and `requirements_local.txt` (#358)

* update requirements to match poetry

* update with extras

* remove requirements

* disable pretty exceptions (#367)

* Updated documentation for users (#365)


---------

Co-authored-by: Vivian Fang <hi@vivi.sh>

* Create pull_request_template.md (#368)

* Create pull_request_template.md

* Add pymemgpt-nightly workflow (#373)

* Add pymemgpt-nightly workflow

* change token name

* Update lmstudio.md (#382)

* Update lmstudio.md

* Update lmstudio.md

* Update lmstudio.md to show the Prompt Formatting Option (#384)

* Update lmstudio.md to show the Prompt Formatting Option

* Update lmstudio.md Update the screenshot

* Swap asset location from #384 (#385)

* Update poetry with `pg8000` and include `pgvector` in docs  (#390)

* Allow overriding config location with `MEMGPT_CONFIG_PATH` (#383)

* Always default to local embeddings if not OpenAI or Azure  (#387)

* Add support for larger archival memory stores (#359)

* Replace `memgpt run` flags error with warning + remove custom embedding endpoint option + add agent create time (#364)

* Update webui.md (#397)

turn emoji warning into markdown warning

* Update webui.md (#398)

* dont hard code embeddings

* formatting

* black

* add full deps

* remove changes

* update poetry

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
Co-authored-by: Vivian Fang <hi@vivi.sh>
Co-authored-by: MSZ-MGS <65172063+MSZ-MGS@users.noreply.github.com>
2023-11-09 12:11:19 -08:00
Charles Packer
6d7a1652e6 Update webui.md (#398) 2023-11-09 09:46:51 -08:00
Charles Packer
1d69ca9f0d Update webui.md (#397)
turn emoji warning into markdown warning
2023-11-09 09:42:02 -08:00
Sarah Wooders
5464cb8ace Replace memgpt run flags error with warning + remove custom embedding endpoint option + add agent create time (#364) 2023-11-09 09:10:17 -08:00
Sarah Wooders
cef4d8489d Add support for larger archival memory stores (#359) 2023-11-09 09:09:57 -08:00
Sarah Wooders
974fa56ee7 Always default to local embeddings if not OpenAI or Azure (#387) 2023-11-08 21:35:48 -08:00
Sarah Wooders
f47247d4b4 Allow overriding config location with MEMGPT_CONFIG_PATH (#383) 2023-11-08 21:35:35 -08:00
Sarah Wooders
b068cb97b1 Update poetry with pg8000 and include pgvector in docs (#390) 2023-11-08 21:34:24 -08:00
Charles Packer
7121bc91f6 Swap asset location from #384 (#385) 2023-11-08 12:38:51 -08:00
MSZ-MGS
197d46c66a Update lmstudio.md to show the Prompt Formatting Option (#384)
* Update lmstudio.md to show the Prompt Formatting Option

* Update lmstudio.md Update the screenshot
2023-11-08 12:34:45 -08:00
Charles Packer
4fca38a56d Update lmstudio.md (#382)
* Update lmstudio.md

* Update lmstudio.md
2023-11-08 11:54:56 -08:00
Vivian Fang
1378e5ed5e Add pymemgpt-nightly workflow (#373)
* Add pymemgpt-nightly workflow

* change token name
2023-11-07 23:09:59 -08:00
Charles Packer
6768c1a4e1 Create pull_request_template.md (#368)
* Create pull_request_template.md
2023-11-07 17:52:45 -08:00
Charles Packer
29d67ba848 Updated documentation for users (#365)
---------

Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-07 17:30:14 -08:00
Charles Packer
267697e98e disable pretty exceptions (#367) 2023-11-07 17:27:31 -08:00
Sarah Wooders
a733203bce Remove requirements.txt and requirements_local.txt (#358)
* update requirements to match poetry

* update with extras

* remove requirements
2023-11-07 17:26:11 -08:00
Vivian Fang
a66d05ba9d Bump version to 0.1.18 (#351) 2023-11-06 22:07:18 -08:00
Vivian Fang
5b8812c822 Revert relaxing verify_first_message_correctness, still add archival_memory_search as an exception (#350)
* Revert "Relax verify_first_message_correctness to accept any function call (#340)"

This reverts commit 30e911057d755f5946d7bc2ba54619b5f2e08dc3.

* add archival_memory_search as an exception for verify
2023-11-06 22:00:40 -08:00
Charles Packer
a4e9240316 add gpt-4-turbo (#349)
* add gpt-4-turbo

* add in another place

* change to 3.5 16k
2023-11-06 21:53:49 -08:00
Vivian Fang
1942ecee76 Add autogen example that lets you chat with docs (#342)
* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access

* Support attach in memgpt autogen agent

* Add docs example

* Add documentation, cleanup
2023-11-06 21:06:13 -08:00
Sarah Wooders
4fed48f75c Update poetry.lock (#346)
* mark depricated API section

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* add readme

* CLI bug fixes for azure

* check azure before running

* Update README.md

* Update README.md

* bug fix with persona loading

* remove print

* make errors for cli flags more clear

* format

* fix imports

* fix imports

* add prints

* update lock
2023-11-06 21:02:56 -08:00
Vivian Fang
86f500d513 Relax verify_first_message_correctness to accept any function call (#340)
* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access
2023-11-06 20:53:40 -08:00
Sarah Wooders
fb29290dd4 Dependency management (#337)
* Divides dependencies into `pip install pymemgpt[legacy,local,postgres,dev]`. 
* Update docs
2023-11-06 19:45:44 -08:00
Sarah Wooders
3f61969cc2 Avoid throwing error for older ~/.memgpt/config files due to missing section archival_storage (#344)
* avoid error if has old config type
2023-11-06 18:49:49 -08:00
Sarah Wooders
e07a7ac1ef Fix config tests (#343)
Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-06 18:43:23 -08:00
Sarah Wooders
8700158222 Specify model inference and embedding endpoint separately (#286) 2023-11-06 17:19:45 -08:00
Mo Nuaimat
8adef204e6 Fixing some dict value checking for function_call (#249) 2023-11-06 15:44:51 -08:00
Vivian Fang
87fc69baae Better error message printing for function call failing (#291)
* Better error message printing for function call failing

* only one import traceback

* don't forward entire stack trace to memgpt
2023-11-06 15:32:58 -08:00
Vivian Fang
4edba17419 Better interface output for function calls (#296)
Co-authored-by: Charles Packer <packercharles@gmail.com>
2023-11-06 15:21:30 -08:00