Commit Graph

48 Commits

Author SHA1 Message Date
Sarah Wooders
77e72d18da Merge branch 'main' into cherry-pick-storage-refactor 2023-12-27 13:47:27 +04:00
Charles Packer
42e2a301eb fix: increase the func return char limit (#714)
* increase the funct return limit

* disable truncation for base search functions

* added stdout suppression to remove MockLLM warning
2023-12-27 01:33:30 -08:00
Sarah Wooders
3f8b9063a1 Run black formatter 2023-12-26 17:53:57 +04:00
Sarah Wooders
b64f3aca4b Support metadata table via storage connectors for data sources 2023-12-26 17:06:58 +04:00
Sarah Wooders
deb7f3c1e1 Enable loading local agents with refactored recall memory + update MemGPTConfig to default to sqlite + chroma 2023-12-26 17:06:56 +04:00
Sarah Wooders
8a3681297a Cleanup agent function linking 2023-12-26 17:05:26 +04:00
Sarah Wooders
223e196da8 Support recall and archival memory for postgres
working test
2023-12-26 17:05:24 +04:00
Charles Packer
4f23934e04 feat: Add new wrapper defaults (#656) 2023-12-21 17:05:38 +04:00
Charles Packer
27eb05e7e8 Add safeguard on tokens returned by functions (#576)
* swapping out hardcoded str for prefix (forgot to include in #569)

* add extra failout when the summarizer tries to run on a single message

* added function response validation code, currently will truncate responses based on character count

* added return type hints (functions/tools should either return strings or None)

* discuss function output length in custom function section

* made the truncation more informative
2023-12-13 21:57:50 -08:00
Charles Packer
0d8b95e2a7 AutoGen misc fixes (#603)
* don't add anything except for assistant messages to the global autogen message historoy

* properly format autogen messages when using local llms (allow naming to get passed through to the prompt formatter)

* add extra handling of autogen's name field in step()

* comments
2023-12-10 20:52:21 -08:00
Charles Packer
f2a2942c66 sort agents by directory-last-modified time (#574)
* sort agents by directory-last-modified time

* only save agent config when agent is saved

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2023-12-04 14:16:03 -08:00
Claudio Cambra
14fbfa1406 Fix crash from bad key access into response_message without function_call (#437)
Signed-off-by: Claudio Cambra <developer@claudiocambra.com>
2023-12-04 13:58:36 -08:00
Charles Packer
df999de4c1 use a consistent warning prefix across codebase (#569) 2023-12-04 11:38:51 -08:00
Sarah Wooders
2d23370930 Removing dead code + legacy commands (#536) 2023-11-30 13:37:11 -08:00
Charles Packer
5e7676e133 Remove openai package and migrate to requests (#534) 2023-11-30 13:00:13 -08:00
Sarah Wooders
b05b09439f Add user field for vLLM endpoint (#531) 2023-11-29 12:30:42 -08:00
Charles Packer
f4a1455c24 Clean memory error messages (#523)
* Raise a custom keyerror instead of basic keyerror to clarify issue to LLM processor

* remove self value from error message passed to LLM processor

* simplify error message propogated to llm processor
2023-11-27 16:41:42 -08:00
Charles Packer
188236b457 add core memory char limits to text shown in core memory (#508)
* add core memory char limits to text shown in core memory

* include char limit in xml tag

* add flag to allow reverting to old version
2023-11-26 20:00:01 -08:00
Charles Packer
9989fd9a52 Fix #487 (summarize call uses OpenAI even with local LLM config) (#488)
* use new chatcompletion function that takes agent config inside of summarize

* patch issue with model now missing
2023-11-19 14:54:12 -08:00
cpacker
88cc33244a patch bad merge 2023-11-14 16:09:13 -08:00
Sarah Wooders
ec2bda4966 Refactor config + determine LLM via config.model_endpoint_type (#422)
* mark depricated API section

* CLI bug fixes for azure

* check azure before running

* Update README.md

* Update README.md

* bug fix with persona loading

* remove print

* make errors for cli flags more clear

* format

* fix imports

* fix imports

* add prints

* update lock

* update config fields

* cleanup config loading

* commit

* remove asserts

* refactor configure

* put into different functions

* add embedding default

* pass in config

* fixes

* allow overriding openai embedding endpoint

* black

* trying to patch tests (some circular import errors)

* update flags and docs

* patched support for local llms using endpoint and endpoint type passed via configs, not env vars

* missing files

* fix naming

* fix import

* fix two runtime errors

* patch ollama typo, move ollama model question pre-wrapper, modify question phrasing to include link to readthedocs, also have a default ollama model that has a tag included

* disable debug messages

* made error message for failed load more informative

* don't print dynamic linking function warning unless --debug

* updated tests to work with new cli workflow (disabled openai config test for now)

* added skips for tests when vars are missing

* update bad arg

* revise test to soft pass on empty string too

* don't run configure twice

* extend timeout (try to pass against nltk download)

* update defaults

* typo with endpoint type default

* patch runtime errors for when model is None

* catching another case of 'x in model' when model is None (preemptively)

* allow overrides to local llm related config params

* made model wrapper selection from a list vs raw input

* update test for select instead of input

* Fixed bug in endpoint when using local->openai selection, also added validation loop to manual endpoint entry

* updated error messages to be more informative with links to readthedocs

* add back gpt3.5-turbo

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-14 15:58:19 -08:00
Charles Packer
442a0ca8bf always cast config.context_window to int before use (#444)
* always cast config.context_window to int before use

* extra code to be super safe if self.config.context_window is somehow None
2023-11-14 15:12:00 -08:00
Charles Packer
e5add4e430 Configurable presets to support easy extension of MemGPT's function set (#420)
* partial

* working schema builder, tested that it matches the hand-written schemas

* correct another schema diff

* refactor

* basic working test

* refactored preset creation to use yaml files

* added docstring-parser

* add code for dynamic function linking in agent loading

* pretty schema diff printer

* support pulling from ~/.memgpt/functions/*.py

* clean

* allow looking for system prompts in ~/.memgpt/system_prompts

* create ~/.memgpt/system_prompts if it doesn't exist

* pull presets from ~/.memgpt/presets in addition to examples folder

* add support for loading agent configs that have additional keys

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2023-11-13 10:43:28 -08:00
Charles Packer
dab47001a9 Fix max tokens constant (#374)
* stripped LLM_MAX_TOKENS constant, instead it's a dictionary, and context_window is set via the config (defaults to 8k)

* pass context window in the calls to local llm APIs

* safety check

* remove dead imports

* context_length -> context_window

* add default for agent.load

* in configure, ask for the model context window if not specified via dictionary

* fix default, also make message about OPENAI_API_BASE missing more informative

* make openai default embedding if openai is default llm

* make openai on top of list

* typo

* also make local the default for embeddings if you're using localllm instead of the locallm endpoint

* provide --context_window flag to memgpt run

* fix runtime error

* stray comments

* stray comment
2023-11-09 17:59:03 -08:00
Vivian Fang
11326ec24e Remove AsyncAgent and async from cli (#400)
* Remove AsyncAgent and async from cli

Refactor agent.py memory.py

Refactor interface.py

Refactor main.py

Refactor openai_tools.py

Refactor cli/cli.py

stray asyncs

save

make legacy embeddings not use async

Refactor presets

Remove deleted function from import

* remove stray prints

* typo

* another stray print

* patch test

---------

Co-authored-by: cpacker <packercharles@gmail.com>
2023-11-09 14:51:12 -08:00
Sarah Wooders
cef4d8489d Add support for larger archival memory stores (#359) 2023-11-09 09:09:57 -08:00
Vivian Fang
5b8812c822 Revert relaxing verify_first_message_correctness, still add archival_memory_search as an exception (#350)
* Revert "Relax verify_first_message_correctness to accept any function call (#340)"

This reverts commit 30e911057d755f5946d7bc2ba54619b5f2e08dc3.

* add archival_memory_search as an exception for verify
2023-11-06 22:00:40 -08:00
Vivian Fang
86f500d513 Relax verify_first_message_correctness to accept any function call (#340)
* Relax verify_first_message_correctness to accept any function call

* Also allow missing internal monologue if request_heartbeat

* Cleanup

* get instead of raw dict access
2023-11-06 20:53:40 -08:00
Vivian Fang
87fc69baae Better error message printing for function call failing (#291)
* Better error message printing for function call failing

* only one import traceback

* don't forward entire stack trace to memgpt
2023-11-06 15:32:58 -08:00
Vivian Fang
4edba17419 Better interface output for function calls (#296)
Co-authored-by: Charles Packer <packercharles@gmail.com>
2023-11-06 15:21:30 -08:00
Charles Packer
caba2f468c Create docs pages (#328)
* Create docs  (#323)

* Create .readthedocs.yaml

* Update mkdocs.yml

* update

* revise

* syntax

* syntax

* syntax

* syntax

* revise

* revise

* spacing

* Docs (#327)

* add stuff

* patch homepage

* more docs

* updated

* updated

* refresh

* refresh

* refresh

* update

* refresh

* refresh

* refresh

* refresh

* missing file

* refresh

* refresh

* refresh

* refresh

* fix black

* refresh

* refresh

* refresh

* refresh

* add readme for just the docs

* Update README.md

* add more data loading docs

* cleanup data sources

* refresh

* revised

* add search

* make prettier

* revised

* updated

* refresh

* favi

* updated

---------

Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2023-11-06 12:38:49 -08:00
Charles Packer
cc1ce0ce33 Remove embeddings as argument in archival_memory.insert (#284) 2023-11-05 12:48:22 -08:00
Sarah Wooders
b9ce763fda VectorDB support (pgvector) for archival memory (#226) 2023-11-03 16:19:15 -07:00
Charles Packer
25dd225d04 strip '/' and use osp.join (Windows support) (#283)
* strip '/' and use osp.join

* grepped for MEMGPT_DIR, found more places to replace '/'

* typo

* grep pass over filesep

---------

Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-03 13:54:29 -07:00
Charles Packer
fde0087a19 Patch summarize when running with local llms (#213)
* trying to patch summarize when running with local llms

* moved token magic numbers to constants, made special localllm exception class (TODO catch these for retry), fix summarize bug where it exits early if empty list

* missing file

* raise an exception on no-op summary

* changed summarization logic to walk forwards in list until fraction of tokens in buffer is reached

* added same diff to sync agent

* reverted default max tokens to 8k, cleanup + more error wrapping for better error messages that get caught on retry

* patch for web UI context limit error propogation, using best guess for what the web UI error message is

* add webui token length exception

* remove print

* make no wrapper warning only pop up once

* cleanup

* Add errors to other wrappers

---------

Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-02 23:44:02 -07:00
Charles Packer
f889f24643 Allow MemGPT to read/write text files + make HTTP requests (#174)
* added file read/write

* added HTTP requests

* black on utils.py
2023-11-01 01:08:44 -07:00
Sarah Wooders
23f3d42fae Refactoring CLI to use config file, connect to Llama Index data sources, and allow for multiple agents (#154)
* Migrate to `memgpt run` and `memgpt configure` 
* Add Llama index data sources via `memgpt load` 
* Save config files for defaults and agents
2023-10-30 16:47:54 -07:00
Vivian Fang
e7caf22f49 Fix summarizer bug in async agent (#195) 2023-10-29 22:43:17 -07:00
tractorjuice
fcbf21419f Update agent.py (#185)
Fix missing bracket in construct_system_with_memory
2023-10-29 21:46:23 -07:00
Charles Packer
a9a6a2629a hotfix for broken sync agent code (#171) 2023-10-28 20:15:55 -07:00
Vivian Fang
53cacad075 Add synchronous memgpt agent (#156) 2023-10-27 16:48:14 -07:00
Sarah Wooders
5c44790ad0 add black to poetry and reformat 2023-10-26 15:33:50 -07:00
cpacker
f9c689db2a address review 2023-10-20 14:17:18 -07:00
Charles Packer
3c3a5ae0e2 Merge branch 'main' into gpt35-patch 2023-10-20 14:05:39 -07:00
cpacker
dff7a444aa gpt35 adjustments 2023-10-16 20:48:24 -07:00
Vivian Fang
86d52c4cdf fix summarizer 2023-10-15 21:07:45 -07:00
Vivian Fang
15540c24ac fix paging bug, implement llamaindex api search on top of memgpt 2023-10-15 16:45:41 -07:00
Charles Packer
257c3998f7 init commit 2023-10-12 18:48:58 -07:00