Commit Graph

12 Commits

Author SHA1 Message Date
Charles Packer
8a7a64c7f9 patch web UI (#484)
* patch web UI

* set truncation_length
2023-11-19 14:56:10 -08:00
Charles Packer
576795ffdb move webui to new openai completions endpoint, but also provide existing functionality via webui-legacy backend (#468) 2023-11-15 23:08:30 -08:00
Charles Packer
7f950b05e8 Patch local LLMs with context_window (#416)
* patch

* patch ollama

* patch lmstudio

* patch kobold
2023-11-10 12:06:41 -08:00
Charles Packer
6b4008c72e more stop tokens (#288) 2023-11-03 12:25:37 -07:00
Charles Packer
437306388f Improvements to JSON handling for local LLMs (#269)
* some extra json hacks

* add 'smart' json loader to other wrapers

* added chatml related stop tokens by default
2023-11-03 00:18:31 -07:00
Charles Packer
fde0087a19 Patch summarize when running with local llms (#213)
* trying to patch summarize when running with local llms

* moved token magic numbers to constants, made special localllm exception class (TODO catch these for retry), fix summarize bug where it exits early if empty list

* missing file

* raise an exception on no-op summary

* changed summarization logic to walk forwards in list until fraction of tokens in buffer is reached

* added same diff to sync agent

* reverted default max tokens to 8k, cleanup + more error wrapping for better error messages that get caught on retry

* patch for web UI context limit error propogation, using best guess for what the web UI error message is

* add webui token length exception

* remove print

* make no wrapper warning only pop up once

* cleanup

* Add errors to other wrappers

---------

Co-authored-by: Vivian Fang <hi@vivi.sh>
2023-11-02 23:44:02 -07:00
cpacker
205226ea5e bump trunc length assuming mistral 2023-11-01 02:11:45 -07:00
Web 3 Wes
e9e5d341a6 update max new tokens (#182)
Co-authored-by: Clarence Jones <wes@Clarences-MacBook-Pro.local>
2023-10-31 12:16:11 -07:00
Charles Packer
5854307dde missing stop token 2023-10-25 01:10:15 -07:00
Vivian Fang
8c3409cf02 Revert "cleanup"
This reverts commit 85d9fba811f237fc0c625e920d4ee5995a9308f6, reversing
changes made to a7e06d0acc1b69b311fb16e386c4867337fe76f8.
2023-10-25 01:02:43 -07:00
cpacker
4d667de8f4 hotfix for airoboros prompt formatting template (was missing stop string) 2023-10-24 16:50:27 -07:00
Charles Packer
faaa9a04fa refactored + updated the airo wrapper a bit 2023-10-23 00:41:10 -07:00