Commit Graph

17 Commits

Author SHA1 Message Date
Charles Packer
e67c98eedb feat: add tests for prompt caching + fix anthropic prompt caching [LET-6373] (#6454)
* feat: add tests for prompt caching

* fix: add cache control breakpoints for anthropic + fix tests

* fix: silence logging

* fix: patch token counting error

* fix: same patch on non-streaming path
2025-12-15 12:02:19 -08:00
Charles Packer
4af6465226 feat(core+web): store raw usage data on streams (and visualize properly in ADE) (#6452)
* feat(core): store raw usage data on streams

* fix(web): various fixes to deal w/ hardcoding against openai
2025-12-15 12:02:19 -08:00
Charles Packer
88a3743cc8 fix(core): distinguish between null and 0 for prompt caching (#6451)
* fix(core): distinguish between null and 0 for prompt caching

* fix: runtime errors

* fix: just publish just sgate
2025-12-15 12:02:19 -08:00
Charles Packer
131891e05f feat: add tracking of advanced usage data (eg caching) [LET-6372] (#6449)
* feat: init refactor

* feat: add helper code

* fix: missing file + test

* fix: just state/publish api
2025-12-15 12:02:19 -08:00
jnjpng
c6df306ccf fix: upgrade google-genai sdk version and fix gemini 3 streaming (#6437)
* base

* base

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-12-15 12:02:18 -08:00
Ari Webb
30dab0abb9 fix: handle llm error during streaming [LET-6280] (#6341)
handle llm error during streaming

Co-authored-by: Ari Webb <ari@letta.com>
2025-11-24 19:10:27 -08:00
Charles Packer
18029250d0 fix(core): sanitize messages to anthropic in the main path the same way (or similar) to how we do it in the token counter (#6044)
* fix(core): sanitize messages to anthropic in the main path the same way (or similar) to how we do it in the token counter

* fix: also patch poison error in backend by filtering lazily

* fix: remap streaming errors (what the fuck)

* fix: dedupe tool clals

* fix: cleanup, removed try/catch
2025-11-13 15:36:55 -08:00
Matthew Zhou
ff81f4153b feat: Support parallel tool calling streaming for OpenAI chat completions [LET-4594] (#5865)
* Finish chat completions parallel tool calling

* Undo comments

* Add comments

* Remove test file
2025-11-13 15:36:14 -08:00
Ari Webb
48cc73175b feat: parallel tool calling for openai non streaming [LET-4593] (#5773)
* first hack

* clean up

* first implementation working

* revert package-lock

* remove openai test

* error throw

* typo

* Update integration_test_send_message_v2.py

* Update integration_test_send_message_v2.py

* refine test

* Only make changes for openai non streaming

* Add tests

---------

Co-authored-by: Ari Webb <ari@letta.com>
Co-authored-by: Matt Zhou <mattzh1314@gmail.com>
2025-11-13 15:36:14 -08:00
Matthew Zhou
bb8a7889e0 feat: Add parallel tool call streaming for anthropic [LET-4601] (#5225)
* wip

* Fix parallel tool calling interface

* wip

* wip adapt using id field

* Integrate new multi tool return schemas into parallel tool calling

* Remove example script

* Reset changes to llm stream adapter since old agent loop should not enable parallel tool calling

* Clean up fallback logic for extracting tool calls

* Remove redundant check

* Simplify logic

* Clean up logic in handle ai response

* Fix tests

* Write anthropic dict conversion to be back compatible

* wip

* Double write tool call id for legacy reasons

* Fix override args failures

* Patch for approvals

* Revert comments

* Remove extraneous prints
2025-10-24 15:11:31 -07:00
Matthew Zhou
7511b0f4fe feat: Write anthropic streaming interface that supports parallel tool calling [LET-5355] (#5295)
Write anthropic streaming interface that supports parallel tool calling
2025-10-09 15:25:21 -07:00
cthomas
1d611d92b9 feat: update assistant content parts union (#5115)
* feat: update assistant content parts union

* api sync

* just use the base object since updating assistant breaks frontend
2025-10-07 17:50:48 -07:00
cthomas
f7755d837a feat: add gemini streaming to new agent loop (#5109)
* feat: add gemini streaming to new agent loop

* add google as required dependency

* support storing all content parts

* remove extra google references
2025-10-07 17:50:48 -07:00
Sarah Wooders
ef07e03ee3 feat: add run_id to input messages and step_id to messages (#5099) 2025-10-07 17:50:48 -07:00
cthomas
67f8e46619 feat: add run id to streamed messages (#5037) 2025-10-07 17:50:47 -07:00
Matthew Zhou
d3c5d0c330 feat: Add missing import for SimpleOpenAIResponsesStreamingInterface (#5036)
Add missing import
2025-10-07 17:50:47 -07:00
cthomas
76d1bc8cbc feat: move new streaming adapters into own files (#5001) 2025-10-07 17:50:47 -07:00