Commit Graph

42 Commits

Author SHA1 Message Date
Sarah Wooders
221b4e6279 refactor: add extract_usage_statistics returning LettaUsageStatistics (#9065)
👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-29 12:44:04 -08:00
cthomas
c162de5127 fix: use shared event + .athrow() to properly set stream_was_cancelle… (#9019)
fix: use shared event + .athrow() to properly set stream_was_cancelled flag

**Problem:**
When a run is cancelled via /cancel endpoint, `stream_was_cancelled` remained
False because `RunCancelledException` was raised in the consumer code (wrapper),
which closes the generator from outside. This causes Python to skip the
generator's except blocks and jump directly to finally with the wrong flag value.

**Solution:**
1. Shared `asyncio.Event` registry for cross-layer cancellation signaling
2. `cancellation_aware_stream_wrapper` sets the event when cancellation detected
3. Wrapper uses `.athrow()` to inject exception INTO generator (not consumer-side raise)
4. All streaming interfaces check event in `finally` block to set flag correctly
5. `streaming_service.py` handles `RunCancelledException` gracefully, yields [DONE]

**Changes:**
- streaming_response.py: Event registry + .athrow() injection + graceful handling
- openai_streaming_interface.py: 3 classes check event in finally
- gemini_streaming_interface.py: Check event in finally
- anthropic_*.py: Catch RunCancelledException
- simple_llm_stream_adapter.py: Create & pass event to interfaces
- streaming_service.py: Handle RunCancelledException, yield [DONE], skip double-update
- routers/v1/{conversations,runs}.py: Pass event to wrapper
- integration_test_human_in_the_loop.py: New test for approval + cancellation

**Tests:**
- test_tool_call with cancellation (OpenAI models) 
- test_approve_with_cancellation (approval flow + concurrent cancel) 

**Known cosmetic warnings (pre-existing):**
- "Run already in terminal state" - agent loop tries to update after /cancel
- "Stream ended without terminal event" - background streaming timing race

👾 Generated with [Letta Code](https://letta.com)

Co-authored-by: Letta <noreply@letta.com>
2026-01-29 12:44:04 -08:00
Ari Webb
851798d71a fix: step_id is none (#8528) 2026-01-19 15:54:37 -08:00
Charles Packer
1f7165afc4 fix: patch counting of tokens for anthropic (#6458)
* fix: patch counting of tokens for anthropic

* fix: patch ui to be simpler

* fix: patch undercounting bug in anthropic when caching is on
2025-12-15 12:02:19 -08:00
Matthew Zhou
6f57ae829a fix: Reduce string growth for anthropic (#5974)
Reduce string growth for anthropic
2025-11-13 15:36:55 -08:00
Ari Webb
9d5fdc6de7 feat: migrate integration test mcp serverspy to use 1.0 client [LET-5945] (#5814)
* new test first hack, should still break

---------

Co-authored-by: Ari Webb <ari@letta.com>
2025-11-13 15:36:14 -08:00
cthomas
4823416af9 feat: default unpack assistant message content [LET-5404] (#5707)
feat: default unpack assistant message content
2025-10-24 15:14:20 -07:00
Kian Jones
c2e474e03a feat: refactor logs to parse as a single log line each and filter out 404s from sentry (#5242)
* add multiline log auto detect

* implement logger.exception()

* filter out 404

* remove potentially problematic changes
2025-10-24 15:11:31 -07:00
Matthew Zhou
5593f1450b feat: Double write to ToolCallMessage's new list tool_calls field (#5268)
* Add new tool_calls field to ToolCallMessage

* fern autogen

* Double write to new tool_calls field

* Update straggling instances
2025-10-09 13:20:52 -07:00
Sarah Wooders
ef07e03ee3 feat: add run_id to input messages and step_id to messages (#5099) 2025-10-07 17:50:48 -07:00
cthomas
67f8e46619 feat: add run id to streamed messages (#5037) 2025-10-07 17:50:47 -07:00
cthomas
f235dfb356 feat: add tool call test for new agent loop (#5034) 2025-10-07 17:50:47 -07:00
Charles Packer
a4041879a4 feat: add new agent loop (squash rebase of OSS PR) (#4815)
* feat: squash rebase of OSS PR

* fix: revert changes that weren't on manual rebase

* fix: caught another one

* fix: disable force

* chore: drop print

* fix: just stage-api && just publish-api

* fix: make agent_type consistently an arg in the client

* fix: patch multi-modal support

* chore: put in todo stub

* fix: disable hardcoding for tests

* fix: patch validate agent sync (#4882)

patch validate agent sync

* fix: strip bad merge diff

* fix: revert unrelated diff

* fix: react_v2 naming -> letta_v1 naming

* fix: strip bad merge

---------

Co-authored-by: Kevin Lin <klin5061@gmail.com>
2025-10-07 17:50:45 -07:00
Kian Jones
b8e9a80d93 merge this (#4759)
* wait I forgot to comit locally

* cp the entire core directory and then rm the .git subdir
2025-09-17 15:47:40 -07:00
Kian Jones
22f70ca07c chore: officially migrate to submodule (#4502)
* remove apps/core and apps/fern

* fix precommit

* add submodule updates in workflows

* submodule

* remove core tests

* update core revision

* Add submodules: true to all GitHub workflows

- Ensure all workflows can access git submodules
- Add submodules support to deployment, test, and CI workflows
- Fix YAML syntax issues in workflow files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove core-lint

* upgrade core with latest main of oss

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 12:45:53 -07:00
cthomas
cb7296c81d fix: approval request for streaming (#4445)
* fix: approval request for streaming

* fix: claude code attempt, unit test passing (add on to #4445) (#4448)

* fix: claude code attempt, unit test passing

* chore: update locks to 0.1.314 from 0.1.312

* chore: just stage-api && just publish-api

* chore: drop dead poetry lock

---------

Co-authored-by: Charles Packer <packercharles@gmail.com>
2025-09-05 17:43:21 -07:00
Charles Packer
e741f84add fix: patch bug w/ extended thinking mode involving text leaking into reasoning (#4341)
* fix: patch for bad native reasoning behavior w/ sonnet

* fix: cleanup

* fix: cleanup

* fix: another prompt tune for less flaking
2025-09-01 20:26:24 -07:00
cthomas
db41f01ac2 feat: continue stream processing on client cancel (#3796) 2025-08-07 13:17:36 -07:00
Andy Li
ca6f474c4e feat: track metrics for runs in db 2025-08-06 15:46:50 -07:00
cthomas
7d33254f5f feat: log stream cancellation to sentry (#3759) 2025-08-05 16:07:30 -07:00
Andy Li
33c1f26ab6 feat: support for agent loop job cancelation (#2837) 2025-07-02 14:31:16 -07:00
Sarah Wooders
5fa52a2c38 fix: avoid calling model_dump on stop reason messages twice (#2811) 2025-06-13 18:25:35 -07:00
cthomas
1405464a1c feat: send stop reason in letta APIs (#2789) 2025-06-13 16:04:48 -07:00
Andy Li
33bfd14017 fix: metric tracking (#2785) 2025-06-13 13:53:10 -07:00
cthomas
605a1f410c feat: consolidate logic for finish tokens (#2779) 2025-06-12 15:24:06 -07:00
Matthew Zhou
881506d574 fix: Turn off parallel tool calling for Claude (#2736) 2025-06-10 13:04:20 -07:00
Andy Li
d2252f2953 feat: otel metrics and expanded collecting (#2647)
(passed tests in last run)
2025-06-05 17:20:14 -07:00
cthomas
22c66da7bc fix: add temp hack to gracefully handle parallel tool calling (#2654) 2025-06-05 14:43:46 -07:00
cthomas
6d094fd196 fix: send message tests (#2656) 2025-06-05 13:57:43 -07:00
cthomas
da49024a5a fix: inner thoughts kwarg should never be streamed (#2644) 2025-06-05 12:30:55 -07:00
cthomas
904ccd65b6 fix: remove separate tool call id in streaming path (#2641) 2025-06-04 17:35:55 -07:00
cthomas
a8f394d675 feat: populate tool call name and id in when token streaming (#2639) 2025-06-04 17:06:44 -07:00
Matthew Zhou
7debadb3b9 fix: Change enum to fix composio tests (#2488) 2025-05-28 10:24:22 -07:00
Matthew Zhou
dba4cc9ea0 feat: Add TTFT latency from provider in traces (#2481) 2025-05-28 10:06:16 -07:00
cthomas
871e171b44 feat: add tracing to streaming interface (#2477) 2025-05-27 16:20:05 -07:00
cthomas
fe592eda72 feat: protect against anthropic nested tool args (#2250) 2025-05-19 16:01:59 -07:00
Andy Li
a78abc610e feat: track llm provider traces and tracking steps in async agent loop (#2219) 2025-05-19 15:50:56 -07:00
cthomas
5e025e29d8 feat: add otid to new async loop (#2185) 2025-05-14 15:28:53 -07:00
cthomas
a22d274e9c fix: anthropic streaming thinking tag (#2164) 2025-05-13 21:33:08 -07:00
Andy Li
3d94adbac3 fix: user messages on new agent loop are not processed in ADE (includes new json parser) (#1934) 2025-04-30 18:07:42 -07:00
Matthew Zhou
1d57f4c938 feat: Strip out thinking XML from persisted messages (#1596) 2025-04-07 10:39:45 -07:00
Matthew Zhou
038c40244f feat: Enable Anthropic streaming on new agent loop (#1550) 2025-04-03 19:40:48 -07:00