Commit Graph

19 Commits

Author SHA1 Message Date
Kian Jones
f5c4ab50f4 chore: add ty + pre-commit hook and repeal even more ruff rules (#9504)
* auto fixes

* auto fix pt2 and transitive deps and undefined var checking locals()

* manual fixes (ignored or letta-code fixed)

* fix circular import

* remove all ignores, add FastAPI rules and Ruff rules

* add ty and precommit

* ruff stuff

* ty check fixes

* ty check fixes pt 2

* error on invalid
2026-02-24 10:55:11 -08:00
Kian Jones
25d54dd896 chore: enable F821, F401, W293 (#9503)
* auto fixes

* auto fix pt2 and transitive deps and undefined var checking locals()

* manual fixes (ignored or letta-code fixed)

* fix circular import
2026-02-24 10:55:08 -08:00
Kian Jones
4126fdadea fix(core): preserve thought_signature on TextContent in Gemini streaming path (#9461)
get_content() was only setting signature on ReasoningContent items.
When Gemini returns a function call with thought_signature but no
ReasoningContent (e.g. include_thoughts=False), the signature was
stored on self.thinking_signature but never attached to TextContent.
This caused "missing thought_signature in functionCall parts" errors
when the message was echoed back to Gemini on the next turn.

🐾 Generated with [Letta Code](https://letta.com)

Co-authored-by: Letta <noreply@letta.com>
2026-02-24 10:52:07 -08:00
Sarah Wooders
221b4e6279 refactor: add extract_usage_statistics returning LettaUsageStatistics (#9065)
👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-29 12:44:04 -08:00
cthomas
c162de5127 fix: use shared event + .athrow() to properly set stream_was_cancelle… (#9019)
fix: use shared event + .athrow() to properly set stream_was_cancelled flag

**Problem:**
When a run is cancelled via /cancel endpoint, `stream_was_cancelled` remained
False because `RunCancelledException` was raised in the consumer code (wrapper),
which closes the generator from outside. This causes Python to skip the
generator's except blocks and jump directly to finally with the wrong flag value.

**Solution:**
1. Shared `asyncio.Event` registry for cross-layer cancellation signaling
2. `cancellation_aware_stream_wrapper` sets the event when cancellation detected
3. Wrapper uses `.athrow()` to inject exception INTO generator (not consumer-side raise)
4. All streaming interfaces check event in `finally` block to set flag correctly
5. `streaming_service.py` handles `RunCancelledException` gracefully, yields [DONE]

**Changes:**
- streaming_response.py: Event registry + .athrow() injection + graceful handling
- openai_streaming_interface.py: 3 classes check event in finally
- gemini_streaming_interface.py: Check event in finally
- anthropic_*.py: Catch RunCancelledException
- simple_llm_stream_adapter.py: Create & pass event to interfaces
- streaming_service.py: Handle RunCancelledException, yield [DONE], skip double-update
- routers/v1/{conversations,runs}.py: Pass event to wrapper
- integration_test_human_in_the_loop.py: New test for approval + cancellation

**Tests:**
- test_tool_call with cancellation (OpenAI models) 
- test_approve_with_cancellation (approval flow + concurrent cancel) 

**Known cosmetic warnings (pre-existing):**
- "Run already in terminal state" - agent loop tries to update after /cancel
- "Stream ended without terminal event" - background streaming timing race

👾 Generated with [Letta Code](https://letta.com)

Co-authored-by: Letta <noreply@letta.com>
2026-01-29 12:44:04 -08:00
Kian Jones
82e5d70807 fix: prevent empty reasoning messages in streaming interfaces (#7207)
* fix: prevent empty reasoning messages in streaming interfaces

Prevents empty "Thinking..." indicators from appearing in clients by
filtering out reasoning messages with no content at the source.

Changes:
- Gemini: Don't emit ReasoningMessage when only thought_signature exists
- Gemini: Only emit reasoning content if text is non-empty
- Anthropic: Don't emit ReasoningMessage for BetaSignatureDelta
- Anthropic: Only emit reasoning content if thinking text is non-empty

This fixes the issue where providers send signature metadata before
actual thinking content, causing empty reasoning blocks to appear
in the UI after responses complete.

Affects: Gemini reasoning, Anthropic extended thinking

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: handle Anthropic thinking signature correctly

- Only include 'signature' in Anthropic message payload if it is not None (fixes BadRequestError).
- Capture and attach 'signature' to ReasoningMessage in streaming interface.

* fix(anthropic): attach signature to last reasoning message in stream

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-12 10:57:19 -08:00
Charles Packer
4af6465226 feat(core+web): store raw usage data on streams (and visualize properly in ADE) (#6452)
* feat(core): store raw usage data on streams

* fix(web): various fixes to deal w/ hardcoding against openai
2025-12-15 12:02:19 -08:00
Charles Packer
88a3743cc8 fix(core): distinguish between null and 0 for prompt caching (#6451)
* fix(core): distinguish between null and 0 for prompt caching

* fix: runtime errors

* fix: just publish just sgate
2025-12-15 12:02:19 -08:00
Charles Packer
131891e05f feat: add tracking of advanced usage data (eg caching) [LET-6372] (#6449)
* feat: init refactor

* feat: add helper code

* fix: missing file + test

* fix: just state/publish api
2025-12-15 12:02:19 -08:00
Charles Packer
e142d440d5 fix: patch gemini token counting (#6445)
fix: use usage_metadata.candidates_token_count for counting total tokens
2025-12-15 12:02:18 -08:00
Matthew Zhou
72e80395cc fix: Fix gemini streaming interface string growth [LET-6067] (#5975)
* Fix gemini streaming interface

* Add comments
2025-11-13 15:36:55 -08:00
Matthew Zhou
7b3cb0224a feat: Add gemini parallel tool call streaming for gemini [LET-6027] (#5913)
* Make changes to gemini streaming interface to support parallel tool calling

* Finish send message integration test

* Add comments
2025-11-13 15:36:39 -08:00
cthomas
c67bdd9c64 fix: special approval message otid for gemini streaming (#5742) 2025-10-24 15:14:39 -07:00
Kian Jones
c2e474e03a feat: refactor logs to parse as a single log line each and filter out 404s from sentry (#5242)
* add multiline log auto detect

* implement logger.exception()

* filter out 404

* remove potentially problematic changes
2025-10-24 15:11:31 -07:00
Matthew Zhou
5593f1450b feat: Double write to ToolCallMessage's new list tool_calls field (#5268)
* Add new tool_calls field to ToolCallMessage

* fern autogen

* Double write to new tool_calls field

* Update straggling instances
2025-10-09 13:20:52 -07:00
cthomas
cc913df27c feat: add signature to content parts (#5134)
* feat: add signature to content parts

* always base64 encode thought signature

* propagate thought signature back to request
2025-10-07 17:50:49 -07:00
cthomas
93d9ff01c6 feat: add gemini native thinking (#5124)
* feat: add gemini native thinking

* update test

* revert comments
2025-10-07 17:50:49 -07:00
cthomas
3e17b4289a feat: gracefully handle gemini empty content parts (#5116) 2025-10-07 17:50:48 -07:00
cthomas
f7755d837a feat: add gemini streaming to new agent loop (#5109)
* feat: add gemini streaming to new agent loop

* add google as required dependency

* support storing all content parts

* remove extra google references
2025-10-07 17:50:48 -07:00