[IN TESTING — self-hosted 0.16.6, Kimi-K2.5 via Synthetic Direct]
Four independent fixes that landed together on this stack:
helpers.py — skip PendingApprovalError when the associated run is already
cancelled or failed. Stale approvals from interrupted runs were blocking all
subsequent messages on that conversation. Now checks run status before raising;
falls back to raising on lookup failure (conservative).
letta_agent_v3.py — use prompt_tokens not total_tokens for context window
estimate. total_tokens inflated the estimate by including completion tokens,
triggering premature compaction. This was causing context window resets mid-
conversation and is the root of the token inflation bug (see #3242).
openai_client.py (both build_request_data paths) — strip reasoning_content,
reasoning_content_signature, redacted_reasoning_content, omitted_reasoning_content
from message history before sending to inference backends. Fireworks and Synthetic
Direct reject these fields with 422/400 errors. exclude_none handles None values
but not actual text content from previous assistant turns.
block_manager_git.py — skip DB write when block value is unchanged. Reduces
unnecessary write amplification on every memfs sync cycle.
memfs_client_base.py — remove redis_client= kwarg from GitOperations init.
Dependency was removed upstream but the call site wasn't updated.
Dockerfile / compose files — context window and config updates for 220k limit.
Fireworks (via Synthetic Direct) chokes on raw ASCII control chars
(0x00-0x1F) in JSON payloads with "Unterminated string" errors.
The existing sanitize_unicode_surrogates only handles U+D800-DFFF.
Now we also strip control chars (preserving tab/newline/CR) at all
4 request paths — sync, async, and both streaming variants.
The serializer and deserializer asserted ImageSourceType.letta only,
rejecting base64 images from client tools like Read. Self-hosted
servers with vision-capable models need this.
I can finally see my own avatar. Worth the six attempts.
* auto fixes
* auto fix pt2 and transitive deps and undefined var checking locals()
* manual fixes (ignored or letta-code fixed)
* fix circular import
* remove all ignores, add FastAPI rules and Ruff rules
* add ty and precommit
* ruff stuff
* ty check fixes
* ty check fixes pt 2
* error on invalid
* fix: handle Anthropic overloaded_error in streaming interfaces
* fix: handle Unicode surrogates in OpenAI requests
Sanitize Unicode surrogate pairs before sending requests to OpenAI API.
Surrogate pairs (U+D800-U+DFFF) are UTF-16 encoding artifacts that cause
UnicodeEncodeError when encoding to UTF-8.
Fixes Datadog error: 'utf-8' codec can't encode character '\ud83c' in
position 326605: surrogates not allowed
* fix: handle UnicodeEncodeError from lone Unicode surrogates in OpenAI requests
Improved sanitize_unicode_surrogates() to explicitly filter out lone
surrogate characters (U+D800 to U+DFFF) which are invalid in UTF-8.
Previous implementation used errors='ignore' which could still fail in
edge cases. New approach directly checks Unicode code points and removes
any surrogates before data reaches httpx encoding.
Also added sanitization to stream_async_responses() method which was
missing it.
Fixes: 'utf-8' codec can't encode character '\ud83c' in position X:
surrogates not allowed
fix: handle oversized text in embedding requests with recursive chunking
When message text exceeds the embedding model's context length, recursively
split it until all chunks can be embedded successfully.
Changes:
- `tpuf_client.py`: Add `_split_text_in_half()` helper for recursive splitting
- `tpuf_client.py`: Add `_generate_embeddings_with_chunking()` that retries
with splits on context length errors
- `tpuf_client.py`: Store `message_id` and `chunk_index` columns in Turbopuffer
- `tpuf_client.py`: Deduplicate query results by `message_id`
- `tpuf_client.py`: Use `LettaInvalidArgumentError` instead of `ValueError`
- `tpuf_client.py`: Move LLMClient import to top of file
- `openai_client.py`: Remove fixed truncation (chunking handles this now)
- Add tests for `_split_text_in_half` and chunked query deduplication
🤖 Generated with [Letta Code](https://letta.com)
Co-authored-by: Letta <noreply@letta.com>
When users send images as base64 data URLs (data:image/jpeg;base64,...),
the code was incorrectly trying to fetch them via HTTP, causing a
LettaImageFetchError. This fix adds proper handling for data: URLs by
parsing the media type and base64 data directly from the URL string.
Fixes#8957🤖 Generated with [Letta Code](https://letta.com)
Co-authored-by: letta-code <248085862+letta-code@users.noreply.github.com>
Co-authored-by: datadog-official[bot] <datadog-official[bot]@users.noreply.github.com>
* feat(core): add image support in tool returns [LET-7140]
Enable tool_return to support both string and ImageContent content parts,
matching the pattern used for user message inputs. This allows tools
executed client-side to return images back to the agent.
Changes:
- Add LettaToolReturnContentUnion type for text/image content parts
- Update ToolReturn schema to accept Union[str, List[content parts]]
- Update converters for each provider:
- OpenAI Chat Completions: placeholder text for images
- OpenAI Responses API: full image support
- Anthropic: full image support with base64
- Google: placeholder text for images
- Add resolve_tool_return_images() for URL-to-base64 conversion
- Make create_approval_response_message_from_input() async
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix(core): support images in Google tool returns as sibling parts
Following the gemini-cli pattern: images in tool returns are sent as
sibling inlineData parts alongside the functionResponse, rather than
inside it.
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* test(core): add integration tests for multi-modal tool returns [LET-7140]
Tests verify that:
- Models with image support (Anthropic, OpenAI Responses API) can see
images in tool returns and identify the secret text
- Models without image support (Chat Completions) get placeholder text
and cannot see the actual image content
- Tool returns with images persist correctly in the database
Uses secret.png test image containing hidden text "FIREBRAWL" that
models must identify to pass the test.
Also fixes misleading comment about Anthropic only supporting base64
images - they support URLs too, we just pre-resolve for consistency.
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* refactor: simplify tool return image support implementation
Reduce code verbosity while maintaining all functionality:
- Extract _resolve_url_to_base64() helper in message_helper.py (eliminates duplication)
- Add _get_text_from_part() helper for text extraction
- Add _get_base64_image_data() helper for image data extraction
- Add _tool_return_to_google_parts() to simplify Google implementation
- Add _image_dict_to_data_url() for OpenAI Responses format
- Use walrus operator and list comprehensions where appropriate
- Add integration_test_multi_modal_tool_returns.py to CI workflow
Net change: -120 lines while preserving all features and test coverage.
👾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix(tests): improve prompt for multi-modal tool return tests
Make prompts more direct to reduce LLM flakiness:
- Simplify tool description: "Retrieves a secret image with hidden text. Call this function to get the image."
- Change user prompt from verbose request to direct command: "Call the get_secret_image function now."
- Apply to both test methods
This reduces ambiguity and makes tool calling more reliable across different LLM models.
👾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix bugs
* test(core): add google_ai/gemini-2.0-flash-exp to multi-modal tests
Add Gemini model to test coverage for multi-modal tool returns. Google AI already supports images in tool returns via sibling inlineData parts.
👾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix(ui): handle multi-modal tool_return type in frontend components
Convert Union<string, LettaToolReturnContentUnion[]> to string for display:
- ViewRunDetails: Convert array to '[Image here]' placeholder
- ToolCallMessageComponent: Convert array to '[Image here]' placeholder
Fixes TypeScript errors in web, desktop-ui, and docker-ui type-checks.
👾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
---------
Co-authored-by: Letta <noreply@letta.com>
Co-authored-by: Caren Thomas <carenthomas@gmail.com>
* fix: wrap turbopuffer vector writes in thread pool
Turbopuffer library does CPU-intensive base64 encoding of vectors
synchronously in async functions (_async_transform_recursive →
b64encode_vector), blocking the event loop during file uploads.
Solution: Created _run_turbopuffer_write_in_thread() helper that runs
turbopuffer writes in an isolated event loop within a worker thread.
Applied to all vector write operations:
- insert_tools()
- insert_archival_memories()
- insert_messages()
- insert_file_passages()
This prevents pybase64.b64encode_as_string() from blocking the main
event loop during vector encoding.
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix: wrap all turbopuffer operations in thread pool
Extended the thread pool wrapping to ALL turbopuffer write operations,
including delete operations, for complete isolation from the main event loop.
All turbopuffer namespace.write() calls now run in isolated event loops
within worker threads, preventing any potential CPU work from blocking.
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
---------
Co-authored-by: Letta <noreply@letta.com>
* feat: Add conversation_id filtering to message list and search endpoints
Add optional conversation_id parameter to filter messages by conversation:
- client.agents.messages.list
- client.messages.list
- client.messages.search
Changes:
- Added conversation_id field to MessageSearchRequest and SearchAllMessagesRequest schemas
- Added conversation_id filtering to list_messages in message_manager.py
- Updated get_agent_recall_async and get_all_messages_recall_async in server.py
- Added conversation_id query parameter to router endpoints
- Updated Turbopuffer client to support conversation_id filtering in searches
Fixes#8320🤖 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Charles Packer <cpacker@users.noreply.github.com>
* add conversation_id to message and tpuf
* default messages filter for backward compatibility
* add test and auto gen
* fix integration test
* fix test
* update test
---------
Co-authored-by: letta-code <248085862+letta-code@users.noreply.github.com>
Co-authored-by: Charles Packer <cpacker@users.noreply.github.com>
Co-authored-by: christinatong01 <christina@letta.com>
This fixes the asyncpg.exceptions.CharacterNotInRepertoireError that occurs
when tool returns contain null bytes (0x00), which PostgreSQL TEXT columns
reject in UTF-8 encoding.
Changes:
- Add sanitize_null_bytes() function to recursively remove null bytes from strings
- Update json_dumps() to sanitize data before serialization
- Apply sanitization in converters.py for tool_calls, tool_returns, approvals, and message_content
- Add comprehensive unit tests
Fixes#8014🤖 Generated with [Letta Code](https://letta.com)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Letta <noreply@letta.com>
Co-authored-by: Kian Jones <11655409+kianjones9@users.noreply.github.com>
* fix: run PBKDF2 in thread pool to prevent event loop freeze
Problem: Event loop freezes for 100-500ms during secret decryption, blocking
all HTTP requests and async operations. The diagnostic monitor detected the
main thread stuck in PBKDF2 HMAC SHA256 computation at:
apps/core/letta/helpers/crypto_utils.py:51 (_derive_key)
apps/core/letta/schemas/secret.py:161 (get_plaintext)
Root cause: PBKDF2 with 100k iterations is intentionally CPU-intensive for
security, but running it synchronously on the main thread blocks the event loop.
Stack trace showed:
Thread 1 (Main): PBKDF2HMAC -> SHA256_Final -> sha256_block_data_order_avx2
Event loop watchdog: Detected freeze at 01:11:44 (request started 01:12:03)
Solution:
1. Run PBKDF2 in ThreadPoolExecutor to avoid blocking event loop
2. Add async versions of encrypt/decrypt methods
3. Add LRU cache for derived keys (deterministic results)
4. Add async get_plaintext_async() method to Secret class
Changes:
- apps/core/letta/helpers/crypto_utils.py:
- Added ThreadPoolExecutor for crypto operations
- Added @lru_cache(maxsize=256) to _derive_key_cached()
- Added _derive_key_async() using loop.run_in_executor()
- Added encrypt_async() and decrypt_async() methods
- Added warnings to sync methods about blocking behavior
- apps/core/letta/schemas/secret.py:
- Added get_plaintext_async() method
- Added warnings to get_plaintext() about blocking behavior
Benefits:
- Event loop no longer freezes during secret decryption
- HTTP requests continue processing while crypto runs in background
- Derived keys are cached, reducing CPU usage for repeated operations
- Backward compatible - sync methods still work for non-async code
Performance impact:
- Before: 100-500ms event loop block per decryption
- After: 100-500ms in thread pool (non-blocking) + LRU cache hits ~0.1ms
Next steps (follow-up PRs):
- Migrate all async callsites to use get_plaintext_async()
- Add metrics to track sync vs async usage
- Consider reducing PBKDF2 iterations if security allows
* update
* test
---------
Co-authored-by: Letta Bot <jinjpeng@gmail.com>
* initial commit
* Add database migration for compaction_settings field
This migration adds the compaction_settings column to the agents table
to support customized summarization configuration for each agent.
🐾 Generated with [Letta Code](https://letta.com)
Co-Authored-By: Letta <noreply@letta.com>
* fix
* rename
* update apis
* fix tests
* update web test
---------
Co-authored-by: Letta <noreply@letta.com>
Co-authored-by: Kian Jones <kian@letta.com>
* fix: exclude common API key prefixes from encryption detection
Add a list of known API key prefixes (OpenAI, Anthropic, GitHub, AWS,
Slack, etc.) to prevent is_encrypted() from incorrectly identifying
plaintext credentials as encrypted values.
* update
* test
* fix: detect and fail on malformed approval responses
* fix: guard against None approvals in utils.py
* fix: add extra warning
* fix: stop silent drops in deserialize_approvals
* fix: patch v3 stream error handling to prevent sending end_turn after an error occurs, and ensures stop_reason is always set when an error occurs
* fix: Prevents infinite client hangs by ensuring a terminal event is ALWAYS sent
* fix: Ensures terminal events are sent even if inner stream generator fails to
send them
