Commit Graph

32 Commits

Author SHA1 Message Date
Kian Jones
0099a95a43 fix(sec): first pass of ensuring actor id is required everywhere (#9126)
first pass of ensuring actor id is required
2026-01-29 12:44:04 -08:00
jnjpng
ccfd3d1432 fix: asyncify encrypt on write (#8339)
* base

* update

* import
2026-01-12 10:57:48 -08:00
jnjpng
e56c5c5b49 fix: global sandbox environment variables not injected on tool execution (#8310)
* base

* test
2026-01-12 10:57:48 -08:00
jnjpng
591420876a fix: correct decorator order for trace_method and raise_on_invalid_id (#7226)
Swap the order of @trace_method and @raise_on_invalid_id decorators
across all service managers so that @trace_method is always the first
wrapper applied to the function (positioned directly above the method).

This ensures the ID validation happens before tracing begins, which is
the intended execution order.

Files modified:
- agent_manager.py (23 occurrences)
- archive_manager.py (11 occurrences)
- block_manager.py (7 occurrences)
- file_manager.py (6 occurrences)
- group_manager.py (9 occurrences)
- identity_manager.py (10 occurrences)
- job_manager.py (7 occurrences)
- message_manager.py (2 occurrences)
- provider_manager.py (3 occurrences)
- sandbox_config_manager.py (7 occurrences)
- source_manager.py (5 occurrences)
- step_manager.py (13 occurrences)
2025-12-17 17:31:02 -08:00
jnjpng
25d75d6528 fix: update aysnc get plaintext callsites (#7069)
* base

* resolve

* fix

* fix
2025-12-17 17:31:02 -08:00
Kian Jones
bce1749408 fix: run PBKDF2 in thread pool to prevent event loop freeze (#6763)
* fix: run PBKDF2 in thread pool to prevent event loop freeze

Problem: Event loop freezes for 100-500ms during secret decryption, blocking
all HTTP requests and async operations. The diagnostic monitor detected the
main thread stuck in PBKDF2 HMAC SHA256 computation at:
  apps/core/letta/helpers/crypto_utils.py:51 (_derive_key)
  apps/core/letta/schemas/secret.py:161 (get_plaintext)

Root cause: PBKDF2 with 100k iterations is intentionally CPU-intensive for
security, but running it synchronously on the main thread blocks the event loop.

Stack trace showed:
  Thread 1 (Main): PBKDF2HMAC -> SHA256_Final -> sha256_block_data_order_avx2
  Event loop watchdog: Detected freeze at 01:11:44 (request started 01:12:03)

Solution:
1. Run PBKDF2 in ThreadPoolExecutor to avoid blocking event loop
2. Add async versions of encrypt/decrypt methods
3. Add LRU cache for derived keys (deterministic results)
4. Add async get_plaintext_async() method to Secret class

Changes:
- apps/core/letta/helpers/crypto_utils.py:
  - Added ThreadPoolExecutor for crypto operations
  - Added @lru_cache(maxsize=256) to _derive_key_cached()
  - Added _derive_key_async() using loop.run_in_executor()
  - Added encrypt_async() and decrypt_async() methods
  - Added warnings to sync methods about blocking behavior

- apps/core/letta/schemas/secret.py:
  - Added get_plaintext_async() method
  - Added warnings to get_plaintext() about blocking behavior

Benefits:
- Event loop no longer freezes during secret decryption
- HTTP requests continue processing while crypto runs in background
- Derived keys are cached, reducing CPU usage for repeated operations
- Backward compatible - sync methods still work for non-async code

Performance impact:
- Before: 100-500ms event loop block per decryption
- After: 100-500ms in thread pool (non-blocking) + LRU cache hits ~0.1ms

Next steps (follow-up PRs):
- Migrate all async callsites to use get_plaintext_async()
- Add metrics to track sync vs async usage
- Consider reducing PBKDF2 iterations if security allows

* update

* test

---------

Co-authored-by: Letta Bot <jinjpeng@gmail.com>
2025-12-15 12:03:09 -08:00
jnjpng
4be813b956 fix: migrate sandbox and agent environment variables to encrypted only (#6623)
* base

* remove unnnecessary db migration

* update

* fix

* update

* update

* comments

* fix

* revert

* anotha

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-12-15 12:03:08 -08:00
Kian Jones
45065297a0 feat: runtime validation for ids for internal managers calls (#5544)
* claude coded first pass

* fix test cases to expect errors instead

* fix this

* let's see how letta-code did

* claude

* fix tests, remove dangling comments, retrofit all managers functions with decorator

* revert to main for these since we are not erroring on invalid tool and block ids

* reorder decorators

* finish refactoring test cases

* reorder agent_manager decorators and fix test tool manager

* add decorator on missing managers

* fix id sources

* remove redundant check

* uses enum now

* move to enum
2025-10-24 15:13:47 -07:00
jnjpng
b3fef4b5a8 feat: double write to all encrypted columns and decrypt on read (#5265)
* base

* use secret field

* fix

* auth code

* stage publish

* decouple backfill

* revert uncomment

* providers and agent vars

* mcp

* mcp

* stage and publish

* fix oauth

* double encrypt

* sandbox

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-10-24 15:11:31 -07:00
Sarah Wooders
4df0a27eb0 chore: remove sync db (#4873) 2025-10-07 17:50:45 -07:00
Kian Jones
b8e9a80d93 merge this (#4759)
* wait I forgot to comit locally

* cp the entire core directory and then rm the .git subdir
2025-09-17 15:47:40 -07:00
Kian Jones
22f70ca07c chore: officially migrate to submodule (#4502)
* remove apps/core and apps/fern

* fix precommit

* add submodule updates in workflows

* submodule

* remove core tests

* update core revision

* Add submodules: true to all GitHub workflows

- Ensure all workflows can access git submodules
- Add submodules support to deployment, test, and CI workflows
- Fix YAML syntax issues in workflow files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove core-lint

* upgrade core with latest main of oss

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 12:45:53 -07:00
Kian Jones
fecf6decfb chore: migrate to ruff (#4305)
* base requirements

* autofix

* Configure ruff for Python linting and formatting

- Set up minimal ruff configuration with basic checks (E, W, F, I)
- Add temporary ignores for common issues during migration
- Configure pre-commit hooks to use ruff with pass_filenames
- This enables gradual migration from black to ruff

* Delete sdj

* autofixed only

* migrate lint action

* more autofixed

* more fixes

* change precommit

* try changing the hook

* try this stuff
2025-08-29 11:11:19 -07:00
Andy Li
73e74270f4 feat: modal sandbox 2025-07-30 11:25:40 -07:00
Andy Li
d2252f2953 feat: otel metrics and expanded collecting (#2647)
(passed tests in last run)
2025-06-05 17:20:14 -07:00
cthomas
78c82f1ad9 fix: move db logic inside sessions (#2553) 2025-05-30 14:02:25 -07:00
cthomas
8c64bb7fec feat(asyncify): convert get composio key and sandbox config manager (#2436) 2025-05-26 13:25:33 -07:00
cthomas
4778180658 fix: async sandbox db writes (#2351) 2025-05-22 19:17:04 -07:00
cthomas
6c1b189080 feat: add db tracing to otel (#2337) 2025-05-22 13:47:49 -07:00
cthomas
7162ed5ecc feat(asyncify): migrate sandbox config routes (#2233) 2025-05-17 23:24:34 -07:00
Andy Li
600ccb35e7 feat: async db client (#2076) 2025-05-12 17:15:14 -07:00
cthomas
05b701e0bc feat: enable mounting local tools dir to container (#1119) 2025-03-04 13:18:23 -08:00
Matthew Zhou
b5e09536ae feat: Serialize agent state simple fields and messages (#1012) 2025-02-18 11:01:10 -08:00
Matthew Zhou
9ee53a0eaa feat: Composio tools execute on-the-fly (#999) 2025-02-13 16:13:29 -08:00
mlong93
d6be7809d0 fix: deprecate cursor param and replace with before, after (#736)
Co-authored-by: Mindy Long <mindy@letta.com>
2025-01-23 14:17:25 -08:00
cthomas
f5bcef2657 chore: rename metadata_ field to metadata in pydantic (#732) 2025-01-22 19:05:41 -08:00
Matthew Zhou
ebfda389e5 feat: Adjust tool execution sandbox to be under ~/.letta (#591) 2025-01-10 15:17:16 -08:00
Matthew Zhou
dfa8c0091d feat: Add sandbox_type filter when listing sandbox configurations (#567) 2025-01-09 12:48:21 -08:00
Matthew Zhou
9cd326ddb8 feat: Allow per-agent tool execution environment variables (#509) 2025-01-05 21:06:38 -08:00
Caren Thomas
fd8961c39e run black, add isort config to pyproject.toml 2024-12-26 19:43:11 -08:00
Caren Thomas
ea0cca7bad run isort on apps/core 2024-12-26 19:27:09 -08:00
Shubham Naik
5a743d1dc4 Add 'apps/core/' from commit 'ea2a7395f4023f5b9fab03e6273db3b64a1181d5'
git-subtree-dir: apps/core
git-subtree-mainline: a8963e11e7a5a0059acbc849ce768e1eee80df61
git-subtree-split: ea2a7395f4023f5b9fab03e6273db3b64a1181d5
2024-12-22 20:31:22 -08:00