Commit Graph

151 Commits

Author SHA1 Message Date
Kian Jones
a92e868ee6 feat: centralize telemetry logging at LLM client level (#8815)
* feat: centralize telemetry logging at LLM client level

Moves telemetry logging from individual adapters to LLMClientBase:
- Add TelemetryStreamWrapper for streaming telemetry on stream close
- Add request_async_with_telemetry() for non-streaming requests
- Add stream_async_with_telemetry() for streaming requests
- Add set_telemetry_context() to configure agent_id, run_id, step_id

Updates adapters and agents to use new pattern:
- LettaLLMAdapter now accepts agent_id/run_id in constructor
- Adapters call set_telemetry_context() before LLM requests
- Removes duplicate telemetry logging from adapters
- Enriches traces with agent_id, run_id, call_type metadata

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: accumulate streaming response content for telemetry

TelemetryStreamWrapper now extracts actual response data from chunks:
- Content text (concatenated from deltas)
- Tool calls (id, name, arguments)
- Model name, finish reason, usage stats

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* refactor: move streaming telemetry to caller (option 3)

- Remove TelemetryStreamWrapper class
- Add log_provider_trace_async() helper to LLMClientBase
- stream_async_with_telemetry() now just returns raw stream
- Callers log telemetry after processing with rich interface data

Updated callers:
- summarizer.py: logs content + usage after stream processing
- letta_agent.py: logs tool_call, reasoning, model, usage

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: pass agent_id and run_id to parent adapter class

LettaLLMStreamAdapter was not passing agent_id/run_id to parent,
causing "unexpected keyword argument" errors.

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-19 15:54:43 -08:00
Kian Jones
9418ab9815 feat: add provider trace backend abstraction for multi-backend telemetry (#8814)
* feat: add provider trace backend abstraction for multi-backend telemetry

Introduces a pluggable backend system for provider traces:
- Base class with async/sync create and read interfaces
- PostgreSQL backend (existing behavior)
- ClickHouse backend (via OTEL instrumentation)
- Socket backend (writes to Unix socket for crouton sidecar)
- Factory for instantiating backends from config

Refactors TelemetryManager to use backends with support for:
- Multi-backend writes (concurrent via asyncio.gather)
- Primary backend for reads (first in config list)
- Graceful error handling per backend

Config: LETTA_TELEMETRY_PROVIDER_TRACE_BACKEND (comma-separated)
Example: "postgres,socket" for dual-write to Postgres and crouton

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* feat: add protocol version to socket backend records

Adds PROTOCOL_VERSION constant to socket backend:
- Included in every telemetry record sent to crouton
- Must match ProtocolVersion in apps/crouton/main.go
- Enables crouton to detect and reject incompatible messages

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: remove organization_id from ProviderTraceCreate calls

The organization_id is now handled via the actor parameter in the
telemetry manager, not through ProviderTraceCreate schema. This fixes
validation errors after changing ProviderTraceCreate to inherit from
BaseProviderTrace which forbids extra fields.

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* consolidate provider trace

* add clickhouse-connect to fix bug on main lmao

* auto generated sdk changes, and deployment details, and clikchouse prefix bug and added fields to runs trace return api

* auto generated sdk changes, and deployment details, and clikchouse prefix bug and added fields to runs trace return api

* consolidate provider trace

* consolidate provider trace bug fix

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-19 15:54:43 -08:00
jnjpng
a98bc31bf3 fix: refactor enable strict mode for structured output (#8840)
* base

* test
2026-01-19 15:54:42 -08:00
jnjpng
25d75d6528 fix: update aysnc get plaintext callsites (#7069)
* base

* resolve

* fix

* fix
2025-12-17 17:31:02 -08:00
jnjpng
4be813b956 fix: migrate sandbox and agent environment variables to encrypted only (#6623)
* base

* remove unnnecessary db migration

* update

* fix

* update

* update

* comments

* fix

* revert

* anotha

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-12-15 12:03:08 -08:00
Sarah Wooders
91e3dd8b3e feat: fix new summarizer code and add more tests (#6461) 2025-12-15 12:02:19 -08:00
Charles Packer
88a3743cc8 fix(core): distinguish between null and 0 for prompt caching (#6451)
* fix(core): distinguish between null and 0 for prompt caching

* fix: runtime errors

* fix: just publish just sgate
2025-12-15 12:02:19 -08:00
Charles Packer
131891e05f feat: add tracking of advanced usage data (eg caching) [LET-6372] (#6449)
* feat: init refactor

* feat: add helper code

* fix: missing file + test

* fix: just state/publish api
2025-12-15 12:02:19 -08:00
cthomas
73dcc0d4b7 feat: latest hitl + parallel tool call changes (#5565) 2025-10-24 15:12:49 -07:00
cthomas
69b15d606c feat: support approval requests for parallel tool calls (#5538) 2025-10-24 15:12:11 -07:00
jnjpng
b3fef4b5a8 feat: double write to all encrypted columns and decrypt on read (#5265)
* base

* use secret field

* fix

* auth code

* stage publish

* decouple backfill

* revert uncomment

* providers and agent vars

* mcp

* mcp

* stage and publish

* fix oauth

* double encrypt

* sandbox

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-10-24 15:11:31 -07:00
Matthew Zhou
20ce885e07 feat: Simplify create_letta_messages_from_llm_response [LET-4681] (#5204)
Simplify create_letta_messages_from_llm_response
2025-10-07 17:50:50 -07:00
Sarah Wooders
e07a589796 chore: rm composio (#5151) 2025-10-07 17:50:49 -07:00
Sarah Wooders
354205f581 feat: create new runs table [LET-4467] (#4841) 2025-10-07 17:50:47 -07:00
Charles Packer
a4041879a4 feat: add new agent loop (squash rebase of OSS PR) (#4815)
* feat: squash rebase of OSS PR

* fix: revert changes that weren't on manual rebase

* fix: caught another one

* fix: disable force

* chore: drop print

* fix: just stage-api && just publish-api

* fix: make agent_type consistently an arg in the client

* fix: patch multi-modal support

* chore: put in todo stub

* fix: disable hardcoding for tests

* fix: patch validate agent sync (#4882)

patch validate agent sync

* fix: strip bad merge diff

* fix: revert unrelated diff

* fix: react_v2 naming -> letta_v1 naming

* fix: strip bad merge

---------

Co-authored-by: Kevin Lin <klin5061@gmail.com>
2025-10-07 17:50:45 -07:00
jnjpng
39a8959989 feat: _handle_ai_response activity skeleton (#4760)
* base

* skeleton

* TODO

* unpack

* unpack

* cleanup

* cleanup

* merge

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-10-07 17:50:43 -07:00
Kian Jones
b8e9a80d93 merge this (#4759)
* wait I forgot to comit locally

* cp the entire core directory and then rm the .git subdir
2025-09-17 15:47:40 -07:00
Kian Jones
22f70ca07c chore: officially migrate to submodule (#4502)
* remove apps/core and apps/fern

* fix precommit

* add submodule updates in workflows

* submodule

* remove core tests

* update core revision

* Add submodules: true to all GitHub workflows

- Ensure all workflows can access git submodules
- Add submodules support to deployment, test, and CI workflows
- Fix YAML syntax issues in workflow files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove core-lint

* upgrade core with latest main of oss

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 12:45:53 -07:00
Matthew Zhou
acaf820009 feat: Add project_id to message filtering/inserting to Turbopuffer [LET-4252] (#4466)
* Add project_id

* Fern autogen
2025-09-08 14:35:15 -07:00
cthomas
cb7296c81d fix: approval request for streaming (#4445)
* fix: approval request for streaming

* fix: claude code attempt, unit test passing (add on to #4445) (#4448)

* fix: claude code attempt, unit test passing

* chore: update locks to 0.1.314 from 0.1.312

* chore: just stage-api && just publish-api

* chore: drop dead poetry lock

---------

Co-authored-by: Charles Packer <packercharles@gmail.com>
2025-09-05 17:43:21 -07:00
Matthew Zhou
2ef47d8002 feat: Add search messages endpoint [LET-4144] (#4434)
* Add search messages endpoint

* Run fern autogen and fix tests
2025-09-05 14:28:27 -07:00
Matthew Zhou
cbf2e09e13 feat: Add project id to message schema [LET-4166] (#4433)
* Add project id

* Propogate through update message by id async

* Add project id testing
2025-09-04 16:50:41 -07:00
Matthew Zhou
6e633bd8f9 feat: Change namespace to be org scoped and filter on agent_id [LET-4163] (#4368)
* Change to org scoped and agent_id filtering

* Finish modifying conversation search tool

* Fix failing tests

* Get rid of bad imports
2025-09-03 10:00:19 -07:00
Charles Packer
9d49eff204 fix: patch the streaming issue in the openai client for when inner_thoughts_in_kwargs is off [LET-4146] (#4350)
fix: patch the streaming issue in the openai client for when inner_thoughts_in_kwargs is off
2025-09-02 12:44:25 -07:00
cthomas
315d07568b feat: handle message persistence for approvals flows (#4338) 2025-09-01 14:10:02 -07:00
cthomas
8d1389146a feat: support deny tool call request (#4334) 2025-09-01 10:33:18 -07:00
cthomas
86b073d726 feat: support approve tool call request (#4333) 2025-08-31 11:26:21 -07:00
cthomas
b8c2f42d33 feat: add support for approval request creation (#4313) 2025-08-29 15:23:02 -07:00
cthomas
bb87cc68c7 feat: add approval create input to messages endpoints [LET-4110] (#4309)
* feat: add approval create input to messages endpoints

* rename discriminator tag

* add base class with default

* add field validator

* exclude new type field from agent file schema
2025-08-29 13:16:03 -07:00
Kian Jones
fecf6decfb chore: migrate to ruff (#4305)
* base requirements

* autofix

* Configure ruff for Python linting and formatting

- Set up minimal ruff configuration with basic checks (E, W, F, I)
- Add temporary ignores for common issues during migration
- Configure pre-commit hooks to use ruff with pass_filenames
- This enables gradual migration from black to ruff

* Delete sdj

* autofixed only

* migrate lint action

* more autofixed

* more fixes

* change precommit

* try changing the hook

* try this stuff
2025-08-29 11:11:19 -07:00
cthomas
3e6a223d17 feat: add invalid_llm_response stop reason [LET-4083] (#4269)
* feat: add invalid_llm_response stop reason

* add sqllite support

* simply skip for sqllite

* fix imports

* fix isort
2025-08-27 17:07:47 -07:00
cthomas
11b447a02b feat: add gating to provider trace persistence in db (#4223)
* feat: make provider trace fetch result nullable

* feat: add flag for persisting provider trace to db
2025-08-26 15:58:26 -07:00
Charles Packer
bb57f9cca4 fix: allow for send_message to be unterminated if the user requests it (#4169) 2025-08-25 21:11:14 -07:00
Sarah Wooders
f10f27a9ce fix: avoid using summarization agent when not using voice (#4097) 2025-08-25 13:23:24 -07:00
Sarah Wooders
ea26680674 fix: add handling for summarizer llm call (#4028) 2025-08-20 16:25:02 -07:00
cthomas
2d4511fd21 feat: add null check to step logging LET-3890 (#4004) 2025-08-19 11:52:34 -07:00
jnjpng
0179901e79 feat: record step metrics to table
Co-authored-by: Jin Peng <jinjpeng@Jins-MacBook-Pro.local>
2025-08-13 11:29:18 -07:00
Andy Li
c6002744e6 feat: track run state in stream 2025-08-08 16:31:15 -07:00
jnjpng
7645310570 fix: clean up tool execution print statement
Co-authored-by: Jin Peng <jinjpeng@Jins-MacBook-Pro.local>
2025-08-08 15:08:12 -07:00
Matthew Zhou
617ec9e87b feat: Add comprehensive error tracking to steps table (#3765) 2025-08-06 20:19:29 -07:00
Andy Li
ca6f474c4e feat: track metrics for runs in db 2025-08-06 15:46:50 -07:00
cthomas
b4810633af chore: deprecate orm enum file (#3724) 2025-08-04 11:59:55 -07:00
jnjpng
6b082f0447 fix: manually count tokens for streaming lmstudio models
Co-authored-by: Jin Peng <jinjpeng@Jins-MacBook-Pro.local>
Co-authored-by: Charles Packer <packercharles@gmail.com>
2025-07-29 18:12:42 -07:00
Matthew Zhou
272b36c63f feat: Scrub inner thoughts from history on toggle (#3607) 2025-07-28 21:43:36 -07:00
Matthew Zhou
11540a4586 fix: Adjust summarize_agent_conversation endpoint to return 204 (#3595) 2025-07-28 14:29:09 -07:00
Andy Li
fbc69e6a0c fix: error logging for stop reasons 2025-07-24 16:51:31 -07:00
cthomas
6d56aecf95 feat: add special message update helper for agent (#3442) 2025-07-20 23:00:57 -07:00
Andy Li
60c4e23904 feat: stop reasons and error messages and sentry fixes 2025-07-18 11:56:20 -07:00
cthomas
7939d2c366 feat: remove concurrent db connection spawning (#3380) 2025-07-17 14:01:38 -07:00
Andy Li
ed198a547d fix: llm_request_ms tracing (#3277) 2025-07-10 14:55:37 -07:00