Commit Graph

157 Commits

Author SHA1 Message Date
Kian Jones
9418ab9815 feat: add provider trace backend abstraction for multi-backend telemetry (#8814)
* feat: add provider trace backend abstraction for multi-backend telemetry

Introduces a pluggable backend system for provider traces:
- Base class with async/sync create and read interfaces
- PostgreSQL backend (existing behavior)
- ClickHouse backend (via OTEL instrumentation)
- Socket backend (writes to Unix socket for crouton sidecar)
- Factory for instantiating backends from config

Refactors TelemetryManager to use backends with support for:
- Multi-backend writes (concurrent via asyncio.gather)
- Primary backend for reads (first in config list)
- Graceful error handling per backend

Config: LETTA_TELEMETRY_PROVIDER_TRACE_BACKEND (comma-separated)
Example: "postgres,socket" for dual-write to Postgres and crouton

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* feat: add protocol version to socket backend records

Adds PROTOCOL_VERSION constant to socket backend:
- Included in every telemetry record sent to crouton
- Must match ProtocolVersion in apps/crouton/main.go
- Enables crouton to detect and reject incompatible messages

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: remove organization_id from ProviderTraceCreate calls

The organization_id is now handled via the actor parameter in the
telemetry manager, not through ProviderTraceCreate schema. This fixes
validation errors after changing ProviderTraceCreate to inherit from
BaseProviderTrace which forbids extra fields.

🐙 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* consolidate provider trace

* add clickhouse-connect to fix bug on main lmao

* auto generated sdk changes, and deployment details, and clikchouse prefix bug and added fields to runs trace return api

* auto generated sdk changes, and deployment details, and clikchouse prefix bug and added fields to runs trace return api

* consolidate provider trace

* consolidate provider trace bug fix

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-19 15:54:43 -08:00
Sarah Wooders
b888c4c17a feat: allow for conversation-level isolation of blocks (#8684)
* feat: add conversation_id parameter to context endpoint [LET-6989]

Add optional conversation_id query parameter to retrieve_agent_context_window.
When provided, the endpoint uses messages from the specific conversation
instead of the agent's default message_ids.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* chore: regenerate SDK after context endpoint update [LET-6989]

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* feat: add isolated blocks support for conversations

Allows conversations to have their own copies of specific memory blocks (e.g., todo_list) that override agent defaults, enabling conversation-specific state isolation.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* undo

* update apis

* test

* cleanup

* fix tests

* simplify

* move override logic

* patch

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-19 15:54:39 -08:00
cthomas
ab4ccfca31 feat: add tags support to blocks (#8474)
* feat: add tags support to blocks

* fix: add timestamps and org scoping to blocks_tags

Addresses PR feedback:

1. Migration: Added timestamps (created_at, updated_at), soft delete
   (is_deleted), audit fields (_created_by_id, _last_updated_by_id),
   and organization_id to blocks_tags table for filtering support.
   Follows SQLite baseline pattern (composite PK of block_id+tag, no
   separate id column) to avoid insert failures.

2. ORM: Relationship already correct with lazy="raise" to prevent
   implicit joins and passive_deletes=True for efficient CASCADE deletes.

3. Schema: Changed normalize_tags() from Any to dict for type safety.

4. SQLite: Added blocks_tags to SQLite baseline schema to prevent
   table-not-found errors.

5. Code: Updated all tag row inserts to include organization_id.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: add ORM columns and update SQLite baseline for blocks_tags

Fixes test failures (CompileError: Unconsumed column names: organization_id):

1. ORM: Added organization_id, timestamps, audit fields to BlocksTags
   ORM model to match database schema from migrations.

2. SQLite baseline: Added full column set to blocks_tags (organization_id,
   timestamps, audit fields) to match PostgreSQL schema.

3. Test: Added 'tags' to expected Block schema fields.

This ensures SQLite and PostgreSQL have matching schemas and the ORM
can consume all columns that the code inserts.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* revert change to existing alembic migration

* fix: remove passive_deletes and SQLite support for blocks_tags

1. Removed passive_deletes=True from Block.tags relationship to match
   AgentsTags pattern (neither have ondelete CASCADE in DB schema).

2. Removed SQLite branch from _replace_block_pivot_rows_async since
   blocks_tags table is PostgreSQL-only (migration skips SQLite).

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* api sync

---------

Co-authored-by: Letta <noreply@letta.com>
2026-01-19 15:54:38 -08:00
Sarah Wooders
87d920782f feat: add conversation and conversation_messages tables for concurrent messaging (#8182) 2026-01-12 10:57:48 -08:00
Ari Webb
4d8d9757aa feat: add request-id for steps [LET-6587] (#7349)
* feat: add request-id for steps

* order revisions correctly

* stage publish api
2026-01-12 10:57:19 -08:00
Ari Webb
79c2319403 chore: add project constraint on tools db (#7360) 2025-12-17 17:32:27 -08:00
Sarah Wooders
812bfd16dd Revert "feat: project_id uniqueness for tools" (#7007)
Revert "feat: project_id uniqueness for tools (#6604)"

This reverts commit 2c4b6397041e2c965493525fc52e056f10d1bdb6.
2025-12-15 12:03:09 -08:00
Ari Webb
c1aa01db6f feat: project_id uniqueness for tools (#6604)
* feat: project_id uniqueness for tools

* prevent double upsert of global tools

* use default project if no header for sdk

* reorder unique constraint for performance

* use separate session for check conflict

* feature flag adding project id header in cloud api

* add my migration after one on main

* remove comment

* stage and publish api

* web set project id just for tools

* includes instead of startswith
2025-12-15 12:03:09 -08:00
Sarah Wooders
7ea297231a feat: add compaction_settings to agents (#6625)
* initial commit

* Add database migration for compaction_settings field

This migration adds the compaction_settings column to the agents table
to support customized summarization configuration for each agent.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix

* rename

* update apis

* fix tests

* update web test

---------

Co-authored-by: Letta <noreply@letta.com>
Co-authored-by: Kian Jones <kian@letta.com>
2025-12-15 12:02:34 -08:00
Ari Webb
4092820f3a feat: add project id scoping for tools backend changes (#6529) 2025-12-15 12:02:34 -08:00
Sarah Wooders
4f1fbe45aa feat: add index and concurrency control for tools (fixed alembic) (#6552) 2025-12-15 12:02:34 -08:00
Sarah Wooders
c8c06168e2 Revert "feat: add index and concurrency control for tools " (#6551)
Revert "feat: add index and concurrency control for tools  (#6547)"

This reverts commit f4abf8e061bc2f5e08853b5ce5775a7f8626463a.
2025-12-15 12:02:34 -08:00
Sarah Wooders
a2d3011d84 feat: add index and concurrency control for tools (#6547) 2025-12-15 12:02:34 -08:00
Charles Packer
131891e05f feat: add tracking of advanced usage data (eg caching) [LET-6372] (#6449)
* feat: init refactor

* feat: add helper code

* fix: missing file + test

* fix: just state/publish api
2025-12-15 12:02:19 -08:00
Kian Jones
7b7b89ecfa fix: fix alembic on main (#6116)
fix alembic on main
2025-11-13 15:36:56 -08:00
Kian Jones
78600fac23 chore: index suggestions from planetscale suggestions (#6111)
suggestions from ps dashboard
2025-11-13 15:36:56 -08:00
Sarah Wooders
982501f6fa feat: create model table to persist available models (#5835)
---------

Co-authored-by: Ari Webb <arijwebb@gmail.com>
Co-authored-by: Ari Webb <ari@letta.com>
2025-11-13 15:36:56 -08:00
Kian Jones
e665768085 chore: alembic upgrade for step metrics run id index (#6109)
alembic upgrade for step metrics run id index
2025-11-13 15:36:56 -08:00
Kian Jones
ba19142669 fix: address high query latency with step id index on messages (#6068)
* add step id index on messages

* simple index

* fix sources padding

* fix import

* fix passage manager

* fix ci

* I think fixed

* disable turbopufer for managers to avoid failing passages tests
2025-11-13 15:36:56 -08:00
Christina Tong
ef3df907c5 feat: add last_stop_reason to AgentState [LET-5911] (#5772)
* feat: add last_stop_reason to AgentState [LET-5911]

* undo agent loop changes, use update_run_by_id_async

* add run manager test

* add integration tests

* remove comment

* remove duplicate test
2025-11-13 15:36:55 -08:00
Matthew Zhou
e7e86124f9 feat: Support embedding config on the archive [LET-5832] (#5714)
* Add embedding config field to archives

* Fix alembic script

* Simplify archive manager

* Fern autogen

* Fix failing tests

* Fix alembic
2025-10-24 15:14:21 -07:00
Ari Webb
c7c0d7507c feat: add new mcp_servers routes [LET-4321] (#5675)
---------

Co-authored-by: Ari Webb <ari@letta.com>
Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2025-10-24 15:14:21 -07:00
Ari Webb
abbd1b5595 Revert "feat: revise mcp tool routes [LET-4321]" (#5652)
Revert "feat: revise mcp tool routes [LET-4321] (#5631)"

This reverts commit e15f120078652b2160d64a1e300317b95eccb163.
2025-10-24 15:13:47 -07:00
Ari Webb
747e7eae0e feat: revise mcp tool routes [LET-4321] (#5631)
* start adding tools mapping

* ready to start testing new table migration

* alembic generated code

* just stage api

* small update

* integration tests

---------

Co-authored-by: Ari Webb <ari@letta.com>
2025-10-24 15:13:47 -07:00
Christina Tong
79f49c33d8 feat: add tool_used field to run_metrics [LET-5419] (#5540)
* feat: add tool_used field to run_metrics [LET-5419]

* change to tool name

* use tool ids over names

* add generated files

* chore: update OpenAPI schema to include tools_used field in RunMetrics

* update alembic file
2025-10-24 15:12:11 -07:00
Matthew Zhou
ddd1686ad1 feat: Add flag to tools to signify if they are race safe [LET-5611] (#5548)
* wip add race unsafe param

* Alembic migration

* Fern autogen

* Set default to True

* Adjust fern

* Rename field

* Run fern autogen

* Add enable_parallel_execution to Tool Create and Update
2025-10-24 15:12:11 -07:00
jnjpng
fea13051e6 feat: backfill providers, sandbox_environment_variables, mcp_oauth *_enc columns [LET-5458] (#5382)
* base

* remove print

---------

Co-authored-by: Letta Bot <noreply@letta.com>
2025-10-24 15:11:31 -07:00
cthomas
1c80e1c11f feat: add approvals persistence to message orm (#5309)
* feat: add approvals persistence to message orm

* fix imports in alembic migration

* missing import
2025-10-24 15:11:31 -07:00
cthomas
9cb5156ac7 chore: sync 0.12.0 version (#3023)
Co-authored-by: Matthew Zhou <mattzh1314@gmail.com>
Co-authored-by: Kian Jones <11655409+kianjones9@users.noreply.github.com>
Co-authored-by: Sarah Wooders <sarahwooders@gmail.com>
2025-10-08 16:10:51 -07:00
Matthew Zhou
932b418681 feat: Add FK cascades, remove app-side label propagation [LET-4690] (#5219)
add FK cascades, remove app-side label propagation
2025-10-07 17:50:50 -07:00
jnjpng
7930bfeb0b feat: add encrypted columns for providers, sandbox/agent variables, mcp auth code [LET-4246] (#5187)
add encrypted columns

Co-authored-by: Letta Bot <noreply@letta.com>
2025-10-07 17:50:49 -07:00
Sarah Wooders
4168506dd2 feat: replace composite runs index to not include is_err (#5177) 2025-10-07 17:50:49 -07:00
Sarah Wooders
e07a589796 chore: rm composio (#5151) 2025-10-07 17:50:49 -07:00
Sarah Wooders
300c32456e feat: add composite message index and reduce pool timeout (#5156) 2025-10-07 17:50:49 -07:00
Sarah Wooders
65b33ae27f feat: add index for run_id on messages and project_id on agents (#5117) 2025-10-07 17:50:49 -07:00
Sarah Wooders
354205f581 feat: create new runs table [LET-4467] (#4841) 2025-10-07 17:50:47 -07:00
Charles Packer
d748d86059 feat: add more indexes (#4849) 2025-10-07 17:50:45 -07:00
Sarah Wooders
a952a1266b feat: add various indexes (#4811) 2025-10-07 17:50:44 -07:00
Sarah Wooders
c85bfefa52 feat: add agents_runs table (#4768) 2025-10-07 17:50:44 -07:00
Kian Jones
b8e9a80d93 merge this (#4759)
* wait I forgot to comit locally

* cp the entire core directory and then rm the .git subdir
2025-09-17 15:47:40 -07:00
Kian Jones
22f70ca07c chore: officially migrate to submodule (#4502)
* remove apps/core and apps/fern

* fix precommit

* add submodule updates in workflows

* submodule

* remove core tests

* update core revision

* Add submodules: true to all GitHub workflows

- Ensure all workflows can access git submodules
- Add submodules support to deployment, test, and CI workflows
- Fix YAML syntax issues in workflow files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove core-lint

* upgrade core with latest main of oss

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 12:45:53 -07:00
Matthew Zhou
ee63c59682 feat: Fix alembic migration for vector db type on sources for pg8000 driver (#4482)
Add commit after as well
2025-09-08 19:50:49 -07:00
Matthew Zhou
6aa2a92537 feat: Add vector db type column to source [LET-4203] (#4469)
* Add vector db type column to source

* Add comments
2025-09-08 15:09:29 -07:00
cthomas
57e69a35bc feat: add step checkpointing for progress tracking (#4458)
* feat: add step checkpointing for progress tracking

* openapi sync
2025-09-08 10:30:44 -07:00
Shubham Naik
eb21668f35 feat: add hidden property to group and blocks [PRO-1145] (#4415)
* feat: add hidden property to group and blocks

* feat: add hidden property to group and blocks

* chore: bup

* chore: add hidden property

* chore: next

---------

Co-authored-by: Shubham Naik <shub@memgpt.ai>
2025-09-04 10:53:16 -07:00
Matthew Zhou
29502ac3aa feat: Add association for the vector namespace [LET-4148] (#4354)
Add association for the namespace
2025-09-02 13:21:42 -07:00
cthomas
315d07568b feat: handle message persistence for approvals flows (#4338) 2025-09-01 14:10:02 -07:00
Kian Jones
fecf6decfb chore: migrate to ruff (#4305)
* base requirements

* autofix

* Configure ruff for Python linting and formatting

- Set up minimal ruff configuration with basic checks (E, W, F, I)
- Add temporary ignores for common issues during migration
- Configure pre-commit hooks to use ruff with pass_filenames
- This enables gradual migration from black to ruff

* Delete sdj

* autofixed only

* migrate lint action

* more autofixed

* more fixes

* change precommit

* try changing the hook

* try this stuff
2025-08-29 11:11:19 -07:00
Matthew Zhou
c1f8c48818 feat: Support arbitrary string tagging filtering [LET-3467] (#4285)
* Finish tagging

* Add comprehensive tags functionality

* Add fern autogen

* Create passage tags table

* Add indices

* Add comments explaining dual storage

* Fix alembic heads

* Fix alembic

---------

Co-authored-by: Kian Jones <11655409+kianjones9@users.noreply.github.com>
2025-08-28 16:57:36 -07:00
cthomas
e3d3bc09eb feat: add new default_requires_approval flag on tools (#4287) 2025-08-28 15:38:59 -07:00