Commit Graph

21 Commits

Author SHA1 Message Date
cthomas
416ffc7cd7 Add billing context to LLM telemetry traces (#9745)
* feat: add billing context to LLM telemetry traces

Add billing metadata (plan type, cost source, customer ID) to LLM traces in ClickHouse for cost analytics and attribution.

**Data Flow:**
- Cloud-API: Extract billing info from subscription in rate limiting, set x-billing-* headers
- Core: Parse headers into BillingContext object via dependencies
- Adapters: Flow billing_context through all LLM adapters (blocking & streaming)
- Agent: Pass billing_context to step() and stream() methods
- ClickHouse: Store in billing_plan_type, billing_cost_source, billing_customer_id columns

**Changes:**
- Add BillingContext schema to provider_trace.py
- Add billing columns to llm_traces ClickHouse table DDL
- Update getCustomerSubscription to fetch stripeCustomerId from organization_billing_details
- Propagate billing_context through agent step flow, adapters, and streaming service
- Update ProviderTrace and LLMTrace to include billing metadata
- Regenerate SDK with autogen

**Production Deployment:**
Requires env vars: LETTA_PROVIDER_TRACE_BACKEND=clickhouse, LETTA_STORE_LLM_TRACES=true, CLICKHOUSE_*

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix: add billing_context parameter to agent step methods

- Add billing_context to BaseAgent and BaseAgentV2 abstract methods
- Update LettaAgent, LettaAgentV2, LettaAgentV3 step methods
- Update multi-agent groups: SleeptimeMultiAgentV2, V3, V4
- Fix test_utils.py to include billing header parameters
- Import BillingContext in all affected files

* fix: add billing_context to stream methods

- Add billing_context parameter to BaseAgentV2.stream()
- Add billing_context parameter to LettaAgentV2.stream()
- LettaAgentV3.stream() already has it from previous commit

* fix: exclude billing headers from OpenAPI spec

Mark billing headers as internal (include_in_schema=False) so they don't appear in the public API.
These are internal headers between cloud-api and core, not part of the public SDK.

Regenerated SDK with stage-api - removes 10,650 lines of bloat that was causing OOM during Next.js build.

* refactor: return billing context from handleUnifiedRateLimiting instead of mutating req

Instead of passing req into handleUnifiedRateLimiting and mutating headers inside it:
- Return billing context fields (billingPlanType, billingCostSource, billingCustomerId) from handleUnifiedRateLimiting
- Set headers in handleMessageRateLimiting (middleware layer) after getting the result
- This fixes step-orchestrator compatibility since it doesn't have a real Express req object

* chore: remove extra gencode

* p

---------

Co-authored-by: Letta <noreply@letta.com>
2026-03-03 18:34:13 -08:00
Kian Jones
f5c4ab50f4 chore: add ty + pre-commit hook and repeal even more ruff rules (#9504)
* auto fixes

* auto fix pt2 and transitive deps and undefined var checking locals()

* manual fixes (ignored or letta-code fixed)

* fix circular import

* remove all ignores, add FastAPI rules and Ruff rules

* add ty and precommit

* ruff stuff

* ty check fixes

* ty check fixes pt 2

* error on invalid
2026-02-24 10:55:11 -08:00
Kian Jones
25d54dd896 chore: enable F821, F401, W293 (#9503)
* auto fixes

* auto fix pt2 and transitive deps and undefined var checking locals()

* manual fixes (ignored or letta-code fixed)

* fix circular import
2026-02-24 10:55:08 -08:00
Sarah Wooders
354205f581 feat: create new runs table [LET-4467] (#4841) 2025-10-07 17:50:47 -07:00
Kian Jones
b8e9a80d93 merge this (#4759)
* wait I forgot to comit locally

* cp the entire core directory and then rm the .git subdir
2025-09-17 15:47:40 -07:00
Kian Jones
22f70ca07c chore: officially migrate to submodule (#4502)
* remove apps/core and apps/fern

* fix precommit

* add submodule updates in workflows

* submodule

* remove core tests

* update core revision

* Add submodules: true to all GitHub workflows

- Ensure all workflows can access git submodules
- Add submodules support to deployment, test, and CI workflows
- Fix YAML syntax issues in workflow files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* remove core-lint

* upgrade core with latest main of oss

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-09-09 12:45:53 -07:00
Andy Li
33c1f26ab6 feat: support for agent loop job cancelation (#2837) 2025-07-02 14:31:16 -07:00
Andy Li
98a79afbdd fix: bugfixes (and bump) (#3071) 2025-06-27 14:23:42 -07:00
cthomas
99e112e486 feat: make create_async route consistent with other message routes (#2877) 2025-06-19 13:51:51 -07:00
cthomas
605a1f410c feat: consolidate logic for finish tokens (#2779) 2025-06-12 15:24:06 -07:00
cthomas
a855532e4b feat: set request heartbeat for max steps (#2739) 2025-06-10 15:26:07 -07:00
Sarah Wooders
3e6729214b feat: add include_return_message_types to LettaRequest to filter down requests (#2666) 2025-06-06 15:48:27 -07:00
Andy Li
d2252f2953 feat: otel metrics and expanded collecting (#2647)
(passed tests in last run)
2025-06-05 17:20:14 -07:00
Andy Li
3f28a5c8df fix: SleeptimeMultiAgentV2.step() unexpected arg (#2599) 2025-06-03 13:21:22 -07:00
Sarah Wooders
268285c688 fix: pass in argument to sleeptime (#2566) 2025-06-01 13:55:23 -07:00
Matthew Zhou
87f4bcad9a feat: Add summarization for more scenarios (#2499) 2025-05-29 11:10:13 -07:00
cthomas
2bdeeccc41 feat: add step stream to async sleeptime agent (#2379) 2025-05-23 14:38:25 -07:00
cthomas
db520f9a22 feat: add sleeptime to new agent loop (#2263) 2025-05-22 23:22:51 -07:00
cthomas
1325cacbfe feat(asyncify): migrate list messages (#2272) 2025-05-20 16:52:11 -07:00
cthomas
3644a17426 fix: unset timezone for async db (#2220) 2025-05-16 13:11:15 -07:00
cthomas
d4f9fd3cd2 feat: add sleeptime to new agent loop (#1900) 2025-04-26 10:23:59 -07:00