Add billing context to LLM telemetry traces (#9745)
* feat: add billing context to LLM telemetry traces Add billing metadata (plan type, cost source, customer ID) to LLM traces in ClickHouse for cost analytics and attribution. **Data Flow:** - Cloud-API: Extract billing info from subscription in rate limiting, set x-billing-* headers - Core: Parse headers into BillingContext object via dependencies - Adapters: Flow billing_context through all LLM adapters (blocking & streaming) - Agent: Pass billing_context to step() and stream() methods - ClickHouse: Store in billing_plan_type, billing_cost_source, billing_customer_id columns **Changes:** - Add BillingContext schema to provider_trace.py - Add billing columns to llm_traces ClickHouse table DDL - Update getCustomerSubscription to fetch stripeCustomerId from organization_billing_details - Propagate billing_context through agent step flow, adapters, and streaming service - Update ProviderTrace and LLMTrace to include billing metadata - Regenerate SDK with autogen **Production Deployment:** Requires env vars: LETTA_PROVIDER_TRACE_BACKEND=clickhouse, LETTA_STORE_LLM_TRACES=true, CLICKHOUSE_* 🐾 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta <noreply@letta.com> * fix: add billing_context parameter to agent step methods - Add billing_context to BaseAgent and BaseAgentV2 abstract methods - Update LettaAgent, LettaAgentV2, LettaAgentV3 step methods - Update multi-agent groups: SleeptimeMultiAgentV2, V3, V4 - Fix test_utils.py to include billing header parameters - Import BillingContext in all affected files * fix: add billing_context to stream methods - Add billing_context parameter to BaseAgentV2.stream() - Add billing_context parameter to LettaAgentV2.stream() - LettaAgentV3.stream() already has it from previous commit * fix: exclude billing headers from OpenAPI spec Mark billing headers as internal (include_in_schema=False) so they don't appear in the public API. These are internal headers between cloud-api and core, not part of the public SDK. Regenerated SDK with stage-api - removes 10,650 lines of bloat that was causing OOM during Next.js build. * refactor: return billing context from handleUnifiedRateLimiting instead of mutating req Instead of passing req into handleUnifiedRateLimiting and mutating headers inside it: - Return billing context fields (billingPlanType, billingCostSource, billingCustomerId) from handleUnifiedRateLimiting - Set headers in handleMessageRateLimiting (middleware layer) after getting the result - This fixes step-orchestrator compatibility since it doesn't have a real Express req object * chore: remove extra gencode * p --------- Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
@@ -24,6 +24,9 @@ def test_get_headers_user_id_allows_none():
|
||||
letta_v1_agent=None,
|
||||
letta_v1_agent_message_async=None,
|
||||
modal_sandbox=None,
|
||||
billing_plan_type=None,
|
||||
billing_cost_source=None,
|
||||
billing_customer_id=None,
|
||||
)
|
||||
assert isinstance(headers, HeaderParams)
|
||||
|
||||
@@ -40,6 +43,9 @@ def test_get_headers_user_id_rejects_invalid_format():
|
||||
letta_v1_agent=None,
|
||||
letta_v1_agent_message_async=None,
|
||||
modal_sandbox=None,
|
||||
billing_plan_type=None,
|
||||
billing_cost_source=None,
|
||||
billing_customer_id=None,
|
||||
)
|
||||
|
||||
|
||||
@@ -54,6 +60,9 @@ def test_get_headers_user_id_accepts_valid_format():
|
||||
letta_v1_agent=None,
|
||||
letta_v1_agent_message_async=None,
|
||||
modal_sandbox=None,
|
||||
billing_plan_type=None,
|
||||
billing_cost_source=None,
|
||||
billing_customer_id=None,
|
||||
)
|
||||
assert headers.actor_id == "user-123e4567-e89b-42d3-8456-426614174000"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user