feat: centralize telemetry logging at LLM client level (#8815)
* feat: centralize telemetry logging at LLM client level Moves telemetry logging from individual adapters to LLMClientBase: - Add TelemetryStreamWrapper for streaming telemetry on stream close - Add request_async_with_telemetry() for non-streaming requests - Add stream_async_with_telemetry() for streaming requests - Add set_telemetry_context() to configure agent_id, run_id, step_id Updates adapters and agents to use new pattern: - LettaLLMAdapter now accepts agent_id/run_id in constructor - Adapters call set_telemetry_context() before LLM requests - Removes duplicate telemetry logging from adapters - Enriches traces with agent_id, run_id, call_type metadata 🐙 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta <noreply@letta.com> * fix: accumulate streaming response content for telemetry TelemetryStreamWrapper now extracts actual response data from chunks: - Content text (concatenated from deltas) - Tool calls (id, name, arguments) - Model name, finish reason, usage stats 🐙 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta <noreply@letta.com> * refactor: move streaming telemetry to caller (option 3) - Remove TelemetryStreamWrapper class - Add log_provider_trace_async() helper to LLMClientBase - stream_async_with_telemetry() now just returns raw stream - Callers log telemetry after processing with rich interface data Updated callers: - summarizer.py: logs content + usage after stream processing - letta_agent.py: logs tool_call, reasoning, model, usage 🐙 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta <noreply@letta.com> * fix: pass agent_id and run_id to parent adapter class LettaLLMStreamAdapter was not passing agent_id/run_id to parent, causing "unexpected keyword argument" errors. 🐙 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta <noreply@letta.com> --------- Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
committed by
Sarah Wooders
parent
9418ab9815
commit
a92e868ee6
@@ -952,7 +952,13 @@ async def generate_tool_from_prompt(
|
||||
llm_config,
|
||||
tools=[tool],
|
||||
)
|
||||
response_data = await llm_client.request_async(request_data, llm_config)
|
||||
from letta.services.telemetry_manager import TelemetryManager
|
||||
|
||||
llm_client.set_telemetry_context(
|
||||
telemetry_manager=TelemetryManager(),
|
||||
call_type="tool_generation",
|
||||
)
|
||||
response_data = await llm_client.request_async_with_telemetry(request_data, llm_config)
|
||||
response = await llm_client.convert_response_to_chat_completion(response_data, input_messages, llm_config)
|
||||
|
||||
# Validate that we got a tool call response
|
||||
|
||||
Reference in New Issue
Block a user