From 9c1c2698ab0121d100a23515cbc9ddadc903f960 Mon Sep 17 00:00:00 2001 From: Kian Jones <11655409+kianjones9@users.noreply.github.com> Date: Wed, 17 Dec 2025 13:02:20 -0500 Subject: [PATCH] feat: enable Datadog LLM Observability for memgpt-server (#7258) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix: prevent empty reasoning messages in streaming interfaces Prevents empty "Thinking..." indicators from appearing in clients by filtering out reasoning messages with no content at the source. Changes: - Gemini: Don't emit ReasoningMessage when only thought_signature exists - Gemini: Only emit reasoning content if text is non-empty - Anthropic: Don't emit ReasoningMessage for BetaSignatureDelta - Anthropic: Only emit reasoning content if thinking text is non-empty This fixes the issue where providers send signature metadata before actual thinking content, causing empty reasoning blocks to appear in the UI after responses complete. Affects: Gemini reasoning, Anthropic extended thinking 👾 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta * feat: enable Datadog LLM Observability for memgpt-server Enables DD_LLMOBS to track LLM calls, prompts, completions, and costs in production for memgpt-server. Changes: - Add DD_LLMOBS_ENABLED=1 and DD_LLMOBS_ML_APP=memgpt-server in: - .github/workflows/deploy-core.yml (GitHub Actions deployment) - justfile (Helm deployment secrets) - apps/core/letta/server/rest_api/app.py (runtime config) This provides visibility into: - LLM API calls and latency - Prompt/completion content and tokens - Model costs and usage - Error rates per model/provider 👾 Generated with [Letta Code](https://letta.com) Co-Authored-By: Letta * dd llmobs * Revert "fix: prevent empty reasoning messages in streaming interfaces" This reverts commit a900228b3611de49eb5f740f68dc76a657fc9b14. --------- Co-authored-by: Letta --- letta/server/rest_api/app.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/letta/server/rest_api/app.py b/letta/server/rest_api/app.py index ccf76ff9..56772be7 100644 --- a/letta/server/rest_api/app.py +++ b/letta/server/rest_api/app.py @@ -241,6 +241,10 @@ def create_application() -> "FastAPI": os.environ.setdefault("DD_PROFILING_MEMORY_ENABLED", str(telemetry_settings.datadog_profiling_memory_enabled).lower()) os.environ.setdefault("DD_PROFILING_HEAP_ENABLED", str(telemetry_settings.datadog_profiling_heap_enabled).lower()) + # Enable LLM Observability for tracking LLM calls, prompts, and completions + os.environ.setdefault("DD_LLMOBS_ENABLED", "1") + os.environ.setdefault("DD_LLMOBS_ML_APP", "memgpt-server") + # Note: DD_LOGS_INJECTION, DD_APPSEC_ENABLED, DD_IAST_ENABLED, DD_APPSEC_SCA_ENABLED # are set via deployment configs and automatically picked up by ddtrace