feat: enable Datadog LLM Observability for memgpt-server (#7258)

* fix: prevent empty reasoning messages in streaming interfaces

Prevents empty "Thinking..." indicators from appearing in clients by
filtering out reasoning messages with no content at the source.

Changes:
- Gemini: Don't emit ReasoningMessage when only thought_signature exists
- Gemini: Only emit reasoning content if text is non-empty
- Anthropic: Don't emit ReasoningMessage for BetaSignatureDelta
- Anthropic: Only emit reasoning content if thinking text is non-empty

This fixes the issue where providers send signature metadata before
actual thinking content, causing empty reasoning blocks to appear
in the UI after responses complete.

Affects: Gemini reasoning, Anthropic extended thinking

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* feat: enable Datadog LLM Observability for memgpt-server

Enables DD_LLMOBS to track LLM calls, prompts, completions, and costs
in production for memgpt-server.

Changes:
- Add DD_LLMOBS_ENABLED=1 and DD_LLMOBS_ML_APP=memgpt-server in:
  - .github/workflows/deploy-core.yml (GitHub Actions deployment)
  - justfile (Helm deployment secrets)
  - apps/core/letta/server/rest_api/app.py (runtime config)

This provides visibility into:
- LLM API calls and latency
- Prompt/completion content and tokens
- Model costs and usage
- Error rates per model/provider

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* dd llmobs

* Revert "fix: prevent empty reasoning messages in streaming interfaces"

This reverts commit a900228b3611de49eb5f740f68dc76a657fc9b14.

---------

Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
Kian Jones
2025-12-17 13:02:20 -05:00
committed by Caren Thomas
parent 350f3a751c
commit 9c1c2698ab

View File

@@ -241,6 +241,10 @@ def create_application() -> "FastAPI":
os.environ.setdefault("DD_PROFILING_MEMORY_ENABLED", str(telemetry_settings.datadog_profiling_memory_enabled).lower())
os.environ.setdefault("DD_PROFILING_HEAP_ENABLED", str(telemetry_settings.datadog_profiling_heap_enabled).lower())
# Enable LLM Observability for tracking LLM calls, prompts, and completions
os.environ.setdefault("DD_LLMOBS_ENABLED", "1")
os.environ.setdefault("DD_LLMOBS_ML_APP", "memgpt-server")
# Note: DD_LOGS_INJECTION, DD_APPSEC_ENABLED, DD_IAST_ENABLED, DD_APPSEC_SCA_ENABLED
# are set via deployment configs and automatically picked up by ddtrace