fix: catch contextwindowexceeded error on gemini (#9450)

* catch contextwindowexceeded error

* fix(core): detect Google token limit errors as ContextWindowExceededError

Google's error message says "input token count exceeds the maximum
number of tokens allowed" which doesn't contain the word "context",
so it was falling through to generic LLMBadRequestError instead of
ContextWindowExceededError. This means compaction won't auto-trigger.

Expands the detection to also match "token count" and "tokens allowed"
in addition to the existing "context" keyword.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): add missing message arg to LLMBadRequestError in OpenAI client

The generic 400 path in handle_llm_error was constructing
LLMBadRequestError without the required message positional arg,
causing TypeError in prod during summarization.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* ci: add adapters/ test suite to core unit test matrix

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(tests): update adapter error handling test expectations to match actual behavior

The streaming adapter's error handling double-wraps errors: the
AnthropicStreamingInterface calls handle_llm_error first, then the
adapter catches the result and calls handle_llm_error again, which
falls through to the base class LLMError. Updated test expectations
to match this behavior.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): prevent double-wrapping of LLMError in stream adapter

The AnthropicStreamingInterface.process() already transforms raw
provider errors into LLMError subtypes via handle_llm_error. The
adapter was catching the result and calling handle_llm_error again,
which didn't recognize the already-transformed LLMError and wrapped
it in a generic LLMError("Unhandled LLM error"). This downgraded
specific error types (LLMConnectionError, LLMServerError, etc.)
and broke retry logic that matches on specific subtypes.

Now the adapter checks if the error is already an LLMError and
re-raises it as-is. Tests restored to original correct expectations.

🐾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

---------

Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
Kian Jones
2026-02-11 22:49:35 -08:00
committed by Caren Thomas
parent 05073ba837
commit b9c4ed3b15
4 changed files with 55 additions and 2 deletions

View File

@@ -1,10 +1,12 @@
import anthropic
import httpx
import pytest
from google.genai import errors as google_errors
from letta.adapters.letta_llm_stream_adapter import LettaLLMStreamAdapter
from letta.errors import ContextWindowExceededError, LLMConnectionError, LLMServerError
from letta.errors import ContextWindowExceededError, LLMBadRequestError, LLMConnectionError, LLMError, LLMServerError
from letta.llm_api.anthropic_client import AnthropicClient
from letta.llm_api.google_vertex_client import GoogleVertexClient
from letta.schemas.enums import LLMCallType
from letta.schemas.llm_config import LLMConfig
@@ -188,3 +190,48 @@ def test_anthropic_client_handle_llm_error_request_too_large_string():
assert isinstance(result, ContextWindowExceededError)
assert "request_too_large" in result.message.lower() or "context window exceeded" in result.message.lower()
@pytest.mark.parametrize(
"error_message",
[
"The input token count exceeds the maximum number of tokens allowed 1048576.",
"Token count of 1500000 exceeds the model limit of 1048576 tokens allowed.",
],
ids=["gemini-token-count-exceeds", "gemini-tokens-allowed-limit"],
)
def test_google_client_handle_llm_error_token_limit_returns_context_window_exceeded(error_message):
"""Google 400 errors about token limits should map to ContextWindowExceededError."""
client = GoogleVertexClient.__new__(GoogleVertexClient)
response_json = {
"message": f'{{"error": {{"code": 400, "message": "{error_message}", "status": "INVALID_ARGUMENT"}}}}',
"status": "Bad Request",
}
error = google_errors.ClientError(400, response_json)
result = client.handle_llm_error(error)
assert isinstance(result, ContextWindowExceededError)
def test_google_client_handle_llm_error_context_exceeded_returns_context_window_exceeded():
"""Google 400 errors with 'context' + 'exceeded' should map to ContextWindowExceededError."""
client = GoogleVertexClient.__new__(GoogleVertexClient)
response_json = {
"message": '{"error": {"code": 400, "message": "Request context window exceeded the limit.", "status": "INVALID_ARGUMENT"}}',
"status": "Bad Request",
}
error = google_errors.ClientError(400, response_json)
result = client.handle_llm_error(error)
assert isinstance(result, ContextWindowExceededError)
def test_google_client_handle_llm_error_generic_400_returns_bad_request():
"""Google 400 errors without token/context keywords should map to LLMBadRequestError."""
client = GoogleVertexClient.__new__(GoogleVertexClient)
response_json = {
"message": '{"error": {"code": 400, "message": "Invalid argument: unsupported parameter.", "status": "INVALID_ARGUMENT"}}',
"status": "Bad Request",
}
error = google_errors.ClientError(400, response_json)
result = client.handle_llm_error(error)
assert isinstance(result, LLMBadRequestError)
assert not isinstance(result, ContextWindowExceededError)