fix(core): handle Google 499 CANCELLED as client disconnect, not server error (#9363)

The google.genai.errors.ClientError with code 499 (CANCELLED) indicates the
client disconnected, not a server-side failure. Previously this fell through
to the generic ClientError handler and was classified as LLMServerError,
causing false 500s in Datadog error tracking.

- Add explicit 499 handling in handle_llm_error: log at info level, return
  LLMConnectionError instead of LLMServerError
- Catch 499 during stream iteration in stream_async and end gracefully
  instead of propagating the error

Datadog: https://us5.datadoghq.com/error-tracking/issue/c8453aaa-d559-11f0-81c6-da7ad0900000

🤖 Generated with [Letta Code](https://letta.com)

Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
Kian Jones
2026-02-06 17:25:35 -08:00
committed by Caren Thomas
parent f20fdc73d1
commit 2c0cddf9f5

View File

@@ -204,8 +204,16 @@ class GoogleVertexClient(LLMClientBase):
raise e
# Direct yield - keeps response alive in generator's local scope throughout iteration
# This is required because the SDK's connection lifecycle is tied to the response object
async for chunk in response:
yield chunk
try:
async for chunk in response:
yield chunk
except errors.ClientError as e:
if e.code == 499:
logger.info(f"{self._provider_prefix()} Stream cancelled by client (499): {e}")
return
raise self.handle_llm_error(e)
except errors.APIError as e:
raise self.handle_llm_error(e)
@staticmethod
def add_dummy_model_messages(messages: List[dict]) -> List[dict]:
@@ -801,6 +809,14 @@ class GoogleVertexClient(LLMClientBase):
def handle_llm_error(self, e: Exception) -> Exception:
# Handle Google GenAI specific errors
if isinstance(e, errors.ClientError):
if e.code == 499:
logger.info(f"{self._provider_prefix()} Request cancelled by client (499): {e}")
return LLMConnectionError(
message=f"Request to {self._provider_name()} was cancelled (client disconnected): {str(e)}",
code=ErrorCode.INTERNAL_SERVER_ERROR,
details={"status_code": 499, "cause": "client_cancelled"},
)
logger.warning(f"{self._provider_prefix()} Client error ({e.code}): {e}")
# Handle specific error codes