fix: allow explicit null for max_tokens on GPT-5 models (#9562)

The Pydantic validator `set_model_specific_defaults` was checking
`values.get("max_tokens") is None`, which matched both "field not
provided" and "field explicitly set to null". This meant users could
not disable the max output tokens limit for GPT-5/GPT-4.1 models -
the validator would always override null with a default value during
request deserialization.

Changed to `"max_tokens" not in values` so that an explicit
`max_tokens: null` is preserved while still applying defaults when
the field is omitted entirely.
This commit is contained in:
jnjpng
2026-02-19 15:14:10 -08:00
committed by Caren Thomas
parent 828c89c76f
commit eb4a0daabd

View File

@@ -137,13 +137,12 @@ class LLMConfig(BaseModel):
if model is None:
return values
# Set max_tokens defaults based on model
if values.get("max_tokens") is None:
# Set max_tokens defaults based on model (only if not explicitly provided)
if "max_tokens" not in values:
if model.startswith("gpt-5"): # Covers both gpt-5 and gpt-5.1
values["max_tokens"] = 16384
elif model == "gpt-4.1":
values["max_tokens"] = 8192
# For other models, the field default of 4096 will be used
# Set context_window defaults if not provided
if values.get("context_window") is None: