feat: parallel tool calling for openai non streaming [LET-4593] (#5773)

* first hack

* clean up

* first implementation working

* revert package-lock

* remove openai test

* error throw

* typo

* Update integration_test_send_message_v2.py

* Update integration_test_send_message_v2.py

* refine test

* Only make changes for openai non streaming

* Add tests

---------

Co-authored-by: Ari Webb <ari@letta.com>
Co-authored-by: Matt Zhou <mattzh1314@gmail.com>
This commit is contained in:
Ari Webb
2025-10-30 15:52:11 -07:00
committed by Caren Thomas
parent 7cc9471f40
commit 48cc73175b
6 changed files with 84 additions and 24 deletions

View File

@@ -78,6 +78,13 @@ class SimpleLLMStreamAdapter(LettaLLMStreamAdapter):
use_responses = "input" in request_data and "messages" not in request_data
# No support for Responses API proxy
is_proxy = self.llm_config.provider_name == "lmstudio_openai"
# Use parallel tool calling interface if enabled in config
use_parallel = self.llm_config.parallel_tool_calls and tools and not use_responses and not is_proxy
# TODO: Temp, remove
if use_parallel:
raise RuntimeError("Parallel tool calling not supported for OpenAI streaming")
if use_responses and not is_proxy:
self.interface = SimpleOpenAIResponsesStreamingInterface(
is_openai_proxy=False,