Problem: When executing a tool that sends messages to many agents matching
tags, the code used asyncio.gather to process all agents concurrently. Each
agent processing creates database operations (run creation, message storage),
leading to N concurrent database connections.
Example: If 100 agents match the tags, 100 simultaneous database connections
are created, exhausting the connection pool and causing errors.
Root cause: asyncio.gather(*[_process_agent(...) for agent in agents])
creates all coroutines and runs them concurrently, each opening a DB session.
Solution: Process agents sequentially instead of concurrently. While this is
slower, it prevents database connection pool exhaustion. The operation is
still async, so it won't block the event loop.
Changes:
- apps/core/letta/services/tool_executor/multi_agent_tool_executor.py:
- Replaced asyncio.gather with sequential for loop
- Added explanatory comment about why sequential processing is needed
Impact: With 100 matching agents:
- Before: 100 concurrent DB connections (pool exhaustion)
- After: 1 DB connection at a time (no pool exhaustion)
Note: This follows the same pattern as PR #6617 which fixed a similar issue
in file attachment operations.
15cede7281