fix: prevent db connection pool exhaustion in file status checks (#6620)
Problem: When listing files with status checking enabled, the code used asyncio.gather to check and update status for all files concurrently. Each status check may update the file in the database (e.g., for timeouts or embedding completion), leading to N concurrent database connections. Example: Listing 100 files with status checking creates 100 simultaneous database update operations, exhausting the connection pool. Root cause: asyncio.gather(*[check_and_update_file_status(f) for f in files]) processes all files concurrently, each potentially creating DB updates. Solution: Check and update file status sequentially instead of concurrently. While this is slower, it prevents database connection pool exhaustion when listing many files. Changes: - apps/core/letta/services/file_manager.py: - Replaced asyncio.gather with sequential for loop - Added explanatory comment about db pool exhaustion prevention Impact: With 100 files: - Before: Up to 100 concurrent DB connections (pool exhaustion) - After: 1 DB connection at a time (no pool exhaustion) Note: This follows the same pattern as PR #6617 and #6619 which fixed similar issues in file attachment and multi-agent tool execution.
This commit is contained in:
@@ -449,11 +449,15 @@ class FileManager:
|
||||
*[file.to_pydantic_async(include_content=include_content, strip_directory_prefix=strip_directory_prefix) for file in files]
|
||||
)
|
||||
|
||||
# if status checking is enabled, check all files concurrently
|
||||
# if status checking is enabled, check all files sequentially to avoid db pool exhaustion
|
||||
# Each status check may update the file in the database, so concurrent checks with many
|
||||
# files can create too many simultaneous database connections
|
||||
if check_status_updates:
|
||||
file_metadatas = await asyncio.gather(
|
||||
*[self.check_and_update_file_status(file_metadata, actor) for file_metadata in file_metadatas]
|
||||
)
|
||||
updated_file_metadatas = []
|
||||
for file_metadata in file_metadatas:
|
||||
updated_metadata = await self.check_and_update_file_status(file_metadata, actor)
|
||||
updated_file_metadatas.append(updated_metadata)
|
||||
file_metadatas = updated_file_metadatas
|
||||
|
||||
return file_metadatas
|
||||
|
||||
|
||||
Reference in New Issue
Block a user