feat: git smart HTTP for agent memory repos (#9257)

* feat(core): add git-backed memory repos and block manager

Introduce a GCS-backed git repository per agent as the source of truth for core
memory blocks. Add a GitEnabledBlockManager that writes block updates to git and
syncs values back into Postgres as a cache.

Default newly-created memory repos to the `main` branch.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* feat(core): serve memory repos over git smart HTTP

Run dulwich's WSGI HTTPGitApplication on a local sidecar port and proxy
/v1/git/* through FastAPI to support git clone/fetch/push directly against
GCS-backed memory repos.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): create memory repos on demand and stabilize git HTTP

- Ensure MemoryRepoManager creates the git repo on first write (instead of 500ing)
  and avoids rewriting history by only auto-creating on FileNotFoundError.
- Simplify dulwich-thread async execution and auto-create empty repos on first
  git clone.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): make dulwich optional for CI installs

Guard dulwich imports in the git smart HTTP router so the core server can boot
(and CI tests can run) without installing the memory-repo extra.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): guard git HTTP WSGI init when dulwich missing

Avoid instantiating dulwich's HTTPGitApplication at import time when dulwich
isn't installed (common in CI installs).

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): avoid masking send_message errors in finally

Initialize `result` before the agent loop so error paths (e.g. approval
validation) don't raise UnboundLocalError in the run-tracking finally block.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): stop event loop watchdog on FastAPI shutdown

Ensure the EventLoopWatchdog thread is stopped during FastAPI lifespan
shutdown to avoid daemon threads logging during interpreter teardown (seen in CI
unit tests).

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* chore(core): remove send_*_message_to_agent from SyncServer

Drop send_message_to_agent and send_group_message_to_agent from SyncServer and
route internal fire-and-forget messaging through send_messages helpers instead.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): backfill git memory repo when tag added

When an agent is updated to include the git-memory-enabled tag, ensure the
git-backed memory repo is created and initialized from the agent's current
blocks. Also support configuring the memory repo object store via
LETTA_OBJECT_STORE_URI.

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): preserve block tags on git-enabled updates

When updating a block for a git-memory-enabled agent, keep block tags in sync
with PostgreSQL (tags are not currently stored in the git repo).

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* chore(core): remove git-state legacy shims

- Rename optional dependency extra from memory-repo to git-state
- Drop legacy object-store env aliases and unused region config
- Simplify memory repo metadata to a single canonical format
- Remove unused repo-cache invalidation helper

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

* fix(core): keep PR scope for git-backed blocks

- Revert unrelated change in fire-and-forget multi-agent send helper
- Route agent block updates-by-label through injected block manager only when needed

👾 Generated with [Letta Code](https://letta.com)

Co-Authored-By: Letta <noreply@letta.com>

---------

Co-authored-by: Letta <noreply@letta.com>

letta/server/rest_api/app.py

@@ -192,6 +192,17 @@ async def lifespan(app_: FastAPI):
     logger.info(f"[Worker {worker_id}] Starting scheduler with leader election")
     global server
     await server.init_async(init_with_default_org_and_user=not settings.no_default_actor)
+
+    # Set server instance for git HTTP endpoints and start dulwich sidecar
+    try:
+        from letta.server.rest_api.routers.v1.git_http import set_server_instance, start_dulwich_server
+
+        set_server_instance(server)
+        start_dulwich_server()
+        logger.info(f"[Worker {worker_id}] Git HTTP server instance set (dulwich sidecar started)")
+    except Exception as e:
+        logger.warning(f"[Worker {worker_id}] Failed to start git HTTP sidecar: {e}")
+
     try:
         await start_scheduler_with_leader_election(server)
         logger.info(f"[Worker {worker_id}] Scheduler initialization completed")
@@ -203,6 +214,15 @@ async def lifespan(app_: FastAPI):
     # Cleanup on shutdown
     logger.info(f"[Worker {worker_id}] Starting lifespan shutdown")
 
+    # Stop watchdog thread (important for clean test/worker shutdown)
+    try:
+        from letta.monitoring.event_loop_watchdog import stop_watchdog
+
+        stop_watchdog()
+        logger.info(f"[Worker {worker_id}] Event loop watchdog stopped")
+    except Exception as e:
+        logger.warning(f"[Worker {worker_id}] Failed to stop watchdog: {e}")
+
     try:
         from letta.jobs.scheduler import shutdown_scheduler_and_release_lock
 
@@ -221,17 +241,6 @@ async def lifespan(app_: FastAPI):
         except Exception as e:
             logger.warning(f"[Worker {worker_id}] SQLAlchemy instrumentation shutdown failed: {e}")
 
-    # Shutdown LLM raw trace writer (closes ClickHouse connection)
-    if settings.store_llm_traces:
-        try:
-            from letta.services.llm_trace_writer import get_llm_trace_writer
-
-            writer = get_llm_trace_writer()
-            await writer.shutdown_async()
-            logger.info(f"[Worker {worker_id}] LLM raw trace writer shutdown completed")
-        except Exception as e:
-            logger.warning(f"[Worker {worker_id}] LLM raw trace writer shutdown failed: {e}")
-
     logger.info(f"[Worker {worker_id}] Lifespan shutdown completed")
 
 
@@ -715,6 +724,8 @@ def create_application() -> "FastAPI":
     # /api/auth endpoints
     app.include_router(setup_auth_router(server, interface, random_password), prefix=API_PREFIX)
 
+    # Git smart HTTP is served by a dulwich sidecar and proxied by the /v1/git router.
+
     # / static files
     mount_static_files(app)