feat: memory filesystem sync (#905)

Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
Charles Packer
2026-02-10 18:06:05 -08:00
committed by GitHub
parent eaa813ddb9
commit d1a6eeb40a
13 changed files with 1085 additions and 3079 deletions

View File

@@ -9,81 +9,150 @@ Agents with the `git-memory-enabled` tag have their memory blocks stored in git
**Features:**
- Stored in cloud (GCS)
- Accessible via `https://api.letta.com/v1/git/<agent-id>/state.git`
- Bidirectional sync: API Git (webhook-triggered, ~2-3s delay)
- Accessible via `$LETTA_BASE_URL/v1/git/<agent-id>/state.git`
- Bidirectional sync: API <-> Git (webhook-triggered, ~2-3s delay)
- Structure: `memory/system/*.md` for system blocks
## Setup Authentication (One-Time)
## What the CLI Harness Does Automatically
Configure git credential helper to authenticate with Letta API:
When memfs is enabled, the Letta Code CLI automatically:
1. Adds the `git-memory-enabled` tag to the agent (triggers backend to create the git repo)
2. Clones the repo into `~/.letta/agents/<agent-id>/memory/` (git root is the memory directory)
3. Configures a **local** credential helper in `memory/.git/config` (so `git push`/`git pull` work without auth ceremony)
4. Installs a **pre-commit hook** that validates frontmatter before each commit (see below)
5. On subsequent startups: pulls latest changes, reconfigures credentials and hook (self-healing)
6. During sessions: periodically checks `git status` and reminds you (the agent) to commit/push if dirty
If any of these steps fail, you can replicate them manually using the sections below.
## Authentication
The harness configures a per-repo credential helper during clone. To verify or reconfigure:
```bash
export LETTA_API_KEY="your-api-key"
cd ~/.letta/agents/<agent-id>/memory
git config --global credential.https://api.letta.com.helper '!f() {
echo "username=letta";
echo "password=$LETTA_API_KEY";
}; f'
# Check if configured
git config --get credential.$LETTA_BASE_URL.helper
# Reconfigure (e.g. after API key rotation)
git config credential.$LETTA_BASE_URL.helper \
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
```
After setup, git operations will automatically use your API key for authentication.
For cloning a *different* agent's repo (e.g. during memory migration), set up a global helper:
```bash
git config --global credential.$LETTA_BASE_URL.helper \
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
```
## Pre-Commit Hook (Frontmatter Validation)
The harness installs a git pre-commit hook that validates `.md` files under `memory/` before each commit. This prevents pushes that the server would reject.
**Rules:**
- Every `.md` file must have YAML frontmatter (`---` header and closing `---`)
- Required fields: `description` (non-empty string), `limit` (positive integer)
- `read_only` is a **protected field**: you (the agent) cannot add, remove, or change it. Files with `read_only: true` cannot be modified at all. Only the server/user sets this field.
- Unknown frontmatter keys are rejected
**Valid file format:**
```markdown
---
description: What this block contains
limit: 20000
---
Block content goes here.
```
If the hook rejects a commit, read the error message — it tells you exactly which file and which rule was violated. Fix the file and retry.
## Clone Agent Memory
```bash
# Clone agent's memory repo
git clone "https://api.letta.com/v1/git/<agent-id>/state.git" ~/my-agent-memory
git clone "$LETTA_BASE_URL/v1/git/<agent-id>/state.git" ~/my-agent-memory
# View memory blocks
ls ~/my-agent-memory/memory/system/
cat ~/my-agent-memory/memory/system/human.md
```
## Enabling Git Memory (Manual)
If the harness `/memfs enable` failed, you can replicate it:
```bash
AGENT_ID="<your-agent-id>"
AGENT_DIR=~/.letta/agents/$AGENT_ID
MEMORY_REPO_DIR="$AGENT_DIR/memory"
# 1. Add git-memory-enabled tag (IMPORTANT: preserve existing tags!)
# First GET the agent to read current tags, then PATCH with the new tag appended.
# The harness code does: tags = [...existingTags, "git-memory-enabled"]
curl -X PATCH "$LETTA_BASE_URL/v1/agents/$AGENT_ID" \
-H "Authorization: Bearer $LETTA_API_KEY" \
-H "Content-Type: application/json" \
-d '{"tags": ["origin:letta-code", "git-memory-enabled"]}'
# 2. Clone the repo into memory/
mkdir -p "$MEMORY_REPO_DIR"
git clone "$LETTA_BASE_URL/v1/git/$AGENT_ID/state.git" "$MEMORY_REPO_DIR"
# 3. Configure local credential helper
cd "$MEMORY_REPO_DIR"
git config credential.$LETTA_BASE_URL.helper \
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
```
## Bidirectional Sync
### API Edit Git Pull
### API Edit -> Git Pull
```bash
# 1. Edit block via API (or use memory tools)
# 2. Pull to get changes (webhook creates commit automatically)
cd ~/my-agent-memory
git pull --ff-only
cd ~/.letta/agents/<agent-id>/memory
git pull
```
Changes made via the API are automatically committed to git within 2-3 seconds.
### Git Push API Update
### Git Push -> API Update
```bash
cd ~/.letta/agents/<agent-id>/memory
# 1. Edit files locally
echo "Updated info" > memory/system/human.md
echo "Updated info" > system/human.md
# 2. Commit and push
git add memory/system/human.md
git commit -m "update human block"
git add system/human.md
git commit -m "fix: update human block"
git push
# 3. API automatically reflects changes (webhook-triggered, ~2-3s delay)
```
Changes pushed to git are automatically synced to the API within 2-3 seconds.
## Conflict Resolution
When both API and git have diverged:
```bash
cd ~/my-agent-memory
cd ~/.letta/agents/<agent-id>/memory
# 1. Try to push (will be rejected)
git push # "fetch first"
git push # -> "fetch first"
# 2. Pull to create merge conflict
git pull --no-rebase
# CONFLICT in memory/system/human.md
# -> CONFLICT in system/human.md
# 3. View conflict markers
cat memory/system/human.md
cat system/human.md
# <<<<<<< HEAD
# your local changes
# =======
@@ -91,13 +160,13 @@ cat memory/system/human.md
# >>>>>>> <commit>
# 4. Resolve
echo "final resolved content" > memory/system/human.md
git add memory/system/human.md
git commit -m "resolved conflict"
echo "final resolved content" > system/human.md
git add system/human.md
git commit -m "fix: resolved conflict in human block"
# 5. Push resolution
git push
# API automatically updates with resolved content
# -> API automatically updates with resolved content
```
## Block Management
@@ -106,36 +175,37 @@ git push
```bash
# Create file in system/ directory (automatically attached to agent)
echo "My new block content" > memory/system/new-block.md
git add memory/system/new-block.md
git commit -m "add new block"
echo "My new block content" > system/new-block.md
git add system/new-block.md
git commit -m "feat: add new block"
git push
# Block automatically created and attached to agent
# -> Block automatically created and attached to agent
```
### Delete/Detach Block
```bash
# Remove file from system/ directory
git rm memory/system/persona.md
git commit -m "remove persona block"
git rm system/persona.md
git commit -m "chore: remove persona block"
git push
# Block automatically detached from agent
# -> Block automatically detached from agent
```
## Directory Structure
```
repo/
~/.letta/agents/<agent-id>/
├── .letta/
│ └── config.json # Repo metadata
└── memory/
── system/ # System blocks (attached to agent)
│ └── config.json # Agent metadata
└── memory/ # Git repo root
── .git/ # Git repo data
└── system/ # System blocks (attached to agent)
├── human.md
└── persona.md
```
**System blocks** (`memory/system/`) are attached to the agent and appear in the agent's memory.
**System blocks** (`memory/system/`) are attached to the agent and appear in the agent's system prompt.
## Requirements
@@ -146,15 +216,19 @@ repo/
## Troubleshooting
**Clone fails with "Authentication failed":**
- Verify credential helper is set: `git config --global --get credential.https://api.letta.com.helper`
- Verify API key is exported: `echo $LETTA_API_KEY`
- Reconfigure: Run setup command again with your API key
- Check credential helper: `git config --get credential.$LETTA_BASE_URL.helper`
- Reconfigure: see Authentication section above
- Verify the endpoint is reachable: `curl -u letta:$LETTA_API_KEY $LETTA_BASE_URL/v1/git/<agent-id>/state.git/info/refs?service=git-upload-pack`
**Push/pull doesn't update API:**
- Wait 2-3 seconds for webhook processing
- Verify agent has `git-memory-enabled` tag
- Check if you have write access to the agent
**Harness setup failed (no .git/ after /memfs enable):**
- Check debug logs (`LETTA_DEBUG=1`)
- Follow "Enabling Git Memory (Manual)" steps above
**Can't see changes immediately:**
- Bidirectional sync has a 2-3 second delay for webhook processing
- Use `git pull` to get latest API changes