feat: memory filesystem sync (#905)
Co-authored-by: Letta <noreply@letta.com>
This commit is contained in:
@@ -9,81 +9,150 @@ Agents with the `git-memory-enabled` tag have their memory blocks stored in git
|
||||
|
||||
**Features:**
|
||||
- Stored in cloud (GCS)
|
||||
- Accessible via `https://api.letta.com/v1/git/<agent-id>/state.git`
|
||||
- Bidirectional sync: API ↔ Git (webhook-triggered, ~2-3s delay)
|
||||
- Accessible via `$LETTA_BASE_URL/v1/git/<agent-id>/state.git`
|
||||
- Bidirectional sync: API <-> Git (webhook-triggered, ~2-3s delay)
|
||||
- Structure: `memory/system/*.md` for system blocks
|
||||
|
||||
## Setup Authentication (One-Time)
|
||||
## What the CLI Harness Does Automatically
|
||||
|
||||
Configure git credential helper to authenticate with Letta API:
|
||||
When memfs is enabled, the Letta Code CLI automatically:
|
||||
|
||||
1. Adds the `git-memory-enabled` tag to the agent (triggers backend to create the git repo)
|
||||
2. Clones the repo into `~/.letta/agents/<agent-id>/memory/` (git root is the memory directory)
|
||||
3. Configures a **local** credential helper in `memory/.git/config` (so `git push`/`git pull` work without auth ceremony)
|
||||
4. Installs a **pre-commit hook** that validates frontmatter before each commit (see below)
|
||||
5. On subsequent startups: pulls latest changes, reconfigures credentials and hook (self-healing)
|
||||
6. During sessions: periodically checks `git status` and reminds you (the agent) to commit/push if dirty
|
||||
|
||||
If any of these steps fail, you can replicate them manually using the sections below.
|
||||
|
||||
## Authentication
|
||||
|
||||
The harness configures a per-repo credential helper during clone. To verify or reconfigure:
|
||||
|
||||
```bash
|
||||
export LETTA_API_KEY="your-api-key"
|
||||
cd ~/.letta/agents/<agent-id>/memory
|
||||
|
||||
git config --global credential.https://api.letta.com.helper '!f() {
|
||||
echo "username=letta";
|
||||
echo "password=$LETTA_API_KEY";
|
||||
}; f'
|
||||
# Check if configured
|
||||
git config --get credential.$LETTA_BASE_URL.helper
|
||||
|
||||
# Reconfigure (e.g. after API key rotation)
|
||||
git config credential.$LETTA_BASE_URL.helper \
|
||||
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
|
||||
```
|
||||
|
||||
After setup, git operations will automatically use your API key for authentication.
|
||||
For cloning a *different* agent's repo (e.g. during memory migration), set up a global helper:
|
||||
|
||||
```bash
|
||||
git config --global credential.$LETTA_BASE_URL.helper \
|
||||
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
|
||||
```
|
||||
|
||||
## Pre-Commit Hook (Frontmatter Validation)
|
||||
|
||||
The harness installs a git pre-commit hook that validates `.md` files under `memory/` before each commit. This prevents pushes that the server would reject.
|
||||
|
||||
**Rules:**
|
||||
- Every `.md` file must have YAML frontmatter (`---` header and closing `---`)
|
||||
- Required fields: `description` (non-empty string), `limit` (positive integer)
|
||||
- `read_only` is a **protected field**: you (the agent) cannot add, remove, or change it. Files with `read_only: true` cannot be modified at all. Only the server/user sets this field.
|
||||
- Unknown frontmatter keys are rejected
|
||||
|
||||
**Valid file format:**
|
||||
```markdown
|
||||
---
|
||||
description: What this block contains
|
||||
limit: 20000
|
||||
---
|
||||
|
||||
Block content goes here.
|
||||
```
|
||||
|
||||
If the hook rejects a commit, read the error message — it tells you exactly which file and which rule was violated. Fix the file and retry.
|
||||
|
||||
## Clone Agent Memory
|
||||
|
||||
```bash
|
||||
# Clone agent's memory repo
|
||||
git clone "https://api.letta.com/v1/git/<agent-id>/state.git" ~/my-agent-memory
|
||||
git clone "$LETTA_BASE_URL/v1/git/<agent-id>/state.git" ~/my-agent-memory
|
||||
|
||||
# View memory blocks
|
||||
ls ~/my-agent-memory/memory/system/
|
||||
cat ~/my-agent-memory/memory/system/human.md
|
||||
```
|
||||
|
||||
## Enabling Git Memory (Manual)
|
||||
|
||||
If the harness `/memfs enable` failed, you can replicate it:
|
||||
|
||||
```bash
|
||||
AGENT_ID="<your-agent-id>"
|
||||
AGENT_DIR=~/.letta/agents/$AGENT_ID
|
||||
MEMORY_REPO_DIR="$AGENT_DIR/memory"
|
||||
|
||||
# 1. Add git-memory-enabled tag (IMPORTANT: preserve existing tags!)
|
||||
# First GET the agent to read current tags, then PATCH with the new tag appended.
|
||||
# The harness code does: tags = [...existingTags, "git-memory-enabled"]
|
||||
curl -X PATCH "$LETTA_BASE_URL/v1/agents/$AGENT_ID" \
|
||||
-H "Authorization: Bearer $LETTA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"tags": ["origin:letta-code", "git-memory-enabled"]}'
|
||||
|
||||
# 2. Clone the repo into memory/
|
||||
mkdir -p "$MEMORY_REPO_DIR"
|
||||
git clone "$LETTA_BASE_URL/v1/git/$AGENT_ID/state.git" "$MEMORY_REPO_DIR"
|
||||
|
||||
# 3. Configure local credential helper
|
||||
cd "$MEMORY_REPO_DIR"
|
||||
git config credential.$LETTA_BASE_URL.helper \
|
||||
'!f() { echo "username=letta"; echo "password=$LETTA_API_KEY"; }; f'
|
||||
```
|
||||
|
||||
## Bidirectional Sync
|
||||
|
||||
### API Edit → Git Pull
|
||||
### API Edit -> Git Pull
|
||||
|
||||
```bash
|
||||
# 1. Edit block via API (or use memory tools)
|
||||
# 2. Pull to get changes (webhook creates commit automatically)
|
||||
cd ~/my-agent-memory
|
||||
git pull --ff-only
|
||||
cd ~/.letta/agents/<agent-id>/memory
|
||||
git pull
|
||||
```
|
||||
|
||||
Changes made via the API are automatically committed to git within 2-3 seconds.
|
||||
|
||||
### Git Push → API Update
|
||||
### Git Push -> API Update
|
||||
|
||||
```bash
|
||||
cd ~/.letta/agents/<agent-id>/memory
|
||||
|
||||
# 1. Edit files locally
|
||||
echo "Updated info" > memory/system/human.md
|
||||
echo "Updated info" > system/human.md
|
||||
|
||||
# 2. Commit and push
|
||||
git add memory/system/human.md
|
||||
git commit -m "update human block"
|
||||
git add system/human.md
|
||||
git commit -m "fix: update human block"
|
||||
git push
|
||||
|
||||
# 3. API automatically reflects changes (webhook-triggered, ~2-3s delay)
|
||||
```
|
||||
|
||||
Changes pushed to git are automatically synced to the API within 2-3 seconds.
|
||||
|
||||
## Conflict Resolution
|
||||
|
||||
When both API and git have diverged:
|
||||
|
||||
```bash
|
||||
cd ~/my-agent-memory
|
||||
cd ~/.letta/agents/<agent-id>/memory
|
||||
|
||||
# 1. Try to push (will be rejected)
|
||||
git push # → "fetch first"
|
||||
git push # -> "fetch first"
|
||||
|
||||
# 2. Pull to create merge conflict
|
||||
git pull --no-rebase
|
||||
# → CONFLICT in memory/system/human.md
|
||||
# -> CONFLICT in system/human.md
|
||||
|
||||
# 3. View conflict markers
|
||||
cat memory/system/human.md
|
||||
cat system/human.md
|
||||
# <<<<<<< HEAD
|
||||
# your local changes
|
||||
# =======
|
||||
@@ -91,13 +160,13 @@ cat memory/system/human.md
|
||||
# >>>>>>> <commit>
|
||||
|
||||
# 4. Resolve
|
||||
echo "final resolved content" > memory/system/human.md
|
||||
git add memory/system/human.md
|
||||
git commit -m "resolved conflict"
|
||||
echo "final resolved content" > system/human.md
|
||||
git add system/human.md
|
||||
git commit -m "fix: resolved conflict in human block"
|
||||
|
||||
# 5. Push resolution
|
||||
git push
|
||||
# → API automatically updates with resolved content
|
||||
# -> API automatically updates with resolved content
|
||||
```
|
||||
|
||||
## Block Management
|
||||
@@ -106,36 +175,37 @@ git push
|
||||
|
||||
```bash
|
||||
# Create file in system/ directory (automatically attached to agent)
|
||||
echo "My new block content" > memory/system/new-block.md
|
||||
git add memory/system/new-block.md
|
||||
git commit -m "add new block"
|
||||
echo "My new block content" > system/new-block.md
|
||||
git add system/new-block.md
|
||||
git commit -m "feat: add new block"
|
||||
git push
|
||||
# → Block automatically created and attached to agent
|
||||
# -> Block automatically created and attached to agent
|
||||
```
|
||||
|
||||
### Delete/Detach Block
|
||||
|
||||
```bash
|
||||
# Remove file from system/ directory
|
||||
git rm memory/system/persona.md
|
||||
git commit -m "remove persona block"
|
||||
git rm system/persona.md
|
||||
git commit -m "chore: remove persona block"
|
||||
git push
|
||||
# → Block automatically detached from agent
|
||||
# -> Block automatically detached from agent
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
repo/
|
||||
~/.letta/agents/<agent-id>/
|
||||
├── .letta/
|
||||
│ └── config.json # Repo metadata
|
||||
└── memory/
|
||||
└── system/ # System blocks (attached to agent)
|
||||
│ └── config.json # Agent metadata
|
||||
└── memory/ # Git repo root
|
||||
├── .git/ # Git repo data
|
||||
└── system/ # System blocks (attached to agent)
|
||||
├── human.md
|
||||
└── persona.md
|
||||
```
|
||||
|
||||
**System blocks** (`memory/system/`) are attached to the agent and appear in the agent's memory.
|
||||
**System blocks** (`memory/system/`) are attached to the agent and appear in the agent's system prompt.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -146,15 +216,19 @@ repo/
|
||||
## Troubleshooting
|
||||
|
||||
**Clone fails with "Authentication failed":**
|
||||
- Verify credential helper is set: `git config --global --get credential.https://api.letta.com.helper`
|
||||
- Verify API key is exported: `echo $LETTA_API_KEY`
|
||||
- Reconfigure: Run setup command again with your API key
|
||||
- Check credential helper: `git config --get credential.$LETTA_BASE_URL.helper`
|
||||
- Reconfigure: see Authentication section above
|
||||
- Verify the endpoint is reachable: `curl -u letta:$LETTA_API_KEY $LETTA_BASE_URL/v1/git/<agent-id>/state.git/info/refs?service=git-upload-pack`
|
||||
|
||||
**Push/pull doesn't update API:**
|
||||
- Wait 2-3 seconds for webhook processing
|
||||
- Verify agent has `git-memory-enabled` tag
|
||||
- Check if you have write access to the agent
|
||||
|
||||
**Harness setup failed (no .git/ after /memfs enable):**
|
||||
- Check debug logs (`LETTA_DEBUG=1`)
|
||||
- Follow "Enabling Git Memory (Manual)" steps above
|
||||
|
||||
**Can't see changes immediately:**
|
||||
- Bidirectional sync has a 2-3 second delay for webhook processing
|
||||
- Use `git pull` to get latest API changes
|
||||
|
||||
Reference in New Issue
Block a user