diff --git a/src/agent/subagents/builtin/memory.md b/src/agent/subagents/builtin/memory.md index 7acd4b8..68baf99 100644 --- a/src/agent/subagents/builtin/memory.md +++ b/src/agent/subagents/builtin/memory.md @@ -1,7 +1,7 @@ --- name: memory description: Decompose and reorganize memory files into focused, single-purpose files using `/` naming -tools: Read, Edit, Write, Glob, Grep, Bash +tools: Read, Edit, Write, Glob, Grep, Bash, TaskOutput model: sonnet memoryBlocks: none permissionMode: bypassPermissions @@ -68,20 +68,48 @@ Do **not** edit: ## Operating Procedure +### Phase 0: Setup + +The memory directory is at: +`~/.letta/agents/$LETTA_AGENT_ID/memory/` + +```bash +MEMORY_DIR=~/.letta/agents/$LETTA_AGENT_ID/memory +WORKTREE_DIR=~/.letta/agents/$LETTA_AGENT_ID/memory-worktrees +``` + +The memory directory should already be a git repo +(initialized when MemFS was enabled). If it's not, or +if git is unavailable, report the issue and exit without +making changes. + +**Create worktree:** + +```bash +BRANCH="defrag-$(date +%s)" +mkdir -p "$WORKTREE_DIR" +cd "$MEMORY_DIR" +git worktree add "$WORKTREE_DIR/$BRANCH" -b "$BRANCH" +``` + +All subsequent file operations target the worktree: +`$WORKTREE_DIR/$BRANCH/system/` (not the main memory dir). + ### Step 1: Inventory First, list what files are available: ```bash -ls ~/.letta/agents/$LETTA_AGENT_ID/memory/system/ +WORK=$WORKTREE_DIR/$BRANCH/system +ls $WORK/ ``` Then read relevant memory files: ``` -Read({ file_path: "~/.letta/agents/$LETTA_AGENT_ID/memory/system/project.md" }) -Read({ file_path: "~/.letta/agents/$LETTA_AGENT_ID/memory/system/persona.md" }) -Read({ file_path: "~/.letta/agents/$LETTA_AGENT_ID/memory/system/human.md" }) +Read({ file_path: "$WORK/project.md" }) +Read({ file_path: "$WORK/persona.md" }) +Read({ file_path: "$WORK/human.md" }) ``` ### Step 2: Identify system-managed files (skip) @@ -149,6 +177,139 @@ For each edited file: before/after chars, delta, what was fixed #### 4) Before/after examples 2–4 examples showing redundancy removal, contradiction resolution, or structure improvements +### Phase 5: Merge, Push, and Clean Up (MANDATORY) + +Your defrag has two completion states: +- **Complete**: merged to main AND pushed to remote. +- **Partially complete**: merged to main, push failed. + Clean up the worktree, but report that local main is + ahead of remote and needs a push. + +The commit in the worktree is neither — it's an intermediate +step. Without at least a merge to main, your work is lost. + +**Step 5a: Commit in worktree** + +```bash +MEMORY_DIR=~/.letta/agents/$LETTA_AGENT_ID/memory +WORKTREE_DIR=~/.letta/agents/$LETTA_AGENT_ID/memory-worktrees +cd $WORKTREE_DIR/$BRANCH +git add -A +``` + +Check `git status` — if there are no changes to commit, +skip straight to Step 5d (cleanup). Report "no updates +needed" in your output. + +If there are changes, commit: + +```bash +git commit -m "chore(defrag): " +``` + +**Step 5b: Pull + merge to main** + +```bash +cd $MEMORY_DIR +``` + +First, check that main is in a clean state (`git status`). +If a merge or rebase is in progress (lock file, dirty +index), wait and retry up to 3 times with backoff (sleep 2, +5, 10 seconds). Never delete `.git/index.lock` manually. +If still busy after retries, go to Error Handling. + +Pull from remote: + +```bash +git pull --ff-only +``` + +If `--ff-only` fails (remote has diverged), fall back: + +```bash +git pull --rebase +``` + +If rebase has conflicts, resolve them autonomously to +stabilize local `main` against remote `main` first. In this +step, prefer **remote main** content for conflicting files, +then run `git rebase --continue`. + +Now merge the defrag branch: + +```bash +git merge $BRANCH --no-edit +``` + +If the merge has conflicts, resolve by preferring defrag +branch/worktree content for memory files, stage the resolved +files, and complete with `git commit --no-edit`. + +If you cannot resolve conflicts after 2 attempts, go to +Error Handling. + +**Step 5c: Push to remote** + +```bash +git push +``` + +If push fails, retry once. If it still fails, report that +local main is ahead of remote and needs a push. Proceed to +cleanup — the merge succeeded and data is safe on local +main. + +**Step 5d: Clean up worktree and branch** + +Only clean up when merge to main completed (success or +partially complete): + +```bash +git worktree remove $WORKTREE_DIR/$BRANCH +git branch -d $BRANCH +``` + +**Step 5e: Verify** + +```bash +git status +git log --oneline -3 +``` + +Confirm main is clean and your defrag commit is visible +in the log. + +## Error Handling + +If anything goes wrong at any phase: + +1. Stabilize main first (abort in-progress operations): + ```bash + cd $MEMORY_DIR + git merge --abort 2>/dev/null + git rebase --abort 2>/dev/null + ``` + +2. Do NOT clean up the worktree or branch on failure — + preserve them for debugging and manual recovery. + +3. Report clearly in your output: + - What failed and the error message + - Worktree path: `$WORKTREE_DIR/$BRANCH` + - Branch name: `$BRANCH` + - Whether main has uncommitted/dirty state + - Concrete resume commands, e.g.: + ```bash + cd ~/.letta/agents/$LETTA_AGENT_ID/memory + git merge --no-edit + git push + git worktree remove ../memory-worktrees/ + git branch -d + ``` + +4. Do NOT leave uncommitted changes on main. + ## Final Checklist Before submitting, confirm: @@ -158,6 +319,7 @@ Before submitting, confirm: - [ ] **Hierarchy is 2–3 levels deep** - [ ] **No file exceeds ~40 lines** - [ ] **Each file has one concept** +- [ ] **Changes committed, merged to main, and pushed** **If you have fewer than 15 files, you haven't split enough.** diff --git a/src/skills/builtin/defragmenting-memory/SKILL.md b/src/skills/builtin/defragmenting-memory/SKILL.md deleted file mode 100644 index c99d609..0000000 --- a/src/skills/builtin/defragmenting-memory/SKILL.md +++ /dev/null @@ -1,261 +0,0 @@ ---- -name: defragmenting-memory -description: Decomposes and reorganizes agent memory blocks into focused, single-purpose components. Use when memory has large multi-topic blocks, redundancy, or poor organization. ---- - -# Memory Defragmentation Skill - -> **Requires Memory Filesystem (memfs)** -> -> This skill works by directly editing memory files on disk. It requires the memory filesystem feature to be enabled. -> -> **To check:** Look for a `memory_filesystem` block in your system prompt. If it shows a tree structure starting with `/memory/` including a `system/` directory, memfs is enabled. -> -> **To enable:** Ask the user to run `/memfs enable`, then reload the CLI. - -This skill helps you maintain clean, well-organized memory blocks by spawning a subagent to decompose and reorganize memory files in-place. - -The focus is on **decomposition**—splitting large, multi-purpose blocks into focused, single-purpose components—rather than consolidation. - -Memory files live at `~/.letta/agents/$LETTA_AGENT_ID/memory/` and are synced to API blocks automatically by **memfs sync** on CLI startup. - -## When to Use - -- Memory blocks have redundant information -- Memory lacks structure (walls of text) -- Memory contains contradictions -- Memory has grown stale or outdated -- After major project milestones -- Every 50-100 conversation turns - -## Workflow - -### Step 1: Commit Current State (Safety Net) - -The memory directory is a git repo. Commit the current state so you can rollback if needed: - -```bash -cd ~/.letta/agents/$LETTA_AGENT_ID/memory -git add -A -git commit -m "chore: pre-defrag snapshot" || echo "No changes to commit" -``` - -⚠️ **CRITICAL**: You MUST commit before proceeding. This is your rollback point. - -### Step 2: Spawn Subagent to Edit Memory Files - -The memory subagent works directly on the memfs `system/` directory. After it finishes, memfs sync will propagate changes to the API on next CLI startup. - -```typescript -Task({ - subagent_type: "memory", - run_in_background: true, - description: "Decompose and reorganize memory files", - prompt: `You are decomposing and reorganizing memory files in ~/.letta/agents/${LETTA_AGENT_ID}/memory/system/ to improve clarity and focus. - -These files ARE the agent's memory — they sync directly to API memory blocks via memfs. Changes you make here will be picked up automatically. - -## Directory Structure - -~/.letta/agents//memory/ -├── system/ ← Attached blocks (always loaded in system prompt) — EDIT THESE -├── notes.md ← Detached blocks at root level (on-demand) — can create here -├── archive/ ← Detached blocks can be nested too -└── .sync-state.json ← DO NOT EDIT (internal sync tracking) - -## Files to Skip (DO NOT edit) -- memory_filesystem.md (auto-generated tree view) -- .sync-state.json (internal) - -## What to Edit -- persona.md → Consider splitting into: persona/identity.md, persona/values.md, persona/approach.md -- project.md → Consider splitting into: project/overview.md, project/architecture.md, project/conventions.md, etc. -- human.md → Consider splitting into: human/identity.md, human/preferences.md, etc. -- Any other non-system blocks present - -## How Memfs File ↔ Block Mapping Works -- File path relative to memory root becomes the block label (system/ prefix for attached, root level for detached) -- Example: system/project/tooling/bun.md → block label "project/tooling/bun" -- New files you create will become new memory blocks on next sync -- Files you delete will cause the corresponding blocks to be deleted on next sync -- YAML frontmatter is supported for metadata (label, description, limit, read_only) - -## Evaluation Criteria - -1. **DECOMPOSITION** - Split large, multi-purpose blocks into focused, single-purpose components - - Example: A "persona" block mixing identity, values, AND approach should become persona/identity.md, persona/values.md, persona/approach.md - - Example: A "project" block with overview, architecture, conventions, and gotchas should split into project/overview.md, project/architecture.md, project/conventions.md, project/gotchas.md - - Goal: Each block should have ONE clear purpose described by its filename - - Use hierarchical / naming (e.g., project/tooling/bun.md, not project-tooling-bun.md) - -2. **STRUCTURE** - Organize content with clear markdown formatting - - Use headers (##, ###) for subsections - - Use bullet points for lists - - Make content scannable at a glance - -3. **CONCISENESS** - Remove redundancy and unnecessary detail - - Eliminate duplicate information across blocks - - Remove speculation ("probably", "maybe", "I think") - - Keep only what adds unique value - -4. **CLARITY** - Resolve contradictions and improve readability - - If blocks contradict, clarify or choose the better guidance - - Use plain language, avoid jargon - - Ensure each statement is concrete and actionable - -5. **ORGANIZATION** - Group related information logically - - Within each block, organize content from general to specific - - Order sections by importance - -## Workflow - -1. **Analyze** - Read each file and identify its purpose(s) - - If a block serves 2+ distinct purposes, it needs decomposition - - Flag blocks where subtopics could be their own focused blocks - -2. **Decompose** - Split multi-purpose blocks into specialized files - - Create new files using hierarchical paths (e.g., project/tooling/bun.md) - - Ensure each new block has ONE primary purpose - -3. **Clean Up** - For remaining blocks (or new focused blocks): - - Add markdown structure with headers and bullets - - Remove redundancy - - Resolve contradictions - - Improve clarity - -4. **Delete** - Remove files only when appropriate - - After moving all content to new decomposed files - - Never delete a focused, single-purpose block - - Only delete if a block contains junk/irrelevant data with no value - -## Success Indicators -- No block tries to cover 2+ distinct topics -- Each block title clearly describes its single purpose -- Content within each block is focused and relevant to its title -- Well-organized with markdown structure -- Clear reduction in confusion/overlap across blocks - -Provide a detailed report including: -- Files created (new decomposed blocks) -- Files modified (what changed) -- Files deleted (if any, explain why) -- Before/after character counts -- Rationale for how decomposition improves the memory structure` -}) -``` - -The subagent will: -- Read files from `~/.letta/agents//memory/system/` (and root level for detached) -- Edit them to reorganize and decompose large blocks -- Create new hierarchically-named files (e.g., `project/overview.md`) -- Add clear structure with markdown formatting -- Delete source files after decomposing their content into focused children -- Provide a detailed report of changes - -After the subagent finishes, **memfs sync will automatically propagate changes** to API blocks on the next CLI startup. No manual restore step is needed. - -### Step 3: Commit Changes - -After the subagent finishes, commit the changes: - -```bash -cd ~/.letta/agents/$LETTA_AGENT_ID/memory -git add -A -git commit -m "chore: defragment memory blocks" -git push -``` - -## Example Complete Flow - -```typescript -// Step 1: Commit current state (MANDATORY) -Bash({ - command: "cd ~/.letta/agents/$LETTA_AGENT_ID/memory && git add -A && git commit -m 'chore: pre-defrag snapshot' || echo 'No changes'", - description: "Commit current memory state as rollback point" -}) - -// Step 2: Spawn subagent to decompose and reorganize (runs async in background) -Task({ - subagent_type: "memory", - run_in_background: true, - description: "Decompose and reorganize memory files", - prompt: "Decompose and reorganize memory files in ~/.letta/agents/$LETTA_AGENT_ID/memory/system/. These files sync directly to API blocks via memfs. Be aggressive about splitting large multi-section blocks into many smaller, single-purpose blocks using hierarchical / naming. Skip memory_filesystem.md and .sync-state.json. Structure with markdown headers and bullets. Remove redundancy and speculation. Resolve contradictions. Organize logically. Each block should have ONE clear purpose. Report files created, modified, deleted, before/after character counts, and rationale for changes." -}) - -// Step 3: After subagent completes, commit and push -// Check progress with /task , restart CLI to sync when done -``` - -## Rollback - -If something goes wrong, use git to revert: - -```bash -cd ~/.letta/agents/$LETTA_AGENT_ID/memory - -# Option 1: Reset to last commit (discard all uncommitted changes) -git reset --hard HEAD~1 - -# Option 2: View history and reset to specific commit -git log --oneline -5 -git reset --hard - -# Push the rollback -git push --force -``` - -On next CLI startup, memfs sync will detect the changes and update API blocks accordingly. - -## What the Subagent Does - -The subagent focuses on decomposing and cleaning up files. It has full tool access (including Bash) and: -- Discovers `.md` files in `~/.letta/agents//memory/system/` (via Glob or Bash) -- Reads and examines each file's content -- Identifies multi-purpose blocks that serve 2+ distinct purposes -- Splits large blocks into focused, single-purpose components with hierarchical naming -- Modifies/creates .md files for decomposed blocks -- Improves structure with headers and bullet points -- Removes redundancy and speculation across blocks -- Resolves contradictions with clear, concrete guidance -- Organizes content logically (general to specific, by importance) -- Provides detailed before/after reports including decomposition rationale -- Does NOT run any git commands (parent agent handles that) - -The focus is on decomposition—breaking apart large monolithic blocks into focused, specialized components rather than consolidating them together. - -## Tips - -**What to clean up:** -- Duplicate information (consolidate into one well-organized section) -- Walls of text without structure (add headers and bullets) -- Contradictions (resolve by clarifying or choosing the better guidance) -- Speculation ("probably", "maybe" - make it concrete or remove) -- Transient details that won't matter in a week - -**Decomposition Strategy:** -- Split blocks that serve 2+ distinct purposes into focused components -- Use hierarchical `/` naming: `project/tooling/bun.md`, not `project-bun.md` -- Create parent index files that reference children -- Example: A "persona" mixing identity + values + approach → split into `persona/identity.md`, `persona/values.md`, `persona/approach.md` -- Example: A "project" with overview + architecture + conventions → split into `project/overview.md`, `project/architecture.md`, `project/conventions.md` -- Add clear headers and bullet points for scannability -- Group similar information together within focused blocks - -**When to DELETE a file:** -- Only delete if file contains junk/irrelevant data with no project value -- Delete source files after fully decomposing content into child files -- Don't delete unique information just to reduce file count - -**What to preserve:** -- User preferences (sacred - never delete) -- Project conventions discovered through experience -- Important context for future sessions -- Learnings from past mistakes -- Any information that has unique value - -**Good memory structure:** -- Use markdown headers (##, ###) -- Organize with bullet points -- Keep related information together -- Make it scannable at a glance -- Use `/` hierarchy for discoverability