# RedFlag Directory Structure Migration - SIMPLIFIED (v0.1.18 Only)

**Date**: 2025-12-16
**Status**: Simplified implementation ready
**Discovery**: Legacy v0.1.18 uses `/etc/aggregator` and `/var/lib/aggregator` - NO intermediate broken versions in the wild
**Version Jump**: v0.1.18 → v0.2.0 (breaking change)

---

## Migration Simplification Analysis

### **Critical Discovery**

**Legacy v0.1.18 paths:**
```
/etc/aggregator/config.json
/var/lib/aggregator/
```

**Current dev paths (unreleased):**
```
/etc/redflag/config.json
/var/lib/redflag-agent/ (broken, inconsistent)
/var/lib/redflag/ (inconsistent)
```

**Implication:** Only need to migrate from v0.1.18. Can ignore broken v0.1.19-v0.1.23 states.

**Timeline reduction:** 6h 50m → **3h 45m**

---

## Simplified Implementation Phases

### **Phase 1: Create Centralized Path Constants** (30 min)

**File:** `aggregator-agent/internal/constants/paths.go` (NEW)

```go
package constants

import (
    "runtime"
    "path/filepath"
)

// Base directories
const (
    LinuxBaseDir   = "/var/lib/redflag"
    WindowsBaseDir = "C:\\ProgramData\\RedFlag"
)

// Subdirectory structure
const (
    AgentDir         = "agent"
    ServerDir        = "server"
    CacheSubdir      = "cache"
    StateSubdir      = "state"
    MigrationSubdir  = "migration_backups"
)

// Config paths
const (
    LinuxConfigBase   = "/etc/redflag"
    WindowsConfigBase = "C:\\ProgramData\\RedFlag"
    ConfigFile        = "config.json"
)

// Log paths
const (
    LinuxLogBase = "/var/log/redflag"
)

// Legacy paths for migration
const (
    LegacyConfigPath = "/etc/aggregator/config.json"
    LegacyStatePath  = "/var/lib/aggregator"
)

// GetBaseDir returns platform-specific base directory
func GetBaseDir() string {
    if runtime.GOOS == "windows" {
        return WindowsBaseDir
    }
    return LinuxBaseDir
}

// GetAgentStateDir returns /var/lib/redflag/agent/state
func GetAgentStateDir() string {
    return filepath.Join(GetBaseDir(), AgentDir, StateSubdir)
}

// GetAgentCacheDir returns /var/lib/redflag/agent/cache
func GetAgentCacheDir() string {
    return filepath.Join(GetBaseDir(), AgentDir, CacheSubdir)
}

// GetMigrationBackupDir returns /var/lib/redflag/agent/migration_backups
func GetMigrationBackupDir() string {
    return filepath.Join(GetBaseDir(), AgentDir, MigrationSubdir)
}

// GetAgentConfigPath returns /etc/redflag/agent/config.json
func GetAgentConfigPath() string {
    if runtime.GOOS == "windows" {
        return filepath.Join(WindowsConfigBase, AgentDir, ConfigFile)
    }
    return filepath.Join(LinuxConfigBase, AgentDir, ConfigFile)
}

// GetAgentConfigDir returns /etc/redflag/agent
func GetAgentConfigDir() string {
    if runtime.GOOS == "windows" {
        return filepath.Join(WindowsConfigBase, AgentDir)
    }
    return filepath.Join(LinuxConfigBase, AgentDir)
}

// GetAgentLogDir returns /var/log/redflag/agent
func GetAgentLogDir() string {
    return filepath.Join(LinuxLogBase, AgentDir)
}

// GetLegacyAgentConfigPath returns legacy /etc/aggregator/config.json
func GetLegacyAgentConfigPath() string {
    return LegacyConfigPath
}

// GetLegacyAgentStatePath returns legacy /var/lib/aggregator
func GetLegacyAgentStatePath() string {
    return LegacyStatePath
}
```

### **Phase 2: Update Agent Main** (30 min)

**File:** `aggregator-agent/cmd/agent/main.go`

**Changes:**
```go
// 1. Remove these functions (lines 40-54):
//    - getConfigPath()
//    - getStatePath()

// 2. Add import:
import "github.com/Fimeg/RedFlag/aggregator-agent/internal/constants"

// 3. Update all references:
// Line 88: cfg.Save(getConfigPath()) → cfg.Save(constants.GetAgentConfigPath())
// Line 240: BackupPath: filepath.Join(getStatePath(), "migration_backups") → constants.GetMigrationBackupDir()
// Line 49: cfg.Save(getConfigPath()) → cfg.Save(constants.GetAgentConfigPath())

// 4. Remove: import "runtime" (no longer needed in main.go)

// 5. Remove: import "path/filepath" (unless used elsewhere)
```

### **Phase 3: Update Cache System** (15 min)

**File:** `aggregator-agent/internal/cache/local.go`

**Changes:**
```go
// 1. Add imports:
import (
    "path/filepath"
    "github.com/Fimeg/RedFlag/aggregator-agent/internal/constants"
)

// 2. Remove constant (line 26):
// OLD: const CacheDir = "/var/lib/redflag-agent"

// 3. Update cacheFile (line 29):
// OLD: const CacheFile = "last_scan.json"
// NEW: const cacheFile = "last_scan.json"  // unexported

// 4. Update GetCachePath():
// OLD:
func GetCachePath() string {
    return filepath.Join(CacheDir, CacheFile)
}

// NEW:
func GetCachePath() string {
    return filepath.Join(constants.GetAgentCacheDir(), cacheFile)
}

// 5. Update Load() and Save() to use constants.GetAgentCacheDir() instead of CacheDir
```

### **Phase 4: Update Migration Detection** (20 min)

**File:** `aggregator-agent/internal/migration/detection.go`

**Changes:**
```go
// 1. Add imports:
import (
    "path/filepath"
    "github.com/Fimeg/RedFlag/aggregator-agent/internal/constants"
)

// 2. Update NewFileDetectionConfig():
func NewFileDetectionConfig() *FileDetectionConfig {
    return &FileDetectionConfig{
        OldConfigPath:    "/etc/aggregator",  // v0.1.18 legacy
        OldStatePath:     "/var/lib/aggregator",  // v0.1.18 legacy
        NewConfigPath:    constants.GetAgentConfigDir(),
        NewStatePath:     constants.GetAgentStateDir(),
        BackupDirPattern: filepath.Join(constants.GetMigrationBackupDir(), "%d"),
    }
}

// 3. Update DetectLegacyInstallation() to ONLY check for v0.1.18 paths:
func (d *Detector) DetectLegacyInstallation() (bool, error) {
    // Check for v0.1.18 legacy paths ONLY
    if d.fileExists(constants.GetLegacyAgentConfigPath()) {
        log.Info("Detected legacy v0.1.18 installation")
        return true, nil
    }
    return false, nil
}
```

### **Phase 5: Update Installer Template** (30 min)

**File:** `aggregator-server/internal/services/templates/install/scripts/linux.sh.tmpl`

**Key Changes:**
```bash
# Update header (lines 16-49):
AGENT_USER="redflag-agent"
BASE_DIR="/var/lib/redflag"
AGENT_HOME="/var/lib/redflag/agent"
CONFIG_DIR="/etc/redflag"
AGENT_CONFIG_DIR="/etc/redflag/agent"
LOG_DIR="/var/log/redflag"
AGENT_LOG_DIR="/var/log/redflag/agent"

# Update directory creation (lines 175-179):
sudo mkdir -p "${BASE_DIR}"
sudo mkdir -p "${AGENT_HOME}"
sudo mkdir -p "${AGENT_HOME}/cache"
sudo mkdir -p "${AGENT_HOME}/state"
sudo mkdir -p "${AGENT_CONFIG_DIR}"
sudo mkdir -p "${AGENT_LOG_DIR}"

# Update ReadWritePaths (line 269):
ReadWritePaths=/var/lib/redflag /var/lib/redflag/agent /var/lib/redflag/agent/cache /var/lib/redflag/agent/state /var/lib/redflag/agent/migration_backups /etc/redflag /var/log/redflag

# Update backup path (line 46):
BACKUP_DIR="${AGENT_CONFIG_DIR}/backups/backup.$(date +%s)"
```

### **Phase 6: Simplified Migration Logic** (20 min)

**File:** `aggregator-agent/internal/migration/executor.go`

```go
// Simplified migration - only handles v0.1.18
func (e *Executor) RunMigration() error {
    log.Info("Checking for legacy v0.1.18 installation...")

    // Only check for v0.1.18 legacy paths
    if !e.fileExists(constants.GetLegacyAgentConfigPath()) {
        log.Info("No legacy installation found, fresh install")
        return nil
    }

    // Create backup
    backupDir := filepath.Join(
        constants.GetMigrationBackupDir(),
        fmt.Sprintf("pre_v0.2.0_migration_%d", time.Now().Unix()))

    if err := e.createBackup(backupDir); err != nil {
        return fmt.Errorf("failed to create backup: %w", err)
    }

    log.Info("Migrating from v0.1.18 to v0.2.0...")

    // Migrate config
    if err := e.migrateConfig(); err != nil {
        return e.rollback(backupDir, err)
    }

    // Migrate state
    if err := e.migrateState(); err != nil {
        return e.rollback(backupDir, err)
    }

    log.Info("Migration completed successfully")
    return nil
}

// Helper methods remain similar but simplified for v0.1.18 only
```

### **Phase 7: Update Acknowledgment System** (10 min)

**File:** `aggregator-agent/internal/acknowledgment/tracker.go`

```go
// Add import:
import "github.com/Fimeg/RedFlag/aggregator-agent/internal/constants"

// Update Save():
func (t *Tracker) Save() error {
    stateDir := constants.GetAgentStateDir()
    if err := os.MkdirAll(stateDir, 0755); err != nil {
        return err
    }

    ackFile := filepath.Join(stateDir, "pending_acks.json")
    // ... save logic
}
```

### **Phase 8: Update Version** (5 min)

**File:** `aggregator-agent/cmd/agent/main.go`

```go
// Line 32:
const AgentVersion = "0.2.0"  // Breaking: Directory structure reorganization
```

---

## Simplified Testing Requirements

### **Test Matrix (Reduced Complexity)**

**Fresh Installation Tests:**
- [ ] Agent installs cleanly on Ubuntu 22.04
- [ ] Agent installs cleanly on RHEL 9
- [ ] Agent installs cleanly on Windows Server 2022
- [ ] All directories created: `/var/lib/redflag/agent/{cache,state}`
- [ ] Config created: `/etc/redflag/agent/config.json`
- [ ] Logs created: `/var/log/redflag/agent/agent.log`
- [ ] Agent starts and functions correctly

**Migration Tests (v0.1.18 only):**
- [ ] v0.1.18 → v0.2.0 migration succeeds
- [ ] Config migrated from `/etc/aggregator/config.json`
- [ ] State migrated from `/var/lib/aggregator/`
- [ ] Backup created in `/var/lib/redflag/agent/migration_backups/`
- [ ] Rollback works if migration fails
- [ ] Agent starts after migration

**Runtime Tests:**
- [ ] Acknowledgments persist (writes to `/var/lib/redflag/agent/state/`)
- [ ] Cache functions (reads/writes to `/var/lib/redflag/agent/cache/`)
- [ ] Migration backups can be created (systemd allowed)
- [ ] No permission errors under systemd

### **Migration Testing Script**

```bash
#!/bin/bash
# test_migration.sh

# Setup legacy v0.1.18 structure
sudo mkdir -p /etc/aggregator
sudo mkdir -p /var/lib/aggregator
echo '{"agent_id":"test-123","version":18}' | sudo tee /etc/aggregator/config.json

# Run migration with new agent
./aggregator-agent --config /etc/redflag/agent/config.json

# Verify migration
if [ -f "/etc/redflag/agent/config.json" ]; then
    echo "✓ Config migrated"
fi

if [ -d "/var/lib/redflag/agent/state" ]; then
    echo "✓ State structure created"
fi

if [ -d "/var/lib/redflag/agent/migration_backups" ]; then
    echo "✓ Backup created"
fi

# Cleanup test
sudo rm -rf /etc/aggregator /var/lib/aggregator
```

---

## Timeline: 3 Hours 30 Minutes

| Phase | Task | Time | Status |
|-------|------|------|--------|
| 1 | Create constants | 30 min | Pending |
| 2 | Update main.go | 30 min | Pending |
| 3 | Update cache | 15 min | Pending |
| 4 | Update migration | 20 min | Pending |
| 5 | Update installer | 30 min | Pending |
| 6 | Update tracker | 10 min | Pending |
| 7 | Update version | 5 min | Pending |
| 8 | Testing | 60 min | Pending |
| **Total** | | **3h 30m** | **Not started** |

---

## Pre-Integration Checklist (Simplified)

✅ **Completed:**
- [x] Path constants centralized
- [x] Security review: No unauthenticated endpoints
- [x] Backup/restore paths defined
- [x] Idempotency: Only v0.1.18 → v0.2.0 (one-time)
- [x] Error logging throughout

**Remaining for v0.2.0 release:**
- [ ] Implementation complete
- [ ] Fresh install tested (Ubuntu, RHEL, Windows)
- [ ] Migration tested (v0.1.18 → v0.2.0)
- [ ] History table logging added
- [ ] Documentation updated
- [ ] CHANGELOG.md created
- [ ] Release notes drafted

---

## Risk Assessment

**Risk Level: LOW**

**Factors reducing risk:**
- Only one legacy path to support (v0.1.18)
- No broken intermediate versions in the wild
- Migration has rollback capability
- Fresh installs are clean, no legacy debt
- Small user base (~20 users) for controlled rollout

**Mitigation:**
- Rollback script auto-generated with each migration
- Backup created before any migration changes
- Idempotent migration (can detect already-migrated state)
- Extensive logging for debugging

---

## What We Learned

**The power of checking legacy code:**
- Saved 3+ hours of unnecessary migration complexity
- Eliminated need for v0.1.19-v0.1.23 broken state handling
- Reduced testing surface area significantly
- Clarified actual legacy state (not assumed)

**Lesson:** Always verify legacy paths BEFORE designing migration.

---

## Implementation Decision

**Recommended approach:** Full nested structure implementation

**Rationale:**
- Only 3.5 hours vs. original 6h 50m estimate
- Aligns with Ethos #3 (Resilience) and #5 (No BS)
- Permanent architectural improvement
- Future-proof for server component
- Clean slate - no intermediate version debt

**Coffee level required:** 1-2 cups

**Break points:** Stop after any phase, pick up next session

**Ready to implement?**

*- Ani, having done the homework before building*