Fimeg/Redflag
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

Add docs and project files - force for Culurien

Browse Source
This commit is contained in:
Fimeg
2026-03-28 20:46:24 -04:00
parent dc61797423
commit 484a7f77ce
343 changed files with 119530 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

399
docs/3_BACKLOG/P4-004_Directory-Path-Standardization.md Normal file
View File

@@ -0,0 +1,399 @@
# P4-004: Directory Path Standardization
**Priority:** P4 (Technical Debt)
**Source Reference:** From analysis of needsfixingbeforepush.md lines 1584-1609 and DEVELOPMENT_TODOS.md lines 580-607
**Date Identified:** 2025-11-12
## Problem Description
Mixed directory naming creates confusion and maintenance issues throughout the codebase. Both `/var/lib/aggregator` and `/var/lib/redflag` paths are used inconsistently across agent and server code, leading to operational complexity, backup/restore challenges, and potential path conflicts.
## Impact
- **User Confusion:** Inconsistent file locations make system administration difficult
- **Maintenance Overhead:** Multiple path patterns increase development complexity
- **Backup Complexity:** Mixed paths complicate backup and restore procedures
- **Documentation Conflicts:** Documentation shows different paths than actual usage
- **Migration Issues:** Path inconsistencies break upgrade processes
## Current Path Inconsistencies
### Agent Code
- **Config:** `/etc/aggregator/config.json` (old) vs `/etc/redflag/config.json` (new)
- **State:** `/var/lib/aggregator/` (old) vs `/var/lib/redflag/` (new)
- **Logs:** Mixed usage in different files
### Server Code
- **Install Scripts:** References to both old and new paths
- **Documentation:** Inconsistent path examples
- **Templates:** Mixed path usage in install script templates
### File References
```go
// Found in codebase:
STATE_DIR = "/var/lib/aggregator" // aggregator-agent/cmd/agent/main.go:47
CONFIG_PATH = "/etc/redflag/config.json" // Some newer files
STATE_DIR = "/var/lib/redflag" // Other files
```
## Proposed Solution
Standardize on `/var/lib/redflag` and `/etc/redflag` throughout the entire codebase:
### 1. Centralized Path Constants
```go
// aggregator/internal/paths/paths.go
package paths
const (
// Standard paths for RedFlag
ConfigDir = "/etc/redflag"
StateDir = "/var/lib/redflag"
LogDir = "/var/log/redflag"
BackupDir = "/var/lib/redflag/backups"
CacheDir = "/var/lib/redflag/cache"
// Specific files
ConfigFile = ConfigDir + "/config.json"
StateFile = StateDir + "/last_scan.json"
AckFile = StateDir + "/pending_acks.json"
HistoryFile = StateDir + "/command_history.json"
// Legacy paths (for migration)
LegacyConfigDir = "/etc/aggregator"
LegacyStateDir = "/var/lib/aggregator"
LegacyLogDir = "/var/log/aggregator"
)
type PathConfig struct {
Config string
State string
Log string
Backup string
Cache string
}
func GetStandardPaths() PathConfig {
return PathConfig{
Config: ConfigDir,
State: StateDir,
Log: LogDir,
Backup: BackupDir,
Cache: CacheDir,
}
}
func GetStandardFiles() map[string]string {
return map[string]string{
"config": ConfigFile,
"state": StateFile,
"acknowledgments": AckFile,
"history": HistoryFile,
}
}
```
### 2. Path Migration System
```go
// aggregator/internal/paths/migration.go
package paths
type PathMigrator struct {
Standard PathConfig
Legacy PathConfig
DryRun bool
Backup bool
}
func NewPathMigrator(backup bool) *PathMigrator {
return &PathMigrator{
Standard: GetStandardPaths(),
Legacy: PathConfig{
Config: LegacyConfigDir,
State: LegacyStateDir,
Log: LegacyLogDir,
},
Backup: backup,
}
}
func (pm *PathMigrator) MigrateAll() error {
// Migrate configuration directory
if err := pm.migrateDirectory(pm.Legacy.Config, pm.Standard.Config); err != nil {
return fmt.Errorf("config migration failed: %w", err)
}
// Migrate state directory
if err := pm.migrateDirectory(pm.Legacy.State, pm.Standard.State); err != nil {
return fmt.Errorf("state migration failed: %w", err)
}
// Migrate log directory
if err := pm.migrateDirectory(pm.Legacy.Log, pm.Standard.Log); err != nil {
return fmt.Errorf("log migration failed: %w", err)
}
return nil
}
func (pm *PathMigrator) migrateDirectory(legacyPath, standardPath string) error {
// Check if legacy path exists
if _, err := os.Stat(legacyPath); os.IsNotExist(err) {
return nil // No migration needed
}
// Check if standard path already exists
if _, err := os.Stat(standardPath); err == nil {
return fmt.Errorf("standard path already exists: %s", standardPath)
}
if pm.DryRun {
log.Printf("[DRY RUN] Would migrate %s -> %s", legacyPath, standardPath)
return nil
}
// Create backup if requested
if pm.Backup {
backupPath := standardPath + ".backup." + time.Now().Format("20060102-150405")
if err := copyDirectory(legacyPath, backupPath); err != nil {
return fmt.Errorf("backup creation failed: %w", err)
}
}
// Perform migration
if err := os.Rename(legacyPath, standardPath); err != nil {
return fmt.Errorf("directory rename failed: %w", err)
}
log.Printf("Migrated directory: %s -> %s", legacyPath, standardPath)
return nil
}
```
### 3. Install Script Template Updates
```go
// Update linux.sh.tmpl to use standard paths
const LinuxInstallTemplate = `
#!/bin/bash
set -euo pipefail
# Standard RedFlag paths
CONFIG_DIR="{{ .ConfigDir }}"
STATE_DIR="{{ .StateDir }}"
LOG_DIR="{{ .LogDir }}"
AGENT_USER="redflag-agent"
# Create directories with proper permissions
for dir in "$CONFIG_DIR" "$STATE_DIR" "$LOG_DIR"; do
if [ ! -d "$dir" ]; then
mkdir -p "$dir"
chown "$AGENT_USER:$AGENT_USER" "$dir"
chmod 755 "$dir"
fi
done
# Check for legacy paths and migrate
LEGACY_CONFIG_DIR="/etc/aggregator"
LEGACY_STATE_DIR="/var/lib/aggregator"
if [ -d "$LEGACY_CONFIG_DIR" ] && [ ! -d "$CONFIG_DIR" ]; then
echo "Migrating configuration from legacy path..."
mv "$LEGACY_CONFIG_DIR" "$CONFIG_DIR"
fi
if [ -d "$LEGACY_STATE_DIR" ] && [ ! -d "$STATE_DIR" ]; then
echo "Migrating state from legacy path..."
mv "$LEGACY_STATE_DIR" "$STATE_DIR"
fi
`
```
### 4. Agent Configuration Integration
```go
// Update agent config to use path constants
type AgentConfig struct {
Server string `json:"server_url"`
AgentID string `json:"agent_id"`
Token string `json:"registration_token,omitempty"`
Paths PathConfig `json:"paths,omitempty"`
}
func DefaultAgentConfig() AgentConfig {
return AgentConfig{
Paths: paths.GetStandardPaths(),
}
}
// Usage in agent code
func (a *Agent) getStateFilePath() string {
if a.config.Paths.State != "" {
return filepath.Join(a.config.Paths.State, "last_scan.json")
}
return paths.StateFile
}
```
### 5. Code Updates Strategy
#### Phase 1: Introduce Path Constants
```bash
# Find all hardcoded paths
grep -r "/var/lib/aggregator" aggregator-agent/
grep -r "/etc/aggregator" aggregator-agent/
grep -r "/var/lib/redflag" aggregator-agent/
grep -r "/etc/redflag" aggregator-agent/
# Replace with path constants
find . -name "*.go" -exec sed -i 's|"/var/lib/aggregator"|paths.StateDir|g' {} \;
```
#### Phase 2: Update Import Statements
```go
// Add to files using paths
import (
"github.com/redflag/redflag/internal/paths"
)
```
#### Phase 3: Update Documentation
```markdown
## Installation Paths
RedFlag uses standardized paths for all installations:
- **Configuration:** `/etc/redflag/config.json`
- **State Data:** `/var/lib/redflag/`
- **Log Files:** `/var/log/redflag/`
- **Backups:** `/var/lib/redflag/backups/`
### File Structure
```
/etc/redflag/
└── config.json # Agent configuration
/var/lib/redflag/
├── last_scan.json # Last scan results
├── pending_acks.json # Pending acknowledgments
├── command_history.json # Command history
└── backups/ # Backup directory
/var/log/redflag/
└── agent.log # Agent log files
```
```
## Definition of Done
- [ ] All hardcoded paths replaced with centralized constants
- [ ] Path migration system handles legacy installations
- [ ] Install script templates use standard paths
- [ ] Documentation updated with correct paths
- [ ] Server code updated for consistency
- [ ] Agent code uses path constants throughout
- [ ] SystemD service files updated with correct paths
- [ ] Migration process tested on existing installations
## Implementation Details
### Files Requiring Updates
#### Agent Code
- `aggregator-agent/cmd/agent/main.go` - STATE_DIR and CONFIG_PATH constants
- `aggregator-agent/internal/config/config.go` - Default paths
- `aggregator-agent/internal/orchestrator/*.go` - File path references
- `aggregator-agent/internal/installer/*.go` - Installation paths
#### Server Code
- `aggregator-server/internal/api/handlers/downloads.go` - Install script templates
- `aggregator-server/internal/services/templates/install/scripts/linux.sh.tmpl`
- `aggregator-server/internal/services/templates/install/scripts/windows.ps1.tmpl`
#### Configuration Files
- Dockerfiles and docker-compose.yml
- SystemD unit files
- Documentation files
### Migration Process
1. **Backup:** Create backup of existing installations
2. **Path Detection:** Detect which paths are currently in use
3. **Migration:** Move files to standard locations
4. **Permission Updates:** Ensure correct ownership and permissions
5. **Validation:** Verify all files are accessible after migration
### Testing Strategy
- Test migration from legacy paths to standard paths
- Verify fresh installations use standard paths
- Test that existing installations continue to work
- Validate SystemD service files work with new paths
## Testing Scenarios
### 1. Fresh Installation Test
```bash
# Fresh install should create standard paths
curl -sSL http://localhost:8080/api/v1/install/linux | sudo bash
# Verify standard paths exist
ls -la /etc/redflag/
ls -la /var/lib/redflag/
```
### 2. Migration Test
```bash
# Simulate legacy installation
sudo mkdir -p /etc/aggregator /var/lib/aggregator
echo "legacy config" | sudo tee /etc/aggregator/config.json
echo "legacy state" | sudo tee /var/lib/aggregator/last_scan.json
# Run migration
sudo /usr/local/bin/redflag-agent --migrate-paths
# Verify files moved to standard paths
ls -la /etc/redflag/config.json
ls -la /var/lib/redflag/last_scan.json
# Verify legacy paths removed
! test -d /etc/aggregator
! test -d /var/lib/aggregator
```
### 3. Service Integration Test
```bash
# Ensure SystemD service works with new paths
sudo systemctl restart redflag-agent
sudo systemctl status redflag-agent
sudo journalctl -u redflag-agent -n 20
```
## Prerequisites
- Path detection and migration system implemented
- Backup system for safe migrations
- Install script template system available
- Configuration system supports path overrides
## Effort Estimate
**Complexity:** Medium
**Effort:** 2-3 days
- Day 1: Create path constants and migration system
- Day 2: Update agent code and test migration
- Day 3: Update server code, templates, and documentation
## Success Metrics
- Zero hardcoded paths remaining in codebase
- All installations use consistent paths
- Migration成功率 for existing installations >95%
- No data loss during migration process
- Documentation matches actual implementation
- SystemD service integration works seamlessly
## Rollback Plan
If issues arise during migration:
1. Stop all RedFlag services
2. Restore from backups created during migration
3. Update configuration to point to legacy paths
4. Restart services
5. Document issues for future improvement