Fimeg/Redflag
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
Redflag/docs/4_LOG/November_2025/analysis/TECHNICAL_DEBT.md

16 KiB
Raw Permalink Blame History

📝 Technical Debt & Future Enhancements

This file tracks non-critical improvements, optimizations, and technical debt items that can be addressed in future sessions.

🔧 Performance Optimizations

Docker Registry Cache Cleanup (Low Priority)

File: aggregator-agent/internal/scanner/registry.go Lines: 249-259 Status: Optional Enhancement Added: 2025-10-12 (Session 2)

Current Behavior:

  • The cleanupExpired() method exists but is never called
  • Cache entries are cleaned up lazily during get() operations (line 229-232)
  • Expired entries remain in memory until accessed

Enhancement: Add a background goroutine to periodically clean up expired cache entries.

Implementation Approach:

// In NewRegistryClient() or Scan():
go func() {
    ticker := time.NewTicker(10 * time.Minute)
    defer ticker.Stop()
    for range ticker.C {
        c.cache.cleanupExpired()
    }
}()

Impact:

  • Benefit: Reduces memory footprint for long-running agents
  • Risk: Adds goroutine management complexity
  • Priority: LOW (current lazy cleanup works fine)

Why It's Not Critical:

  • Cache TTL is only 5 minutes
  • Expired entries are deleted on next access (line 231)
  • Typical agents won't accumulate enough entries for this to matter
  • Memory leak risk is minimal

When to Implement:

  • If agent runs for weeks/months without restart
  • If scanning hundreds of different images
  • If memory profiling shows cache bloat

🔐 Security Enhancements

Private Registry Authentication (Medium Priority)

File: aggregator-agent/internal/scanner/registry.go Lines: 125-128 Status: TODO Added: 2025-10-12 (Session 2)

Current Behavior:

  • Only Docker Hub anonymous pull authentication is implemented
  • Custom registries default to unauthenticated requests
  • Private registries will fail with 401 errors

Enhancement: Implement authentication for private/custom registries:

  • Basic auth (username/password)
  • Bearer token auth (custom tokens)
  • Registry-specific credential storage
  • Docker config.json credential loading

Implementation Approach:

  1. Add config options for registry credentials
  2. Implement basic auth header construction
  3. Support reading from Docker's ~/.docker/config.json
  4. Handle registry-specific auth flows (ECR, GCR, ACR, etc.)

Impact:

  • Benefit: Support for private Docker registries
  • Risk: Credential management complexity
  • Priority: MEDIUM (many homelabbers use private registries)

🧪 Testing

Unit Tests for Registry Client (Medium Priority)

Files:

  • aggregator-agent/internal/scanner/registry.go
  • aggregator-agent/internal/scanner/docker.go

Status: Not Implemented Added: 2025-10-12 (Session 2)

Current State:

  • Manual testing with real Docker images ✅
  • No automated unit tests ❌
  • No mock registry for testing ❌

Needed Tests:

  1. Cache tests:
    • Cache hit/miss behavior
    • Expiration logic
    • Thread-safety with concurrent access
  2. parseImageName tests:
    • Official images: nginx → library/nginx
    • User images: user/repo → user/repo
    • Custom registries: gcr.io/proj/img
  3. Error handling tests:
    • 429 rate limiting
    • 401 unauthorized
    • Network failures
    • Malformed responses
  4. Integration tests:
    • Mock registry server
    • Token authentication flow
    • Digest comparison logic

Implementation Approach:

# Create test files
touch aggregator-agent/internal/scanner/registry_test.go
touch aggregator-agent/internal/scanner/docker_test.go

Priority: MEDIUM (important for production confidence)

📊 Features & Functionality

Multi-Architecture Manifest Support (Low Priority)

File: aggregator-agent/internal/scanner/registry.go Lines: 200-215 Status: Not Implemented Added: 2025-10-12 (Session 2)

Current Behavior:

  • Uses Docker-Content-Digest header (manifest digest)
  • Falls back to config.digest from manifest body
  • Doesn't handle multi-arch manifest lists

Enhancement: Support Docker manifest lists (multi-architecture images):

  • Detect manifest list media type
  • Select correct architecture-specific manifest
  • Compare digests for the correct platform

Impact:

  • Benefit: Accurate updates for multi-arch images (arm64, amd64, etc.)
  • Risk: Complexity in manifest parsing
  • Priority: LOW (current approach works for most cases)

🗄️ Data Persistence

Persistent Cache (Low Priority)

File: aggregator-agent/internal/scanner/registry.go Lines: 20-29 Status: In-Memory Only Added: 2025-10-12 (Session 2)

Current Behavior:

  • Cache is in-memory only
  • Lost on agent restart
  • No disk persistence

Enhancement: Add optional persistent cache:

  • Save cache to disk (JSON or SQLite)
  • Load cache on startup
  • Reduces registry queries after restart

Implementation Approach:

# Cache storage location
/var/cache/aggregator/registry_cache.json

Impact:

  • Benefit: Faster scans after restart, fewer registry requests
  • Risk: Stale data if not properly invalidated
  • Priority: LOW (5-minute TTL makes persistence less valuable)

🎨 User Experience

Settings Page Theme Issues (Medium Priority)

Files: aggregator-web/src/pages/Settings.tsx, related CSS/styling files Status: Needs Cleanup Added: 2025-10-15 (Session 5)

Current Issues:

  • Settings page has ugly theming code attempting to mimic system dark mode
  • Multiple theme options implemented when only one theme should exist
  • Theme switching functionality creates visual inconsistency
  • Timezone and Dashboard settings don't match established theme

Required Cleanup:

  1. Remove Theme Switching Code

    • Remove dark/light mode toggle functionality
    • Remove system theme detection code
    • Simplify to single consistent theme

  2. Fix Visual Consistency

    • Ensure Settings page matches established design theme
    • Make Timezone and Dashboard settings consistent with rest of UI
    • Remove any CSS/styling that creates visual dissonance

  3. Streamline Settings UI

    • Keep only functional settings (timezone, dashboard preferences)
    • Remove theme-related settings entirely
    • Ensure clean, consistent visual presentation

Implementation Approach:

// Remove theme switching components
// Keep only essential settings:
- Timezone selection
- Dashboard refresh intervals
- Notification preferences
- Agent management settings

// Ensure consistent styling with rest of application

Impact:

  • Benefit: Cleaner UI, reduced complexity, consistent user experience
  • Risk: Low (removing unused functionality)
  • Priority: MEDIUM (visual polish, user experience improvement)

Why This Matters:

  • Current theming code creates visual inconsistency
  • Adds unnecessary complexity to the codebase
  • Distracts from core functionality
  • Users expect consistent design across all pages

Local Agent CLI Features ✅ COMPLETED

File: aggregator-agent/cmd/agent/main.go, aggregator-agent/internal/cache/local.go, aggregator-agent/internal/display/terminal.go Status: IMPLEMENTED - Full local visibility and control Added: 2025-10-12 (Session 2) Completed: 2025-10-13 (Session 3) Priority: HIGH (critical for user experience)

✅ IMPLEMENTED FEATURES:

  1. --scan flag - Run scan NOW and display results locally

    sudo ./aggregator-agent --scan
# Beautiful color output with severity indicators, package counts
# Works without server registration for local scans

  2. --status flag - Show agent status and last scan

    ./aggregator-agent --status
# Shows Agent ID, Server URL, Last check-in, Last scan, Update count

  3. --list-updates flag - Show detailed update list

    ./aggregator-agent --list-updates
# Full details including CVEs, sizes, descriptions, versions

  4. --export flag - Export results to JSON/CSV

    ./aggregator-agent --scan --export=json > updates.json
./aggregator-agent --list-updates --export=csv > updates.csv

  5. Local cache/database - Store last scan results

    # Stored in: /var/lib/aggregator/last_scan.json
# Enables offline viewing and status tracking

Implementation Completed:

  • ✅ Local cache system with JSON storage (cache/local.go - 129 lines)
  • ✅ Beautiful terminal output with colors/icons (display/terminal.go - 372 lines)
  • ✅ All 4 CLI flags implemented in main.go (360 lines)
  • ✅ Export functionality (JSON/CSV) for automation
  • ✅ Cache expiration and safety checks
  • ✅ Error handling for unregistered agents

Benefits Delivered:

  • ✅ Better UX for single-machine users
  • ✅ Quick local checks without dashboard
  • ✅ Offline viewing capability
  • ✅ Audit trail of past scans
  • ✅ Reduces server dependency
  • ✅ Aligns with self-hoster philosophy
  • ✅ Production-ready with proper error handling

Impact Assessment:

  • Benefit: ✅ MAJOR UX improvement for target audience
  • Risk: ✅ LOW (additive feature, doesn't break existing functionality)
  • Status: ✅ COMPLETED (Session 3)
  • Actual Effort: ~3 hours
  • Code Added: ~680 lines across 3 files

Future Enhancements: 6. React Native Desktop App (Future - Preferred over TUI)

  • Cross-platform GUI (Linux, Windows, macOS)
  • Use React Native for Desktop or Electron + React
  • Navigate updates, approve, view details
  • Standalone mode (no server needed)
  • Better UX than terminal TUI for most users
  • Code sharing with web dashboard (same React components)
  • Note: CLI features (#1-5 above) are still valuable for scripting/automation

🐳 Docker Networking & Environment Configuration (High Priority)

Docker Network Integration & Hostname Resolution

Files:

  • aggregator-web/.env, aggregator-server/.env, aggregator-agent/.env
  • docker-compose.yml (when created)
  • All API client configuration files

Status: CRITICAL for Docker deployment Added: 2025-10-13 (Session 4)

Current Behavior:

  • Web dashboard hardcoded to http://localhost:8080/api/v1
  • No Docker network hostname resolution support
  • Server/agent communication assumes localhost or static IP
  • Environment configuration minimal

Required Enhancements:

  1. Docker Network Support:

    • Support for Docker Compose service names (e.g., redflag-server:8080)
    • Automatic hostname resolution in Docker networks
    • Configurable API URLs via environment variables

  2. Environment Configuration:

    • .env templates for different deployment scenarios
    • Docker-specific environment variables
    • Network-aware service discovery

  3. Multi-Network Support:

    • Support for custom Docker networks
    • Bridge network configuration
    • Host network mode options

Implementation Approach:

# Environment variables to add:
VITE_API_URL=http://redflag-server:8080/api/v1  # Docker hostname
SERVER_HOST=0.0.0.0  # Listen on all interfaces
AGENT_SERVER_URL=http://redflag-server:8080/api/v1  # Docker hostname

Docker Compose Network Structure:

networks:
  redflag-network:
    driver: bridge
services:
  redflag-server:
    networks:
      - redflag-network
  redflag-web:
    networks:
      - redflag-network
  redflag-agent:
    networks:
      - redflag-network

Impact:

  • Benefit: Essential for Docker deployment and production use
  • Risk: Configuration complexity, potential hostname resolution issues
  • Priority: HIGH (blocker for Docker deployment)

When to Implement:

  • Session 5: Before Docker Compose deployment
  • Required for: Any containerized deployment
  • Critical for: Multi-service architecture

GitHub Repository Setup & Documentation (Medium Priority)

Files: README.md, .gitignore, all documentation files Status: Ready for initial commit Added: 2025-10-13 (Session 4)

Current State:

  • Repository not yet initialized with git
  • GitHub references point to non-existent repo
  • README needs "development status" warnings
  • No .gitignore configured for Go/Node projects

Required Setup:

  1. Initialize Git Repository:

    git init
git add .
git commit -m "Initial commit - RedFlag update management platform"
git branch -M main
git remote add origin git@github.com:Fimeg/RedFlag.git
git push -u origin main

  2. Update README.md:

    • Add clear "IN ACTIVE DEVELOPMENT" warnings
    • Specify current functionality status
    • Add "NOT PRODUCTION READY" disclaimers
    • Update GitHub links to correct repository

  3. Create .gitignore:

    • Go-specific ignores (build artifacts, vendor/)
    • Node.js ignores (node_modules, build/)
    • Environment files (.env, .env.local)
    • IDE files (.vscode/, etc.)
    • OS files (DS_Store, Thumbs.db)

Implementation Priority:

  • Before any public sharing: Repository setup
  • Before Session 5: Complete documentation updates
  • Critical for: Collaboration and contribution

📝 Documentation

Update README with Development Status (Medium Priority)

Status: README exists but needs development warnings Added: 2025-10-13 (Session 4)

Required Updates:

  1. Development Status Warnings:

    • "🚧 IN ACTIVE DEVELOPMENT - NOT PRODUCTION READY"
    • "Alpha software - use at your own risk"
    • "Breaking changes expected"

  2. Current Functionality:

    • What works (server, agent, web dashboard)
    • What doesn't work (update installation, real-time updates)
    • Installation requirements and prerequisites

  3. Setup Instructions:

    • Current working setup process
    • Development environment requirements
    • Known limitations and issues

  4. GitHub Repository Links:

    • Update to correct repository URL
    • Add issue tracker link
    • Add contribution guidelines

🚀 Next Session Recommendations

Based on MVP checklist progress and completed Session 5 work:

Completed in Session 5:

  • ✅ JWT Authentication Fixed - Secret mismatch resolved, debug logging added
  • ✅ Docker API Implementation - Complete container management endpoints
  • ✅ Docker Model Architecture - Full container and stats models
  • ✅ Agent Architecture Decision - Universal strategy documented
  • ✅ Compilation Issues Resolved - All JSONB and model reference fixes

High Priority (Session 6):

  1. System Domain Reorganization (update categorization by type)
    • OS & System, Applications & Services, Container Images, Development Tools
    • Critical for making the update system usable and organized
  2. Agent Status Display Fixes (last check-in time updates)
    • User feedback indicates status display issues
  3. Rate Limiting & Security (critical security gap vs PatchMon)
    • Must be implemented before production deployment
  4. UI/UX Cleanup (remove duplicate fields, layout improvements)
    • Improves user experience and reduces confusion

Medium Priority: 5. YUM/DNF Support (expand beyond Debian/Ubuntu) 6. Update Installation (APT packages first) 7. Deployment Improvements (Docker, one-line installer, systemd)

Future High Priority: 8. Proxmox Integration ⭐⭐⭐ (KILLER FEATURE - See PROXMOX_INTEGRATION_SPEC.md)

  • Auto-discover LXC containers across Proxmox clusters
  • Hierarchical view: Proxmox host → LXC → Docker containers
  • Bulk operations by cluster/node
  • User priority: 2 Proxmox clusters, many LXCs, many Docker containers
  • Impact: THE differentiator for homelab users

Medium Priority: 9. Add CVE enrichment for APT packages 10. Private Docker registry authentication 11. Unit tests for scanners 12. Host grouping (complements Proxmox hierarchy)

Low Priority: 13. Cache cleanup goroutine 14. Multi-arch manifest support 15. Persistent cache 16. Multi-user/RBAC (not needed for self-hosters)

Last Updated: 2025-10-13 (Post-Session 3 - Proxmox priorities updated) Next Review: Before Session 4 (Web Dashboard with Proxmox in mind)

Reference in New Issue View Git Blame Copy Permalink