29 Commits

Author SHA1 Message Date
Fimeg
62697df112 v0.1.27 release: Complete implementation
Features:
- Error logging system with ETHOS #1 compliance
- Command factory pattern with UUID generation
- Hardware binding with machine fingerprint validation
- Ed25519 cryptographic signing for updates
- Deduplication and idempotency for commands
- Circuit breakers and retry logic
- Frontend error logging integration

Bug Fixes:
- Version display using compile-time injection
- Migration 017 CONCURRENTLY issue resolved
- Docker build context fixes
- Rate limiting implementation verified

Documentation:
- README updated to reflect actual implementation
- v0.1.27 inventory analysis added
2025-12-20 13:47:36 -05:00
Fimeg
54c554ac7c fix: Remove misleading TODO comment from handleUpdateAgent
The TODO claimed update system was a "placeholder" but the code that
follows implements complete functionality:
- downloadUpdatePackage() downloads the package
- computeSHA256() verifies checksum
- verifyBinarySignature() with ed25519.Verify() checks signature
- installNewBinary() performs atomic installation
- Full rollback/cleanup with defer on failure

The comment was incorrect and misleading about the code quality.
2025-12-19 22:30:19 -05:00
Fimeg
2da93e442e fix: migration runner and scan logging fixes
- Fix migration conflicts and duplicate key errors
- Remove duplicate scan logging from agents
- Fix AgentHealth UI and Storage page triggers
- Prevent scans from appearing on wrong pages

Fixes duplicate key violations on fresh installs and
storage scans appearing on Updates page.
2025-12-19 20:59:12 -05:00
Fimeg
6b3ab6d6fc fix: Remove duplicate scan logging to prevent storage/system scans on Updates page
BREAKING CHANGE: Storage and system scans no longer create entries in update_logs

**Problem**
- Storage scans were appearing on Updates page (mixed with package updates)
- System scans were appearing on Updates page (mixed with package updates)
- Duplicate "Scan All" entries from collective + individual logging

**Root Cause**
Scan handlers were calling both ReportLog() and dedicated endpoints:
- reportLogWithAck → POST /api/v1/agents/:id/logs → update_logs table
- This caused storage/system metrics to appear alongside package updates

**Fix**
Removed ALL ReportLog() calls from scan handlers:
1. handleScanUpdatesV2 (lines 44-46): Removed collective logging
2. handleScanStorage (lines 103-105): Use only ReportStorageMetrics
3. handleScanSystem (lines 189-191): Use only ReportMetrics
4. handleScanDocker (lines 269-271): Use only ReportDockerImages

**Verification**
- All 4 handlers have working dedicated endpoints (verified via subagent)
- Routes already registered: POST /storage-metrics, POST /metrics, etc.
- Frontend queries correct endpoints (verified)
- No data loss: dedicated endpoints store in proper tables

**Result**
- Storage scans → storage_metrics table → Storage page only 
- System scans → system reporting → System page only 
- Package updates → update_logs table → Updates page only 
- No duplicate "Scan All" entries 

**Files Changed**
- aggregator-agent/cmd/agent/subsystem_handlers.go: Removed 20 lines of ReportLog calls
- internal/api/handlers/agents.go: Command recovery enhancements
- internal/api/handlers/updates.go: Subsystem extraction logic
- internal/database/queries/commands.go: GetStuckCommands query
2025-12-19 15:11:32 -05:00
Fimeg
a90692f1d8 fix: Complete AgentHealth improvements and build fixes
- Update Update scanner default from 15min to 12 hours (backend)
- Add 1 week and 2 week frequency options (frontend)
- Rename AgentScanners to AgentHealth component
- Add OS-aware package manager badges (APT, DNF, Windows/Winget, Docker)
- Fix all build errors (types, imports, storage metrics)
- Add useMemo optimization for enabled/auto-run counts
2025-12-17 21:30:10 -05:00
Fimeg
0fff047cb5 Implement proper storage metrics (P0-009)\n\n- Add dedicated storage_metrics table\n- Create StorageMetricReport models with proper field names\n- Add ReportStorageMetrics to agent client\n- Update storage scanner to use new method\n- Implement server-side handlers and queries\n- Register new routes and update UI\n- Remove legacy Scan() method\n- Follow ETHOS principles: honest naming, clean architecture 2025-12-17 17:00:05 -05:00
Fimeg
3f0838affc refactor: replace 899 lines of script generation with templates
Created InstallTemplateService with clean template-based script generation.
Added linux.sh.tmpl and windows.ps1.tmpl for install scripts.
Removed massive generateLinuxScript and generateWindowsScript functions.
Downloads handler now uses template service (1073 lines → 174 lines).
Templates easily maintainable without modifying Go code.
2025-11-10 22:41:47 -05:00
Fimeg
c95cc7d91f cleanup: remove 2,369 lines of dead code
Removed backup files and unused legacy scanner function.
All code verified as unreferenced.
2025-11-10 21:20:42 -05:00
Fimeg
e6ac0b1ec4 feat: implement agent migration system
- Fix config version inflation bug in main.go
- Add dynamic subsystem checking to prevent false change detection
- Implement migration detection and execution system
- Add directory migration from /etc/aggregator to /etc/redflag
- Update all path references across codebase to use new directories
- Add configuration schema versioning and automatic migration
- Implement backup and rollback capabilities
- Add security feature detection and hardening
- Update installation scripts and sudoers for new paths
- Complete Phase 1 migration system
2025-11-04 14:25:53 -05:00
Fimeg
95f70bd9bb feat: bump to v0.1.23 with security metrics and UI improvements
- Bump agent and server versions to 0.1.23
- Implement security metrics collection (bound agents, command processing, version compliance)
- Add dismiss button for timed out commands in agent status
- Add config sync endpoint for server->agent configuration updates
- Add ignored updates workflow in AgentUpdatesEnhanced (approve/reject workflow)
- Swap AgentScanners layout (subsystems top, security bottom)
- Replace placeholder security data with database metrics
- Add backpressure detection based on pending command ratios
2025-11-04 09:41:27 -05:00
Fimeg
38894f64d3 feat: add config sync endpoint and security UI updates
- Add GET /api/v1/agents/:id/config endpoint for server configuration
- Agent fetches config during check-in and applies updates
- Add version tracking to prevent unnecessary config applications
- Clean separation: config sync independent of commands
- Fix agent UI subsystem settings to actually control agent behavior
- Update Security Health UI with frosted glass styling and tooltips
2025-11-03 22:36:26 -05:00
Fimeg
eccc38d7c9 feat: separate data classification architecture
- Create separate scanner interfaces for storage, system, and docker data
- Add dedicated endpoints for metrics and docker images instead of misclassifying as updates
- Implement proper database tables for storage metrics and docker images
- Fix storage/system metrics appearing incorrectly as package updates
- Add scanner types with proper data structures for each subsystem
- Update agent handlers to use correct endpoints for each data type
2025-11-03 21:44:48 -05:00
Fimeg
57be3754c6 fix: agent acknowledgment recursion and subsystem UI improvements
- Fix recursive call in reportLogWithAck that caused infinite loop
- Add machine binding and security API endpoints
- Enhance AgentScanners component with security status display
- Update scheduler and timeout service reliability
- Remove deprecated install.sh script
- Add subsystem configuration and logging improvements
2025-11-03 21:02:57 -05:00
Fimeg
d0f13e5da7 bump: agent version 0.1.20 -> 0.1.22
v0.1.22 minimum version for security features
machine binding and version enforcement

agent now sends version during registration
no bootstrap gap
2025-11-02 14:00:55 -05:00
Fimeg
ec3ba88459 feat: machine binding and version enforcement
migration 017 adds machine_id to agents table
middleware validates X-Machine-ID header on authed routes
agent client sends machine ID with requests
MIN_AGENT_VERSION config defaults 0.1.22
version utils added for comparison

blocks config copying attacks via hardware fingerprint
old agents get 426 upgrade required
breaking: <0.1.22 agents rejected
2025-11-02 09:30:04 -05:00
Fimeg
99480f3fe3 fix: resolve frontend approval error and add invalid command handling
- Added missing approveMultiple function to updateApi
- Fixed API endpoint from /updates/bulk-approve to /updates/approve
- Enhanced invalid command handling in both Linux and Windows agents
- Agents now report unknown command types as failed commands back to server
2025-11-01 21:56:31 -04:00
Fimeg
3690472396 feat: granular subsystem commands with parallel scanner execution
Split monolithic scan_updates into individual subsystems (updates/storage/system/docker).
Scanners now run in parallel via goroutines - cuts scan time roughly in half, maybe more.

Agent changes:
- Orchestrator pattern for scanner management
- New scanners: storage (disk metrics), system (cpu/mem/processes)
- New commands: scan_storage, scan_system, scan_docker
- Wrapped existing scanners (APT/DNF/Docker/Windows/Winget) with common interface
- Version bump to 0.1.20

Server changes:
- Migration 015: agent_subsystems table with trigger for auto-init
- Subsystem CRUD: enable/disable, interval (5min-24hr), auto-run toggle
- API routes: /api/v1/agents/:id/subsystems/* (9 endpoints)
- Stats tracking per subsystem

Web UI changes:
- ChatTimeline shows subsystem-specific labels and icons
- AgentScanners got interactive toggles, interval dropdowns, manual trigger buttons
- TypeScript types added for subsystems

Backward compatible with legacy scan_updates - for now. Bugs probably exist somewhere.
2025-11-01 21:34:26 -04:00
Fimeg
bf4d46529f feat: add resilience and reliability features for agent subsystems
Added circuit breakers with configurable timeouts for all subsystems (APT, DNF, Docker, Windows, Winget, Storage). Replaces cron-based scheduler with priority queue that should scale beyond 1000+ agents if your homelab is that big.

Command acknowledgment system ensures results aren't lost on network failures or restarts. Agent tracks pending acknowledgments with persistent state and automatic retry.

- Circuit breakers: 3 failures in 1min opens circuit, 30s cooldown
- Per-subsystem timeouts: 30s-10min depending on scanner
- Priority queue scheduler: O(log n), worker pool, jitter, backpressure
- Acknowledgments: at-least-once delivery, max 10 retries over 24h
- All tests passing (26/26)
2025-11-01 18:42:41 -04:00
Fimeg
01c09cefab feat: agent UI redesign and version bump to 0.1.18
- Redesign AgentUpdatesEnhanced with tab-based workflow (pending/approved/installing/installed)
- Add AgentStorage component with disk partition table
- Add AgentScanners component for agent health monitoring
- Fix agent removal not refreshing list (cache invalidation)
- Bump agent version to 0.1.18 (enhanced disk detection)
- Update server default version to 0.1.18
- Add command source tracking (system/manual) migration
- Improve Linux disk detection for all physical mount points
2025-11-01 09:27:58 -04:00
Fimeg
e72e9fc16f feat: add host restart detection and fix agent version display
Potential fixes for issues #4 and #6.

Agent version display:
- Set CurrentVersion during registration instead of waiting for first check-in
- Update UI to show "Initial Registration" instead of "Unknown"

Host restart detection:
- Added reboot_required, last_reboot_at, reboot_reason fields to agents table
- Agent now detects pending reboots (Debian/Ubuntu via /var/run/reboot-required, RHEL/Fedora via needs-restarting)
- New reboot command type with 1-minute grace period
- UI shows restart alerts and adds restart button in quick actions
- Restart indicator badge in agent list

The reboot detection runs during system info collection and gets reported back to the server automatically.

Using shutdown command for now until we make the restart mechanism user-adjustable later - need to think on that.
Also need to come up with a Windows derivative outside of reading event log for detecting reboots.
2025-10-31 15:03:59 -04:00
Fimeg
a92ac0ed78 v0.1.17: UI fixes, Linux improvements, documentation overhaul
UI/UX:
- Fix heartbeat auto-refresh and rate-limiting page
- Add navigation breadcrumbs to settings pages
- New screenshots added

Linux Agent v0.1.17:
- Fix disk detection for multiple mount points
- Improve installer idempotency
- Prevent duplicate registrations

Documentation:
- README rewrite: 538→229 lines, homelab-focused
- Split docs: API.md, CONFIGURATION.md, DEVELOPMENT.md
- Add NOTICE for Apache 2.0 attribution
2025-10-30 22:17:48 -04:00
Fimeg
e40cb14945 Fix module paths for GitHub repository structure
- Update go.mod files to use github.com/Fimeg/RedFlag module path
- Fix all import statements across server and agent code
- Resolves build errors when cloning from GitHub
- Utils package (version comparison) is actually needed and working
2025-10-29 11:53:20 -04:00
Fimeg
03fee29760 v0.1.16: Security overhaul and systematic deployment preparation
Breaking changes for clean alpha releases:
- JWT authentication with user-provided secrets (no more development defaults)
- Registration token system for secure agent enrollment
- Rate limiting with user-adjustable settings
- Enhanced agent configuration with proxy support
- Interactive server setup wizard (--setup flag)
- Heartbeat architecture separation for better UX
- Package status synchronization fixes
- Accurate timestamp tracking for RMM features

Setup process for new installations:
1. docker-compose up -d postgres
2. ./redflag-server --setup
3. ./redflag-server --migrate
4. ./redflag-server
5. Generate tokens via admin UI
6. Deploy agents with registration tokens
2025-10-29 10:38:18 -04:00
Fimeg
b3e1b9e52f Bump agent version to 0.1.8 2025-10-25 12:40:16 -04:00
Fimeg
d1c5cb9597 feat: Updates page UI improvements and Windows agent enhancements
- Redesigned statistics cards with combined layout
- Added quick filters for Installing, Installed, Failed, Dependencies
- Implemented column sorting for all table headers
- Added package name truncation to prevent layout stretching
- Fixed TypeScript types for new update statuses
- Updated screenshots and README
2025-10-17 22:40:40 -04:00
Fimeg
4ef5216c89 Windows agent improvements and dependency workflow fixes
- Added system info reporting to agent main loop
- Updated README with current project status and screenshots
- Fixed a few workflow quirks
2025-10-17 16:26:21 -04:00
Fimeg
2ade509b63 Update README with current features and screenshots
- Cross-platform support (Windows/Linux) with Windows Updates and Winget
- Added dependency confirmation workflow and refresh token authentication
- New screenshots: History, Live Operations, Windows Agent Details
- Local CLI features with terminal output and cache system
- Updated known limitations - Proxmox integration is broken
- Organized docs to docs/ folder and updated .gitignore
- Probably introduced a dozen bugs with Windows agents - stay tuned
2025-10-17 15:28:22 -04:00
Fimeg
61294ba514 Add screenshots and update gitignore for alpha release
- Fixed gitignore to allow Screenshots/*.png files
- Added all screenshots for README documentation
- Fixed gitignore to be less restrictive with image files
- Includes dashboard, agent, updates, and docker screenshots
2025-10-16 09:16:05 -04:00
Fimeg
55b7d03010 Session 4 complete - RedFlag update management platform
🚩 Private development - version retention only

 Complete web dashboard (React + TypeScript + TailwindCSS)
 Production-ready server backend (Go + Gin + PostgreSQL)
 Linux agent with APT + Docker scanning + local CLI tools
 JWT authentication and REST API
 Update discovery and approval workflow

🚧 Status: Alpha software - active development
📦 Purpose: Version retention during development
⚠️  Not for public use or deployment
2025-10-13 16:46:31 -04:00