Fimeg/Redflag
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

v0.1.16: Security overhaul and systematic deployment preparation

Browse Source 
Breaking changes for clean alpha releases:
- JWT authentication with user-provided secrets (no more development defaults)
- Registration token system for secure agent enrollment
- Rate limiting with user-adjustable settings
- Enhanced agent configuration with proxy support
- Interactive server setup wizard (--setup flag)
- Heartbeat architecture separation for better UX
- Package status synchronization fixes
- Accurate timestamp tracking for RMM features

Setup process for new installations:
1. docker-compose up -d postgres
2. ./redflag-server --setup
3. ./redflag-server --migrate
4. ./redflag-server
5. Generate tokens via admin UI
6. Deploy agents with registration tokens
This commit is contained in:
Fimeg
2025-10-29 10:38:18 -04:00
parent b3e1b9e52f
commit 03fee29760
50 changed files with 5807 additions and 466 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
.gitignore vendored
View File

@@ -404,4 +404,19 @@ secrets/
# =============================================================================
# AI / LLM Development Files
# =============================================================================
.claude/
.claude/
# =============================================================================
# Development and deployment environments
# =============================================================================
website/
deployment/
# =============================================================================
# Generated development documentation
# =============================================================================
docs/
*.md
!README.md
!LICENSE
!.env.example

419
README.md
View File

@@ -1,15 +1,333 @@
# RedFlag (Aggregator)
⚠️ PRIVATE DEVELOPMENT - NOT FOR PUBLIC USE
This is a private development repository for version retention only.
**ALPHA RELEASE - v0.1.16**
Self-hosted update management platform for homelabs and small teams
## Status
- **Active Development**: In progress
- **Not Production Ready**: Do not use
- **Breaking Changes Expected**: APIs will change
- **No Support Available**: This is not released software
- **Core Features Working**: Update management, agent registration, web dashboard
- **Alpha Deployment Ready**: Setup wizard and configuration system implemented
- **Cross-Platform Support**: Linux and Windows agents
- **In Development**: Enhanced features and polish
- **Alpha Software**: Expect some rough edges, backup your data
## What RedFlag Is
A self-hosted, cross-platform update management platform built for homelabs and small teams:
- Go Server Backend with PostgreSQL database
- React Web Dashboard with real-time updates
- Cross-Platform Agents (Linux APT/DNF/Docker, Windows Updates/Winget)
- Secure Authentication with registration tokens and refresh tokens
- Professional Monitoring with real-time status and audit trails
- Enterprise-Grade Security with rate limiting and TLS support
## Key Features
### Alpha Features
- Secure Server Setup: `./redflag-server --setup` with user-provided secrets
- Registration Token System: One-time tokens for secure agent enrollment
- Rate Limiting: User-adjustable API security with professional defaults
- Cross-Platform Agents: Linux and Windows with unified architecture
- Real-Time Heartbeat: Rapid polling for interactive operations
- Dependency Management: Safe update installation with dry-run checking
- Audit Logging: Complete activity tracking and history
- Proxy Support: Enterprise networking with HTTP/HTTPS/SOCKS5 proxies
### Update Management
- Package Managers: APT, DNF, Docker images, Windows Updates, Winget
- Update Discovery: Automatic scanning with severity classification
- Approval Workflow: Controlled update deployment with confirmation
- Bulk Operations: Multi-agent management and batch operations
- Rollback Support: Failed update tracking and retry capabilities
### Deployment
- Configuration Management: CLI flags → environment → config file → defaults
- Service Integration: systemd service management on Linux
- Cross-Platform Installers: One-liner deployment scripts
- Container Support: Docker and Kubernetes deployment options
## Architecture
```
┌─────────────────┐
│ Web Dashboard │ React + TypeScript + TailwindCSS
│ + Rate Limiting │ + Registration Token Management
└────────┬────────┘
│ HTTPS with TLS + User Authentication
┌────────▼────────┐
│ Server (Go) │ Alpha with PostgreSQL
│ + Rate Limits │ + Registration Tokens + Setup Wizard
│ + JWT Auth │ + Heartbeat System + Comprehensive API
└────────┬────────┘
│ Pull-based (agents check in every 5 min) + Rapid Polling
┌────┴────┬────────┐
│ │ │
┌───▼──┐ ┌──▼──┐ ┌──▼───┐
│Linux │ │Windows│ │Linux │
│Agent │ │Agent │ │Agent │
│+Proxy│ │+Proxy│ │+Proxy│
└──────┘ └───────┘ └──────┘
```
## Quick Start
### 1. Server Setup (Linux)
```bash
# Clone and build
git clone https://github.com/Fimeg/RedFlag.git
cd RedFlag/aggregator-server
go build -o redflag-server cmd/server/main.go
# Interactive setup wizard
sudo ./redflag-server --setup
# Follow prompts for:
# - Admin credentials
# - Database configuration
# - Server settings
# - Agent seat limits
# Start database
docker-compose up -d postgres
# Run migrations
./redflag-server --migrate
# Start server
./redflag-server
# Server: http://redflag.wiuf.net:8080
# Dashboard: http://redflag.wiuf.net:8080
```
### 2. Agent Deployment (Linux)
```bash
# Option 1: One-liner with registration token
sudo bash -c 'curl -sfL https://redflag.wiuf.net/install | bash -s -- rf-tok-abc123'
# Option 2: Manual installation
sudo ./install.sh --server https://redflag.wiuf.net:8080 --token rf-tok-abc123
# Option 3: Advanced configuration with proxy
sudo ./redflag-agent --server https://redflag.wiuf.net:8080 \
--token rf-tok-abc123 \
--proxy-http http://proxy.company.com:8080 \
--organization "my-homelab" \
--tags "production,webserver"
```
### 3. Windows Agent Deployment
```powershell
# PowerShell one-liner
iwr https://redflag.wiuf.net/install.ps1 | iex -Arguments '--server https://redflag.wiuf.net:8080 --token rf-tok-abc123'
# Or manual download and install
.\redflag-agent.exe --server https://redflag.wiuf.net:8080 --token rf-tok-abc123
```
## Agent Configuration Options
### CLI Flags (Highest Priority)
```bash
./redflag-agent --server https://redflag.wiuf.net \
--token rf-tok-abc123 \
--proxy-http http://proxy.company.com:8080 \
--proxy-https https://proxy.company.com:8080 \
--log-level debug \
--organization "my-homelab" \
--tags "production,webserver" \
--name "redflag-server-01" \
--insecure-tls
```
### Environment Variables
```bash
export REDFLAG_SERVER_URL="https://redflag.wiuf.net"
export REDFLAG_REGISTRATION_TOKEN="rf-tok-abc123"
export REDFLAG_HTTP_PROXY="http://proxy.company.com:8080"
export REDFLAG_HTTPS_PROXY="https://proxy.company.com:8080"
export REDFLAG_NO_PROXY="localhost,127.0.0.1"
export REDFLAG_LOG_LEVEL="info"
export REDFLAG_ORGANIZATION="my-homelab"
```
### Configuration File
```json
{
"server_url": "https://redflag.wiuf.net",
"registration_token": "rf-tok-abc123",
"proxy": {
"enabled": true,
"http": "http://proxy.company.com:8080",
"https": "https://proxy.company.com:8080",
"no_proxy": "localhost,127.0.0.1"
},
"network": {
"timeout": "30s",
"retry_count": 3,
"retry_delay": "5s"
},
"logging": {
"level": "info",
"max_size": 100,
"max_backups": 3
},
"tags": ["production", "webserver"],
"organization": "my-homelab",
"display_name": "redflag-server-01"
}
```
## Web Dashboard Features
### Agent Management
- Real-time Status: Online/offline with heartbeat indicators
- System Information: CPU, memory, disk usage, OS details
- Version Tracking: Agent versions and update availability
- Metadata Management: Tags, organizations, display names
- Bulk Operations: Multi-agent scanning and updates
### Update Management
- Severity Classification: Critical, high, medium, low priority updates
- Approval Workflow: Controlled update deployment with dependencies
- Dependency Resolution: Safe installation with conflict checking
- Batch Operations: Approve/install multiple updates
- Audit Trail: Complete history of all operations
### Settings & Administration
- Registration Tokens: Generate and manage secure enrollment tokens
- Rate Limiting: User-adjustable API security settings
- Authentication: Secure login with session management
- Audit Logging: Comprehensive activity tracking
- Server Configuration: Admin settings and system controls
## API Reference
### Registration Token Management
```bash
# Generate registration token
curl -X POST https://redflag.wiuf.net/api/v1/admin/registration-tokens \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-d '{"label": "Production Servers", "expires_in": "24h"}'
# List tokens
curl -X GET https://redflag.wiuf.net/api/v1/admin/registration-tokens \
-H "Authorization: Bearer $ADMIN_TOKEN"
# Revoke token
curl -X DELETE https://redflag.wiuf.net/api/v1/admin/registration-tokens/rf-tok-abc123 \
-H "Authorization: Bearer $ADMIN_TOKEN"
```
### Rate Limit Management
```bash
# View current settings
curl -X GET https://redflag.wiuf.net/api/v1/admin/rate-limits \
-H "Authorization: Bearer $ADMIN_TOKEN"
# Update settings
curl -X PUT https://redflag.wiuf.net/api/v1/admin/rate-limits \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-d '{
"agent_registration": {"requests": 10, "window": "1m", "enabled": true},
"admin_operations": {"requests": 200, "window": "1m", "enabled": true}
}'
```
## Security
### Authentication & Authorization
- Registration Tokens: One-time use tokens prevent unauthorized agent enrollment
- Refresh Token Authentication: 90-day sliding window with 24h access tokens
- SHA-256 token hashing for secure storage
- Admin authentication for server access and management
### Network Security
- Rate Limiting: Configurable API protection with professional defaults
- TLS Support: Certificate validation and client certificate support
- Pull-based Model: Agents poll server (firewall-friendly)
- HTTPS Required: Production deployments must use TLS
### System Hardening
- Minimal Privilege Execution: Agents run with least required privileges
- Command Validation: Whitelisted commands only
- Secure Defaults: Hardened configurations out of the box
- Security Hardening: Minimal privilege execution and sudoers management
### Audit & Monitoring
- Audit Trails: Complete logging of all activities
- Token Renewal: `/renew` endpoint prevents daily re-registration
- Activity Tracking: Comprehensive monitoring and alerting
- Access Logs: Full audit trail of user and agent actions
## Docker Deployment
```yaml
# docker-compose.yml
version: '3.8'
services:
redflag-server:
build: ./aggregator-server
ports:
- "8080:8080"
environment:
- REDFLAG_SERVER_HOST=0.0.0.0
- REDFLAG_SERVER_PORT=8080
- REDFLAG_DB_HOST=postgres
- REDFLAG_DB_PORT=5432
- REDFLAG_DB_NAME=redflag
- REDFLAG_DB_USER=redflag
- REDFLAG_DB_PASSWORD=secure-password
depends_on:
- postgres
volumes:
- ./redflag-data:/etc/redflag
- ./logs:/app/logs
postgres:
image: postgres:15
environment:
POSTGRES_DB: redflag
POSTGRES_USER: redflag
POSTGRES_PASSWORD: secure-password
volumes:
- postgres-data:/var/lib/postgresql/data
ports:
- "5432:5432"
```
## Project Structure
```
RedFlag/
├── aggregator-server/ # Go server backend
│ ├── cmd/server/ # Main server entry point
│ ├── internal/
│ │ ├── api/ # REST API handlers and middleware
│ │ │ └── handlers/ # API endpoint implementations
│ │ ├── database/ # Database layer with migrations
│ │ │ ├── migrations/ # Database schema migrations
│ │ │ └── queries/ # Database query functions
│ │ ├── models/ # Data models and structs
│ │ ├── services/ # Business logic services
│ │ └── config/ # Configuration management
│ └── redflag-server # Server binary
├── aggregator-agent/ # Cross-platform Go agent
│ ├── cmd/agent/ # Agent main entry point
│ ├── internal/
│ │ ├── client/ # HTTP client with token renewal
│ │ ├── config/ # Enhanced configuration system
│ │ ├── scanner/ # Update scanners for each platform
│ │ ├── installer/ # Package installers
│ │ └── system/ # System information collection
│ ├── install.sh # Linux installation script
│ └── redflag-agent # Agent binary
├── aggregator-web/ # React dashboard
├── docker-compose.yml # Development environment
├── Makefile # Common tasks
└── README.md # This file
```
## What This Is
@@ -45,9 +363,8 @@ A self-hosted, cross-platform update management platform built with:
- Event-sourced database architecture for scalability
### Known Limitations
- No rate limiting on API endpoints (security improvement needed)
- No real-time WebSocket updates
- Proxmox integration is broken (needs complete rewrite)
- Proxmox integration is not implemented in this version (planned for future release)
- Authentication system works but needs security hardening
## Screenshots
@@ -81,81 +398,6 @@ This repository contains:
- **Web dashboard** (`aggregator-web/`)
- **Database migrations** and configuration
## Architecture
```
┌─────────────────┐
│ Web Dashboard │ React + TypeScript + TailwindCSS
└────────┬────────┘
│ HTTPS
┌────────▼────────┐
│ Server (Go) │ Production Ready with PostgreSQL
│ + PostgreSQL │
└────────┬────────┘
│ Pull-based (agents check in every 5 min)
┌────┴────┬────────┐
│ │ │
┌───▼──┐ ┌──▼──┐ ┌──▼───┐
│Linux │ │Windows│ │Linux │
│Agent │ │Agent │ │Agent │
└──────┘ └───────┘ └──────┘
```
## Project Structure
```
RedFlag/
├── aggregator-server/ # Go server (Gin + PostgreSQL)
│ ├── cmd/server/ # Main entry point
│ ├── internal/
│ │ ├── api/ # HTTP handlers & middleware
│ │ │ └── handlers/ # API endpoint handlers
│ │ ├── database/ # Database layer & migrations
│ │ │ ├── migrations/ # Database schema migrations
│ │ │ └── queries/ # Database query functions
│ │ ├── models/ # Data models and structs
│ │ ├── services/ # Business logic services
│ │ ├── utils/ # Utility functions
│ │ └── config/ # Configuration management
│ └── go.mod
├── aggregator-agent/ # Go agent (cross-platform)
│ ├── cmd/agent/ # Main entry point
│ ├── internal/
│ │ ├── cache/ # Local cache system for offline viewing
│ │ ├── client/ # API client with token renewal
│ │ ├── config/ # Configuration management
│ │ ├── display/ # Terminal output formatting
│ │ ├── installer/ # Update installers
│ │ │ ├── apt.go # APT package installer
│ │ │ ├── dnf.go # DNF package installer
│ │ │ ├── docker.go # Docker image installer
│ │ │ ├── windows.go # Windows installer base
│ │ │ ├── winget.go # Winget package installer
│ │ │ ├── security.go # Security utilities
│ │ │ └── sudoers.go # Sudo management
│ │ ├── scanner/ # Update scanners
│ │ │ ├── apt.go # APT package scanner
│ │ │ ├── dnf.go # DNF package scanner
│ │ │ ├── docker.go # Docker image scanner
│ │ │ ├── registry.go # Docker registry client
│ │ │ ├── windows.go # Windows Update scanner
│ │ │ ├── winget.go # Winget package scanner
│ │ │ └── windows_*.go # Windows Update API components
│ │ ├── system/ # System information collection
│ │ │ ├── info.go # System metrics
│ │ │ └── windows.go # Windows system info
│ │ └── executor/ # Command execution
│ ├── install.sh # Linux installation script
│ ├── uninstall.sh # Linux uninstallation script
│ └── go.mod
├── aggregator-web/ # React dashboard
├── docker-compose.yml # PostgreSQL for local dev
├── Makefile # Common tasks
└── README.md # This file
```
## Database Schema
Key Tables:
@@ -260,15 +502,6 @@ curl -X POST http://localhost:8080/api/v1/updates/{update-id}/approve
curl -X POST http://localhost:8080/api/v1/updates/{update-id}/confirm-dependencies
```
## Security
- Agent Authentication: Refresh token system with 90-day sliding window + 24h access tokens
- SHA-256 token hashing for secure storage
- Pull-based Model: Agents poll server (firewall-friendly)
- Command Validation: Whitelisted commands only
- TLS Required: Production deployments must use HTTPS
- Token Renewal: `/renew` endpoint prevents daily re-registration
## License
MIT License - see LICENSE file for details.

BIN
aggregator-agent/aggregator-agent Executable file
View File

Binary file not shown.

283
aggregator-agent/cmd/agent/main.go
View File

@@ -21,7 +21,7 @@ import (
)
const (
AgentVersion = "0.1.8" // Added dnf makecache to security allowlist, retry tracking
AgentVersion = "0.1.16" // Enhanced configuration system with proxy support and registration tokens
)
// getConfigPath returns the platform-specific config path
@@ -32,6 +32,26 @@ func getConfigPath() string {
return "/etc/aggregator/config.json"
}
// getCurrentPollingInterval returns the appropriate polling interval based on rapid mode
func getCurrentPollingInterval(cfg *config.Config) int {
// Check if rapid polling mode is active and not expired
if cfg.RapidPollingEnabled && time.Now().Before(cfg.RapidPollingUntil) {
return 5 // Rapid polling: 5 seconds
}
// Check if rapid polling has expired and clean up
if cfg.RapidPollingEnabled && time.Now().After(cfg.RapidPollingUntil) {
cfg.RapidPollingEnabled = false
cfg.RapidPollingUntil = time.Time{}
// Save the updated config to clean up expired rapid mode
if err := cfg.Save(getConfigPath()); err != nil {
log.Printf("Warning: Failed to cleanup expired rapid polling mode: %v", err)
}
}
return cfg.CheckInInterval // Normal polling: 5 minutes (300 seconds) by default
}
// getDefaultServerURL returns the default server URL with environment variable support
func getDefaultServerURL() string {
// Check environment variable first
@@ -48,16 +68,65 @@ func getDefaultServerURL() string {
}
func main() {
// Define CLI flags
registerCmd := flag.Bool("register", false, "Register agent with server")
scanCmd := flag.Bool("scan", false, "Scan for updates and display locally")
statusCmd := flag.Bool("status", false, "Show agent status")
listUpdatesCmd := flag.Bool("list-updates", false, "List detailed update information")
serverURL := flag.String("server", getDefaultServerURL(), "Server URL")
versionCmd := flag.Bool("version", false, "Show version information")
serverURL := flag.String("server", "", "Server URL")
registrationToken := flag.String("token", "", "Registration token for secure enrollment")
proxyHTTP := flag.String("proxy-http", "", "HTTP proxy URL")
proxyHTTPS := flag.String("proxy-https", "", "HTTPS proxy URL")
proxyNoProxy := flag.String("proxy-no", "", "Comma-separated hosts to bypass proxy")
logLevel := flag.String("log-level", "", "Log level (debug, info, warn, error)")
configFile := flag.String("config", "", "Configuration file path")
tagsFlag := flag.String("tags", "", "Comma-separated tags for agent")
organization := flag.String("organization", "", "Organization/group name")
displayName := flag.String("name", "", "Display name for agent")
insecureTLS := flag.Bool("insecure-tls", false, "Skip TLS certificate verification")
exportFormat := flag.String("export", "", "Export format: json, csv")
flag.Parse()
// Load configuration
cfg, err := config.Load(getConfigPath())
// Handle version command
if *versionCmd {
fmt.Printf("RedFlag Agent v%s\n", AgentVersion)
fmt.Printf("Self-hosted update management platform\n")
os.Exit(0)
}
// Parse tags from comma-separated string
var tags []string
if *tagsFlag != "" {
tags = strings.Split(*tagsFlag, ",")
for i, tag := range tags {
tags[i] = strings.TrimSpace(tag)
}
}
// Create CLI flags structure
cliFlags := &config.CLIFlags{
ServerURL: *serverURL,
RegistrationToken: *registrationToken,
ProxyHTTP: *proxyHTTP,
ProxyHTTPS: *proxyHTTPS,
ProxyNoProxy: *proxyNoProxy,
LogLevel: *logLevel,
ConfigFile: *configFile,
Tags: tags,
Organization: *organization,
DisplayName: *displayName,
InsecureTLS: *insecureTLS,
}
// Determine config path
configPath := getConfigPath()
if *configFile != "" {
configPath = *configFile
}
// Load configuration with priority: CLI > env > file > defaults
cfg, err := config.Load(configPath, cliFlags)
if err != nil {
log.Fatal("Failed to load configuration:", err)
}
@@ -313,6 +382,24 @@ func runAgent(cfg *config.Config) error {
}
}
// Add heartbeat status to metrics metadata if available
if metrics != nil && cfg.RapidPollingEnabled {
// Check if rapid polling is still valid
if time.Now().Before(cfg.RapidPollingUntil) {
// Include heartbeat metadata in metrics
if metrics.Metadata == nil {
metrics.Metadata = make(map[string]interface{})
}
metrics.Metadata["rapid_polling_enabled"] = true
metrics.Metadata["rapid_polling_until"] = cfg.RapidPollingUntil.Format(time.RFC3339)
metrics.Metadata["rapid_polling_duration_minutes"] = int(time.Until(cfg.RapidPollingUntil).Minutes())
} else {
// Heartbeat expired, disable it
cfg.RapidPollingEnabled = false
cfg.RapidPollingUntil = time.Time{}
}
}
// Get commands from server (with optional metrics)
commands, err := apiClient.GetCommands(cfg.AgentID, metrics)
if err != nil {
@@ -320,7 +407,7 @@ func runAgent(cfg *config.Config) error {
newClient, renewErr := renewTokenIfNeeded(apiClient, cfg, err)
if renewErr != nil {
log.Printf("Check-in unsuccessful and token renewal failed: %v\n", renewErr)
time.Sleep(time.Duration(cfg.CheckInInterval) * time.Second)
time.Sleep(time.Duration(getCurrentPollingInterval(cfg)) * time.Second)
continue
}
@@ -331,12 +418,12 @@ func runAgent(cfg *config.Config) error {
commands, err = apiClient.GetCommands(cfg.AgentID, metrics)
if err != nil {
log.Printf("Check-in unsuccessful even after token renewal: %v\n", err)
time.Sleep(time.Duration(cfg.CheckInInterval) * time.Second)
time.Sleep(time.Duration(getCurrentPollingInterval(cfg)) * time.Second)
continue
}
} else {
log.Printf("Check-in unsuccessful: %v\n", err)
time.Sleep(time.Duration(cfg.CheckInInterval) * time.Second)
time.Sleep(time.Duration(getCurrentPollingInterval(cfg)) * time.Second)
continue
}
}
@@ -375,13 +462,23 @@ func runAgent(cfg *config.Config) error {
log.Printf("Error confirming dependencies: %v\n", err)
}
case "enable_heartbeat":
if err := handleEnableHeartbeat(apiClient, cfg, cmd.ID, cmd.Params); err != nil {
log.Printf("[Heartbeat] Error enabling heartbeat: %v\n", err)
}
case "disable_heartbeat":
if err := handleDisableHeartbeat(apiClient, cfg, cmd.ID); err != nil {
log.Printf("[Heartbeat] Error disabling heartbeat: %v\n", err)
}
default:
log.Printf("Unknown command type: %s\n", cmd.Type)
}
}
// Wait for next check-in
time.Sleep(time.Duration(cfg.CheckInInterval) * time.Second)
time.Sleep(time.Duration(getCurrentPollingInterval(cfg)) * time.Second)
}
}
@@ -743,9 +840,9 @@ func handleInstallUpdates(apiClient *client.Client, cfg *config.Config, commandI
// Perform installation based on what's specified
if packageName != "" {
action = "install"
log.Printf("Installing package: %s (type: %s)", packageName, packageType)
result, err = inst.Install(packageName)
action = "update"
log.Printf("Updating package: %s (type: %s)", packageName, packageType)
result, err = inst.UpdatePackage(packageName)
} else if len(params) > 1 {
// Multiple packages might be specified in various ways
var packageNames []string
@@ -774,15 +871,15 @@ func handleInstallUpdates(apiClient *client.Client, cfg *config.Config, commandI
}
if err != nil {
// Report installation failure
// Report installation failure with actual command output
logReport := client.LogReport{
CommandID: commandID,
Action: action,
Result: "failed",
Stdout: "",
Stderr: fmt.Sprintf("Installation error: %v", err),
ExitCode: 1,
DurationSeconds: 0,
Stdout: result.Stdout,
Stderr: result.Stderr,
ExitCode: result.ExitCode,
DurationSeconds: result.DurationSeconds,
}
if reportErr := apiClient.ReportLog(cfg.AgentID, logReport); reportErr != nil {
@@ -991,21 +1088,22 @@ func handleConfirmDependencies(apiClient *client.Client, cfg *config.Config, com
allPackages := append([]string{packageName}, dependencies...)
result, err = inst.InstallMultiple(allPackages)
} else {
action = "install"
action = "upgrade"
log.Printf("Installing package: %s (no dependencies)", packageName)
result, err = inst.Install(packageName)
// Use UpdatePackage instead of Install to handle existing packages
result, err = inst.UpdatePackage(packageName)
}
if err != nil {
// Report installation failure
// Report installation failure with actual command output
logReport := client.LogReport{
CommandID: commandID,
Action: action,
Result: "failed",
Stdout: "",
Stderr: fmt.Sprintf("Installation error: %v", err),
ExitCode: 1,
DurationSeconds: 0,
Stdout: result.Stdout,
Stderr: result.Stderr,
ExitCode: result.ExitCode,
DurationSeconds: result.DurationSeconds,
}
if reportErr := apiClient.ReportLog(cfg.AgentID, logReport); reportErr != nil {
@@ -1051,6 +1149,145 @@ func handleConfirmDependencies(apiClient *client.Client, cfg *config.Config, com
return nil
}
// handleEnableHeartbeat handles enable_heartbeat command
func handleEnableHeartbeat(apiClient *client.Client, cfg *config.Config, commandID string, params map[string]interface{}) error {
// Parse duration parameter (default to 10 minutes)
durationMinutes := 10
if duration, ok := params["duration_minutes"]; ok {
if durationFloat, ok := duration.(float64); ok {
durationMinutes = int(durationFloat)
}
}
// Calculate when heartbeat should expire
expiryTime := time.Now().Add(time.Duration(durationMinutes) * time.Minute)
log.Printf("[Heartbeat] Enabling rapid polling for %d minutes (expires: %s)", durationMinutes, expiryTime.Format(time.RFC3339))
// Update agent config to enable rapid polling
cfg.RapidPollingEnabled = true
cfg.RapidPollingUntil = expiryTime
// Save config to persist heartbeat settings
if err := cfg.Save(getConfigPath()); err != nil {
log.Printf("[Heartbeat] Warning: Failed to save config: %v", err)
}
// Create log report for heartbeat enable
logReport := client.LogReport{
CommandID: commandID,
Action: "enable_heartbeat",
Result: "success",
Stdout: fmt.Sprintf("Heartbeat enabled for %d minutes", durationMinutes),
Stderr: "",
ExitCode: 0,
DurationSeconds: 0,
}
if reportErr := apiClient.ReportLog(cfg.AgentID, logReport); reportErr != nil {
log.Printf("[Heartbeat] Failed to report heartbeat enable: %v", reportErr)
}
// Send immediate check-in to update heartbeat status in UI
log.Printf("[Heartbeat] Sending immediate check-in to update status")
sysMetrics, err := system.GetLightweightMetrics()
if err == nil {
metrics := &client.SystemMetrics{
CPUPercent: sysMetrics.CPUPercent,
MemoryPercent: sysMetrics.MemoryPercent,
MemoryUsedGB: sysMetrics.MemoryUsedGB,
MemoryTotalGB: sysMetrics.MemoryTotalGB,
DiskUsedGB: sysMetrics.DiskUsedGB,
DiskTotalGB: sysMetrics.DiskTotalGB,
DiskPercent: sysMetrics.DiskPercent,
Uptime: sysMetrics.Uptime,
Version: AgentVersion,
}
// Include heartbeat metadata to show enabled state
metrics.Metadata = map[string]interface{}{
"rapid_polling_enabled": true,
"rapid_polling_until": expiryTime.Format(time.RFC3339),
}
// Send immediate check-in with updated heartbeat status
_, checkinErr := apiClient.GetCommands(cfg.AgentID, metrics)
if checkinErr != nil {
log.Printf("[Heartbeat] Failed to send immediate check-in: %v", checkinErr)
} else {
log.Printf("[Heartbeat] Immediate check-in sent successfully")
}
} else {
log.Printf("[Heartbeat] Failed to get system metrics for immediate check-in: %v", err)
}
log.Printf("[Heartbeat] Rapid polling enabled successfully")
return nil
}
// handleDisableHeartbeat handles disable_heartbeat command
func handleDisableHeartbeat(apiClient *client.Client, cfg *config.Config, commandID string) error {
log.Printf("[Heartbeat] Disabling rapid polling")
// Update agent config to disable rapid polling
cfg.RapidPollingEnabled = false
cfg.RapidPollingUntil = time.Time{} // Zero value
// Save config to persist heartbeat settings
if err := cfg.Save(getConfigPath()); err != nil {
log.Printf("[Heartbeat] Warning: Failed to save config: %v", err)
}
// Create log report for heartbeat disable
logReport := client.LogReport{
CommandID: commandID,
Action: "disable_heartbeat",
Result: "success",
Stdout: "Heartbeat disabled",
Stderr: "",
ExitCode: 0,
DurationSeconds: 0,
}
if reportErr := apiClient.ReportLog(cfg.AgentID, logReport); reportErr != nil {
log.Printf("[Heartbeat] Failed to report heartbeat disable: %v", reportErr)
}
// Send immediate check-in to update heartbeat status in UI
log.Printf("[Heartbeat] Sending immediate check-in to update status")
sysMetrics, err := system.GetLightweightMetrics()
if err == nil {
metrics := &client.SystemMetrics{
CPUPercent: sysMetrics.CPUPercent,
MemoryPercent: sysMetrics.MemoryPercent,
MemoryUsedGB: sysMetrics.MemoryUsedGB,
MemoryTotalGB: sysMetrics.MemoryTotalGB,
DiskUsedGB: sysMetrics.DiskUsedGB,
DiskTotalGB: sysMetrics.DiskTotalGB,
DiskPercent: sysMetrics.DiskPercent,
Uptime: sysMetrics.Uptime,
Version: AgentVersion,
}
// Include empty heartbeat metadata to explicitly show disabled state
metrics.Metadata = map[string]interface{}{
"rapid_polling_enabled": false,
"rapid_polling_until": "",
}
// Send immediate check-in with updated heartbeat status
_, checkinErr := apiClient.GetCommands(cfg.AgentID, metrics)
if checkinErr != nil {
log.Printf("[Heartbeat] Failed to send immediate check-in: %v", checkinErr)
} else {
log.Printf("[Heartbeat] Immediate check-in sent successfully")
}
} else {
log.Printf("[Heartbeat] Failed to get system metrics for immediate check-in: %v", err)
}
log.Printf("[Heartbeat] Rapid polling disabled successfully")
return nil
}
// reportSystemInfo collects and reports detailed system information to the server
func reportSystemInfo(apiClient *client.Client, cfg *config.Config) error {
// Collect detailed system information

28
aggregator-agent/install.sh
View File

@@ -29,6 +29,16 @@ create_user() {
useradd -r -s /bin/false -d "$AGENT_HOME" -m "$AGENT_USER"
echo "✓ User $AGENT_USER created"
fi
# Add user to docker group for Docker update scanning
if getent group docker &>/dev/null; then
echo "Adding $AGENT_USER to docker group..."
usermod -aG docker "$AGENT_USER"
echo "✓ User $AGENT_USER added to docker group"
else
echo "⚠ Docker group not found - Docker updates will not be available"
echo " (Install Docker first, then reinstall the agent to enable Docker support)"
fi
}
# Function to build agent binary
@@ -58,19 +68,19 @@ install_sudoers() {
# APT package management commands
redflag-agent ALL=(root) NOPASSWD: /usr/bin/apt-get update
redflag-agent ALL=(root) NOPASSWD: /usr/bin/apt-get install -y *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/apt-get upgrade -y
redflag-agent ALL=(root) NOPASSWD: /usr/bin/apt-get upgrade -y *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/apt-get install --dry-run --yes *
# DNF package management commands
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf refresh -y
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf makecache
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf install -y *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf upgrade -y
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf upgrade -y *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/dnf install --assumeno --downloadonly *
# Docker operations (uncomment if needed)
# redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker pull *
# redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker image inspect *
# redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker manifest inspect *
# Docker operations
redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker pull *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker image inspect *
redflag-agent ALL=(root) NOPASSWD: /usr/bin/docker manifest inspect *
EOF
chmod 440 "$SUDOERS_FILE"
@@ -103,10 +113,10 @@ Restart=always
RestartSec=30
# Security hardening
NoNewPrivileges=true
# NoNewPrivileges=true - DISABLED: Prevents sudo from working
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=$AGENT_HOME
ReadWritePaths=$AGENT_HOME /var/log /etc/aggregator
PrivateTmp=true
[Install]

46
aggregator-agent/internal/client/client.go
View File

@@ -16,9 +16,11 @@ import (
// Client handles API communication with the server
type Client struct {
baseURL string
token string
http *http.Client
baseURL string
token string
http *http.Client
RapidPollingEnabled bool
RapidPollingUntil time.Time
}
// NewClient creates a new API client
@@ -159,20 +161,28 @@ type Command struct {
// CommandsResponse contains pending commands
type CommandsResponse struct {
Commands []Command `json:"commands"`
Commands []Command `json:"commands"`
RapidPolling *RapidPollingConfig `json:"rapid_polling,omitempty"`
}
// RapidPollingConfig contains rapid polling configuration from server
type RapidPollingConfig struct {
Enabled bool `json:"enabled"`
Until string `json:"until"` // ISO 8601 timestamp
}
// SystemMetrics represents lightweight system metrics sent with check-ins
type SystemMetrics struct {
CPUPercent float64 `json:"cpu_percent,omitempty"`
MemoryPercent float64 `json:"memory_percent,omitempty"`
MemoryUsedGB float64 `json:"memory_used_gb,omitempty"`
MemoryTotalGB float64 `json:"memory_total_gb,omitempty"`
DiskUsedGB float64 `json:"disk_used_gb,omitempty"`
DiskTotalGB float64 `json:"disk_total_gb,omitempty"`
DiskPercent float64 `json:"disk_percent,omitempty"`
Uptime string `json:"uptime,omitempty"`
Version string `json:"version,omitempty"` // Agent version
CPUPercent float64 `json:"cpu_percent,omitempty"`
MemoryPercent float64 `json:"memory_percent,omitempty"`
MemoryUsedGB float64 `json:"memory_used_gb,omitempty"`
MemoryTotalGB float64 `json:"memory_total_gb,omitempty"`
DiskUsedGB float64 `json:"disk_used_gb,omitempty"`
DiskTotalGB float64 `json:"disk_total_gb,omitempty"`
DiskPercent float64 `json:"disk_percent,omitempty"`
Uptime string `json:"uptime,omitempty"`
Version string `json:"version,omitempty"` // Agent version
Metadata map[string]interface{} `json:"metadata,omitempty"` // Additional metadata
}
// GetCommands retrieves pending commands from the server
@@ -219,6 +229,16 @@ func (c *Client) GetCommands(agentID uuid.UUID, metrics *SystemMetrics) ([]Comma
return nil, err
}
// Handle rapid polling configuration if provided
if result.RapidPolling != nil {
// Parse the timestamp
if until, err := time.Parse(time.RFC3339, result.RapidPolling.Until); err == nil {
// Update client's rapid polling configuration
c.RapidPollingEnabled = result.RapidPolling.Enabled
c.RapidPollingUntil = until
}
}
return result.Commands, nil
}

330
aggregator-agent/internal/config/config.go
View File

@@ -5,21 +5,152 @@ import (
"fmt"
"os"
"path/filepath"
"time"
"github.com/google/uuid"
)
// Config holds agent configuration
type Config struct {
ServerURL string `json:"server_url"`
AgentID uuid.UUID `json:"agent_id"`
Token string `json:"token"` // Short-lived access token (24h)
RefreshToken string `json:"refresh_token"` // Long-lived refresh token (90d)
CheckInInterval int `json:"check_in_interval"`
// ProxyConfig holds proxy configuration
type ProxyConfig struct {
Enabled bool `json:"enabled"`
HTTP string `json:"http,omitempty"` // HTTP proxy URL
HTTPS string `json:"https,omitempty"` // HTTPS proxy URL
NoProxy string `json:"no_proxy,omitempty"` // Comma-separated hosts to bypass proxy
Username string `json:"username,omitempty"` // Proxy username (optional)
Password string `json:"password,omitempty"` // Proxy password (optional)
}
// Load reads configuration from file
func Load(configPath string) (*Config, error) {
// TLSConfig holds TLS/security configuration
type TLSConfig struct {
InsecureSkipVerify bool `json:"insecure_skip_verify"` // Skip TLS certificate verification
CertFile string `json:"cert_file,omitempty"` // Client certificate file
KeyFile string `json:"key_file,omitempty"` // Client key file
CAFile string `json:"ca_file,omitempty"` // CA certificate file
}
// NetworkConfig holds network-related configuration
type NetworkConfig struct {
Timeout time.Duration `json:"timeout"` // Request timeout
RetryCount int `json:"retry_count"` // Number of retries
RetryDelay time.Duration `json:"retry_delay"` // Delay between retries
MaxIdleConn int `json:"max_idle_conn"` // Maximum idle connections
}
// LoggingConfig holds logging configuration
type LoggingConfig struct {
Level string `json:"level"` // Log level (debug, info, warn, error)
File string `json:"file,omitempty"` // Log file path (optional)
MaxSize int `json:"max_size"` // Max log file size in MB
MaxBackups int `json:"max_backups"` // Max number of log file backups
MaxAge int `json:"max_age"` // Max age of log files in days
}
// Config holds agent configuration
type Config struct {
// Server Configuration
ServerURL string `json:"server_url"`
RegistrationToken string `json:"registration_token,omitempty"` // One-time registration token
// Agent Authentication
AgentID uuid.UUID `json:"agent_id"`
Token string `json:"token"` // Short-lived access token (24h)
RefreshToken string `json:"refresh_token"` // Long-lived refresh token (90d)
// Agent Behavior
CheckInInterval int `json:"check_in_interval"`
// Rapid polling mode for faster response during operations
RapidPollingEnabled bool `json:"rapid_polling_enabled"`
RapidPollingUntil time.Time `json:"rapid_polling_until"`
// Network Configuration
Network NetworkConfig `json:"network,omitempty"`
// Proxy Configuration
Proxy ProxyConfig `json:"proxy,omitempty"`
// Security Configuration
TLS TLSConfig `json:"tls,omitempty"`
// Logging Configuration
Logging LoggingConfig `json:"logging,omitempty"`
// Agent Metadata
Tags []string `json:"tags,omitempty"` // User-defined tags
Metadata map[string]string `json:"metadata,omitempty"` // Custom metadata
DisplayName string `json:"display_name,omitempty"` // Human-readable name
Organization string `json:"organization,omitempty"` // Organization/group
}
// Load reads configuration from multiple sources with priority order:
// 1. CLI flags
// 2. Environment variables
// 3. Configuration file
// 4. Default values
func Load(configPath string, cliFlags *CLIFlags) (*Config, error) {
// Start with defaults
config := getDefaultConfig()
// Load from config file if it exists
if fileConfig, err := loadFromFile(configPath); err == nil {
mergeConfig(config, fileConfig)
}
// Override with environment variables
mergeConfig(config, loadFromEnv())
// Override with CLI flags (highest priority)
if cliFlags != nil {
mergeConfig(config, loadFromFlags(cliFlags))
}
// Validate configuration
if err := validateConfig(config); err != nil {
return nil, fmt.Errorf("invalid configuration: %w", err)
}
return config, nil
}
// CLIFlags holds command line flag values
type CLIFlags struct {
ServerURL string
RegistrationToken string
ProxyHTTP string
ProxyHTTPS string
ProxyNoProxy string
LogLevel string
ConfigFile string
Tags []string
Organization string
DisplayName string
InsecureTLS bool
}
// getDefaultConfig returns default configuration values
func getDefaultConfig() *Config {
return &Config{
ServerURL: "http://localhost:8080",
CheckInInterval: 300, // 5 minutes
Network: NetworkConfig{
Timeout: 30 * time.Second,
RetryCount: 3,
RetryDelay: 5 * time.Second,
MaxIdleConn: 10,
},
Logging: LoggingConfig{
Level: "info",
MaxSize: 100, // 100MB
MaxBackups: 3,
MaxAge: 28, // 28 days
},
Tags: []string{},
Metadata: make(map[string]string),
}
}
// loadFromFile reads configuration from file
func loadFromFile(configPath string) (*Config, error) {
// Ensure directory exists
dir := filepath.Dir(configPath)
if err := os.MkdirAll(dir, 0755); err != nil {
@@ -30,8 +161,7 @@ func Load(configPath string) (*Config, error) {
data, err := os.ReadFile(configPath)
if err != nil {
if os.IsNotExist(err) {
// Return empty config if file doesn't exist
return &Config{}, nil
return getDefaultConfig(), nil // Return defaults if file doesn't exist
}
return nil, fmt.Errorf("failed to read config: %w", err)
}
@@ -44,6 +174,174 @@ func Load(configPath string) (*Config, error) {
return &config, nil
}
// loadFromEnv loads configuration from environment variables
func loadFromEnv() *Config {
config := &Config{}
if serverURL := os.Getenv("REDFLAG_SERVER_URL"); serverURL != "" {
config.ServerURL = serverURL
}
if token := os.Getenv("REDFLAG_REGISTRATION_TOKEN"); token != "" {
config.RegistrationToken = token
}
if proxyHTTP := os.Getenv("REDFLAG_HTTP_PROXY"); proxyHTTP != "" {
config.Proxy.Enabled = true
config.Proxy.HTTP = proxyHTTP
}
if proxyHTTPS := os.Getenv("REDFLAG_HTTPS_PROXY"); proxyHTTPS != "" {
config.Proxy.Enabled = true
config.Proxy.HTTPS = proxyHTTPS
}
if noProxy := os.Getenv("REDFLAG_NO_PROXY"); noProxy != "" {
config.Proxy.NoProxy = noProxy
}
if logLevel := os.Getenv("REDFLAG_LOG_LEVEL"); logLevel != "" {
if config.Logging == (LoggingConfig{}) {
config.Logging = LoggingConfig{}
}
config.Logging.Level = logLevel
}
if org := os.Getenv("REDFLAG_ORGANIZATION"); org != "" {
config.Organization = org
}
if displayName := os.Getenv("REDFLAG_DISPLAY_NAME"); displayName != "" {
config.DisplayName = displayName
}
return config
}
// loadFromFlags loads configuration from CLI flags
func loadFromFlags(flags *CLIFlags) *Config {
config := &Config{}
if flags.ServerURL != "" {
config.ServerURL = flags.ServerURL
}
if flags.RegistrationToken != "" {
config.RegistrationToken = flags.RegistrationToken
}
if flags.ProxyHTTP != "" || flags.ProxyHTTPS != "" {
config.Proxy = ProxyConfig{
Enabled: true,
HTTP: flags.ProxyHTTP,
HTTPS: flags.ProxyHTTPS,
NoProxy: flags.ProxyNoProxy,
}
}
if flags.LogLevel != "" {
config.Logging = LoggingConfig{
Level: flags.LogLevel,
}
}
if len(flags.Tags) > 0 {
config.Tags = flags.Tags
}
if flags.Organization != "" {
config.Organization = flags.Organization
}
if flags.DisplayName != "" {
config.DisplayName = flags.DisplayName
}
if flags.InsecureTLS {
config.TLS = TLSConfig{
InsecureSkipVerify: true,
}
}
return config
}
// mergeConfig merges source config into target config (non-zero values only)
func mergeConfig(target, source *Config) {
if source.ServerURL != "" {
target.ServerURL = source.ServerURL
}
if source.RegistrationToken != "" {
target.RegistrationToken = source.RegistrationToken
}
if source.CheckInInterval != 0 {
target.CheckInInterval = source.CheckInInterval
}
if source.AgentID != uuid.Nil {
target.AgentID = source.AgentID
}
if source.Token != "" {
target.Token = source.Token
}
if source.RefreshToken != "" {
target.RefreshToken = source.RefreshToken
}
// Merge nested configs
if source.Network != (NetworkConfig{}) {
target.Network = source.Network
}
if source.Proxy != (ProxyConfig{}) {
target.Proxy = source.Proxy
}
if source.TLS != (TLSConfig{}) {
target.TLS = source.TLS
}
if source.Logging != (LoggingConfig{}) {
target.Logging = source.Logging
}
// Merge metadata
if source.Tags != nil {
target.Tags = source.Tags
}
if source.Metadata != nil {
if target.Metadata == nil {
target.Metadata = make(map[string]string)
}
for k, v := range source.Metadata {
target.Metadata[k] = v
}
}
if source.DisplayName != "" {
target.DisplayName = source.DisplayName
}
if source.Organization != "" {
target.Organization = source.Organization
}
// Merge rapid polling settings
target.RapidPollingEnabled = source.RapidPollingEnabled
if !source.RapidPollingUntil.IsZero() {
target.RapidPollingUntil = source.RapidPollingUntil
}
}
// validateConfig validates configuration values
func validateConfig(config *Config) error {
if config.ServerURL == "" {
return fmt.Errorf("server_url is required")
}
if config.CheckInInterval < 30 {
return fmt.Errorf("check_in_interval must be at least 30 seconds")
}
if config.CheckInInterval > 3600 {
return fmt.Errorf("check_in_interval cannot exceed 3600 seconds (1 hour)")
}
if config.Network.Timeout <= 0 {
return fmt.Errorf("network timeout must be positive")
}
if config.Network.RetryCount < 0 || config.Network.RetryCount > 10 {
return fmt.Errorf("retry_count must be between 0 and 10")
}
// Validate log level
validLogLevels := map[string]bool{
"debug": true, "info": true, "warn": true, "error": true,
}
if config.Logging.Level != "" && !validLogLevels[config.Logging.Level] {
return fmt.Errorf("invalid log level: %s", config.Logging.Level)
}
return nil
}
// Save writes configuration to file
func (c *Config) Save(configPath string) error {
data, err := json.MarshalIndent(c, "", " ")
@@ -62,3 +360,13 @@ func (c *Config) Save(configPath string) error {
func (c *Config) IsRegistered() bool {
return c.AgentID != uuid.Nil && c.Token != ""
}
// NeedsRegistration checks if the agent needs to register with a token
func (c *Config) NeedsRegistration() bool {
return c.RegistrationToken != "" && c.AgentID == uuid.Nil
}
// HasRegistrationToken checks if the agent has a registration token
func (c *Config) HasRegistrationToken() bool {
return c.RegistrationToken != ""
}

30
aggregator-agent/internal/installer/apt.go
View File

@@ -146,6 +146,36 @@ func (i *APTInstaller) Upgrade() (*InstallResult, error) {
}, nil
}
// UpdatePackage updates a specific package using APT
func (i *APTInstaller) UpdatePackage(packageName string) (*InstallResult, error) {
startTime := time.Now()
// Update specific package using secure executor
updateResult, err := i.executor.ExecuteCommand("apt-get", []string{"install", "--only-upgrade", "-y", packageName})
duration := int(time.Since(startTime).Seconds())
if err != nil {
return &InstallResult{
Success: false,
ErrorMessage: fmt.Sprintf("APT update failed: %v", err),
Stdout: updateResult.Stdout,
Stderr: updateResult.Stderr,
ExitCode: updateResult.ExitCode,
DurationSeconds: duration,
}, err
}
return &InstallResult{
Success: true,
Stdout: updateResult.Stdout,
Stderr: updateResult.Stderr,
ExitCode: updateResult.ExitCode,
DurationSeconds: duration,
PackagesInstalled: []string{packageName},
Action: "update",
}, nil
}
// DryRun performs a dry run installation to check dependencies
func (i *APTInstaller) DryRun(packageName string) (*InstallResult, error) {
startTime := time.Now()

52
aggregator-agent/internal/installer/dnf.go
View File

@@ -31,15 +31,8 @@ func (i *DNFInstaller) IsAvailable() bool {
func (i *DNFInstaller) Install(packageName string) (*InstallResult, error) {
startTime := time.Now()
// Refresh package cache first using secure executor
refreshResult, err := i.executor.ExecuteCommand("dnf", []string{"makecache"})
if err != nil {
refreshResult.DurationSeconds = int(time.Since(startTime).Seconds())
refreshResult.ErrorMessage = fmt.Sprintf("Failed to refresh DNF cache: %v", err)
return refreshResult, fmt.Errorf("dnf refresh failed: %w", err)
}
// Install package using secure executor
// For single package installs, skip makecache to avoid repository conflicts
// Only run makecache when installing multiple packages (InstallMultiple method)
installResult, err := i.executor.ExecuteCommand("dnf", []string{"install", "-y", packageName})
duration := int(time.Since(startTime).Seconds())
@@ -75,17 +68,11 @@ func (i *DNFInstaller) InstallMultiple(packageNames []string) (*InstallResult, e
startTime := time.Now()
// Refresh package cache first using secure executor
refreshResult, err := i.executor.ExecuteCommand("dnf", []string{"makecache"})
if err != nil {
refreshResult.DurationSeconds = int(time.Since(startTime).Seconds())
refreshResult.ErrorMessage = fmt.Sprintf("Failed to refresh DNF cache: %v", err)
return refreshResult, fmt.Errorf("dnf refresh failed: %w", err)
}
// Install all packages in one command using secure executor
args := []string{"install", "-y"}
args = append(args, packageNames...)
// Install all packages in one command using secure executor
installResult, err := i.executor.ExecuteCommand("dnf", args)
duration := int(time.Since(startTime).Seconds())
@@ -299,6 +286,37 @@ func (i *DNFInstaller) extractPackageNameFromDNFLine(line string) string {
return ""
}
// UpdatePackage updates a specific package using DNF
func (i *DNFInstaller) UpdatePackage(packageName string) (*InstallResult, error) {
startTime := time.Now()
// Update specific package using secure executor
// Use 'dnf upgrade' instead of 'dnf install' for existing packages
updateResult, err := i.executor.ExecuteCommand("dnf", []string{"upgrade", "-y", packageName})
duration := int(time.Since(startTime).Seconds())
if err != nil {
return &InstallResult{
Success: false,
ErrorMessage: fmt.Sprintf("DNF upgrade failed: %v", err),
Stdout: updateResult.Stdout,
Stderr: updateResult.Stderr,
ExitCode: updateResult.ExitCode,
DurationSeconds: duration,
}, err
}
return &InstallResult{
Success: true,
Stdout: updateResult.Stdout,
Stderr: updateResult.Stderr,
ExitCode: updateResult.ExitCode,
DurationSeconds: duration,
PackagesInstalled: []string{packageName},
Action: "upgrade",
}, nil
}
// GetPackageType returns type of packages this installer handles
func (i *DNFInstaller) GetPackageType() string {
return "dnf"

6
aggregator-agent/internal/installer/docker.go
View File

@@ -60,6 +60,12 @@ func (i *DockerInstaller) Update(imageName, targetVersion string) (*InstallResul
}, nil
}
// UpdatePackage updates a specific Docker image (alias for Update method)
func (i *DockerInstaller) UpdatePackage(imageName string) (*InstallResult, error) {
// Docker uses same logic for updating as installing
return i.Update(imageName, "")
}
// Install installs a Docker image (alias for Update)
func (i *DockerInstaller) Install(imageName string) (*InstallResult, error) {
return i.Update(imageName, "")

1
aggregator-agent/internal/installer/installer.go
View File

@@ -8,6 +8,7 @@ type Installer interface {
Install(packageName string) (*InstallResult, error)
InstallMultiple(packageNames []string) (*InstallResult, error)
Upgrade() (*InstallResult, error)
UpdatePackage(packageName string) (*InstallResult, error) // New: Update specific package
GetPackageType() string
DryRun(packageName string) (*InstallResult, error) // New: Perform dry run to check dependencies
}

24
aggregator-agent/internal/installer/security.go
View File

@@ -23,6 +23,7 @@ var AllowedCommands = map[string][]string{
},
"dnf": {
"refresh",
"makecache",
"install",
"upgrade",
},
@@ -93,6 +94,9 @@ func (e *SecureCommandExecutor) validateDNFCommand(args []string) error {
if !contains(args, "-y") {
return fmt.Errorf("dnf refresh must include -y flag")
}
case "makecache":
// makecache doesn't require -y flag as it's read-only
return nil
case "install":
// Allow dry-run flags for dependency checking
dryRunFlags := []string{"--assumeno", "--downloadonly"}
@@ -165,12 +169,22 @@ func (e *SecureCommandExecutor) ExecuteCommand(baseCmd string, args []string) (*
}, fmt.Errorf("command validation failed: %w", err)
}
// Log the command for audit purposes (in a real implementation, this would go to a secure log)
fmt.Printf("[AUDIT] Executing command: %s %s\n", baseCmd, strings.Join(args, " "))
// Resolve the full path to the command (required for sudo to match sudoers rules)
fullPath, err := exec.LookPath(baseCmd)
if err != nil {
return &InstallResult{
Success: false,
ErrorMessage: fmt.Sprintf("Command not found: %s", baseCmd),
}, fmt.Errorf("command not found: %w", err)
}
// Execute the command without sudo - it will be handled by sudoers
fullArgs := append([]string{baseCmd}, args...)
cmd := exec.Command(fullArgs[0], fullArgs[1:]...)
// Log the command for audit purposes (in a real implementation, this would go to a secure log)
fmt.Printf("[AUDIT] Executing command: sudo %s %s\n", fullPath, strings.Join(args, " "))
// Execute the command with sudo - requires sudoers configuration
// Use full path to match sudoers rules exactly
fullArgs := append([]string{fullPath}, args...)
cmd := exec.Command("sudo", fullArgs...)
output, err := cmd.CombinedOutput()

6
aggregator-agent/internal/installer/windows.go
View File

@@ -117,6 +117,12 @@ func (i *WindowsUpdateInstaller) installViaPowerShell(packageNames []string) (st
return "Windows Updates installed via PowerShell", nil
}
// UpdatePackage updates a specific Windows update (alias for Install method)
func (i *WindowsUpdateInstaller) UpdatePackage(packageName string) (*InstallResult, error) {
// Windows uses same logic for updating as installing
return i.Install(packageName)
}
// installViaWuauclt uses traditional Windows Update client
func (i *WindowsUpdateInstaller) installViaWuauclt(packageNames []string) (string, error) {
// Force detection of updates

6
aggregator-agent/internal/installer/winget.go
View File

@@ -371,4 +371,10 @@ type WingetPackage struct {
Source string `json:"Source"`
IsPinned bool `json:"IsPinned"`
PinReason string `json:"PinReason,omitempty"`
}
// UpdatePackage updates a specific winget package (alias for Install method)
func (i *WingetInstaller) UpdatePackage(packageName string) (*InstallResult, error) {
// Winget uses same logic for updating as installing
return i.Install(packageName)
}

102
aggregator-server/cmd/server/main.go
View File

@@ -1,6 +1,7 @@
package main
import (
"flag"
"fmt"
"log"
"path/filepath"
@@ -16,6 +17,29 @@ import (
)
func main() {
// Parse command line flags
var setup bool
var migrate bool
var version bool
flag.BoolVar(&setup, "setup", false, "Run setup wizard")
flag.BoolVar(&migrate, "migrate", false, "Run database migrations only")
flag.BoolVar(&version, "version", false, "Show version information")
flag.Parse()
// Handle special commands
if version {
fmt.Printf("RedFlag Server v0.1.0-alpha\n")
fmt.Printf("Self-hosted update management platform\n")
return
}
if setup {
if err := config.RunSetupWizard(); err != nil {
log.Fatal("Setup failed:", err)
}
return
}
// Load configuration
cfg, err := config.Load()
if err != nil {
@@ -23,15 +47,29 @@ func main() {
}
// Set JWT secret
middleware.JWTSecret = cfg.JWTSecret
middleware.JWTSecret = cfg.Admin.JWTSecret
// Build database URL from new config structure
databaseURL := fmt.Sprintf("postgres://%s:%s@%s:%d/%s?sslmode=disable",
cfg.Database.Username, cfg.Database.Password, cfg.Database.Host, cfg.Database.Port, cfg.Database.Database)
// Connect to database
db, err := database.Connect(cfg.DatabaseURL)
db, err := database.Connect(databaseURL)
if err != nil {
log.Fatal("Failed to connect to database:", err)
}
defer db.Close()
// Handle migrate-only flag
if migrate {
migrationsPath := filepath.Join("internal", "database", "migrations")
if err := db.Migrate(migrationsPath); err != nil {
log.Fatal("Migration failed:", err)
}
fmt.Printf("✅ Database migrations completed\n")
return
}
// Run migrations
migrationsPath := filepath.Join("internal", "database", "migrations")
if err := db.Migrate(migrationsPath); err != nil {
@@ -45,18 +83,24 @@ func main() {
updateQueries := queries.NewUpdateQueries(db.DB)
commandQueries := queries.NewCommandQueries(db.DB)
refreshTokenQueries := queries.NewRefreshTokenQueries(db.DB)
registrationTokenQueries := queries.NewRegistrationTokenQueries(db.DB)
// Initialize services
timezoneService := services.NewTimezoneService(cfg)
timeoutService := services.NewTimeoutService(commandQueries, updateQueries)
// Initialize rate limiter
rateLimiter := middleware.NewRateLimiter()
// Initialize handlers
agentHandler := handlers.NewAgentHandler(agentQueries, commandQueries, refreshTokenQueries, cfg.CheckInInterval, cfg.LatestAgentVersion)
updateHandler := handlers.NewUpdateHandler(updateQueries, agentQueries, commandQueries)
authHandler := handlers.NewAuthHandler(cfg.JWTSecret)
updateHandler := handlers.NewUpdateHandler(updateQueries, agentQueries, commandQueries, agentHandler)
authHandler := handlers.NewAuthHandler(cfg.Admin.JWTSecret)
statsHandler := handlers.NewStatsHandler(agentQueries, updateQueries)
settingsHandler := handlers.NewSettingsHandler(timezoneService)
dockerHandler := handlers.NewDockerHandler(updateQueries, agentQueries, commandQueries)
registrationTokenHandler := handlers.NewRegistrationTokenHandler(registrationTokenQueries, agentQueries, cfg)
rateLimitHandler := handlers.NewRateLimitHandler(rateLimiter)
// Setup router
router := gin.Default()
@@ -72,24 +116,26 @@ func main() {
// API routes
api := router.Group("/api/v1")
{
// Authentication routes
api.POST("/auth/login", authHandler.Login)
// Authentication routes (with rate limiting)
api.POST("/auth/login", rateLimiter.RateLimit("public_access", middleware.KeyByIP), authHandler.Login)
api.POST("/auth/logout", authHandler.Logout)
api.GET("/auth/verify", authHandler.VerifyToken)
// Public routes (no authentication required)
api.POST("/agents/register", agentHandler.RegisterAgent)
api.POST("/agents/renew", agentHandler.RenewToken)
// Public routes (no authentication required, with rate limiting)
api.POST("/agents/register", rateLimiter.RateLimit("agent_registration", middleware.KeyByIP), agentHandler.RegisterAgent)
api.POST("/agents/renew", rateLimiter.RateLimit("public_access", middleware.KeyByIP), agentHandler.RenewToken)
// Protected agent routes
agents := api.Group("/agents")
agents.Use(middleware.AuthMiddleware())
{
agents.GET("/:id/commands", agentHandler.GetCommands)
agents.POST("/:id/updates", updateHandler.ReportUpdates)
agents.POST("/:id/logs", updateHandler.ReportLog)
agents.POST("/:id/dependencies", updateHandler.ReportDependencies)
agents.POST("/:id/system-info", agentHandler.ReportSystemInfo)
agents.POST("/:id/updates", rateLimiter.RateLimit("agent_reports", middleware.KeyByAgentID), updateHandler.ReportUpdates)
agents.POST("/:id/logs", rateLimiter.RateLimit("agent_reports", middleware.KeyByAgentID), updateHandler.ReportLog)
agents.POST("/:id/dependencies", rateLimiter.RateLimit("agent_reports", middleware.KeyByAgentID), updateHandler.ReportDependencies)
agents.POST("/:id/system-info", rateLimiter.RateLimit("agent_reports", middleware.KeyByAgentID), agentHandler.ReportSystemInfo)
agents.POST("/:id/rapid-mode", rateLimiter.RateLimit("agent_reports", middleware.KeyByAgentID), agentHandler.SetRapidPollingMode)
agents.DELETE("/:id", agentHandler.UnregisterAgent)
}
// Dashboard/Web routes (protected by web auth)
@@ -101,7 +147,8 @@ func main() {
dashboard.GET("/agents/:id", agentHandler.GetAgent)
dashboard.POST("/agents/:id/scan", agentHandler.TriggerScan)
dashboard.POST("/agents/:id/update", agentHandler.TriggerUpdate)
dashboard.DELETE("/agents/:id", agentHandler.UnregisterAgent)
dashboard.POST("/agents/:id/heartbeat", agentHandler.TriggerHeartbeat)
dashboard.GET("/agents/:id/heartbeat", agentHandler.GetHeartbeatStatus)
dashboard.GET("/updates", updateHandler.ListUpdates)
dashboard.GET("/updates/:id", updateHandler.GetUpdate)
dashboard.GET("/updates/:id/logs", updateHandler.GetUpdateLogs)
@@ -120,6 +167,7 @@ func main() {
dashboard.GET("/commands/recent", updateHandler.GetRecentCommands)
dashboard.POST("/commands/:id/retry", updateHandler.RetryCommand)
dashboard.POST("/commands/:id/cancel", updateHandler.CancelCommand)
dashboard.DELETE("/commands/failed", updateHandler.ClearFailedCommands)
// Settings routes
dashboard.GET("/settings/timezone", settingsHandler.GetTimezone)
@@ -132,6 +180,25 @@ func main() {
dashboard.POST("/docker/containers/:container_id/images/:image_id/approve", dockerHandler.ApproveUpdate)
dashboard.POST("/docker/containers/:container_id/images/:image_id/reject", dockerHandler.RejectUpdate)
dashboard.POST("/docker/containers/:container_id/images/:image_id/install", dockerHandler.InstallUpdate)
// Admin/Registration Token routes (for agent enrollment management)
admin := dashboard.Group("/admin")
{
admin.POST("/registration-tokens", rateLimiter.RateLimit("admin_token_gen", middleware.KeyByUserID), registrationTokenHandler.GenerateRegistrationToken)
admin.GET("/registration-tokens", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.ListRegistrationTokens)
admin.GET("/registration-tokens/active", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.GetActiveRegistrationTokens)
admin.DELETE("/registration-tokens/:token", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.RevokeRegistrationToken)
admin.POST("/registration-tokens/cleanup", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.CleanupExpiredTokens)
admin.GET("/registration-tokens/stats", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.GetTokenStats)
admin.GET("/registration-tokens/validate", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), registrationTokenHandler.ValidateRegistrationToken)
// Rate Limit Management
admin.GET("/rate-limits", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), rateLimitHandler.GetRateLimitSettings)
admin.PUT("/rate-limits", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), rateLimitHandler.UpdateRateLimitSettings)
admin.POST("/rate-limits/reset", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), rateLimitHandler.ResetRateLimitSettings)
admin.GET("/rate-limits/stats", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), rateLimitHandler.GetRateLimitStats)
admin.POST("/rate-limits/cleanup", rateLimiter.RateLimit("admin_operations", middleware.KeyByUserID), rateLimitHandler.CleanupRateLimitEntries)
}
}
}
@@ -166,8 +233,11 @@ func main() {
}()
// Start server
addr := ":" + cfg.ServerPort
fmt.Printf("\n🚩 RedFlag Aggregator Server starting on %s\n\n", addr)
addr := fmt.Sprintf("%s:%d", cfg.Server.Host, cfg.Server.Port)
fmt.Printf("\nRedFlag Aggregator Server starting on %s\n", addr)
fmt.Printf("Admin interface: http://%s:%d/admin\n", cfg.Server.Host, cfg.Server.Port)
fmt.Printf("Dashboard: http://%s:%d\n\n", cfg.Server.Host, cfg.Server.Port)
if err := router.Run(addr); err != nil {
log.Fatal("Failed to start server:", err)
}

1
aggregator-server/go.mod
View File

@@ -9,6 +9,7 @@ require (
github.com/jmoiron/sqlx v1.4.0
github.com/joho/godotenv v1.5.1
github.com/lib/pq v1.10.9
golang.org/x/term v0.33.0
)
require (

2
aggregator-server/go.sum
View File

@@ -92,6 +92,8 @@ golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.33.0 h1:NuFncQrRcaRvVmgRkvM3j/F00gWIAlcmlB8ACEKmGIg=
golang.org/x/term v0.33.0/go.mod h1:s18+ql9tYWp1IfpV9DmCtQDDSRBUjKaw9M1eAv5UeF0=
golang.org/x/text v0.27.0 h1:4fGWRpyh641NLlecmyl4LOe6yDdfaYNrGb2zdfo4JV4=
golang.org/x/text v0.27.0/go.mod h1:1D28KMCvyooCX9hBiosv5Tz/+YLxj0j7XhWjpSUF7CU=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=

431
aggregator-server/internal/api/handlers/agents.go
View File

@@ -1,6 +1,7 @@
package handlers
import (
"fmt"
"log"
"net/http"
"time"
@@ -107,15 +108,16 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
// Try to parse optional system metrics from request body
var metrics struct {
CPUPercent float64 `json:"cpu_percent,omitempty"`
MemoryPercent float64 `json:"memory_percent,omitempty"`
MemoryUsedGB float64 `json:"memory_used_gb,omitempty"`
MemoryTotalGB float64 `json:"memory_total_gb,omitempty"`
DiskUsedGB float64 `json:"disk_used_gb,omitempty"`
DiskTotalGB float64 `json:"disk_total_gb,omitempty"`
DiskPercent float64 `json:"disk_percent,omitempty"`
Uptime string `json:"uptime,omitempty"`
Version string `json:"version,omitempty"`
CPUPercent float64 `json:"cpu_percent,omitempty"`
MemoryPercent float64 `json:"memory_percent,omitempty"`
MemoryUsedGB float64 `json:"memory_used_gb,omitempty"`
MemoryTotalGB float64 `json:"memory_total_gb,omitempty"`
DiskUsedGB float64 `json:"disk_used_gb,omitempty"`
DiskTotalGB float64 `json:"disk_total_gb,omitempty"`
DiskPercent float64 `json:"disk_percent,omitempty"`
Uptime string `json:"uptime,omitempty"`
Version string `json:"version,omitempty"`
Metadata map[string]interface{} `json:"metadata,omitempty"`
}
// Parse metrics if provided (optional, won't fail if empty)
@@ -130,21 +132,21 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
// Always handle version information if provided
if metrics.Version != "" {
// Get current agent to preserve existing metadata
agent, err := h.agentQueries.GetAgentByID(agentID)
if err == nil && agent.Metadata != nil {
// Update agent's current version
if err := h.agentQueries.UpdateAgentVersion(agentID, metrics.Version); err != nil {
log.Printf("Warning: Failed to update agent version: %v", err)
} else {
// Check if update is available
updateAvailable := utils.IsNewerVersion(h.latestAgentVersion, metrics.Version)
// Update agent's current version in database (primary source of truth)
if err := h.agentQueries.UpdateAgentVersion(agentID, metrics.Version); err != nil {
log.Printf("Warning: Failed to update agent version: %v", err)
} else {
// Check if update is available
updateAvailable := utils.IsNewerVersion(h.latestAgentVersion, metrics.Version)
// Update agent's update availability status
if err := h.agentQueries.UpdateAgentUpdateAvailable(agentID, updateAvailable); err != nil {
log.Printf("Warning: Failed to update agent update availability: %v", err)
}
// Update agent's update availability status
if err := h.agentQueries.UpdateAgentUpdateAvailable(agentID, updateAvailable); err != nil {
log.Printf("Warning: Failed to update agent update availability: %v", err)
}
// Get current agent for logging and metadata update
agent, err := h.agentQueries.GetAgentByID(agentID)
if err == nil {
// Log version check
if updateAvailable {
log.Printf("🔄 Agent %s (%s) version %s has update available: %s",
@@ -154,11 +156,20 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
agent.Hostname, agentID, metrics.Version)
}
// Store version in metadata as well
// Store version in metadata as well (for backwards compatibility)
// Initialize metadata if nil
if agent.Metadata == nil {
agent.Metadata = make(models.JSONB)
}
agent.Metadata["reported_version"] = metrics.Version
agent.Metadata["latest_version"] = h.latestAgentVersion
agent.Metadata["update_available"] = updateAvailable
agent.Metadata["version_checked_at"] = time.Now().Format(time.RFC3339)
// Update agent metadata
if err := h.agentQueries.UpdateAgent(agent); err != nil {
log.Printf("Warning: Failed to update agent metadata: %v", err)
}
}
}
}
@@ -179,6 +190,29 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
agent.Metadata["uptime"] = metrics.Uptime
agent.Metadata["metrics_updated_at"] = time.Now().Format(time.RFC3339)
// Process heartbeat metadata from agent check-ins
if metrics.Metadata != nil {
if rapidPollingEnabled, exists := metrics.Metadata["rapid_polling_enabled"]; exists {
if rapidPollingUntil, exists := metrics.Metadata["rapid_polling_until"]; exists {
// Parse the until timestamp
if untilTime, err := time.Parse(time.RFC3339, rapidPollingUntil.(string)); err == nil {
// Validate if rapid polling is still active (not expired)
isActive := rapidPollingEnabled.(bool) && time.Now().Before(untilTime)
// Store heartbeat status in agent metadata
agent.Metadata["rapid_polling_enabled"] = rapidPollingEnabled
agent.Metadata["rapid_polling_until"] = rapidPollingUntil
agent.Metadata["rapid_polling_active"] = isActive
log.Printf("[Heartbeat] Agent %s heartbeat status: enabled=%v, until=%v, active=%v",
agentID, rapidPollingEnabled, rapidPollingUntil, isActive)
} else {
log.Printf("[Heartbeat] Failed to parse rapid_polling_until timestamp for agent %s: %v", agentID, err)
}
}
}
}
// Update agent with new metadata
if err := h.agentQueries.UpdateAgent(agent); err != nil {
log.Printf("Warning: Failed to update agent metrics: %v", err)
@@ -192,6 +226,37 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
return
}
// Process heartbeat metadata from agent check-ins
if metrics.Metadata != nil {
agent, err := h.agentQueries.GetAgentByID(agentID)
if err == nil && agent.Metadata != nil {
if rapidPollingEnabled, exists := metrics.Metadata["rapid_polling_enabled"]; exists {
if rapidPollingUntil, exists := metrics.Metadata["rapid_polling_until"]; exists {
// Parse the until timestamp
if untilTime, err := time.Parse(time.RFC3339, rapidPollingUntil.(string)); err == nil {
// Validate if rapid polling is still active (not expired)
isActive := rapidPollingEnabled.(bool) && time.Now().Before(untilTime)
// Store heartbeat status in agent metadata
agent.Metadata["rapid_polling_enabled"] = rapidPollingEnabled
agent.Metadata["rapid_polling_until"] = rapidPollingUntil
agent.Metadata["rapid_polling_active"] = isActive
log.Printf("[Heartbeat] Agent %s heartbeat status: enabled=%v, until=%v, active=%v",
agentID, rapidPollingEnabled, rapidPollingUntil, isActive)
// Update agent with new metadata
if err := h.agentQueries.UpdateAgent(agent); err != nil {
log.Printf("[Heartbeat] Warning: Failed to update agent heartbeat metadata: %v", err)
}
} else {
log.Printf("[Heartbeat] Failed to parse rapid_polling_until timestamp for agent %s: %v", agentID, err)
}
}
}
}
}
// Check for version updates for agents that don't send version in metrics
// This ensures agents like Metis that don't report version still get update checks
if metrics.Version == "" {
@@ -239,8 +304,97 @@ func (h *AgentHandler) GetCommands(c *gin.Context) {
h.commandQueries.MarkCommandSent(cmd.ID)
}
// Check if rapid polling should be enabled
var rapidPolling *models.RapidPollingConfig
// Enable rapid polling if there are commands to process
if len(commandItems) > 0 {
rapidPolling = &models.RapidPollingConfig{
Enabled: true,
Until: time.Now().Add(10 * time.Minute).Format(time.RFC3339), // 10 minutes default
}
} else {
// Check if agent has rapid polling already configured in metadata
agent, err := h.agentQueries.GetAgentByID(agentID)
if err == nil && agent.Metadata != nil {
if enabled, ok := agent.Metadata["rapid_polling_enabled"].(bool); ok && enabled {
if untilStr, ok := agent.Metadata["rapid_polling_until"].(string); ok {
if until, err := time.Parse(time.RFC3339, untilStr); err == nil && time.Now().Before(until) {
rapidPolling = &models.RapidPollingConfig{
Enabled: true,
Until: untilStr,
}
}
}
}
}
}
// Detect stale heartbeat state: Server thinks it's active, but agent didn't report it
// This happens when agent restarts without heartbeat mode
agent, err := h.agentQueries.GetAgentByID(agentID)
if err == nil && agent.Metadata != nil {
// Check if server metadata shows heartbeat active
if serverEnabled, ok := agent.Metadata["rapid_polling_enabled"].(bool); ok && serverEnabled {
if untilStr, ok := agent.Metadata["rapid_polling_until"].(string); ok {
if until, err := time.Parse(time.RFC3339, untilStr); err == nil && time.Now().Before(until) {
// Server thinks heartbeat is active and not expired
// Check if agent is reporting heartbeat in this check-in
agentReportingHeartbeat := false
if metrics.Metadata != nil {
if agentEnabled, exists := metrics.Metadata["rapid_polling_enabled"]; exists {
agentReportingHeartbeat = agentEnabled.(bool)
}
}
// If agent is NOT reporting heartbeat but server expects it → stale state
if !agentReportingHeartbeat {
log.Printf("[Heartbeat] Stale heartbeat detected for agent %s - server expected active until %s, but agent not reporting heartbeat (likely restarted)",
agentID, until.Format(time.RFC3339))
// Clear stale heartbeat state
agent.Metadata["rapid_polling_enabled"] = false
delete(agent.Metadata, "rapid_polling_until")
if err := h.agentQueries.UpdateAgent(agent); err != nil {
log.Printf("[Heartbeat] Warning: Failed to clear stale heartbeat state: %v", err)
} else {
log.Printf("[Heartbeat] Cleared stale heartbeat state for agent %s", agentID)
// Create audit command to show in history
now := time.Now()
auditCmd := &models.AgentCommand{
ID: uuid.New(),
AgentID: agentID,
CommandType: models.CommandTypeDisableHeartbeat,
Params: models.JSONB{},
Status: models.CommandStatusCompleted,
Result: models.JSONB{
"message": "Heartbeat cleared - agent restarted without active heartbeat mode",
},
CreatedAt: now,
SentAt: &now,
CompletedAt: &now,
}
if err := h.commandQueries.CreateCommand(auditCmd); err != nil {
log.Printf("[Heartbeat] Warning: Failed to create audit command for stale heartbeat: %v", err)
} else {
log.Printf("[Heartbeat] Created audit trail for stale heartbeat cleanup (agent %s)", agentID)
}
}
// Clear rapidPolling response since we just disabled it
rapidPolling = nil
}
}
}
}
}
response := models.CommandsResponse{
Commands: commandItems,
Commands: commandItems,
RapidPolling: rapidPolling,
}
c.JSON(http.StatusOK, response)
@@ -312,6 +466,124 @@ func (h *AgentHandler) TriggerScan(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"message": "scan triggered", "command_id": cmd.ID})
}
// TriggerHeartbeat creates a heartbeat toggle command for an agent
func (h *AgentHandler) TriggerHeartbeat(c *gin.Context) {
idStr := c.Param("id")
agentID, err := uuid.Parse(idStr)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid agent ID"})
return
}
var request struct {
Enabled bool `json:"enabled"`
DurationMinutes int `json:"duration_minutes"`
}
if err := c.ShouldBindJSON(&request); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// Determine command type based on enabled flag
commandType := models.CommandTypeDisableHeartbeat
if request.Enabled {
commandType = models.CommandTypeEnableHeartbeat
}
// Create heartbeat command with duration parameter
cmd := &models.AgentCommand{
ID: uuid.New(),
AgentID: agentID,
CommandType: commandType,
Params: models.JSONB{
"duration_minutes": request.DurationMinutes,
},
Status: models.CommandStatusPending,
}
if err := h.commandQueries.CreateCommand(cmd); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create heartbeat command"})
return
}
// TODO: Clean up previous heartbeat commands for this agent (only for enable commands)
// if request.Enabled {
// // Mark previous heartbeat commands as 'replaced' to clean up Live Operations view
// if err := h.commandQueries.MarkPreviousHeartbeatCommandsReplaced(agentID, cmd.ID); err != nil {
// log.Printf("Warning: Failed to mark previous heartbeat commands as replaced: %v", err)
// // Don't fail the request, just log the warning
// } else {
// log.Printf("[Heartbeat] Cleaned up previous heartbeat commands for agent %s", agentID)
// }
// }
action := "disabled"
if request.Enabled {
action = "enabled"
}
log.Printf("💓 Heartbeat %s command created for agent %s (duration: %d minutes)",
action, agentID, request.DurationMinutes)
c.JSON(http.StatusOK, gin.H{
"message": fmt.Sprintf("heartbeat %s command sent", action),
"command_id": cmd.ID,
"enabled": request.Enabled,
})
}
// GetHeartbeatStatus returns the current heartbeat status for an agent
func (h *AgentHandler) GetHeartbeatStatus(c *gin.Context) {
idStr := c.Param("id")
agentID, err := uuid.Parse(idStr)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid agent ID"})
return
}
// Get agent and their heartbeat metadata
agent, err := h.agentQueries.GetAgentByID(agentID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "agent not found"})
return
}
// Extract heartbeat information from metadata
response := gin.H{
"enabled": false,
"until": nil,
"active": false,
"duration_minutes": 0,
}
if agent.Metadata != nil {
// Check if heartbeat is enabled in metadata
if enabled, exists := agent.Metadata["rapid_polling_enabled"]; exists {
response["enabled"] = enabled.(bool)
// If enabled, get the until time and check if still active
if enabled.(bool) {
if untilStr, exists := agent.Metadata["rapid_polling_until"]; exists {
response["until"] = untilStr.(string)
// Parse the until timestamp to check if still active
if untilTime, err := time.Parse(time.RFC3339, untilStr.(string)); err == nil {
response["active"] = time.Now().Before(untilTime)
}
}
// Get duration if available
if duration, exists := agent.Metadata["rapid_polling_duration_minutes"]; exists {
response["duration_minutes"] = duration.(float64)
}
}
}
}
c.JSON(http.StatusOK, response)
}
// TriggerUpdate creates an update command for an agent
func (h *AgentHandler) TriggerUpdate(c *gin.Context) {
idStr := c.Param("id")
@@ -541,3 +813,114 @@ func (h *AgentHandler) ReportSystemInfo(c *gin.Context) {
c.JSON(http.StatusOK, gin.H{"message": "system info updated successfully"})
}
// EnableRapidPollingMode enables rapid polling for an agent by updating metadata
func (h *AgentHandler) EnableRapidPollingMode(agentID uuid.UUID, durationMinutes int) error {
// Get current agent
agent, err := h.agentQueries.GetAgentByID(agentID)
if err != nil {
return fmt.Errorf("failed to get agent: %w", err)
}
// Calculate new rapid polling end time
newRapidPollingUntil := time.Now().Add(time.Duration(durationMinutes) * time.Minute)
// Update agent metadata with rapid polling settings
if agent.Metadata == nil {
agent.Metadata = models.JSONB{}
}
// Check if rapid polling is already active
if enabled, ok := agent.Metadata["rapid_polling_enabled"].(bool); ok && enabled {
if untilStr, ok := agent.Metadata["rapid_polling_until"].(string); ok {
if currentUntil, err := time.Parse(time.RFC3339, untilStr); err == nil {
// If current heartbeat expires later than the new duration, keep the longer duration
if currentUntil.After(newRapidPollingUntil) {
log.Printf("💓 Heartbeat already active for agent %s (%s), keeping longer duration (expires: %s)",
agent.Hostname, agentID, currentUntil.Format(time.RFC3339))
return nil
}
// Otherwise extend the heartbeat
log.Printf("💓 Extending heartbeat for agent %s (%s) from %s to %s",
agent.Hostname, agentID,
currentUntil.Format(time.RFC3339),
newRapidPollingUntil.Format(time.RFC3339))
}
}
} else {
log.Printf("💓 Enabling heartbeat mode for agent %s (%s) for %d minutes",
agent.Hostname, agentID, durationMinutes)
}
// Set/update rapid polling settings
agent.Metadata["rapid_polling_enabled"] = true
agent.Metadata["rapid_polling_until"] = newRapidPollingUntil.Format(time.RFC3339)
// Update agent in database
if err := h.agentQueries.UpdateAgent(agent); err != nil {
return fmt.Errorf("failed to update agent with rapid polling: %w", err)
}
return nil
}
// SetRapidPollingMode enables rapid polling mode for an agent
// TODO: Rate limiting should be implemented for rapid polling endpoints to prevent abuse (technical debt)
func (h *AgentHandler) SetRapidPollingMode(c *gin.Context) {
idStr := c.Param("id")
agentID, err := uuid.Parse(idStr)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "invalid agent ID"})
return
}
// Check if agent exists
agent, err := h.agentQueries.GetAgentByID(agentID)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{"error": "agent not found"})
return
}
var req struct {
DurationMinutes int `json:"duration_minutes" binding:"required,min=1,max=60"`
Enabled bool `json:"enabled"`
}
if err := c.ShouldBindJSON(&req); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// Calculate rapid polling end time
rapidPollingUntil := time.Now().Add(time.Duration(req.DurationMinutes) * time.Minute)
// Update agent metadata with rapid polling settings
if agent.Metadata == nil {
agent.Metadata = models.JSONB{}
}
agent.Metadata["rapid_polling_enabled"] = req.Enabled
agent.Metadata["rapid_polling_until"] = rapidPollingUntil.Format(time.RFC3339)
// Update agent in database
if err := h.agentQueries.UpdateAgent(agent); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update agent"})
return
}
status := "disabled"
duration := 0
if req.Enabled {
status = "enabled"
duration = req.DurationMinutes
}
log.Printf("🚀 Rapid polling mode %s for agent %s (%s) for %d minutes",
status, agent.Hostname, agentID, duration)
c.JSON(http.StatusOK, gin.H{
"message": fmt.Sprintf("Rapid polling mode %s", status),
"enabled": req.Enabled,
"duration_minutes": req.DurationMinutes,
"rapid_polling_until": rapidPollingUntil,
})
}

2
aggregator-server/internal/api/handlers/docker.go
View File

@@ -378,7 +378,7 @@ func (h *DockerHandler) RejectUpdate(c *gin.Context) {
}
// For now, we'll mark as rejected (this would need a proper reject method in queries)
if err := h.updateQueries.UpdatePackageStatus(update.AgentID, "docker", update.PackageName, "rejected", nil); err != nil {
if err := h.updateQueries.UpdatePackageStatus(update.AgentID, "docker", update.PackageName, "rejected", nil, nil); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to reject Docker update"})
return
}

146
aggregator-server/internal/api/handlers/rate_limits.go Normal file
View File

@@ -0,0 +1,146 @@
package handlers
import (
"fmt"
"net/http"
"time"
"github.com/aggregator-project/aggregator-server/internal/api/middleware"
"github.com/gin-gonic/gin"
)
type RateLimitHandler struct {
rateLimiter *middleware.RateLimiter
}
func NewRateLimitHandler(rateLimiter *middleware.RateLimiter) *RateLimitHandler {
return &RateLimitHandler{
rateLimiter: rateLimiter,
}
}
// GetRateLimitSettings returns current rate limit configuration
func (h *RateLimitHandler) GetRateLimitSettings(c *gin.Context) {
settings := h.rateLimiter.GetSettings()
c.JSON(http.StatusOK, gin.H{
"settings": settings,
"updated_at": time.Now(),
})
}
// UpdateRateLimitSettings updates rate limit configuration
func (h *RateLimitHandler) UpdateRateLimitSettings(c *gin.Context) {
var settings middleware.RateLimitSettings
if err := c.ShouldBindJSON(&settings); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid request format: " + err.Error()})
return
}
// Validate settings
if err := h.validateRateLimitSettings(settings); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// Update rate limiter settings
h.rateLimiter.UpdateSettings(settings)
c.JSON(http.StatusOK, gin.H{
"message": "Rate limit settings updated successfully",
"settings": settings,
"updated_at": time.Now(),
})
}
// ResetRateLimitSettings resets to default values
func (h *RateLimitHandler) ResetRateLimitSettings(c *gin.Context) {
defaultSettings := middleware.DefaultRateLimitSettings()
h.rateLimiter.UpdateSettings(defaultSettings)
c.JSON(http.StatusOK, gin.H{
"message": "Rate limit settings reset to defaults",
"settings": defaultSettings,
"updated_at": time.Now(),
})
}
// GetRateLimitStats returns current rate limit statistics
func (h *RateLimitHandler) GetRateLimitStats(c *gin.Context) {
settings := h.rateLimiter.GetSettings()
// Calculate total requests and windows
stats := gin.H{
"total_configured_limits": 6,
"enabled_limits": 0,
"total_requests_per_minute": 0,
"settings": settings,
}
// Count enabled limits and total requests
for _, config := range []middleware.RateLimitConfig{
settings.AgentRegistration,
settings.AgentCheckIn,
settings.AgentReports,
settings.AdminTokenGen,
settings.AdminOperations,
settings.PublicAccess,
} {
if config.Enabled {
stats["enabled_limits"] = stats["enabled_limits"].(int) + 1
}
stats["total_requests_per_minute"] = stats["total_requests_per_minute"].(int) + config.Requests
}
c.JSON(http.StatusOK, stats)
}
// CleanupRateLimitEntries manually triggers cleanup of expired entries
func (h *RateLimitHandler) CleanupRateLimitEntries(c *gin.Context) {
h.rateLimiter.CleanupExpiredEntries()
c.JSON(http.StatusOK, gin.H{
"message": "Rate limit entries cleanup completed",
"timestamp": time.Now(),
})
}
// validateRateLimitSettings validates the provided rate limit settings
func (h *RateLimitHandler) validateRateLimitSettings(settings middleware.RateLimitSettings) error {
// Validate each configuration
validations := []struct {
name string
config middleware.RateLimitConfig
}{
{"agent_registration", settings.AgentRegistration},
{"agent_checkin", settings.AgentCheckIn},
{"agent_reports", settings.AgentReports},
{"admin_token_generation", settings.AdminTokenGen},
{"admin_operations", settings.AdminOperations},
{"public_access", settings.PublicAccess},
}
for _, validation := range validations {
if validation.config.Requests <= 0 {
return fmt.Errorf("%s: requests must be greater than 0", validation.name)
}
if validation.config.Window <= 0 {
return fmt.Errorf("%s: window must be greater than 0", validation.name)
}
if validation.config.Window > 24*time.Hour {
return fmt.Errorf("%s: window cannot exceed 24 hours", validation.name)
}
if validation.config.Requests > 1000 {
return fmt.Errorf("%s: requests cannot exceed 1000 per window", validation.name)
}
}
// Specific validations for different endpoint types
if settings.AgentRegistration.Requests > 10 {
return fmt.Errorf("agent_registration: requests should not exceed 10 per minute for security")
}
if settings.PublicAccess.Requests > 50 {
return fmt.Errorf("public_access: requests should not exceed 50 per minute for security")
}
return nil
}

284
aggregator-server/internal/api/handlers/registration_tokens.go Normal file
View File

@@ -0,0 +1,284 @@
package handlers
import (
"net/http"
"strconv"
"time"
"github.com/aggregator-project/aggregator-server/internal/config"
"github.com/aggregator-project/aggregator-server/internal/database/queries"
"github.com/gin-gonic/gin"
)
type RegistrationTokenHandler struct {
tokenQueries *queries.RegistrationTokenQueries
agentQueries *queries.AgentQueries
config *config.Config
}
func NewRegistrationTokenHandler(tokenQueries *queries.RegistrationTokenQueries, agentQueries *queries.AgentQueries, config *config.Config) *RegistrationTokenHandler {
return &RegistrationTokenHandler{
tokenQueries: tokenQueries,
agentQueries: agentQueries,
config: config,
}
}
// GenerateRegistrationToken creates a new registration token
func (h *RegistrationTokenHandler) GenerateRegistrationToken(c *gin.Context) {
var request struct {
Label string `json:"label" binding:"required"`
ExpiresIn string `json:"expires_in"` // e.g., "24h", "7d", "168h"
Metadata map[string]interface{} `json:"metadata"`
}
if err := c.ShouldBindJSON(&request); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid request format: " + err.Error()})
return
}
// Check agent seat limit (security, not licensing)
activeAgents, err := h.agentQueries.GetActiveAgentCount()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to check agent count"})
return
}
if activeAgents >= h.config.AgentRegistration.MaxSeats {
c.JSON(http.StatusForbidden, gin.H{
"error": "Maximum agent seats reached",
"limit": h.config.AgentRegistration.MaxSeats,
"current": activeAgents,
})
return
}
// Parse expiration duration
expiresIn := request.ExpiresIn
if expiresIn == "" {
expiresIn = h.config.AgentRegistration.TokenExpiry
}
duration, err := time.ParseDuration(expiresIn)
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid expiration format. Use formats like '24h', '7d', '168h'"})
return
}
expiresAt := time.Now().Add(duration)
if duration > 168*time.Hour { // Max 7 days
c.JSON(http.StatusBadRequest, gin.H{"error": "Token expiration cannot exceed 7 days"})
return
}
// Generate secure token
token, err := config.GenerateSecureToken()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to generate token"})
return
}
// Create metadata with default values
metadata := request.Metadata
if metadata == nil {
metadata = make(map[string]interface{})
}
metadata["server_url"] = c.Request.Host
metadata["expires_in"] = expiresIn
// Store token in database
err = h.tokenQueries.CreateRegistrationToken(token, request.Label, expiresAt, metadata)
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to create token"})
return
}
// Build install command
serverURL := c.Request.Host
if serverURL == "" {
serverURL = "localhost:8080" // Fallback for development
}
installCommand := "curl -sfL https://" + serverURL + "/install | bash -s -- " + token
response := gin.H{
"token": token,
"label": request.Label,
"expires_at": expiresAt,
"install_command": installCommand,
"metadata": metadata,
}
c.JSON(http.StatusCreated, response)
}
// ListRegistrationTokens returns all registration tokens with pagination
func (h *RegistrationTokenHandler) ListRegistrationTokens(c *gin.Context) {
// Parse pagination parameters
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
limit, _ := strconv.Atoi(c.DefaultQuery("limit", "50"))
status := c.Query("status")
// Validate pagination
if limit > 100 {
limit = 100
}
if page < 1 {
page = 1
}
offset := (page - 1) * limit
var tokens []queries.RegistrationToken
var err error
if status != "" {
// TODO: Add filtered queries by status
tokens, err = h.tokenQueries.GetAllRegistrationTokens(limit, offset)
} else {
tokens, err = h.tokenQueries.GetAllRegistrationTokens(limit, offset)
}
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to list tokens"})
return
}
// Get token usage stats
stats, err := h.tokenQueries.GetTokenUsageStats()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to get token stats"})
return
}
response := gin.H{
"tokens": tokens,
"pagination": gin.H{
"page": page,
"limit": limit,
"offset": offset,
},
"stats": stats,
"seat_usage": gin.H{
"current": func() int {
count, _ := h.agentQueries.GetActiveAgentCount()
return count
}(),
"max": h.config.AgentRegistration.MaxSeats,
},
}
c.JSON(http.StatusOK, response)
}
// GetActiveRegistrationTokens returns only active tokens
func (h *RegistrationTokenHandler) GetActiveRegistrationTokens(c *gin.Context) {
tokens, err := h.tokenQueries.GetActiveRegistrationTokens()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to get active tokens"})
return
}
c.JSON(http.StatusOK, gin.H{"tokens": tokens})
}
// RevokeRegistrationToken revokes a registration token
func (h *RegistrationTokenHandler) RevokeRegistrationToken(c *gin.Context) {
token := c.Param("token")
if token == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "Token is required"})
return
}
var request struct {
Reason string `json:"reason"`
}
c.ShouldBindJSON(&request) // Reason is optional
reason := request.Reason
if reason == "" {
reason = "Revoked via API"
}
err := h.tokenQueries.RevokeRegistrationToken(token, reason)
if err != nil {
if err.Error() == "token not found or already used/revoked" {
c.JSON(http.StatusNotFound, gin.H{"error": "Token not found or already used/revoked"})
} else {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to revoke token"})
}
return
}
c.JSON(http.StatusOK, gin.H{"message": "Token revoked successfully"})
}
// ValidateRegistrationToken checks if a token is valid (for testing/debugging)
func (h *RegistrationTokenHandler) ValidateRegistrationToken(c *gin.Context) {
token := c.Query("token")
if token == "" {
c.JSON(http.StatusBadRequest, gin.H{"error": "Token query parameter is required"})
return
}
tokenInfo, err := h.tokenQueries.ValidateRegistrationToken(token)
if err != nil {
c.JSON(http.StatusNotFound, gin.H{
"valid": false,
"error": err.Error(),
})
return
}
c.JSON(http.StatusOK, gin.H{
"valid": true,
"token": tokenInfo,
})
}
// CleanupExpiredTokens performs cleanup of expired tokens
func (h *RegistrationTokenHandler) CleanupExpiredTokens(c *gin.Context) {
count, err := h.tokenQueries.CleanupExpiredTokens()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to cleanup expired tokens"})
return
}
c.JSON(http.StatusOK, gin.H{
"message": "Cleanup completed",
"cleaned": count,
})
}
// GetTokenStats returns comprehensive token usage statistics
func (h *RegistrationTokenHandler) GetTokenStats(c *gin.Context) {
// Get token stats
tokenStats, err := h.tokenQueries.GetTokenUsageStats()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to get token stats"})
return
}
// Get agent count
activeAgentCount, err := h.agentQueries.GetActiveAgentCount()
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to get agent count"})
return
}
response := gin.H{
"token_stats": tokenStats,
"agent_usage": gin.H{
"active_agents": activeAgentCount,
"max_seats": h.config.AgentRegistration.MaxSeats,
"available": h.config.AgentRegistration.MaxSeats - activeAgentCount,
},
"security_limits": gin.H{
"max_tokens_per_request": h.config.AgentRegistration.MaxTokens,
"max_token_duration": "7 days",
"token_expiry_default": h.config.AgentRegistration.TokenExpiry,
},
}
c.JSON(http.StatusOK, response)
}

186
aggregator-server/internal/api/handlers/updates.go
View File

@@ -2,6 +2,7 @@ package handlers
import (
"fmt"
"log"
"net/http"
"strconv"
"time"
@@ -16,16 +17,42 @@ type UpdateHandler struct {
updateQueries *queries.UpdateQueries
agentQueries *queries.AgentQueries
commandQueries *queries.CommandQueries
agentHandler *AgentHandler
}
func NewUpdateHandler(uq *queries.UpdateQueries, aq *queries.AgentQueries, cq *queries.CommandQueries) *UpdateHandler {
func NewUpdateHandler(uq *queries.UpdateQueries, aq *queries.AgentQueries, cq *queries.CommandQueries, ah *AgentHandler) *UpdateHandler {
return &UpdateHandler{
updateQueries: uq,
agentQueries: aq,
commandQueries: cq,
agentHandler: ah,
}
}
// shouldEnableHeartbeat checks if heartbeat is already active for an agent
// Returns true if heartbeat should be enabled (i.e., not already active or expired)
func (h *UpdateHandler) shouldEnableHeartbeat(agentID uuid.UUID, durationMinutes int) (bool, error) {
agent, err := h.agentQueries.GetAgentByID(agentID)
if err != nil {
log.Printf("Warning: Failed to get agent %s for heartbeat check: %v", agentID, err)
return true, nil // Enable heartbeat by default if we can't check
}
// Check if rapid polling is already enabled and not expired
if enabled, ok := agent.Metadata["rapid_polling_enabled"].(bool); ok && enabled {
if untilStr, ok := agent.Metadata["rapid_polling_until"].(string); ok {
until, err := time.Parse(time.RFC3339, untilStr)
if err == nil && until.After(time.Now().Add(5*time.Minute)) {
// Heartbeat is already active for sufficient time
log.Printf("[Heartbeat] Agent %s already has active heartbeat until %s (skipping)", agentID, untilStr)
return false, nil
}
}
}
return true, nil
}
// ReportUpdates handles update reports from agents using event sourcing
func (h *UpdateHandler) ReportUpdates(c *gin.Context) {
agentID := c.MustGet("agent_id").(uuid.UUID)
@@ -172,7 +199,7 @@ func (h *UpdateHandler) ReportLog(c *gin.Context) {
return
}
log := &models.UpdateLog{
logEntry := &models.UpdateLog{
ID: uuid.New(),
AgentID: agentID,
Action: req.Action,
@@ -185,7 +212,7 @@ func (h *UpdateHandler) ReportLog(c *gin.Context) {
}
// Store the log entry
if err := h.updateQueries.CreateUpdateLog(log); err != nil {
if err := h.updateQueries.CreateUpdateLog(logEntry); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to save log"})
return
}
@@ -207,10 +234,34 @@ func (h *UpdateHandler) ReportLog(c *gin.Context) {
}
// Update command status based on log result
if req.Result == "success" {
if req.Result == "success" || req.Result == "completed" {
if err := h.commandQueries.MarkCommandCompleted(commandID, result); err != nil {
fmt.Printf("Warning: Failed to mark command %s as completed: %v\n", commandID, err)
}
// NEW: If this was a successful confirm_dependencies command, mark the package as updated
command, err := h.commandQueries.GetCommandByID(commandID)
if err == nil && command.CommandType == models.CommandTypeConfirmDependencies {
// Extract package info from command params
if packageName, ok := command.Params["package_name"].(string); ok {
if packageType, ok := command.Params["package_type"].(string); ok {
// Extract actual completion timestamp from command result for accurate audit trail
var completionTime *time.Time
if loggedAtStr, ok := command.Result["logged_at"].(string); ok {
if parsed, err := time.Parse(time.RFC3339Nano, loggedAtStr); err == nil {
completionTime = &parsed
}
}
// Update package status to 'updated' with actual completion timestamp
if err := h.updateQueries.UpdatePackageStatus(agentID, packageType, packageName, "updated", nil, completionTime); err != nil {
log.Printf("Warning: Failed to update package status for %s/%s: %v", packageType, packageName, err)
} else {
log.Printf("✅ Package %s (%s) marked as updated after successful installation", packageName, packageType)
}
}
}
}
} else if req.Result == "failed" || req.Result == "dry_run_failed" {
if err := h.commandQueries.MarkCommandFailed(commandID, result); err != nil {
fmt.Printf("Warning: Failed to mark command %s as failed: %v\n", commandID, err)
@@ -304,7 +355,7 @@ func (h *UpdateHandler) UpdatePackageStatus(c *gin.Context) {
return
}
if err := h.updateQueries.UpdatePackageStatus(agentID, req.PackageType, req.PackageName, req.Status, req.Metadata); err != nil {
if err := h.updateQueries.UpdatePackageStatus(agentID, req.PackageType, req.PackageName, req.Status, req.Metadata, nil); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to update package status"})
return
}
@@ -395,7 +446,29 @@ func (h *UpdateHandler) InstallUpdate(c *gin.Context) {
CreatedAt: time.Now(),
}
// Store the command in database
// Check if heartbeat should be enabled (avoid duplicates)
if shouldEnable, err := h.shouldEnableHeartbeat(update.AgentID, 10); err == nil && shouldEnable {
heartbeatCmd := &models.AgentCommand{
ID: uuid.New(),
AgentID: update.AgentID,
CommandType: models.CommandTypeEnableHeartbeat,
Params: models.JSONB{
"duration_minutes": 10,
},
Status: models.CommandStatusPending,
CreatedAt: time.Now(),
}
if err := h.commandQueries.CreateCommand(heartbeatCmd); err != nil {
log.Printf("[Heartbeat] Warning: Failed to create heartbeat command for agent %s: %v", update.AgentID, err)
} else {
log.Printf("[Heartbeat] Command created for agent %s before dry run", update.AgentID)
}
} else {
log.Printf("[Heartbeat] Skipping heartbeat command for agent %s (already active)", update.AgentID)
}
// Store the dry run command in database
if err := h.commandQueries.CreateCommand(command); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create dry run command"})
return
@@ -478,6 +551,28 @@ func (h *UpdateHandler) ReportDependencies(c *gin.Context) {
CreatedAt: time.Now(),
}
// Check if heartbeat should be enabled (avoid duplicates)
if shouldEnable, err := h.shouldEnableHeartbeat(agentID, 10); err == nil && shouldEnable {
heartbeatCmd := &models.AgentCommand{
ID: uuid.New(),
AgentID: agentID,
CommandType: models.CommandTypeEnableHeartbeat,
Params: models.JSONB{
"duration_minutes": 10,
},
Status: models.CommandStatusPending,
CreatedAt: time.Now(),
}
if err := h.commandQueries.CreateCommand(heartbeatCmd); err != nil {
log.Printf("[Heartbeat] Warning: Failed to create heartbeat command for agent %s: %v", agentID, err)
} else {
log.Printf("[Heartbeat] Command created for agent %s before installation", agentID)
}
} else {
log.Printf("[Heartbeat] Skipping heartbeat command for agent %s (already active)", agentID)
}
if err := h.commandQueries.CreateCommand(command); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create installation command"})
return
@@ -536,6 +631,28 @@ func (h *UpdateHandler) ConfirmDependencies(c *gin.Context) {
CreatedAt: time.Now(),
}
// Check if heartbeat should be enabled (avoid duplicates)
if shouldEnable, err := h.shouldEnableHeartbeat(update.AgentID, 10); err == nil && shouldEnable {
heartbeatCmd := &models.AgentCommand{
ID: uuid.New(),
AgentID: update.AgentID,
CommandType: models.CommandTypeEnableHeartbeat,
Params: models.JSONB{
"duration_minutes": 10,
},
Status: models.CommandStatusPending,
CreatedAt: time.Now(),
}
if err := h.commandQueries.CreateCommand(heartbeatCmd); err != nil {
log.Printf("[Heartbeat] Warning: Failed to create heartbeat command for agent %s: %v", update.AgentID, err)
} else {
log.Printf("[Heartbeat] Command created for agent %s before confirm dependencies", update.AgentID)
}
} else {
log.Printf("[Heartbeat] Skipping heartbeat command for agent %s (already active)", update.AgentID)
}
// Store the command in database
if err := h.commandQueries.CreateCommand(command); err != nil {
c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to create confirmation command"})
@@ -684,3 +801,60 @@ func (h *UpdateHandler) GetRecentCommands(c *gin.Context) {
"limit": limit,
})
}
// ClearFailedCommands manually removes failed/timed_out commands with cheeky warning
func (h *UpdateHandler) ClearFailedCommands(c *gin.Context) {
// Get query parameters for filtering
olderThanDaysStr := c.Query("older_than_days")
onlyRetriedStr := c.Query("only_retried")
allFailedStr := c.Query("all_failed")
var count int64
var err error
// Parse parameters
olderThanDays := 7 // default
if olderThanDaysStr != "" {
if days, err := strconv.Atoi(olderThanDaysStr); err == nil && days > 0 {
olderThanDays = days
}
}
onlyRetried := onlyRetriedStr == "true"
allFailed := allFailedStr == "true"
// Build the appropriate cleanup query based on parameters
if allFailed {
// Clear ALL failed commands (most aggressive)
count, err = h.commandQueries.ClearAllFailedCommands(olderThanDays)
} else if onlyRetried {
// Clear only failed commands that have been retried
count, err = h.commandQueries.ClearRetriedFailedCommands(olderThanDays)
} else {
// Clear failed commands older than specified days (default behavior)
count, err = h.commandQueries.ClearOldFailedCommands(olderThanDays)
}
if err != nil {
c.JSON(http.StatusInternalServerError, gin.H{
"error": "failed to clear failed commands",
"details": err.Error(),
})
return
}
// Return success with cheeky message
message := fmt.Sprintf("Archived %d failed commands", count)
if count > 0 {
message += ". WARNING: This shouldn't be necessary if the retry logic is working properly - you might want to check what's causing commands to fail in the first place!"
message += " (History preserved - commands moved to archived status)"
} else {
message += ". No failed commands found matching your criteria. SUCCESS!"
}
c.JSON(http.StatusOK, gin.H{
"message": message,
"count": count,
"cheeky_warning": "Consider this a developer experience enhancement - the system should clean up after itself automatically!",
})
}

279
aggregator-server/internal/api/middleware/rate_limiter.go Normal file
View File

@@ -0,0 +1,279 @@
package middleware
import (
"fmt"
"net/http"
"sync"
"time"
"github.com/gin-gonic/gin"
)
// RateLimitConfig holds configuration for rate limiting
type RateLimitConfig struct {
Requests int `json:"requests"`
Window time.Duration `json:"window"`
Enabled bool `json:"enabled"`
}
// RateLimitEntry tracks requests for a specific key
type RateLimitEntry struct {
Requests []time.Time
mutex sync.RWMutex
}
// RateLimiter implements in-memory rate limiting with user-configurable settings
type RateLimiter struct {
entries sync.Map // map[string]*RateLimitEntry
configs map[string]RateLimitConfig
mutex sync.RWMutex
}
// RateLimitSettings holds all user-configurable rate limit settings
type RateLimitSettings struct {
AgentRegistration RateLimitConfig `json:"agent_registration"`
AgentCheckIn RateLimitConfig `json:"agent_checkin"`
AgentReports RateLimitConfig `json:"agent_reports"`
AdminTokenGen RateLimitConfig `json:"admin_token_generation"`
AdminOperations RateLimitConfig `json:"admin_operations"`
PublicAccess RateLimitConfig `json:"public_access"`
}
// DefaultRateLimitSettings provides sensible defaults
func DefaultRateLimitSettings() RateLimitSettings {
return RateLimitSettings{
AgentRegistration: RateLimitConfig{
Requests: 5,
Window: time.Minute,
Enabled: true,
},
AgentCheckIn: RateLimitConfig{
Requests: 60,
Window: time.Minute,
Enabled: true,
},
AgentReports: RateLimitConfig{
Requests: 30,
Window: time.Minute,
Enabled: true,
},
AdminTokenGen: RateLimitConfig{
Requests: 10,
Window: time.Minute,
Enabled: true,
},
AdminOperations: RateLimitConfig{
Requests: 100,
Window: time.Minute,
Enabled: true,
},
PublicAccess: RateLimitConfig{
Requests: 20,
Window: time.Minute,
Enabled: true,
},
}
}
// NewRateLimiter creates a new rate limiter with default settings
func NewRateLimiter() *RateLimiter {
rl := &RateLimiter{
entries: sync.Map{},
}
// Load default settings
defaults := DefaultRateLimitSettings()
rl.UpdateSettings(defaults)
return rl
}
// UpdateSettings updates rate limit configurations
func (rl *RateLimiter) UpdateSettings(settings RateLimitSettings) {
rl.mutex.Lock()
defer rl.mutex.Unlock()
rl.configs = map[string]RateLimitConfig{
"agent_registration": settings.AgentRegistration,
"agent_checkin": settings.AgentCheckIn,
"agent_reports": settings.AgentReports,
"admin_token_gen": settings.AdminTokenGen,
"admin_operations": settings.AdminOperations,
"public_access": settings.PublicAccess,
}
}
// GetSettings returns current rate limit settings
func (rl *RateLimiter) GetSettings() RateLimitSettings {
rl.mutex.RLock()
defer rl.mutex.RUnlock()
return RateLimitSettings{
AgentRegistration: rl.configs["agent_registration"],
AgentCheckIn: rl.configs["agent_checkin"],
AgentReports: rl.configs["agent_reports"],
AdminTokenGen: rl.configs["admin_token_gen"],
AdminOperations: rl.configs["admin_operations"],
PublicAccess: rl.configs["public_access"],
}
}
// RateLimit creates middleware for a specific rate limit type
func (rl *RateLimiter) RateLimit(limitType string, keyFunc func(*gin.Context) string) gin.HandlerFunc {
return func(c *gin.Context) {
rl.mutex.RLock()
config, exists := rl.configs[limitType]
rl.mutex.RUnlock()
if !exists || !config.Enabled {
c.Next()
return
}
key := keyFunc(c)
if key == "" {
c.Next()
return
}
// Check rate limit
allowed, resetTime := rl.checkRateLimit(key, config)
if !allowed {
c.Header("X-RateLimit-Limit", fmt.Sprintf("%d", config.Requests))
c.Header("X-RateLimit-Remaining", "0")
c.Header("X-RateLimit-Reset", fmt.Sprintf("%d", resetTime.Unix()))
c.Header("Retry-After", fmt.Sprintf("%d", int(resetTime.Sub(time.Now()).Seconds())))
c.JSON(http.StatusTooManyRequests, gin.H{
"error": "Rate limit exceeded",
"limit": config.Requests,
"window": config.Window.String(),
"reset_time": resetTime,
})
c.Abort()
return
}
// Add rate limit headers
remaining := rl.getRemainingRequests(key, config)
c.Header("X-RateLimit-Limit", fmt.Sprintf("%d", config.Requests))
c.Header("X-RateLimit-Remaining", fmt.Sprintf("%d", remaining))
c.Header("X-RateLimit-Reset", fmt.Sprintf("%d", time.Now().Add(config.Window).Unix()))
c.Next()
}
}
// checkRateLimit checks if the request is allowed
func (rl *RateLimiter) checkRateLimit(key string, config RateLimitConfig) (bool, time.Time) {
now := time.Now()
// Get or create entry
entryInterface, _ := rl.entries.LoadOrStore(key, &RateLimitEntry{
Requests: []time.Time{},
})
entry := entryInterface.(*RateLimitEntry)
entry.mutex.Lock()
defer entry.mutex.Unlock()
// Clean old requests outside the window
cutoff := now.Add(-config.Window)
validRequests := make([]time.Time, 0)
for _, reqTime := range entry.Requests {
if reqTime.After(cutoff) {
validRequests = append(validRequests, reqTime)
}
}
// Check if under limit
if len(validRequests) >= config.Requests {
// Find when the oldest request expires
oldestRequest := validRequests[0]
resetTime := oldestRequest.Add(config.Window)
return false, resetTime
}
// Add current request
entry.Requests = append(validRequests, now)
// Clean up expired entries periodically
if len(entry.Requests) == 0 {
rl.entries.Delete(key)
}
return true, time.Time{}
}
// getRemainingRequests calculates remaining requests for the key
func (rl *RateLimiter) getRemainingRequests(key string, config RateLimitConfig) int {
entryInterface, ok := rl.entries.Load(key)
if !ok {
return config.Requests
}
entry := entryInterface.(*RateLimitEntry)
entry.mutex.RLock()
defer entry.mutex.RUnlock()
now := time.Now()
cutoff := now.Add(-config.Window)
count := 0
for _, reqTime := range entry.Requests {
if reqTime.After(cutoff) {
count++
}
}
remaining := config.Requests - count
if remaining < 0 {
remaining = 0
}
return remaining
}
// CleanupExpiredEntries removes expired entries to prevent memory leaks
func (rl *RateLimiter) CleanupExpiredEntries() {
rl.entries.Range(func(key, value interface{}) bool {
entry := value.(*RateLimitEntry)
entry.mutex.Lock()
now := time.Now()
validRequests := make([]time.Time, 0)
for _, reqTime := range entry.Requests {
if reqTime.After(now.Add(-time.Hour)) { // Keep requests from last hour
validRequests = append(validRequests, reqTime)
}
}
if len(validRequests) == 0 {
rl.entries.Delete(key)
} else {
entry.Requests = validRequests
}
entry.mutex.Unlock()
return true
})
}
// Key generation functions
func KeyByIP(c *gin.Context) string {
return c.ClientIP()
}
func KeyByAgentID(c *gin.Context) string {
return c.Param("id")
}
func KeyByUserID(c *gin.Context) string {
// This would extract user ID from JWT or session
// For now, use IP as fallback
return c.ClientIP()
}
func KeyByIPAndPath(c *gin.Context) string {
return c.ClientIP() + ":" + c.Request.URL.Path
}

222
aggregator-server/internal/config/config.go
View File

@@ -1,18 +1,47 @@
package config
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"fmt"
"os"
"strconv"
"strings"
"time"
"github.com/joho/godotenv"
"golang.org/x/term"
)
// Config holds the application configuration
type Config struct {
ServerPort string
DatabaseURL string
JWTSecret string
Server struct {
Host string `env:"REDFLAG_SERVER_HOST" default:"0.0.0.0"`
Port int `env:"REDFLAG_SERVER_PORT" default:"8080"`
TLS struct {
Enabled bool `env:"REDFLAG_TLS_ENABLED" default:"false"`
CertFile string `env:"REDFLAG_TLS_CERT_FILE"`
KeyFile string `env:"REDFLAG_TLS_KEY_FILE"`
}
}
Database struct {
Host string `env:"REDFLAG_DB_HOST" default:"localhost"`
Port int `env:"REDFLAG_DB_PORT" default:"5432"`
Database string `env:"REDFLAG_DB_NAME" default:"redflag"`
Username string `env:"REDFLAG_DB_USER" default:"redflag"`
Password string `env:"REDFLAG_DB_PASSWORD"`
}
Admin struct {
Username string `env:"REDFLAG_ADMIN_USER" default:"admin"`
Password string `env:"REDFLAG_ADMIN_PASSWORD"`
JWTSecret string `env:"REDFLAG_JWT_SECRET"`
}
AgentRegistration struct {
TokenExpiry string `env:"REDFLAG_TOKEN_EXPIRY" default:"24h"`
MaxTokens int `env:"REDFLAG_MAX_TOKENS" default:"100"`
MaxSeats int `env:"REDFLAG_MAX_SEATS" default:"50"`
}
CheckInInterval int
OfflineThreshold int
Timezone string
@@ -24,30 +53,195 @@ func Load() (*Config, error) {
// Load .env file if it exists (for development)
_ = godotenv.Load()
cfg := &Config{}
// Parse server configuration
cfg.Server.Host = getEnv("REDFLAG_SERVER_HOST", "0.0.0.0")
serverPort, _ := strconv.Atoi(getEnv("REDFLAG_SERVER_PORT", "8080"))
cfg.Server.Port = serverPort
cfg.Server.TLS.Enabled = getEnv("REDFLAG_TLS_ENABLED", "false") == "true"
cfg.Server.TLS.CertFile = getEnv("REDFLAG_TLS_CERT_FILE", "")
cfg.Server.TLS.KeyFile = getEnv("REDFLAG_TLS_KEY_FILE", "")
// Parse database configuration
cfg.Database.Host = getEnv("REDFLAG_DB_HOST", "localhost")
dbPort, _ := strconv.Atoi(getEnv("REDFLAG_DB_PORT", "5432"))
cfg.Database.Port = dbPort
cfg.Database.Database = getEnv("REDFLAG_DB_NAME", "redflag")
cfg.Database.Username = getEnv("REDFLAG_DB_USER", "redflag")
cfg.Database.Password = getEnv("REDFLAG_DB_PASSWORD", "")
// Parse admin configuration
cfg.Admin.Username = getEnv("REDFLAG_ADMIN_USER", "admin")
cfg.Admin.Password = getEnv("REDFLAG_ADMIN_PASSWORD", "")
cfg.Admin.JWTSecret = getEnv("REDFLAG_JWT_SECRET", "")
// Parse agent registration configuration
cfg.AgentRegistration.TokenExpiry = getEnv("REDFLAG_TOKEN_EXPIRY", "24h")
maxTokens, _ := strconv.Atoi(getEnv("REDFLAG_MAX_TOKENS", "100"))
cfg.AgentRegistration.MaxTokens = maxTokens
maxSeats, _ := strconv.Atoi(getEnv("REDFLAG_MAX_SEATS", "50"))
cfg.AgentRegistration.MaxSeats = maxSeats
// Parse legacy configuration for backwards compatibility
checkInInterval, _ := strconv.Atoi(getEnv("CHECK_IN_INTERVAL", "300"))
offlineThreshold, _ := strconv.Atoi(getEnv("OFFLINE_THRESHOLD", "600"))
cfg.CheckInInterval = checkInInterval
cfg.OfflineThreshold = offlineThreshold
cfg.Timezone = getEnv("TIMEZONE", "UTC")
cfg.LatestAgentVersion = getEnv("LATEST_AGENT_VERSION", "0.1.16")
cfg := &Config{
ServerPort: getEnv("SERVER_PORT", "8080"),
DatabaseURL: getEnv("DATABASE_URL", "postgres://aggregator:aggregator@localhost:5432/aggregator?sslmode=disable"),
JWTSecret: getEnv("JWT_SECRET", "test-secret-for-development-only"),
CheckInInterval: checkInInterval,
OfflineThreshold: offlineThreshold,
Timezone: getEnv("TIMEZONE", "UTC"),
LatestAgentVersion: getEnv("LATEST_AGENT_VERSION", "0.1.4"),
// Handle missing secrets
if cfg.Admin.Password == "" || cfg.Admin.JWTSecret == "" || cfg.Database.Password == "" {
fmt.Printf("[WARNING] Missing required configuration (admin password, JWT secret, or database password)\n")
fmt.Printf("[INFO] Run: ./redflag-server --setup to configure\n")
return nil, fmt.Errorf("missing required configuration")
}
// Debug: Log what JWT secret we're using (remove in production)
if cfg.JWTSecret == "test-secret-for-development-only" {
fmt.Printf("🔓 Using development JWT secret\n")
// Validate JWT secret is not the development default
if cfg.Admin.JWTSecret == "test-secret-for-development-only" {
fmt.Printf("[SECURITY WARNING] Using development JWT secret\n")
fmt.Printf("[INFO] Run: ./redflag-server --setup to configure production secrets\n")
}
return cfg, nil
}
// RunSetupWizard guides user through initial configuration
func RunSetupWizard() error {
fmt.Printf("RedFlag Server Setup Wizard\n")
fmt.Printf("===========================\n\n")
// Admin credentials
fmt.Printf("Admin Account Setup\n")
fmt.Printf("--------------------\n")
username := promptForInput("Admin username", "admin")
password := promptForPassword("Admin password")
// Database configuration
fmt.Printf("\nDatabase Configuration\n")
fmt.Printf("----------------------\n")
dbHost := promptForInput("Database host", "localhost")
dbPort, _ := strconv.Atoi(promptForInput("Database port", "5432"))
dbName := promptForInput("Database name", "redflag")
dbUser := promptForInput("Database user", "redflag")
dbPassword := promptForPassword("Database password")
// Server configuration
fmt.Printf("\nServer Configuration\n")
fmt.Printf("--------------------\n")
serverHost := promptForInput("Server bind address", "0.0.0.0")
serverPort, _ := strconv.Atoi(promptForInput("Server port", "8080"))
// Agent limits
fmt.Printf("\nAgent Registration\n")
fmt.Printf("------------------\n")
maxSeats, _ := strconv.Atoi(promptForInput("Maximum agent seats (security limit)", "50"))
// Generate JWT secret from admin password
jwtSecret := deriveJWTSecret(username, password)
// Create .env file
envContent := fmt.Sprintf(`# RedFlag Server Configuration
# Generated on %s
# Server Configuration
REDFLAG_SERVER_HOST=%s
REDFLAG_SERVER_PORT=%d
REDFLAG_TLS_ENABLED=false
# REDFLAG_TLS_CERT_FILE=
# REDFLAG_TLS_KEY_FILE=
# Database Configuration
REDFLAG_DB_HOST=%s
REDFLAG_DB_PORT=%d
REDFLAG_DB_NAME=%s
REDFLAG_DB_USER=%s
REDFLAG_DB_PASSWORD=%s
# Admin Configuration
REDFLAG_ADMIN_USER=%s
REDFLAG_ADMIN_PASSWORD=%s
REDFLAG_JWT_SECRET=%s
# Agent Registration
REDFLAG_TOKEN_EXPIRY=24h
REDFLAG_MAX_TOKENS=100
REDFLAG_MAX_SEATS=%d
# Legacy Configuration (for backwards compatibility)
SERVER_PORT=%d
DATABASE_URL=postgres://%s:%s@%s:%d/%s?sslmode=disable
JWT_SECRET=%s
CHECK_IN_INTERVAL=300
OFFLINE_THRESHOLD=600
TIMEZONE=UTC
LATEST_AGENT_VERSION=0.1.8
`, time.Now().Format("2006-01-02 15:04:05"), serverHost, serverPort,
dbHost, dbPort, dbName, dbUser, dbPassword,
username, password, jwtSecret, maxSeats,
serverPort, dbUser, dbPassword, dbHost, dbPort, dbName, jwtSecret)
// Write .env file
if err := os.WriteFile(".env", []byte(envContent), 0600); err != nil {
return fmt.Errorf("failed to write .env file: %w", err)
}
fmt.Printf("\n[OK] Configuration saved to .env file\n")
fmt.Printf("[SECURITY] File permissions set to 0600 (owner read/write only)\n")
fmt.Printf("\nNext steps:\n")
fmt.Printf(" 1. Start database: %s:%d\n", dbHost, dbPort)
fmt.Printf(" 2. Create database: CREATE DATABASE %s;\n", dbName)
fmt.Printf(" 3. Run migrations: ./redflag-server --migrate\n")
fmt.Printf(" 4. Start server: ./redflag-server\n")
fmt.Printf("\nServer will be available at: http://%s:%d\n", serverHost, serverPort)
fmt.Printf("Admin interface: http://%s:%d/admin\n", serverHost, serverPort)
return nil
}
func getEnv(key, defaultValue string) string {
if value := os.Getenv(key); value != "" {
return value
}
return defaultValue
}
func promptForInput(prompt, defaultValue string) string {
fmt.Printf("%s [%s]: ", prompt, defaultValue)
var input string
fmt.Scanln(&input)
if strings.TrimSpace(input) == "" {
return defaultValue
}
return strings.TrimSpace(input)
}
func promptForPassword(prompt string) string {
fmt.Printf("%s: ", prompt)
password, err := term.ReadPassword(int(os.Stdin.Fd()))
if err != nil {
// Fallback to non-hidden input
var input string
fmt.Scanln(&input)
return strings.TrimSpace(input)
}
fmt.Printf("\n")
return strings.TrimSpace(string(password))
}
func deriveJWTSecret(username, password string) string {
// Derive JWT secret from admin credentials
// This ensures JWT secret changes if admin password changes
hash := sha256.Sum256([]byte(username + password + "redflag-jwt-2024"))
return hex.EncodeToString(hash[:])
}
// GenerateSecureToken generates a cryptographically secure random token
func GenerateSecureToken() (string, error) {
bytes := make([]byte, 32)
if _, err := rand.Read(bytes); err != nil {
return "", fmt.Errorf("failed to generate secure token: %w", err)
}
return hex.EncodeToString(bytes), nil
}

9
aggregator-server/internal/database/migrations/009_add_retry_tracking.sql Normal file
View File

@@ -0,0 +1,9 @@
-- Add retry tracking to agent_commands table
-- This allows us to track command retry chains and display retry indicators in the UI
-- Add retried_from_id column to link retries to their original commands
ALTER TABLE agent_commands
ADD COLUMN retried_from_id UUID REFERENCES agent_commands(id) ON DELETE SET NULL;
-- Add index for efficient retry chain lookups
CREATE INDEX idx_commands_retried_from ON agent_commands(retried_from_id) WHERE retried_from_id IS NOT NULL;

9
aggregator-server/internal/database/migrations/010_add_archived_failed_status.sql Normal file
View File

@@ -0,0 +1,9 @@
-- Add 'archived_failed' status to agent_commands status constraint
-- This allows archiving failed/timed_out commands to clean up the active list
-- Drop the existing constraint
ALTER TABLE agent_commands DROP CONSTRAINT IF EXISTS agent_commands_status_check;
-- Add the new constraint with 'archived_failed' included
ALTER TABLE agent_commands ADD CONSTRAINT agent_commands_status_check
CHECK (status IN ('pending', 'sent', 'running', 'completed', 'failed', 'timed_out', 'cancelled', 'archived_failed'));

85
aggregator-server/internal/database/migrations/011_create_registration_tokens_table.sql Normal file
View File

@@ -0,0 +1,85 @@
-- Registration tokens for secure agent enrollment
-- Tokens are one-time use and have configurable expiration
CREATE TABLE registration_tokens (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
token VARCHAR(64) UNIQUE NOT NULL, -- One-time use token
label VARCHAR(255), -- Optional label for token identification
expires_at TIMESTAMP NOT NULL, -- Token expiration time
created_at TIMESTAMP DEFAULT NOW(), -- When token was created
used_at TIMESTAMP NULL, -- When token was used (NULL if unused)
used_by_agent_id UUID NULL, -- Which agent used this token (foreign key)
revoked BOOLEAN DEFAULT FALSE, -- Manual revocation
revoked_at TIMESTAMP NULL, -- When token was revoked
revoked_reason VARCHAR(255) NULL, -- Reason for revocation
-- Token status tracking
status VARCHAR(20) DEFAULT 'active'
CHECK (status IN ('active', 'used', 'expired', 'revoked')),
-- Additional metadata
created_by VARCHAR(100) DEFAULT 'setup_wizard', -- Who created the token
metadata JSONB DEFAULT '{}'::jsonb -- Additional token metadata
);
-- Indexes for performance
CREATE INDEX idx_registration_tokens_token ON registration_tokens(token);
CREATE INDEX idx_registration_tokens_expires_at ON registration_tokens(expires_at);
CREATE INDEX idx_registration_tokens_status ON registration_tokens(status);
CREATE INDEX idx_registration_tokens_used_by_agent ON registration_tokens(used_by_agent_id) WHERE used_by_agent_id IS NOT NULL;
-- Foreign key constraint for used_by_agent_id
ALTER TABLE registration_tokens
ADD CONSTRAINT fk_registration_tokens_agent
FOREIGN KEY (used_by_agent_id) REFERENCES agents(id) ON DELETE SET NULL;
-- Function to clean up expired tokens (called by periodic cleanup job)
CREATE OR REPLACE FUNCTION cleanup_expired_registration_tokens()
RETURNS INTEGER AS $$
DECLARE
deleted_count INTEGER;
BEGIN
UPDATE registration_tokens
SET status = 'expired',
used_at = NOW()
WHERE status = 'active'
AND expires_at < NOW()
AND used_at IS NULL;
GET DIAGNOSTICS deleted_count = ROW_COUNT;
RETURN deleted_count;
END;
$$ LANGUAGE plpgsql;
-- Function to check if a token is valid
CREATE OR REPLACE FUNCTION is_registration_token_valid(token_input VARCHAR)
RETURNS BOOLEAN AS $$
DECLARE
token_valid BOOLEAN;
BEGIN
SELECT (status = 'active' AND expires_at > NOW()) INTO token_valid
FROM registration_tokens
WHERE token = token_input;
RETURN COALESCE(token_valid, FALSE);
END;
$$ LANGUAGE plpgsql;
-- Function to mark token as used
CREATE OR REPLACE function mark_registration_token_used(token_input VARCHAR, agent_id UUID)
RETURNS BOOLEAN AS $$
DECLARE
updated BOOLEAN;
BEGIN
UPDATE registration_tokens
SET status = 'used',
used_at = NOW(),
used_by_agent_id = agent_id
WHERE token = token_input
AND status = 'active'
AND expires_at > NOW();
GET DIAGNOSTICS updated = ROW_COUNT;
RETURN updated > 0;
END;
$$ LANGUAGE plpgsql;

8
aggregator-server/internal/database/queries/agents.go
View File

@@ -196,3 +196,11 @@ func (q *AgentQueries) DeleteAgent(id uuid.UUID) error {
// Commit the transaction
return tx.Commit()
}
// GetActiveAgentCount returns the count of active (online) agents
func (q *AgentQueries) GetActiveAgentCount() (int, error) {
var count int
query := `SELECT COUNT(*) FROM agents WHERE status = 'online'`
err := q.db.Get(&count, query)
return count, err
}

116
aggregator-server/internal/database/queries/commands.go
View File

@@ -21,9 +21,9 @@ func NewCommandQueries(db *sqlx.DB) *CommandQueries {
func (q *CommandQueries) CreateCommand(cmd *models.AgentCommand) error {
query := `
INSERT INTO agent_commands (
id, agent_id, command_type, params, status
id, agent_id, command_type, params, status, retried_from_id
) VALUES (
:id, :agent_id, :command_type, :params, :status
:id, :agent_id, :command_type, :params, :status, :retried_from_id
)
`
_, err := q.db.NamedExec(query, cmd)
@@ -152,14 +152,15 @@ func (q *CommandQueries) RetryCommand(originalID uuid.UUID) (*models.AgentComman
return nil, fmt.Errorf("command must be failed, timed_out, or cancelled to retry")
}
// Create new command with same parameters
// Create new command with same parameters, linking it to the original
newCommand := &models.AgentCommand{
ID: uuid.New(),
AgentID: original.AgentID,
CommandType: original.CommandType,
Params: original.Params,
Status: models.CommandStatusPending,
CreatedAt: time.Now(),
ID: uuid.New(),
AgentID: original.AgentID,
CommandType: original.CommandType,
Params: original.Params,
Status: models.CommandStatusPending,
CreatedAt: time.Now(),
RetriedFromID: &originalID,
}
// Store the new command
@@ -180,20 +181,44 @@ func (q *CommandQueries) GetActiveCommands() ([]models.ActiveCommandInfo, error)
c.id,
c.agent_id,
c.command_type,
c.params,
c.status,
c.created_at,
c.sent_at,
c.result,
c.retried_from_id,
a.hostname as agent_hostname,
COALESCE(ups.package_name, 'N/A') as package_name,
COALESCE(ups.package_type, 'N/A') as package_type
COALESCE(ups.package_type, 'N/A') as package_type,
(c.retried_from_id IS NOT NULL) as is_retry,
EXISTS(SELECT 1 FROM agent_commands WHERE retried_from_id = c.id) as has_been_retried,
COALESCE((
WITH RECURSIVE retry_chain AS (
SELECT id, retried_from_id, 1 as depth
FROM agent_commands
WHERE id = c.id
UNION ALL
SELECT ac.id, ac.retried_from_id, rc.depth + 1
FROM agent_commands ac
JOIN retry_chain rc ON ac.id = rc.retried_from_id
)
SELECT MAX(depth) FROM retry_chain
), 1) - 1 as retry_count
FROM agent_commands c
LEFT JOIN agents a ON c.agent_id = a.id
LEFT JOIN current_package_state ups ON (
c.params->>'update_id' = ups.id::text OR
(c.params->>'package_name' = ups.package_name AND c.params->>'package_type' = ups.package_type)
)
WHERE c.status NOT IN ('completed', 'cancelled')
WHERE c.status NOT IN ('completed', 'cancelled', 'archived_failed')
AND NOT (
c.status IN ('failed', 'timed_out')
AND EXISTS (
SELECT 1 FROM agent_commands retry
WHERE retry.retried_from_id = c.id
AND retry.status = 'completed'
)
)
ORDER BY c.created_at DESC
`
@@ -223,9 +248,24 @@ func (q *CommandQueries) GetRecentCommands(limit int) ([]models.ActiveCommandInf
c.sent_at,
c.completed_at,
c.result,
c.retried_from_id,
a.hostname as agent_hostname,
COALESCE(ups.package_name, 'N/A') as package_name,
COALESCE(ups.package_type, 'N/A') as package_type
COALESCE(ups.package_type, 'N/A') as package_type,
(c.retried_from_id IS NOT NULL) as is_retry,
EXISTS(SELECT 1 FROM agent_commands WHERE retried_from_id = c.id) as has_been_retried,
COALESCE((
WITH RECURSIVE retry_chain AS (
SELECT id, retried_from_id, 1 as depth
FROM agent_commands
WHERE id = c.id
UNION ALL
SELECT ac.id, ac.retried_from_id, rc.depth + 1
FROM agent_commands ac
JOIN retry_chain rc ON ac.id = rc.retried_from_id
)
SELECT MAX(depth) FROM retry_chain
), 1) - 1 as retry_count
FROM agent_commands c
LEFT JOIN agents a ON c.agent_id = a.id
LEFT JOIN current_package_state ups ON (
@@ -243,3 +283,55 @@ func (q *CommandQueries) GetRecentCommands(limit int) ([]models.ActiveCommandInf
return commands, nil
}
// ClearOldFailedCommands archives failed commands older than specified days by changing status to 'archived_failed'
func (q *CommandQueries) ClearOldFailedCommands(days int) (int64, error) {
query := fmt.Sprintf(`
UPDATE agent_commands
SET status = 'archived_failed'
WHERE status IN ('failed', 'timed_out')
AND created_at < NOW() - INTERVAL '%d days'
`, days)
result, err := q.db.Exec(query)
if err != nil {
return 0, fmt.Errorf("failed to archive old failed commands: %w", err)
}
return result.RowsAffected()
}
// ClearRetriedFailedCommands archives failed commands that have been retried and are older than specified days
func (q *CommandQueries) ClearRetriedFailedCommands(days int) (int64, error) {
query := fmt.Sprintf(`
UPDATE agent_commands
SET status = 'archived_failed'
WHERE status IN ('failed', 'timed_out')
AND EXISTS (SELECT 1 FROM agent_commands WHERE retried_from_id = agent_commands.id)
AND created_at < NOW() - INTERVAL '%d days'
`, days)
result, err := q.db.Exec(query)
if err != nil {
return 0, fmt.Errorf("failed to archive retried failed commands: %w", err)
}
return result.RowsAffected()
}
// ClearAllFailedCommands archives all failed commands older than specified days (most aggressive)
func (q *CommandQueries) ClearAllFailedCommands(days int) (int64, error) {
query := fmt.Sprintf(`
UPDATE agent_commands
SET status = 'archived_failed'
WHERE status IN ('failed', 'timed_out')
AND created_at < NOW() - INTERVAL '%d days'
`, days)
result, err := q.db.Exec(query)
if err != nil {
return 0, fmt.Errorf("failed to archive all failed commands: %w", err)
}
return result.RowsAffected()
}

232
aggregator-server/internal/database/queries/registration_tokens.go Normal file
View File

@@ -0,0 +1,232 @@
package queries
import (
"database/sql"
"encoding/json"
"fmt"
"time"
"github.com/google/uuid"
"github.com/jmoiron/sqlx"
)
type RegistrationTokenQueries struct {
db *sqlx.DB
}
type RegistrationToken struct {
ID uuid.UUID `json:"id" db:"id"`
Token string `json:"token" db:"token"`
Label *string `json:"label" db:"label"`
ExpiresAt time.Time `json:"expires_at" db:"expires_at"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
UsedAt *time.Time `json:"used_at" db:"used_at"`
UsedByAgentID *uuid.UUID `json:"used_by_agent_id" db:"used_by_agent_id"`
Revoked bool `json:"revoked" db:"revoked"`
RevokedAt *time.Time `json:"revoked_at" db:"revoked_at"`
RevokedReason *string `json:"revoked_reason" db:"revoked_reason"`
Status string `json:"status" db:"status"`
CreatedBy string `json:"created_by" db:"created_by"`
Metadata map[string]interface{} `json:"metadata" db:"metadata"`
}
type TokenRequest struct {
Label string `json:"label"`
ExpiresIn string `json:"expires_in"` // e.g., "24h", "7d"
Metadata map[string]interface{} `json:"metadata"`
}
type TokenResponse struct {
Token string `json:"token"`
Label string `json:"label"`
ExpiresAt time.Time `json:"expires_at"`
InstallCommand string `json:"install_command"`
}
func NewRegistrationTokenQueries(db *sqlx.DB) *RegistrationTokenQueries {
return &RegistrationTokenQueries{db: db}
}
// CreateRegistrationToken creates a new one-time use registration token
func (q *RegistrationTokenQueries) CreateRegistrationToken(token, label string, expiresAt time.Time, metadata map[string]interface{}) error {
metadataJSON, err := json.Marshal(metadata)
if err != nil {
return fmt.Errorf("failed to marshal metadata: %w", err)
}
query := `
INSERT INTO registration_tokens (token, label, expires_at, metadata)
VALUES ($1, $2, $3, $4)
`
_, err = q.db.Exec(query, token, label, expiresAt, metadataJSON)
if err != nil {
return fmt.Errorf("failed to create registration token: %w", err)
}
return nil
}
// ValidateRegistrationToken checks if a token is valid and unused
func (q *RegistrationTokenQueries) ValidateRegistrationToken(token string) (*RegistrationToken, error) {
var regToken RegistrationToken
query := `
SELECT id, token, label, expires_at, created_at, used_at, used_by_agent_id,
revoked, revoked_at, revoked_reason, status, created_by, metadata
FROM registration_tokens
WHERE token = $1 AND status = 'active' AND expires_at > NOW()
`
err := q.db.Get(&regToken, query, token)
if err != nil {
if err == sql.ErrNoRows {
return nil, fmt.Errorf("invalid or expired token")
}
return nil, fmt.Errorf("failed to validate token: %w", err)
}
return &regToken, nil
}
// MarkTokenUsed marks a token as used by an agent
func (q *RegistrationTokenQueries) MarkTokenUsed(token string, agentID uuid.UUID) error {
query := `
UPDATE registration_tokens
SET status = 'used',
used_at = NOW(),
used_by_agent_id = $1
WHERE token = $2 AND status = 'active' AND expires_at > NOW()
`
result, err := q.db.Exec(query, agentID, token)
if err != nil {
return fmt.Errorf("failed to mark token as used: %w", err)
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return fmt.Errorf("failed to get rows affected: %w", err)
}
if rowsAffected == 0 {
return fmt.Errorf("token not found or already used")
}
return nil
}
// GetActiveRegistrationTokens returns all active tokens
func (q *RegistrationTokenQueries) GetActiveRegistrationTokens() ([]RegistrationToken, error) {
var tokens []RegistrationToken
query := `
SELECT id, token, label, expires_at, created_at, used_at, used_by_agent_id,
revoked, revoked_at, revoked_reason, status, created_by, metadata
FROM registration_tokens
WHERE status = 'active'
ORDER BY created_at DESC
`
err := q.db.Select(&tokens, query)
if err != nil {
return nil, fmt.Errorf("failed to get active tokens: %w", err)
}
return tokens, nil
}
// GetAllRegistrationTokens returns all tokens with pagination
func (q *RegistrationTokenQueries) GetAllRegistrationTokens(limit, offset int) ([]RegistrationToken, error) {
var tokens []RegistrationToken
query := `
SELECT id, token, label, expires_at, created_at, used_at, used_by_agent_id,
revoked, revoked_at, revoked_reason, status, created_by, metadata
FROM registration_tokens
ORDER BY created_at DESC
LIMIT $1 OFFSET $2
`
err := q.db.Select(&tokens, query, limit, offset)
if err != nil {
return nil, fmt.Errorf("failed to get all tokens: %w", err)
}
return tokens, nil
}
// RevokeRegistrationToken revokes a token
func (q *RegistrationTokenQueries) RevokeRegistrationToken(token, reason string) error {
query := `
UPDATE registration_tokens
SET status = 'revoked',
revoked = true,
revoked_at = NOW(),
revoked_reason = $1
WHERE token = $2 AND status = 'active'
`
result, err := q.db.Exec(query, reason, token)
if err != nil {
return fmt.Errorf("failed to revoke token: %w", err)
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return fmt.Errorf("failed to get rows affected: %w", err)
}
if rowsAffected == 0 {
return fmt.Errorf("token not found or already used/revoked")
}
return nil
}
// CleanupExpiredTokens marks expired tokens as expired
func (q *RegistrationTokenQueries) CleanupExpiredTokens() (int, error) {
query := `
UPDATE registration_tokens
SET status = 'expired',
used_at = NOW()
WHERE status = 'active' AND expires_at < NOW() AND used_at IS NULL
`
result, err := q.db.Exec(query)
if err != nil {
return 0, fmt.Errorf("failed to cleanup expired tokens: %w", err)
}
rowsAffected, err := result.RowsAffected()
if err != nil {
return 0, fmt.Errorf("failed to get rows affected: %w", err)
}
return int(rowsAffected), nil
}
// GetTokenUsageStats returns statistics about token usage
func (q *RegistrationTokenQueries) GetTokenUsageStats() (map[string]int, error) {
stats := make(map[string]int)
query := `
SELECT status, COUNT(*) as count
FROM registration_tokens
GROUP BY status
`
rows, err := q.db.Query(query)
if err != nil {
return nil, fmt.Errorf("failed to get token stats: %w", err)
}
defer rows.Close()
for rows.Next() {
var status string
var count int
if err := rows.Scan(&status, &count); err != nil {
return nil, fmt.Errorf("failed to scan token stats row: %w", err)
}
stats[status] = count
}
return stats, nil
}

13
aggregator-server/internal/database/queries/updates.go
View File

@@ -527,7 +527,8 @@ func (q *UpdateQueries) GetPackageHistory(agentID uuid.UUID, packageType, packag
}
// UpdatePackageStatus updates the status of a package and records history
func (q *UpdateQueries) UpdatePackageStatus(agentID uuid.UUID, packageType, packageName, status string, metadata models.JSONB) error {
// completedAt is optional - if nil, uses time.Now(). Pass actual completion time for accurate audit trails.
func (q *UpdateQueries) UpdatePackageStatus(agentID uuid.UUID, packageType, packageName, status string, metadata models.JSONB, completedAt *time.Time) error {
tx, err := q.db.Beginx()
if err != nil {
return fmt.Errorf("failed to begin transaction: %w", err)
@@ -542,13 +543,19 @@ func (q *UpdateQueries) UpdatePackageStatus(agentID uuid.UUID, packageType, pack
return fmt.Errorf("failed to get current state: %w", err)
}
// Use provided timestamp or fall back to server time
timestamp := time.Now()
if completedAt != nil {
timestamp = *completedAt
}
// Update status
updateQuery := `
UPDATE current_package_state
SET status = $1, last_updated_at = $2
WHERE agent_id = $3 AND package_type = $4 AND package_name = $5
`
_, err = tx.Exec(updateQuery, status, time.Now(), agentID, packageType, packageName)
_, err = tx.Exec(updateQuery, status, timestamp, agentID, packageType, packageName)
if err != nil {
return fmt.Errorf("failed to update package status: %w", err)
}
@@ -564,7 +571,7 @@ func (q *UpdateQueries) UpdatePackageStatus(agentID uuid.UUID, packageType, pack
_, err = tx.Exec(historyQuery,
agentID, packageType, packageName, currentState.CurrentVersion,
currentState.AvailableVersion, currentState.Severity,
currentState.RepositorySource, metadata, time.Now(), status)
currentState.RepositorySource, metadata, timestamp, status)
if err != nil {
return fmt.Errorf("failed to record version history: %w", err)
}

57
aggregator-server/internal/models/command.go
View File

@@ -8,20 +8,28 @@ import (
// AgentCommand represents a command to be executed by an agent
type AgentCommand struct {
ID uuid.UUID `json:"id" db:"id"`
AgentID uuid.UUID `json:"agent_id" db:"agent_id"`
CommandType string `json:"command_type" db:"command_type"`
Params JSONB `json:"params" db:"params"`
Status string `json:"status" db:"status"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
SentAt *time.Time `json:"sent_at,omitempty" db:"sent_at"`
CompletedAt *time.Time `json:"completed_at,omitempty" db:"completed_at"`
Result JSONB `json:"result,omitempty" db:"result"`
ID uuid.UUID `json:"id" db:"id"`
AgentID uuid.UUID `json:"agent_id" db:"agent_id"`
CommandType string `json:"command_type" db:"command_type"`
Params JSONB `json:"params" db:"params"`
Status string `json:"status" db:"status"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
SentAt *time.Time `json:"sent_at,omitempty" db:"sent_at"`
CompletedAt *time.Time `json:"completed_at,omitempty" db:"completed_at"`
Result JSONB `json:"result,omitempty" db:"result"`
RetriedFromID *uuid.UUID `json:"retried_from_id,omitempty" db:"retried_from_id"`
}
// CommandsResponse is returned when an agent checks in for commands
type CommandsResponse struct {
Commands []CommandItem `json:"commands"`
Commands []CommandItem `json:"commands"`
RapidPolling *RapidPollingConfig `json:"rapid_polling,omitempty"`
}
// RapidPollingConfig contains rapid polling configuration for the agent
type RapidPollingConfig struct {
Enabled bool `json:"enabled"`
Until string `json:"until"` // ISO 8601 timestamp
}
// CommandItem represents a command in the response
@@ -40,6 +48,8 @@ const (
CommandTypeConfirmDependencies = "confirm_dependencies"
CommandTypeRollback = "rollback_update"
CommandTypeUpdateAgent = "update_agent"
CommandTypeEnableHeartbeat = "enable_heartbeat"
CommandTypeDisableHeartbeat = "disable_heartbeat"
)
// Command statuses
@@ -55,15 +65,20 @@ const (
// ActiveCommandInfo represents information about an active command for UI display
type ActiveCommandInfo struct {
ID uuid.UUID `json:"id" db:"id"`
AgentID uuid.UUID `json:"agent_id" db:"agent_id"`
CommandType string `json:"command_type" db:"command_type"`
Status string `json:"status" db:"status"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
SentAt *time.Time `json:"sent_at,omitempty" db:"sent_at"`
CompletedAt *time.Time `json:"completed_at,omitempty" db:"completed_at"`
Result JSONB `json:"result,omitempty" db:"result"`
AgentHostname string `json:"agent_hostname" db:"agent_hostname"`
PackageName string `json:"package_name" db:"package_name"`
PackageType string `json:"package_type" db:"package_type"`
ID uuid.UUID `json:"id" db:"id"`
AgentID uuid.UUID `json:"agent_id" db:"agent_id"`
CommandType string `json:"command_type" db:"command_type"`
Params JSONB `json:"params" db:"params"`
Status string `json:"status" db:"status"`
CreatedAt time.Time `json:"created_at" db:"created_at"`
SentAt *time.Time `json:"sent_at,omitempty" db:"sent_at"`
CompletedAt *time.Time `json:"completed_at,omitempty" db:"completed_at"`
Result JSONB `json:"result,omitempty" db:"result"`
AgentHostname string `json:"agent_hostname" db:"agent_hostname"`
PackageName string `json:"package_name" db:"package_name"`
PackageType string `json:"package_type" db:"package_type"`
RetriedFromID *uuid.UUID `json:"retried_from_id,omitempty" db:"retried_from_id"`
IsRetry bool `json:"is_retry" db:"is_retry"`
HasBeenRetried bool `json:"has_been_retried" db:"has_been_retried"`
RetryCount int `json:"retry_count" db:"retry_count"`
}

3
aggregator-server/internal/services/timeout.go
View File

@@ -162,7 +162,8 @@ func (ts *TimeoutService) updateRelatedPackageStatus(command *models.AgentComman
command.Params["package_type"].(string),
command.Params["package_name"].(string),
"failed",
metadata)
metadata,
nil) // nil = use time.Now() for timeout operations
}
// extractUpdatePackageID extracts the update package ID from command params

4
aggregator-web/src/App.tsx
View File

@@ -10,6 +10,8 @@ import Docker from '@/pages/Docker';
import LiveOperations from '@/pages/LiveOperations';
import History from '@/pages/History';
import Settings from '@/pages/Settings';
import TokenManagement from '@/pages/TokenManagement';
import RateLimiting from '@/pages/RateLimiting';
import Login from '@/pages/Login';
// Protected route component
@@ -98,6 +100,8 @@ const App: React.FC = () => {
<Route path="/live" element={<LiveOperations />} />
<Route path="/history" element={<History />} />
<Route path="/settings" element={<Settings />} />
<Route path="/settings/tokens" element={<TokenManagement />} />
<Route path="/settings/rate-limiting" element={<RateLimiting />} />
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
</Layout>

129
aggregator-web/src/components/ChatTimeline.tsx
View File

@@ -18,6 +18,7 @@ import {
} from 'lucide-react';
import { useQuery } from '@tanstack/react-query';
import { logApi } from '@/lib/api';
import { useRetryCommand } from '@/hooks/useCommands';
import { cn } from '@/lib/utils';
import toast from 'react-hot-toast';
import { Highlight, themes } from 'prism-react-renderer';
@@ -47,11 +48,80 @@ interface ChatTimelineProps {
externalSearch?: string; // external search query from parent
}
// Helper function to create smart summaries for package operations
const createPackageOperationSummary = (entry: HistoryEntry): string => {
const action = entry.action.replace(/_/g, ' ');
const result = entry.result || 'unknown';
// Extract package name from stdout or params
let packageName = 'unknown package';
if (entry.params?.package_name) {
packageName = entry.params.package_name as string;
} else if (entry.stdout) {
// Look for package patterns in stdout
const packageMatch = entry.stdout.match(/(?:Upgrading|Installing|Package):\s+(\S+)/i);
if (packageMatch) {
packageName = packageMatch[1];
} else {
// Look for "Packages installed: [pkg]" pattern
const installedMatch = entry.stdout.match(/Packages installed:\s*\[([^\]]+)\]/i);
if (installedMatch) {
packageName = installedMatch[1];
}
}
}
// Extract duration if available
let durationInfo = '';
if (entry.logged_at) {
try {
const loggedTime = new Date(entry.logged_at).toLocaleTimeString('en-US', {
hour: '2-digit',
minute: '2-digit'
});
durationInfo = ` at ${loggedTime}`;
if (entry.duration_seconds) {
durationInfo += ` (${entry.duration_seconds}s)`;
}
} catch (e) {
// Ignore date parsing errors
}
}
// Create action-specific summaries
switch (entry.action) {
case 'upgrade':
case 'install':
case 'confirm_dependencies':
if (result === 'success' || result === 'completed') {
return `Successfully ${action}d ${packageName}${durationInfo}`;
} else if (result === 'failed' || result === 'error') {
return `Failed to ${action} ${packageName}${durationInfo}`;
} else {
return `${action.charAt(0).toUpperCase() + action.slice(1)} ${packageName}${durationInfo}`;
}
case 'dry_run_update':
if (result === 'success' || result === 'completed') {
return `Dry run completed for ${packageName}${durationInfo}`;
} else {
return `Dry run for ${packageName}${durationInfo}`;
}
default:
return `${action} ${packageName}${durationInfo}`;
}
};
const ChatTimeline: React.FC<ChatTimelineProps> = ({ agentId, className, isScopedView = false, externalSearch }) => {
const [statusFilter, setStatusFilter] = useState('all'); // 'all', 'success', 'failed', 'pending', 'completed', 'running', 'timed_out'
const [expandedEntries, setExpandedEntries] = useState<Set<string>>(new Set());
const [selectedAgents, setSelectedAgents] = useState<string[]>([]);
// Retry command hook
const retryCommandMutation = useRetryCommand();
// Query parameters for API
const [queryParams, setQueryParams] = useState({
page: 1,
@@ -440,18 +510,23 @@ const ChatTimeline: React.FC<ChatTimelineProps> = ({ agentId, className, isScope
sentence = `System log: ${entry.action}`;
}
} catch {
const lines = entry.stdout.split('\n');
const firstLine = lines[0]?.trim();
// Clean up common prefixes for more elegant system thoughts
if (firstLine) {
sentence = firstLine
.replace(/^(INFO|WARN|ERROR|DEBUG):\s*/i, '')
.replace(/^Step \d+:\s*/i, '')
.replace(/^Command:\s*/i, '')
.replace(/^Output:\s*/i, '')
.trim() || `System log: ${entry.action}`;
// Create smart summary for package management operations
if (['upgrade', 'install', 'confirm_dependencies', 'dry_run_update'].includes(entry.action)) {
sentence = createPackageOperationSummary(entry);
} else {
sentence = `System log: ${entry.action}`;
const lines = entry.stdout.split('\n');
const firstLine = lines[0]?.trim();
// Clean up common prefixes for more elegant system thoughts
if (firstLine) {
sentence = firstLine
.replace(/^(INFO|WARN|ERROR|DEBUG):\s*/i, '')
.replace(/^Step \d+:\s*/i, '')
.replace(/^Command:\s*/i, '')
.replace(/^Output:\s*/i, '')
.trim() || `System log: ${entry.action}`;
} else {
sentence = `System log: ${entry.action}`;
}
}
}
} else {
@@ -564,8 +639,8 @@ const ChatTimeline: React.FC<ChatTimelineProps> = ({ agentId, className, isScope
)}
{narrative.statusType === 'pending' && (
<>
<Clock className="h-3 w-3 text-purple-600" />
<span className="font-mono text-xs bg-purple-100 text-purple-800 px-1.5 py-0.5 rounded">
<Clock className="h-3 w-3 text-amber-600" />
<span className="font-mono text-xs bg-amber-100 text-amber-800 px-1.5 py-0.5 rounded">
PENDING
</span>
</>
@@ -862,15 +937,24 @@ const ChatTimeline: React.FC<ChatTimelineProps> = ({ agentId, className, isScope
{(entry.result === 'failed' || entry.result === 'timed_out') && (
<button
onClick={(e) => {
onClick={async (e) => {
e.stopPropagation();
// Handle retry logic - would integrate with API
toast.success(`Retry command sent to ${entry.hostname || 'agent'}`);
try {
await retryCommandMutation.mutateAsync(entry.id);
toast.success(`Retry command sent to ${entry.hostname || 'agent'}`);
} catch (error: any) {
toast.error(`Failed to retry command: ${error.message || 'Unknown error'}`);
}
}}
disabled={retryCommandMutation.isPending}
className="inline-flex items-center px-2.5 py-1.5 bg-amber-50 text-amber-700 rounded-md hover:bg-amber-100 transition-colors font-medium"
>
<RefreshCw className="h-3 w-3 mr-1" />
Retry Command
{retryCommandMutation.isPending ? (
<RefreshCw className="h-3 w-3 mr-1 animate-spin" />
) : (
<RefreshCw className="h-3 w-3 mr-1" />
)}
{retryCommandMutation.isPending ? 'Retrying...' : 'Retry Command'}
</button>
)}
</div>
@@ -988,10 +1072,11 @@ const ChatTimeline: React.FC<ChatTimelineProps> = ({ agentId, className, isScope
</p>
</div>
) : (
<div className="bg-gray-50 rounded-lg border border-gray-200 p-4">
<div className="space-y-4">
{createTimelineWithDividers(filteredEntries)}
</div>
<div className={cn(
isScopedView ? "bg-gray-50 rounded-lg border border-gray-200 p-4" : "",
"space-y-4"
)}>
{createTimelineWithDividers(filteredEntries)}
</div>
)}

10
aggregator-web/src/hooks/useAgents.ts
View File

@@ -7,8 +7,10 @@ export const useAgents = (params?: ListQueryParams): UseQueryResult<AgentListRes
return useQuery({
queryKey: ['agents', params],
queryFn: () => agentApi.getAgents(params),
staleTime: 30 * 1000, // Consider data stale after 30 seconds
refetchInterval: 60 * 1000, // Auto-refetch every minute
staleTime: 30 * 1000, // Consider data fresh for 30 seconds
refetchInterval: 60 * 1000, // Poll every 60 seconds
refetchIntervalInBackground: false, // Don't poll when tab is inactive
refetchOnWindowFocus: true, // Refresh when window gains focus
});
};
@@ -17,6 +19,10 @@ export const useAgent = (id: string, enabled: boolean = true): UseQueryResult<Ag
queryKey: ['agent', id],
queryFn: () => agentApi.getAgent(id),
enabled: enabled && !!id,
staleTime: 30 * 1000, // Consider data fresh for 30 seconds
refetchInterval: 30 * 1000, // Poll every 30 seconds for selected agent
refetchIntervalInBackground: false, // Don't poll when tab is inactive
refetchOnWindowFocus: true, // Refresh when window gains focus
});
};

22
aggregator-web/src/hooks/useCommands.ts
View File

@@ -15,11 +15,12 @@ interface ActiveCommand {
package_type: string;
}
export const useActiveCommands = (): UseQueryResult<{ commands: ActiveCommand[]; count: number }, Error> => {
export const useActiveCommands = (autoRefresh: boolean = true): UseQueryResult<{ commands: ActiveCommand[]; count: number }, Error> => {
return useQuery({
queryKey: ['activeCommands'],
queryFn: () => updateApi.getActiveCommands(),
refetchInterval: 5000, // Auto-refresh every 5 seconds
refetchInterval: autoRefresh ? 5000 : false, // Auto-refresh every 5 seconds when enabled
staleTime: 0, // Override global staleTime to allow refetchInterval to work
});
};
@@ -54,4 +55,21 @@ export const useCancelCommand = (): UseMutationResult<void, Error, string, unkno
queryClient.invalidateQueries({ queryKey: ['recentCommands'] });
},
});
};
export const useClearFailedCommands = (): UseMutationResult<{ message: string; count: number; cheeky_warning?: string }, Error, {
olderThanDays?: number;
onlyRetried?: boolean;
allFailed?: boolean;
}, unknown> => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: updateApi.clearFailedCommands,
onSuccess: () => {
// Invalidate active and recent commands queries to refresh the UI
queryClient.invalidateQueries({ queryKey: ['activeCommands'] });
queryClient.invalidateQueries({ queryKey: ['recentCommands'] });
},
});
};

63
aggregator-web/src/hooks/useHeartbeat.ts Normal file
View File

@@ -0,0 +1,63 @@
import { useQuery, useQueryClient } from '@tanstack/react-query';
import { agentApi } from '@/lib/api';
import type { UseQueryResult } from '@tanstack/react-query';
export interface HeartbeatStatus {
enabled: boolean;
until: string | null;
active: boolean;
duration_minutes: number;
}
export const useHeartbeatStatus = (agentId: string, enabled: boolean = true): UseQueryResult<HeartbeatStatus, Error> => {
const queryClient = useQueryClient();
return useQuery({
queryKey: ['heartbeat', agentId],
queryFn: () => agentApi.getHeartbeatStatus(agentId),
enabled: enabled && !!agentId,
staleTime: 5000, // Consider data stale after 5 seconds
refetchInterval: (query) => {
// Smart polling: only poll when heartbeat is active
const data = query.state.data as HeartbeatStatus | undefined;
// If heartbeat is enabled and still active, poll every 5 seconds
if (data?.enabled && data?.active) {
return 5000; // 5 seconds
}
// If heartbeat is not active, don't poll
return false;
},
refetchOnWindowFocus: false, // Don't refresh when window gains focus
refetchOnMount: true, // Always refetch when component mounts
});
};
// Hook to manually invalidate heartbeat cache (used after commands)
export const useInvalidateHeartbeat = () => {
const queryClient = useQueryClient();
return (agentId: string) => {
// Invalidate heartbeat cache
queryClient.invalidateQueries({ queryKey: ['heartbeat', agentId] });
// Also invalidate agent cache to synchronize data
queryClient.invalidateQueries({ queryKey: ['agent', agentId] });
queryClient.invalidateQueries({ queryKey: ['agents'] });
};
};
// Hook to synchronize agent data when heartbeat status changes
export const useHeartbeatAgentSync = (agentId: string, heartbeatStatus?: HeartbeatStatus) => {
const queryClient = useQueryClient();
// Sync agent data when heartbeat status changes
return () => {
if (agentId && heartbeatStatus) {
// Invalidate agent cache to get updated last_seen and status
queryClient.invalidateQueries({ queryKey: ['agent', agentId] });
queryClient.invalidateQueries({ queryKey: ['agents'] });
}
};
};

131
aggregator-web/src/hooks/useRateLimits.ts Normal file
View File

@@ -0,0 +1,131 @@
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { toast } from 'react-hot-toast';
import { adminApi } from '@/lib/api';
import {
RateLimitConfig,
RateLimitStats,
RateLimitUsage,
RateLimitSummary
} from '@/types';
// Query keys
export const rateLimitKeys = {
all: ['rate-limits'] as const,
configs: () => [...rateLimitKeys.all, 'configs'] as const,
stats: () => [...rateLimitKeys.all, 'stats'] as const,
usage: () => [...rateLimitKeys.all, 'usage'] as const,
summary: () => [...rateLimitKeys.all, 'summary'] as const,
};
// Hooks
export const useRateLimitConfigs = () => {
return useQuery({
queryKey: rateLimitKeys.configs(),
queryFn: () => adminApi.rateLimits.getConfigs(),
staleTime: 1000 * 60 * 5, // 5 minutes
});
};
export const useRateLimitStats = () => {
return useQuery({
queryKey: rateLimitKeys.stats(),
queryFn: () => adminApi.rateLimits.getStats(),
staleTime: 1000 * 30, // 30 seconds
refetchInterval: 1000 * 30, // Refresh every 30 seconds for real-time monitoring
});
};
export const useRateLimitUsage = () => {
return useQuery({
queryKey: rateLimitKeys.usage(),
queryFn: () => adminApi.rateLimits.getUsage(),
staleTime: 1000 * 15, // 15 seconds
refetchInterval: 1000 * 15, // Refresh every 15 seconds for live usage
});
};
export const useRateLimitSummary = () => {
return useQuery({
queryKey: rateLimitKeys.summary(),
queryFn: () => adminApi.rateLimits.getSummary(),
staleTime: 1000 * 60, // 1 minute
refetchInterval: 1000 * 60, // Refresh every minute
});
};
export const useUpdateRateLimitConfig = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: ({ endpoint, config }: { endpoint: string; config: Partial<RateLimitConfig> }) =>
adminApi.rateLimits.updateConfig(endpoint, config),
onSuccess: (_, { endpoint }) => {
toast.success(`Rate limit configuration for ${endpoint} updated successfully`);
queryClient.invalidateQueries({ queryKey: rateLimitKeys.configs() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.stats() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.usage() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.summary() });
},
onError: (error: any, { endpoint }) => {
console.error(`Failed to update rate limit config for ${endpoint}:`, error);
toast.error(error.response?.data?.message || `Failed to update rate limit configuration for ${endpoint}`);
},
});
};
export const useUpdateAllRateLimitConfigs = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (configs: RateLimitConfig[]) =>
adminApi.rateLimits.updateAllConfigs(configs),
onSuccess: () => {
toast.success('All rate limit configurations updated successfully');
queryClient.invalidateQueries({ queryKey: rateLimitKeys.configs() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.stats() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.usage() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.summary() });
},
onError: (error: any) => {
console.error('Failed to update rate limit configurations:', error);
toast.error(error.response?.data?.message || 'Failed to update rate limit configurations');
},
});
};
export const useResetRateLimitConfigs = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: () => adminApi.rateLimits.resetConfigs(),
onSuccess: () => {
toast.success('Rate limit configurations reset to defaults successfully');
queryClient.invalidateQueries({ queryKey: rateLimitKeys.configs() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.stats() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.usage() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.summary() });
},
onError: (error: any) => {
console.error('Failed to reset rate limit configurations:', error);
toast.error(error.response?.data?.message || 'Failed to reset rate limit configurations');
},
});
};
export const useCleanupRateLimits = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: () => adminApi.rateLimits.cleanup(),
onSuccess: (result) => {
toast.success(`Cleaned up ${result.cleaned} expired rate limit entries`);
queryClient.invalidateQueries({ queryKey: rateLimitKeys.stats() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.usage() });
queryClient.invalidateQueries({ queryKey: rateLimitKeys.summary() });
},
onError: (error: any) => {
console.error('Failed to cleanup rate limits:', error);
toast.error(error.response?.data?.message || 'Failed to cleanup rate limits');
},
});
};

103
aggregator-web/src/hooks/useRegistrationTokens.ts Normal file
View File

@@ -0,0 +1,103 @@
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { toast } from 'react-hot-toast';
import { adminApi } from '@/lib/api';
import {
RegistrationToken,
CreateRegistrationTokenRequest,
RegistrationTokenStats
} from '@/types';
// Query keys
export const registrationTokenKeys = {
all: ['registration-tokens'] as const,
lists: () => [...registrationTokenKeys.all, 'list'] as const,
list: (params: any) => [...registrationTokenKeys.lists(), params] as const,
details: () => [...registrationTokenKeys.all, 'detail'] as const,
detail: (id: string) => [...registrationTokenKeys.details(), id] as const,
stats: () => [...registrationTokenKeys.all, 'stats'] as const,
};
// Hooks
export const useRegistrationTokens = (params?: {
page?: number;
page_size?: number;
is_active?: boolean;
label?: string;
}) => {
return useQuery({
queryKey: registrationTokenKeys.list(params),
queryFn: () => adminApi.tokens.getTokens(params),
staleTime: 1000 * 60, // 1 minute
});
};
export const useRegistrationToken = (id: string) => {
return useQuery({
queryKey: registrationTokenKeys.detail(id),
queryFn: () => adminApi.tokens.getToken(id),
enabled: !!id,
staleTime: 1000 * 60, // 1 minute
});
};
export const useRegistrationTokenStats = () => {
return useQuery({
queryKey: registrationTokenKeys.stats(),
queryFn: () => adminApi.tokens.getStats(),
staleTime: 1000 * 60, // 1 minute
refetchInterval: 1000 * 60 * 5, // Refresh every 5 minutes
});
};
export const useCreateRegistrationToken = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (data: CreateRegistrationTokenRequest) =>
adminApi.tokens.createToken(data),
onSuccess: (newToken) => {
toast.success(`Registration token "${newToken.label}" created successfully`);
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.lists() });
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.stats() });
},
onError: (error: any) => {
console.error('Failed to create registration token:', error);
toast.error(error.response?.data?.message || 'Failed to create registration token');
},
});
};
export const useRevokeRegistrationToken = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: (id: string) => adminApi.tokens.revokeToken(id),
onSuccess: (_, tokenId) => {
toast.success('Registration token revoked successfully');
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.lists() });
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.detail(tokenId) });
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.stats() });
},
onError: (error: any) => {
console.error('Failed to revoke registration token:', error);
toast.error(error.response?.data?.message || 'Failed to revoke registration token');
},
});
};
export const useCleanupRegistrationTokens = () => {
const queryClient = useQueryClient();
return useMutation({
mutationFn: () => adminApi.tokens.cleanup(),
onSuccess: (result) => {
toast.success(`Cleaned up ${result.cleaned} expired tokens`);
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.lists() });
queryClient.invalidateQueries({ queryKey: registrationTokenKeys.stats() });
},
onError: (error: any) => {
console.error('Failed to cleanup registration tokens:', error);
toast.error(error.response?.data?.message || 'Failed to cleanup registration tokens');
},
});
};

186
aggregator-web/src/lib/api.ts
View File

@@ -14,7 +14,14 @@ import {
DockerContainerListResponse,
DockerStats,
DockerUpdateRequest,
BulkDockerUpdateRequest
BulkDockerUpdateRequest,
RegistrationToken,
CreateRegistrationTokenRequest,
RegistrationTokenStats,
RateLimitConfig,
RateLimitStats,
RateLimitUsage,
RateLimitSummary
} from '@/types';
// Base URL for API
@@ -75,6 +82,21 @@ export const agentApi = {
await api.post(`/agents/${id}/scan`);
},
// Trigger heartbeat toggle on single agent
toggleHeartbeat: async (id: string, enabled: boolean, durationMinutes: number = 10): Promise<{ message: string; command_id: string; enabled: boolean }> => {
const response = await api.post(`/agents/${id}/heartbeat`, {
enabled: enabled,
duration_minutes: durationMinutes,
});
return response.data;
},
// Get heartbeat status for single agent
getHeartbeatStatus: async (id: string): Promise<{ enabled: boolean; until: string | null; active: boolean; duration_minutes: number }> => {
const response = await api.get(`/agents/${id}/heartbeat`);
return response.data;
},
// Unregister/remove agent
unregisterAgent: async (id: string): Promise<void> => {
await api.delete(`/agents/${id}`);
@@ -147,6 +169,28 @@ export const updateApi = {
});
return response.data;
},
// Clear failed commands with filtering options
clearFailedCommands: async (options?: {
olderThanDays?: number;
onlyRetried?: boolean;
allFailed?: boolean;
}): Promise<{ message: string; count: number; cheeky_warning?: string }> => {
const params = new URLSearchParams();
if (options?.olderThanDays !== undefined) {
params.append('older_than_days', options.olderThanDays.toString());
}
if (options?.onlyRetried) {
params.append('only_retried', 'true');
}
if (options?.allFailed) {
params.append('all_failed', 'true');
}
const response = await api.delete(`/commands/failed${params.toString() ? '?' + params.toString() : ''}`);
return response.data;
},
};
export const statsApi = {
@@ -351,4 +395,144 @@ export const dockerApi = {
},
};
// Admin API endpoints
export const adminApi = {
// Registration Token Management
tokens: {
// Get all registration tokens
getTokens: async (params?: {
page?: number;
page_size?: number;
is_active?: boolean;
label?: string;
}): Promise<{ tokens: RegistrationToken[]; total: number; page: number; page_size: number }> => {
const response = await api.get('/admin/registration-tokens', { params });
return response.data;
},
// Get single registration token
getToken: async (id: string): Promise<RegistrationToken> => {
const response = await api.get(`/admin/registration-tokens/${id}`);
return response.data;
},
// Create new registration token
createToken: async (request: CreateRegistrationTokenRequest): Promise<RegistrationToken> => {
const response = await api.post('/admin/registration-tokens', request);
return response.data;
},
// Revoke registration token
revokeToken: async (id: string): Promise<void> => {
await api.delete(`/admin/registration-tokens/${id}`);
},
// Get registration token statistics
getStats: async (): Promise<RegistrationTokenStats> => {
const response = await api.get('/admin/registration-tokens/stats');
return response.data;
},
// Cleanup expired tokens
cleanup: async (): Promise<{ cleaned: number }> => {
const response = await api.post('/admin/registration-tokens/cleanup');
return response.data;
},
},
// Rate Limiting Management
rateLimits: {
// Get all rate limit configurations
getConfigs: async (): Promise<RateLimitConfig[]> => {
const response = await api.get('/admin/rate-limits');
return response.data;
},
// Update rate limit configuration
updateConfig: async (endpoint: string, config: Partial<RateLimitConfig>): Promise<RateLimitConfig> => {
const response = await api.put(`/admin/rate-limits/${endpoint}`, config);
return response.data;
},
// Update all rate limit configurations
updateAllConfigs: async (configs: RateLimitConfig[]): Promise<RateLimitConfig[]> => {
const response = await api.put('/admin/rate-limits', { configs });
return response.data;
},
// Reset rate limit configurations to defaults
resetConfigs: async (): Promise<RateLimitConfig[]> => {
const response = await api.post('/admin/rate-limits/reset');
return response.data;
},
// Get rate limit statistics
getStats: async (): Promise<RateLimitStats[]> => {
const response = await api.get('/admin/rate-limits/stats');
return response.data;
},
// Get rate limit usage
getUsage: async (): Promise<RateLimitUsage[]> => {
const response = await api.get('/admin/rate-limits/usage');
return response.data;
},
// Get rate limit summary
getSummary: async (): Promise<RateLimitSummary> => {
const response = await api.get('/admin/rate-limits/summary');
return response.data;
},
// Cleanup expired rate limit data
cleanup: async (): Promise<{ cleaned: number }> => {
const response = await api.post('/admin/rate-limits/cleanup');
return response.data;
},
},
// System Administration
system: {
// Get system health and status
getHealth: async (): Promise<{
status: 'healthy' | 'degraded' | 'unhealthy';
uptime: number;
version: string;
database_status: 'connected' | 'disconnected';
active_agents: number;
active_tokens: number;
rate_limits_enabled: boolean;
}> => {
const response = await api.get('/admin/system/health');
return response.data;
},
// Get active agents
getActiveAgents: async (): Promise<{
agents: Array<{
id: string;
hostname: string;
last_seen: string;
status: string;
}>;
count: number;
}> => {
const response = await api.get('/admin/system/active-agents');
return response.data;
},
// Get system configuration
getConfig: async (): Promise<Record<string, any>> => {
const response = await api.get('/admin/system/config');
return response.data;
},
// Update system configuration
updateConfig: async (config: Record<string, any>): Promise<Record<string, any>> => {
const response = await api.put('/admin/system/config', config);
return response.data;
},
},
};
export default api;

13
aggregator-web/src/lib/utils.ts
View File

@@ -18,6 +18,19 @@ export const formatDate = (dateString: string): string => {
});
};
export const formatDateTime = (dateString: string | null): string => {
if (!dateString) return 'Never';
const date = new Date(dateString);
return date.toLocaleString('en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
});
};
export const formatRelativeTime = (dateString: string): string => {
if (!dateString) return 'Never';

3
aggregator-web/src/main.tsx
View File

@@ -10,7 +10,8 @@ const queryClient = new QueryClient({
defaultOptions: {
queries: {
retry: 2,
staleTime: 10 * 1000, // 10 seconds
staleTime: 0, // Data is always stale to allow real-time updates
refetchOnWindowFocus: false, // Don't refetch on window focus to avoid unnecessary requests
},
},
})

317
aggregator-web/src/pages/Agents.tsx
View File

@@ -1,4 +1,4 @@
import React, { useState, useEffect } from 'react';
import React, { useState, useEffect, useRef } from 'react';
import { useParams, useNavigate } from 'react-router-dom';
import {
Computer,
@@ -6,6 +6,7 @@ import {
Search,
Filter,
ChevronRight as ChevronRightIcon,
ChevronDown,
Activity,
Calendar,
Package,
@@ -23,6 +24,9 @@ import {
} from 'lucide-react';
import { useAgents, useAgent, useScanAgent, useScanMultipleAgents, useUnregisterAgent } from '@/hooks/useAgents';
import { useActiveCommands, useCancelCommand } from '@/hooks/useCommands';
import { useHeartbeatStatus, useInvalidateHeartbeat, useHeartbeatAgentSync } from '@/hooks/useHeartbeat';
import { agentApi } from '@/lib/api';
import { useQueryClient } from '@tanstack/react-query';
import { getStatusColor, formatRelativeTime, isOnline, formatBytes } from '@/lib/utils';
import { cn } from '@/lib/utils';
import toast from 'react-hot-toast';
@@ -32,6 +36,7 @@ import ChatTimeline from '@/components/ChatTimeline';
const Agents: React.FC = () => {
const { id } = useParams<{ id?: string }>();
const navigate = useNavigate();
const queryClient = useQueryClient();
const [searchQuery, setSearchQuery] = useState('');
const [debouncedSearchQuery, setDebouncedSearchQuery] = useState('');
const [statusFilter, setStatusFilter] = useState<string>('all');
@@ -39,6 +44,40 @@ const Agents: React.FC = () => {
const [showFilters, setShowFilters] = useState(false);
const [selectedAgents, setSelectedAgents] = useState<string[]>([]);
const [activeTab, setActiveTab] = useState<'overview' | 'history'>('overview');
const [currentTime, setCurrentTime] = useState(new Date());
const [heartbeatDuration, setHeartbeatDuration] = useState<number>(10); // Default 10 minutes
const [showDurationDropdown, setShowDurationDropdown] = useState(false);
const [heartbeatLoading, setHeartbeatLoading] = useState(false); // Loading state for heartbeat toggle
const [heartbeatCommandId, setHeartbeatCommandId] = useState<string | null>(null); // Track specific heartbeat command
const dropdownRef = useRef<HTMLDivElement>(null);
// Close dropdown when clicking outside
useEffect(() => {
const handleClickOutside = (event: MouseEvent) => {
if (dropdownRef.current && !dropdownRef.current.contains(event.target as Node)) {
setShowDurationDropdown(false);
}
};
document.addEventListener('mousedown', handleClickOutside);
return () => {
document.removeEventListener('mousedown', handleClickOutside);
};
}, []);
// Duration options for heartbeat
const durationOptions = [
{ label: '10 minutes', value: 10 },
{ label: '30 minutes', value: 30 },
{ label: '1 hour', value: 60 },
{ label: 'Permanent', value: -1 },
];
// Get duration label for display
const getDurationLabel = (duration: number) => {
const option = durationOptions.find(opt => opt.value === duration);
return option?.label || '10 minutes';
};
// Debounce search query to avoid API calls on every keystroke
useEffect(() => {
@@ -51,6 +90,18 @@ const Agents: React.FC = () => {
};
}, [searchQuery]);
// Update current time every second for countdown timers
useEffect(() => {
const timer = setInterval(() => {
setCurrentTime(new Date());
}, 1000);
return () => {
clearInterval(timer);
};
}, []);
// Helper function to get system metadata from agent
const getSystemMetadata = (agent: any) => {
const metadata = agent.metadata || {};
@@ -124,8 +175,41 @@ const Agents: React.FC = () => {
return { platform, distribution, version: version.trim() };
};
// Helper function to format heartbeat expiration time
const formatHeartExpiration = (untilString: string) => {
const until = new Date(untilString);
const now = new Date();
const diffMs = until.getTime() - now.getTime();
if (diffMs <= 0) {
return 'expired';
}
const diffMinutes = Math.floor(diffMs / (1000 * 60));
if (diffMinutes < 60) {
return `${diffMinutes} minute${diffMinutes !== 1 ? 's' : ''}`;
}
const diffHours = Math.floor(diffMinutes / 60);
const remainingMinutes = diffMinutes % 60;
if (diffHours < 24) {
return remainingMinutes > 0
? `${diffHours} hour${diffHours !== 1 ? 's' : ''} ${remainingMinutes} min`
: `${diffHours} hour${diffHours !== 1 ? 's' : ''}`;
}
const diffDays = Math.floor(diffHours / 24);
const remainingHours = diffHours % 24;
return remainingHours > 0
? `${diffDays} day${diffDays !== 1 ? 's' : ''} ${remainingHours} hour${remainingHours !== 1 ? 's' : ''}`
: `${diffDays} day${diffDays !== 1 ? 's' : ''}`;
};
// Fetch agents list
const { data: agentsData, isPending, error } = useAgents({
const { data: agentsData, isPending, error, refetch } = useAgents({
search: debouncedSearchQuery || undefined,
status: statusFilter !== 'all' ? statusFilter : undefined,
});
@@ -141,9 +225,31 @@ const Agents: React.FC = () => {
const { data: activeCommandsData, refetch: refetchActiveCommands } = useActiveCommands();
const cancelCommandMutation = useCancelCommand();
const agents = agentsData?.agents || [];
const selectedAgent = selectedAgentData || agents.find(a => a.id === id);
// Get heartbeat status for selected agent (smart polling - only when active)
const { data: heartbeatStatus } = useHeartbeatStatus(selectedAgent?.id || '', !!selectedAgent);
const invalidateHeartbeat = useInvalidateHeartbeat();
const syncAgentData = useHeartbeatAgentSync(selectedAgent?.id || '', heartbeatStatus);
// Simple completion handling - clear loading state quickly
useEffect(() => {
if (!heartbeatCommandId) return;
// Clear loading state quickly since smart polling will handle UI updates
const timeout = setTimeout(() => {
setHeartbeatCommandId(null);
setHeartbeatLoading(false);
}, 2000); // 2 seconds - enough time for command to process
return () => {
clearTimeout(timeout);
};
}, [heartbeatCommandId]);
// Filter agents based on OS
const filteredAgents = agents.filter(agent => {
if (osFilter === 'all') return true;
@@ -224,6 +330,40 @@ const Agents: React.FC = () => {
}
};
// Handle rapid polling toggle
const handleRapidPollingToggle = async (agentId: string, enabled: boolean, durationMinutes?: number) => {
// Prevent multiple clicks
if (heartbeatLoading) return;
setHeartbeatLoading(true);
try {
const duration = durationMinutes || heartbeatDuration;
const result = await agentApi.toggleHeartbeat(agentId, enabled, duration);
// Immediately invalidate cache to force fresh data
invalidateHeartbeat(agentId);
// Store the command ID for minimal tracking
if (result.command_id) {
setHeartbeatCommandId(result.command_id);
}
if (enabled) {
if (duration === -1) {
toast.success('Heartbeat enabled permanently');
} else {
toast.success(`Heartbeat enabled for ${duration} minutes`);
}
} else {
toast.success('Heartbeat disabled');
}
} catch (error: any) {
toast.error(`Failed to send heartbeat command: ${error.message || 'Unknown error'}`);
setHeartbeatLoading(false);
setHeartbeatCommandId(null);
}
};
// Get agent-specific active commands
const getAgentActiveCommands = () => {
if (!selectedAgent || !activeCommandsData?.commands) return [];
@@ -232,11 +372,19 @@ const Agents: React.FC = () => {
// Helper function to get command display info
const getCommandDisplayInfo = (command: any) => {
// Helper to get package name from command params
const getPackageName = (cmd: any) => {
if (cmd.package_name) return cmd.package_name;
if (cmd.params?.package_name) return cmd.params.package_name;
if (cmd.params?.update_id && cmd.update_name) return cmd.update_name;
return 'unknown package';
};
const actionMap: { [key: string]: { icon: React.ReactNode; label: string } } = {
'scan': { icon: <RefreshCw className="h-4 w-4" />, label: 'System scan' },
'install_updates': { icon: <Package className="h-4 w-4" />, label: `Installing ${command.package_name || 'packages'}` },
'dry_run_update': { icon: <Search className="h-4 w-4" />, label: `Checking dependencies for ${command.package_name || 'packages'}` },
'confirm_dependencies': { icon: <CheckCircle className="h-4 w-4" />, label: `Installing confirmed dependencies` },
'install_updates': { icon: <Package className="h-4 w-4" />, label: `Installing ${getPackageName(command)}` },
'dry_run_update': { icon: <Search className="h-4 w-4" />, label: `Checking dependencies for ${getPackageName(command)}` },
'confirm_dependencies': { icon: <CheckCircle className="h-4 w-4" />, label: `Installing ${getPackageName(command)}` },
};
return actionMap[command.command_type] || {
@@ -386,22 +534,88 @@ const Agents: React.FC = () => {
{isOnline(selectedAgent.last_seen) ? 'Online' : 'Offline'}
</span>
</div>
{/* Heartbeat Status Indicator */}
<div className="flex items-center space-x-2">
{(() => {
// Use dedicated heartbeat status instead of general agent metadata
const isRapidPolling = heartbeatStatus?.enabled && heartbeatStatus?.active;
return (
<button
onClick={() => handleRapidPollingToggle(selectedAgent.id, !isRapidPolling)}
disabled={heartbeatLoading}
className={cn(
'flex items-center space-x-1 px-2 py-1 rounded-md text-xs font-medium transition-colors',
heartbeatLoading
? 'bg-gray-100 text-gray-400 border border-gray-200 cursor-not-allowed'
: isRapidPolling
? 'bg-pink-100 text-pink-800 border border-pink-200 hover:bg-pink-200 cursor-pointer'
: 'bg-gray-100 text-gray-600 border border-gray-200 hover:bg-gray-200 cursor-pointer'
)}
title={heartbeatLoading ? 'Sending command...' : `Click to toggle ${isRapidPolling ? 'normal' : 'heartbeat'} mode`}
>
{heartbeatLoading ? (
<RefreshCw className="h-3 w-3 animate-spin" />
) : (
<Activity className={cn(
'h-3 w-3',
isRapidPolling ? 'text-pink-600 animate-pulse' : 'text-gray-400'
)} />
)}
<span>
{heartbeatLoading ? 'Sending...' : isRapidPolling ? 'Heartbeat (5s)' : 'Normal (5m)'}
</span>
</button>
);
})()}
</div>
</div>
{/* Compact Timeline Display */}
<div className="space-y-2 mb-3">
{(() => {
const agentCommands = getAgentActiveCommands();
const activeCommands = agentCommands.filter(cmd =>
// Separate heartbeat commands from other commands
const heartbeatCommands = agentCommands.filter(cmd =>
cmd.command_type === 'enable_heartbeat' || cmd.command_type === 'disable_heartbeat'
);
const otherCommands = agentCommands.filter(cmd =>
cmd.command_type !== 'enable_heartbeat' && cmd.command_type !== 'disable_heartbeat'
);
// For heartbeat commands: only show the MOST RECENT one, but exclude old completed ones
const recentHeartbeatCommands = heartbeatCommands.filter(cmd => {
const createdTime = new Date(cmd.created_at);
const now = new Date();
const hoursOld = (now.getTime() - createdTime.getTime()) / (1000 * 60 * 60);
// Exclude completed/failed heartbeat commands older than 30 minutes
if ((cmd.status === 'completed' || cmd.status === 'failed' || cmd.status === 'timed_out') && hoursOld > 0.5) {
return false;
}
return true;
});
const latestHeartbeatCommand = recentHeartbeatCommands.length > 0
? [recentHeartbeatCommands.reduce((latest, cmd) =>
new Date(cmd.created_at) > new Date(latest.created_at) ? cmd : latest
)]
: [];
// For other commands: show active ones normally
const activeOtherCommands = otherCommands.filter(cmd =>
cmd.status === 'running' || cmd.status === 'sent' || cmd.status === 'pending'
);
const completedCommands = agentCommands.filter(cmd =>
const completedOtherCommands = otherCommands.filter(cmd =>
cmd.status === 'completed' || cmd.status === 'failed' || cmd.status === 'timed_out'
).slice(0, 1); // Only show last completed
const displayCommands = [
...activeCommands.slice(0, 2), // Max 2 active
...completedCommands.slice(0, 1) // Max 1 completed
...latestHeartbeatCommand.slice(0, 1), // Max 1 heartbeat (latest only)
...activeOtherCommands.slice(0, 2), // Max 2 active other commands
...completedOtherCommands.slice(0, 1) // Max 1 completed other command
].slice(0, 3); // Total max 3 entries
if (displayCommands.length === 0) {
@@ -454,7 +668,18 @@ const Agents: React.FC = () => {
</div>
<div className="flex items-center justify-between mt-1">
<span className="text-xs text-gray-500">
{formatRelativeTime(command.created_at)}
{(() => {
const createdTime = new Date(command.created_at);
const now = new Date();
const hoursOld = (now.getTime() - createdTime.getTime()) / (1000 * 60 * 60);
// Show exact time for commands older than 1 hour, relative time for recent ones
if (hoursOld > 1) {
return createdTime.toLocaleString();
} else {
return formatRelativeTime(command.created_at);
}
})()}
</span>
{isActive && (command.status === 'pending' || command.status === 'sent') && (
<button
@@ -479,6 +704,13 @@ const Agents: React.FC = () => {
<span>Last scan: {selectedAgent.last_scan ? formatRelativeTime(selectedAgent.last_scan) : 'Never'}</span>
</div>
{/* Heartbeat Status Info */}
{heartbeatStatus?.enabled && heartbeatStatus?.active && (
<div className="text-xs text-pink-600 bg-pink-50 px-2 py-1 rounded-md mt-2">
Heartbeat active for {formatHeartExpiration(heartbeatStatus.until)}
</div>
)}
{/* Action Button */}
<div className="flex justify-center mt-3 pt-3 border-t border-gray-200">
<button
@@ -650,6 +882,71 @@ const Agents: React.FC = () => {
View All Updates
</button>
{/* Split button for heartbeat with duration */}
<div className="flex space-x-2">
<button
onClick={() => {
// Use dedicated heartbeat status instead of general agent metadata
const isRapidPolling = heartbeatStatus?.enabled && heartbeatStatus?.active;
handleRapidPollingToggle(selectedAgent.id, !isRapidPolling);
}}
disabled={heartbeatLoading}
className={cn(
'flex-1 btn transition-colors',
heartbeatLoading
? 'opacity-50 cursor-not-allowed'
: heartbeatStatus?.enabled && heartbeatStatus?.active
? 'btn-primary' // Use primary style for active heartbeat
: 'btn-secondary' // Use secondary style for normal mode
)}
>
{heartbeatLoading ? (
<RefreshCw className="h-4 w-4 mr-2 animate-spin" />
) : (
<Activity className="h-4 w-4 mr-2" />
)}
{heartbeatLoading
? 'Sending Command...'
: heartbeatStatus?.enabled && heartbeatStatus?.active
? 'Disable Heartbeat'
: 'Enable Heartbeat (5s)'
}
</button>
{/* Duration dropdown */}
<div className="relative" ref={dropdownRef}>
<button
onClick={() => setShowDurationDropdown(!showDurationDropdown)}
className="btn btn-secondary px-3 min-w-[100px]"
>
{getDurationLabel(heartbeatDuration)}
<ChevronDown className="h-4 w-4 ml-1" />
</button>
{showDurationDropdown && (
<div className="absolute right-0 mt-1 w-48 bg-white rounded-lg shadow-lg border border-gray-200 z-10">
<div className="py-1">
{durationOptions.map((option) => (
<button
key={option.value}
onClick={() => {
setHeartbeatDuration(option.value);
setShowDurationDropdown(false);
}}
className={cn(
'w-full px-4 py-2 text-left text-sm hover:bg-gray-100 transition-colors',
heartbeatDuration === option.value ? 'bg-gray-100 font-medium' : 'text-gray-700'
)}
>
{option.label}
</button>
))}
</div>
</div>
)}
</div>
</div>
<button
onClick={() => handleRemoveAgent(selectedAgent.id, selectedAgent.hostname)}
disabled={unregisterAgentMutation.isPending}

235
aggregator-web/src/pages/LiveOperations.tsx
View File

@@ -19,9 +19,10 @@ import {
Eye,
RotateCcw,
X,
Archive,
} from 'lucide-react';
import { useAgents, useUpdates } from '@/hooks/useAgents';
import { useActiveCommands, useRetryCommand, useCancelCommand } from '@/hooks/useCommands';
import { useActiveCommands, useRetryCommand, useCancelCommand, useClearFailedCommands } from '@/hooks/useCommands';
import { getStatusColor, formatRelativeTime, isOnline } from '@/lib/utils';
import { cn } from '@/lib/utils';
import toast from 'react-hot-toast';
@@ -34,7 +35,7 @@ interface LiveOperation {
updateId: string;
packageName: string;
action: 'checking_dependencies' | 'installing' | 'pending_dependencies';
status: 'running' | 'completed' | 'failed' | 'waiting';
status: 'running' | 'completed' | 'failed' | 'pending' | 'sent';
startTime: Date;
duration?: number;
progress?: string;
@@ -42,21 +43,32 @@ interface LiveOperation {
error?: string;
commandId: string;
commandStatus: string;
isRetry?: boolean;
hasBeenRetried?: boolean;
retryCount?: number;
retriedFromId?: string;
}
const LiveOperations: React.FC = () => {
const [expandedOperation, setExpandedOperation] = useState<string | null>(null);
const [expandedOperations, setExpandedOperations] = useState<Set<string>>(new Set());
const [autoRefresh, setAutoRefresh] = useState(true);
const [searchQuery, setSearchQuery] = useState('');
const [statusFilter, setStatusFilter] = useState<string>('all');
const [showFilters, setShowFilters] = useState(false);
const [showCleanupDialog, setShowCleanupDialog] = useState(false);
const [cleanupOptions, setCleanupOptions] = useState({
olderThanDays: 7,
onlyRetried: false,
allFailed: false
});
// Fetch active commands from API
const { data: activeCommandsData, refetch: refetchCommands } = useActiveCommands();
const { data: activeCommandsData, refetch: refetchCommands } = useActiveCommands(autoRefresh);
// Retry and cancel mutations
// Retry, cancel, and cleanup mutations
const retryMutation = useRetryCommand();
const cancelMutation = useCancelCommand();
const clearFailedMutation = useClearFailedCommands();
// Fetch agents for mapping
const { data: agentsData } = useAgents();
@@ -77,7 +89,9 @@ const LiveOperations: React.FC = () => {
if (cmd.status === 'failed' || cmd.status === 'timed_out') {
status = 'failed';
} else if (cmd.status === 'pending') {
status = 'waiting';
status = 'pending';
} else if (cmd.status === 'sent') {
status = 'sent';
} else if (cmd.status === 'completed') {
status = 'completed';
} else {
@@ -105,10 +119,16 @@ const LiveOperations: React.FC = () => {
packageName: cmd.package_name !== 'N/A' ? cmd.package_name : cmd.command_type,
action,
status,
startTime: new Date(cmd.created_at),
startTime: cmd.created_at ? new Date(cmd.created_at) : new Date(),
progress: getStatusText(cmd.command_type, cmd.status),
commandId: cmd.id,
commandStatus: cmd.status,
logOutput: cmd.result?.stdout || cmd.result?.stderr,
error: cmd.result?.error_message,
isRetry: cmd.is_retry || false,
hasBeenRetried: cmd.has_been_retried || false,
retryCount: cmd.retry_count || 0,
retriedFromId: cmd.retried_from_id,
};
});
}, [activeCommandsData, agents]);
@@ -138,6 +158,32 @@ const LiveOperations: React.FC = () => {
}
};
// Handle cleanup failed commands
const handleClearFailedCommands = async () => {
try {
const result = await clearFailedMutation.mutateAsync(cleanupOptions);
toast.success(result.message);
if (result.cheeky_warning) {
// Optional: Show a secondary toast with the cheeky warning
setTimeout(() => {
toast(result.cheeky_warning, {
icon: '⚠️',
style: {
background: '#fef3c7',
color: '#92400e',
},
});
}, 1000);
}
setShowCleanupDialog(false);
} catch (error: any) {
toast.error(`Failed to clear failed commands: ${error.message || 'Unknown error'}`);
}
};
// Count failed operations for display
const failedCount = activeOperations.filter(op => op.status === 'failed').length;
function getStatusText(commandType: string, status: string): string {
if (commandType === 'dry_run_update') {
return status === 'pending' ? 'Pending dependency check...' : 'Checking for required dependencies...';
@@ -172,7 +218,8 @@ const LiveOperations: React.FC = () => {
return <CheckCircle className="h-4 w-4" />;
case 'failed':
return <XCircle className="h-4 w-4" />;
case 'waiting':
case 'pending':
case 'sent':
return <Clock className="h-4 w-4" />;
default:
return <Activity className="h-4 w-4" />;
@@ -235,6 +282,15 @@ const LiveOperations: React.FC = () => {
<RefreshCw className="h-4 w-4" />
<span>Refresh Now</span>
</button>
{failedCount > 0 && (
<button
onClick={() => setShowCleanupDialog(true)}
className="flex items-center space-x-2 px-3 py-2 bg-red-100 text-red-700 rounded-lg hover:bg-red-200 text-sm font-medium transition-colors"
>
<Archive className="h-4 w-4" />
<span>Archive Failed ({failedCount})</span>
</button>
)}
</div>
</div>
@@ -265,9 +321,9 @@ const LiveOperations: React.FC = () => {
<div className="bg-white p-4 rounded-lg border border-amber-200 shadow-sm">
<div className="flex items-center justify-between">
<div>
<p className="text-sm font-medium text-gray-600">Waiting</p>
<p className="text-sm font-medium text-gray-600">Pending</p>
<p className="text-2xl font-bold text-amber-600">
{activeOperations.filter(op => op.status === 'waiting').length}
{activeOperations.filter(op => op.status === 'pending' || op.status === 'sent').length}
</p>
</div>
<Clock className="h-8 w-8 text-amber-400" />
@@ -331,7 +387,8 @@ const LiveOperations: React.FC = () => {
>
<option value="all">All Status</option>
<option value="running">Running</option>
<option value="waiting">Waiting</option>
<option value="pending">Pending</option>
<option value="sent">Sent</option>
<option value="completed">Completed</option>
<option value="failed">Failed</option>
</select>
@@ -371,6 +428,17 @@ const LiveOperations: React.FC = () => {
{getStatusIcon(operation.status)}
<span className="ml-1">{operation.status}</span>
</span>
{operation.isRetry && operation.retryCount && operation.retryCount > 0 && (
<span className="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-purple-100 text-purple-800 border border-purple-200">
<RotateCcw className="h-3 w-3 mr-1" />
Retry #{operation.retryCount}
</span>
)}
{operation.hasBeenRetried && (
<span className="inline-flex items-center px-2 py-0.5 rounded text-xs font-medium bg-gray-100 text-gray-700 border border-gray-300">
Retried
</span>
)}
</div>
<div className="text-sm text-gray-600 flex items-center space-x-1">
<Computer className="h-4 w-4" />
@@ -382,7 +450,15 @@ const LiveOperations: React.FC = () => {
<div className="flex items-center space-x-2">
<button
onClick={() => setExpandedOperation(expandedOperation === operation.id ? null : operation.id)}
onClick={() => {
const newExpanded = new Set(expandedOperations);
if (newExpanded.has(operation.id)) {
newExpanded.delete(operation.id);
} else {
newExpanded.add(operation.id);
}
setExpandedOperations(newExpanded);
}}
className="flex items-center space-x-1 px-3 py-1 text-sm text-gray-600 hover:text-gray-900 hover:bg-gray-100 rounded-md transition-colors"
>
<Eye className="h-4 w-4" />
@@ -390,7 +466,7 @@ const LiveOperations: React.FC = () => {
<ChevronDown
className={cn(
"h-4 w-4 transition-transform",
expandedOperation === operation.id && "rotate-180"
expandedOperations.has(operation.id) && "rotate-180"
)}
/>
</button>
@@ -403,7 +479,7 @@ const LiveOperations: React.FC = () => {
</div>
{/* Expanded details */}
{expandedOperation === operation.id && (
{expandedOperations.has(operation.id) && (
<div className="p-4 bg-gray-50 border-t border-gray-200">
<div className="grid grid-cols-1 lg:grid-cols-2 gap-4">
<div>
@@ -460,14 +536,21 @@ const LiveOperations: React.FC = () => {
{/* Retry button for failed/timed_out commands */}
{operation.commandStatus === 'failed' || operation.commandStatus === 'timed_out' ? (
<button
onClick={() => handleRetryCommand(operation.commandId)}
disabled={retryMutation.isPending}
className="w-full flex items-center justify-center space-x-2 px-3 py-2 bg-green-100 text-green-700 rounded-md hover:bg-green-200 text-sm font-medium transition-colors disabled:opacity-50 disabled:cursor-not-allowed"
>
<RotateCcw className="h-4 w-4" />
<span>{retryMutation.isPending ? 'Retrying...' : 'Retry Command'}</span>
</button>
operation.hasBeenRetried ? (
<div className="w-full flex items-center justify-center space-x-2 px-3 py-2 bg-purple-50 text-purple-700 rounded-md border border-purple-200 text-sm font-medium">
<RotateCcw className="h-4 w-4" />
<span>Already Retried</span>
</div>
) : (
<button
onClick={() => handleRetryCommand(operation.commandId)}
disabled={retryMutation.isPending}
className="w-full flex items-center justify-center space-x-2 px-3 py-2 bg-green-100 text-green-700 rounded-md hover:bg-green-200 text-sm font-medium transition-colors disabled:opacity-50 disabled:cursor-not-allowed"
>
<RotateCcw className="h-4 w-4" />
<span>{retryMutation.isPending ? 'Retrying...' : 'Retry Command'}</span>
</button>
)
) : null}
</div>
</div>
@@ -501,6 +584,114 @@ const LiveOperations: React.FC = () => {
</div>
)}
</div>
{/* Cleanup Confirmation Dialog */}
{showCleanupDialog && (
<div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center z-50">
<div className="bg-white rounded-lg p-6 max-w-md w-full mx-4">
<h3 className="text-lg font-semibold text-gray-900 mb-4">Archive Failed Operations</h3>
<div className="mb-4 p-3 bg-blue-50 border border-blue-200 rounded-md">
<p className="text-sm text-blue-800">
<strong>INFO:</strong> This will remove failed commands from the active operations view, but all history will be preserved in the database for audit trails and continuity.
</p>
</div>
<div className="mb-4 p-3 bg-yellow-50 border border-yellow-200 rounded-md">
<p className="text-sm text-yellow-800">
<strong>WARNING:</strong> This shouldn't be necessary if the retry logic is working properly - you might want to check what's causing commands to fail in the first place!
</p>
</div>
<div className="space-y-4 mb-6">
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">
Clear operations older than
</label>
<div className="flex items-center space-x-2">
<input
type="number"
min="0"
value={cleanupOptions.olderThanDays}
onChange={(e) => setCleanupOptions(prev => ({
...prev,
olderThanDays: parseInt(e.target.value) || 0
}))}
className="w-20 px-3 py-2 border border-gray-300 rounded-md text-sm"
/>
<span className="text-sm text-gray-600">days</span>
</div>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">
Cleanup scope
</label>
<div className="space-y-2">
<label className="flex items-center">
<input
type="radio"
name="cleanupScope"
checked={!cleanupOptions.onlyRetried && !cleanupOptions.allFailed}
onChange={() => setCleanupOptions(prev => ({
...prev,
onlyRetried: false,
allFailed: false
}))}
className="mr-2"
/>
<span className="text-sm text-gray-700">All failed commands older than specified days</span>
</label>
<label className="flex items-center">
<input
type="radio"
name="cleanupScope"
checked={cleanupOptions.onlyRetried}
onChange={() => setCleanupOptions(prev => ({
...prev,
onlyRetried: true,
allFailed: false
}))}
className="mr-2"
/>
<span className="text-sm text-gray-700">Only failed commands that have been retried</span>
</label>
<label className="flex items-center">
<input
type="radio"
name="cleanupScope"
checked={cleanupOptions.allFailed}
onChange={() => setCleanupOptions(prev => ({
...prev,
onlyRetried: false,
allFailed: true
}))}
className="mr-2"
/>
<span className="text-sm text-red-700 font-medium">All failed commands (most aggressive)</span>
</label>
</div>
</div>
</div>
<div className="flex justify-end space-x-3">
<button
onClick={() => setShowCleanupDialog(false)}
className="px-4 py-2 text-sm font-medium text-gray-700 bg-gray-100 rounded-md hover:bg-gray-200 transition-colors"
>
Cancel
</button>
<button
onClick={handleClearFailedCommands}
disabled={clearFailedMutation.isPending}
className="px-4 py-2 text-sm font-medium text-white bg-red-600 rounded-md hover:bg-red-700 transition-colors disabled:opacity-50 disabled:cursor-not-allowed"
>
{clearFailedMutation.isPending ? 'Archiving...' : 'Archive Failed Commands'}
</button>
</div>
</div>
</div>
)}
</div>
);
};

601
aggregator-web/src/pages/RateLimiting.tsx Normal file
View File

@@ -0,0 +1,601 @@
import React, { useState, useMemo } from 'react';
import {
Shield,
RefreshCw,
Save,
RotateCcw,
Activity,
AlertTriangle,
TrendingUp,
BarChart3,
Settings as SettingsIcon,
Eye,
Users,
Search,
Filter
} from 'lucide-react';
import {
useRateLimitConfigs,
useRateLimitStats,
useRateLimitUsage,
useRateLimitSummary,
useUpdateAllRateLimitConfigs,
useResetRateLimitConfigs,
useCleanupRateLimits
} from '../hooks/useRateLimits';
import { RateLimitConfig, RateLimitStats, RateLimitUsage } from '@/types';
const RateLimiting: React.FC = () => {
const [editingMode, setEditingMode] = useState(false);
const [editingConfigs, setEditingConfigs] = useState<RateLimitConfig[]>([]);
const [showAdvanced, setShowAdvanced] = useState(false);
// Search and filter state
const [searchTerm, setSearchTerm] = useState('');
const [statusFilter, setStatusFilter] = useState<'all' | 'enabled' | 'disabled'>('all');
// Queries
const { data: configs, isLoading: isLoadingConfigs, refetch: refetchConfigs } = useRateLimitConfigs();
const { data: stats, isLoading: isLoadingStats } = useRateLimitStats();
const { data: usage, isLoading: isLoadingUsage } = useRateLimitUsage();
const { data: summary, isLoading: isLoadingSummary } = useRateLimitSummary();
// Mutations
const updateAllConfigs = useUpdateAllRateLimitConfigs();
const resetConfigs = useResetRateLimitConfigs();
const cleanupLimits = useCleanupRateLimits();
React.useEffect(() => {
if (configs) {
setEditingConfigs([...configs]);
}
}, [configs]);
// Filtered configurations for display
const filteredConfigs = useMemo(() => {
if (!configs) return [];
return configs.filter((config) => {
const matchesSearch = searchTerm === '' ||
config.endpoint.toLowerCase().includes(searchTerm.toLowerCase()) ||
config.method.toLowerCase().includes(searchTerm.toLowerCase());
const matchesStatus = statusFilter === 'all' ||
(statusFilter === 'enabled' && config.enabled) ||
(statusFilter === 'disabled' && !config.enabled);
return matchesSearch && matchesStatus;
});
}, [configs, searchTerm, statusFilter]);
const handleConfigChange = (index: number, field: keyof RateLimitConfig, value: any) => {
const updatedConfigs = [...editingConfigs];
updatedConfigs[index] = { ...updatedConfigs[index], [field]: value };
setEditingConfigs(updatedConfigs);
};
const handleSaveAllConfigs = () => {
updateAllConfigs.mutate(editingConfigs, {
onSuccess: () => {
setEditingMode(false);
refetchConfigs();
}
});
};
const handleResetConfigs = () => {
if (confirm('Reset all rate limit configurations to defaults? This will overwrite your custom settings.')) {
resetConfigs.mutate(undefined, {
onSuccess: () => {
setEditingMode(false);
refetchConfigs();
}
});
}
};
const handleCleanup = () => {
if (confirm('Clean up expired rate limit data?')) {
cleanupLimits.mutate(undefined, {
onSuccess: () => {
// Refetch stats and usage after cleanup
}
});
}
};
const getUsagePercentage = (endpoint: string) => {
const endpointUsage = usage?.find(u => u.endpoint === endpoint);
if (!endpointUsage) return 0;
return (endpointUsage.current / endpointUsage.limit) * 100;
};
const getUsageColor = (percentage: number) => {
if (percentage >= 90) return 'text-red-600 bg-red-100';
if (percentage >= 70) return 'text-yellow-600 bg-yellow-100';
return 'text-green-600 bg-green-100';
};
const formatEndpointName = (endpoint: string) => {
return endpoint.split('/').pop()?.replace(/_/g, ' ').replace(/\b\w/g, l => l.toUpperCase()) || endpoint;
};
return (
<div className="max-w-7xl mx-auto px-6 py-8">
{/* Header */}
<div className="mb-8">
<div className="flex items-center justify-between">
<div>
<h1 className="text-3xl font-bold text-gray-900">Rate Limiting</h1>
<p className="mt-2 text-gray-600">Configure API rate limits and monitor system usage</p>
</div>
<div className="flex gap-3">
<button
onClick={() => setShowAdvanced(!showAdvanced)}
className="inline-flex items-center gap-2 px-4 py-2 border border-gray-300 text-gray-700 rounded-lg hover:bg-gray-50"
>
<SettingsIcon className="w-4 h-4" />
{showAdvanced ? 'Simple View' : 'Advanced View'}
</button>
<button
onClick={handleCleanup}
disabled={cleanupLimits.isPending}
className="inline-flex items-center gap-2 px-4 py-2 bg-orange-600 text-white rounded-lg hover:bg-orange-700 disabled:opacity-50"
>
<RefreshCw className={`w-4 h-4 ${cleanupLimits.isPending ? 'animate-spin' : ''}`} />
Cleanup Data
</button>
<button
onClick={() => refetchConfigs()}
className="inline-flex items-center gap-2 px-4 py-2 border border-gray-300 text-gray-700 rounded-lg hover:bg-gray-50"
>
<RefreshCw className="w-4 h-4" />
Refresh
</button>
</div>
</div>
</div>
{/* Summary Cards */}
{summary && (
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-5 gap-4 mb-8">
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Active Endpoints</p>
<p className="text-2xl font-bold text-gray-900">{summary.active_endpoints}</p>
<p className="text-xs text-gray-500">of {summary.total_endpoints} total</p>
</div>
<Shield className="w-8 h-8 text-blue-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Total Requests/Min</p>
<p className="text-2xl font-bold text-gray-900">{summary.total_requests_per_minute}</p>
</div>
<Activity className="w-8 h-8 text-green-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Avg Utilization</p>
<p className="text-2xl font-bold text-blue-600">
{Math.round(summary.average_utilization)}%
</p>
</div>
<BarChart3 className="w-8 h-8 text-purple-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Most Active</p>
<p className="text-lg font-bold text-gray-900 truncate">
{formatEndpointName(summary.most_active_endpoint)}
</p>
</div>
<TrendingUp className="w-8 h-8 text-orange-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Status</p>
<p className="text-lg font-bold text-green-600">Enabled</p>
</div>
<Shield className="w-8 h-8 text-green-600" />
</div>
</div>
</div>
)}
{/* Controls */}
{(editingMode || editingConfigs.length > 0) && (
<div className="bg-blue-50 border border-blue-200 rounded-lg p-4 mb-6">
<div className="flex items-center justify-between">
<p className="text-sm text-blue-800">
You have unsaved changes. Click "Save All Changes" to apply them.
</p>
<div className="flex gap-2">
<button
onClick={handleSaveAllConfigs}
disabled={updateAllConfigs.isPending}
className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50"
>
<Save className="w-4 h-4 inline mr-1" />
{updateAllConfigs.isPending ? 'Saving...' : 'Save All Changes'}
</button>
<button
onClick={() => {
setEditingConfigs([...configs!]);
setEditingMode(false);
}}
className="px-4 py-2 bg-gray-200 text-gray-800 rounded-lg hover:bg-gray-300"
>
Discard Changes
</button>
</div>
</div>
</div>
)}
{/* Rate Limit Configurations */}
<div className="bg-white rounded-lg border border-gray-200 mb-8">
<div className="px-6 py-4 border-b border-gray-200">
<div className="flex items-center justify-between">
<h2 className="text-lg font-semibold text-gray-900">Rate Limit Configurations</h2>
<div className="flex gap-2">
{!editingMode && (
<button
onClick={() => setEditingMode(true)}
className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700"
>
<SettingsIcon className="w-4 h-4 inline mr-1" />
Edit All
</button>
)}
<button
onClick={handleResetConfigs}
disabled={resetConfigs.isPending}
className="px-4 py-2 bg-gray-600 text-white rounded-lg hover:bg-gray-700 disabled:opacity-50"
>
<RotateCcw className="w-4 h-4 inline mr-1" />
Reset to Defaults
</button>
</div>
</div>
</div>
{/* Search and Filter Controls */}
<div className="bg-white border border-gray-200 rounded-lg p-4">
<div className="flex flex-col lg:flex-row gap-4">
<div className="flex-1">
<div className="relative">
<Search className="absolute left-3 top-1/2 transform -translate-y-1/2 text-gray-400 w-4 h-4" />
<input
type="text"
placeholder="Search by endpoint or method..."
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
className="w-full pl-10 pr-4 py-2 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
</div>
</div>
<div className="flex gap-2">
<button
onClick={() => setStatusFilter('all')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'all'
? 'bg-gray-100 text-gray-800 border border-gray-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
All
</button>
<button
onClick={() => setStatusFilter('enabled')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'enabled'
? 'bg-green-100 text-green-800 border border-green-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
Enabled
</button>
<button
onClick={() => setStatusFilter('disabled')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'disabled'
? 'bg-red-100 text-red-800 border border-red-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
Disabled
</button>
</div>
</div>
{/* Filter results summary */}
{configs && (
<div className="mt-3 text-sm text-gray-600">
Showing {filteredConfigs.length} of {configs.length} configurations
</div>
)}
</div>
{filteredConfigs.length > 0 ? (
<div className="overflow-x-auto">
<table className="min-w-full divide-y divide-gray-200">
<thead className="bg-gray-50">
<tr>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Endpoint
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Current Usage
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Requests/Min
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Window (min)
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Max Requests
</th>
{showAdvanced && (
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase">
Burst Allowance
</th>
)}
</tr>
</thead>
<tbody className="bg-white divide-y divide-gray-200">
{filteredConfigs.map((config) => {
const originalIndex = editingConfigs.findIndex(c => c.endpoint === config.endpoint);
const usagePercentage = getUsagePercentage(config.endpoint);
const endpointUsage = usage?.find(u => u.endpoint === config.endpoint);
return (
<tr key={config.endpoint} className="hover:bg-gray-50">
<td className="px-6 py-4 whitespace-nowrap">
<div className="text-sm font-medium text-gray-900">
{formatEndpointName(config.endpoint)}
</div>
<div className="text-xs text-gray-500">
{config.endpoint}
</div>
</td>
<td className="px-6 py-4 whitespace-nowrap">
{endpointUsage && (
<div>
<div className={`inline-flex items-center px-2 py-1 rounded-full text-xs font-medium ${getUsageColor(usagePercentage)}`}>
<div className={`w-2 h-2 rounded-full mr-1 ${
usagePercentage >= 90 ? 'bg-red-500' :
usagePercentage >= 70 ? 'bg-yellow-500' : 'bg-green-500'
}`}></div>
{endpointUsage.current} / {endpointUsage.limit}
({Math.round(usagePercentage)}%)
</div>
<div className="w-full bg-gray-200 rounded-full h-2 mt-2">
<div
className={`h-2 rounded-full transition-all ${
usagePercentage >= 90 ? 'bg-red-500' :
usagePercentage >= 70 ? 'bg-yellow-500' : 'bg-green-500'
}`}
style={{ width: `${Math.min(usagePercentage, 100)}%` }}
></div>
</div>
{endpointUsage && (
<div className="flex items-center gap-2 mt-1">
<Eye className="w-3 h-3 text-gray-400" />
<span className="text-xs text-gray-500">
Window: {formatDateTime(endpointUsage.window_start)} - {formatDateTime(endpointUsage.window_end)}
</span>
</div>
)}
</div>
)}
</td>
<td className="px-6 py-4 whitespace-nowrap">
{editingMode ? (
<input
type="number"
min="1"
value={config.requests_per_minute}
onChange={(e) => handleConfigChange(originalIndex, 'requests_per_minute', parseInt(e.target.value))}
className="w-24 px-3 py-1 border border-gray-300 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
) : (
<span className="text-sm text-gray-900">{config.requests_per_minute}</span>
)}
</td>
<td className="px-6 py-4 whitespace-nowrap">
{editingMode ? (
<input
type="number"
min="1"
value={config.window_minutes}
onChange={(e) => handleConfigChange(originalIndex, 'window_minutes', parseInt(e.target.value))}
className="w-20 px-3 py-1 border border-gray-300 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
) : (
<span className="text-sm text-gray-900">{config.window_minutes}</span>
)}
</td>
<td className="px-6 py-4 whitespace-nowrap">
{editingMode ? (
<input
type="number"
min="1"
value={config.max_requests}
onChange={(e) => handleConfigChange(originalIndex, 'max_requests', parseInt(e.target.value))}
className="w-24 px-3 py-1 border border-gray-300 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
) : (
<span className="text-sm text-gray-900">{config.max_requests}</span>
)}
</td>
{showAdvanced && (
<td className="px-6 py-4 whitespace-nowrap">
{editingMode ? (
<input
type="number"
min="0"
value={config.burst_allowance}
onChange={(e) => handleConfigChange(originalIndex, 'burst_allowance', parseInt(e.target.value))}
className="w-24 px-3 py-1 border border-gray-300 rounded focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
) : (
<span className="text-sm text-gray-900">{config.burst_allowance}</span>
)}
</td>
)}
</tr>
);
})}
</tbody>
</table>
</div>
) : configs && configs.length > 0 ? (
<div className="p-12 text-center">
<Activity className="w-16 h-16 text-gray-400 mx-auto mb-4" />
<h3 className="text-lg font-medium text-gray-900 mb-2">No configurations found</h3>
<p className="text-gray-600">
{searchTerm || statusFilter !== 'all'
? 'Try adjusting your search or filter criteria'
: 'No rate limit configurations available'}
</p>
</div>
) : (
<div className="p-8 text-center">
<div className="inline-block animate-spin rounded-full h-8 w-8 border-b-2 border-blue-600"></div>
<p className="mt-2 text-gray-600">Loading rate limit configurations...</p>
</div>
)}
</div>
{/* Rate Limit Statistics */}
{stats && stats.length > 0 && (
<div className="bg-white rounded-lg border border-gray-200">
<div className="px-6 py-4 border-b border-gray-200">
<h2 className="text-lg font-semibold text-gray-900">Rate Limit Statistics</h2>
</div>
<div className="p-6">
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-3 gap-6">
{stats.map((stat) => (
<div key={stat.endpoint} className="border border-gray-200 rounded-lg p-4">
<div className="flex items-center justify-between mb-3">
<h4 className="font-medium text-gray-900">
{formatEndpointName(stat.endpoint)}
</h4>
<Activity className="w-4 h-4 text-yellow-500" />
</div>
<div className="space-y-2 text-sm">
<div className="flex justify-between">
<span className="text-gray-600">Current Requests:</span>
<span className="font-medium">{stat.current_requests}</span>
</div>
<div className="flex justify-between">
<span className="text-gray-600">Limit:</span>
<span className="font-medium">{stat.limit}</span>
</div>
<div className="flex justify-between">
<span className="text-gray-600">Blocked:</span>
<span className="font-medium text-red-600">{stat.blocked_requests}</span>
</div>
<div className="flex justify-between">
<span className="text-gray-600">Window:</span>
<span className="font-medium text-xs">
{new Date(stat.window_start).toLocaleTimeString()} - {new Date(stat.window_end).toLocaleTimeString()}
</span>
</div>
</div>
{stat.top_clients && stat.top_clients.length > 0 && (
<div className="mt-4 pt-3 border-t border-gray-200">
<p className="text-xs text-gray-600 mb-2">Top Clients:</p>
<div className="space-y-1">
{stat.top_clients.slice(0, 3).map((client, index) => (
<div key={index} className="flex justify-between text-xs">
<span className="text-gray-500 truncate mr-2">{client.identifier}</span>
<span className="font-medium">{client.request_count}</span>
</div>
))}
</div>
</div>
)}
</div>
))}
</div>
</div>
</div>
)}
{/* Usage Monitoring */}
{usage && usage.length > 0 && (
<div className="bg-white rounded-lg border border-gray-200">
<div className="px-6 py-4 border-b border-gray-200">
<h2 className="text-lg font-semibold text-gray-900">Usage Monitoring</h2>
</div>
<div className="p-6">
<div className="grid grid-cols-1 md:grid-cols-2 gap-6">
{usage.map((endpointUsage) => (
<div key={endpointUsage.endpoint} className="border border-gray-200 rounded-lg p-4">
<div className="flex items-center justify-between mb-3">
<h4 className="font-medium text-gray-900">
{formatEndpointName(endpointUsage.endpoint)}
</h4>
<BarChart3 className="w-4 h-4 text-blue-500" />
</div>
<div className="space-y-3">
<div>
<div className="flex justify-between text-sm mb-1">
<span className="text-gray-600">Usage</span>
<span className="font-medium">
{endpointUsage.current} / {endpointUsage.limit}
</span>
</div>
<div className="w-full bg-gray-200 rounded-full h-3">
<div
className={`h-3 rounded-full transition-all ${
(endpointUsage.current / endpointUsage.limit) * 100 >= 90 ? 'bg-red-500' :
(endpointUsage.current / endpointUsage.limit) * 100 >= 70 ? 'bg-yellow-500' : 'bg-green-500'
}`}
style={{ width: `${Math.min((endpointUsage.current / endpointUsage.limit) * 100, 100)}%` }}
></div>
</div>
</div>
<div className="text-xs text-gray-600 space-y-1">
<div>Remaining: {endpointUsage.remaining} requests</div>
<div>Reset: {formatDateTime(endpointUsage.reset_time)}</div>
<div>Window: {endpointUsage.window_minutes} minutes</div>
</div>
</div>
</div>
))}
</div>
</div>
</div>
)}
</div>
);
};
export default RateLimiting;

408
aggregator-web/src/pages/Settings.tsx
View File

@@ -1,17 +1,35 @@
import React from 'react';
import { Clock } from 'lucide-react';
import { Link } from 'react-router-dom';
import {
Clock,
User,
Shield,
Server,
Settings as SettingsIcon,
ArrowRight,
AlertTriangle,
CheckCircle,
Activity
} from 'lucide-react';
import { useSettingsStore } from '@/lib/store';
import { useTimezones, useTimezone, useUpdateTimezone } from '../hooks/useSettings';
import { useRegistrationTokenStats } from '../hooks/useRegistrationTokens';
import { useRateLimitSummary } from '../hooks/useRateLimits';
import { formatDateTime } from '@/lib/utils';
const Settings: React.FC = () => {
const { autoRefresh, refreshInterval, setAutoRefresh, setRefreshInterval } = useSettingsStore();
// Timezone settings
const { data: timezones, isLoading: isLoadingTimezones } = useTimezones();
const { data: currentTimezone, isLoading: isLoadingCurrentTimezone } = useTimezone();
const updateTimezone = useUpdateTimezone();
const [selectedTimezone, setSelectedTimezone] = React.useState('');
// Statistics for overview
const { data: tokenStats } = useRegistrationTokenStats();
const { data: rateLimitSummary } = useRateLimitSummary();
React.useEffect(() => {
if (currentTimezone?.timezone) {
setSelectedTimezone(currentTimezone.timezone);
@@ -21,168 +39,302 @@ const Settings: React.FC = () => {
const handleTimezoneChange = async (e: React.ChangeEvent<HTMLSelectElement>) => {
const newTimezone = e.target.value;
setSelectedTimezone(newTimezone);
try {
await updateTimezone.mutateAsync(newTimezone);
} catch (error) {
console.error('Failed to update timezone:', error);
// Revert on error
if (currentTimezone?.timezone) {
setSelectedTimezone(currentTimezone.timezone);
}
}
};
const overviewCards = [
{
title: 'Registration Tokens',
description: 'Create and manage agent registration tokens',
icon: Shield,
href: '/settings/tokens',
stats: tokenStats ? {
total: tokenStats.total_tokens,
active: tokenStats.active_tokens,
used: tokenStats.used_tokens,
color: 'blue'
} : null,
status: 'implemented'
},
{
title: 'Rate Limiting',
description: 'Configure API rate limits and monitor usage',
icon: Activity,
href: '/settings/rate-limiting',
stats: rateLimitSummary ? {
active: rateLimitSummary.active_endpoints,
total: rateLimitSummary.total_endpoints,
utilization: Math.round(rateLimitSummary.average_utilization),
color: 'green'
} : null,
status: 'implemented'
},
{
title: 'System Configuration',
description: 'Server settings and performance tuning',
icon: Server,
href: '/settings/system',
stats: null,
status: 'not-implemented'
},
{
title: 'Agent Management',
description: 'Agent defaults and cleanup policies',
icon: SettingsIcon,
href: '/settings/agents',
stats: null,
status: 'not-implemented'
}
];
return (
<div className="px-4 sm:px-6 lg:px-8 space-y-6">
<div>
<h1 className="text-2xl font-bold text-gray-900">Settings</h1>
<p className="mt-1 text-sm text-gray-600">Configure your RedFlag dashboard preferences</p>
<div className="max-w-6xl mx-auto px-6 py-8">
{/* Header */}
<div className="mb-8">
<h1 className="text-3xl font-bold text-gray-900">Settings</h1>
<p className="mt-2 text-gray-600">Configure your RedFlag deployment and system preferences</p>
</div>
{/* Timezone Settings */}
<div className="bg-white rounded-lg shadow-sm border border-gray-200 p-6">
<div className="flex items-center gap-3 mb-6">
<div className="p-2 bg-gray-100 rounded-lg">
<Clock className="w-5 h-5 text-gray-600" />
{/* Quick Actions */}
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4 mb-8">
<Link
to="/settings/tokens"
className="block p-6 bg-white border border-gray-200 rounded-lg hover:border-blue-300 hover:shadow-sm transition-all"
>
<div className="flex items-center justify-between mb-4">
<Shield className="w-8 h-8 text-blue-600" />
<ArrowRight className="w-5 h-5 text-gray-400" />
</div>
<div>
<h2 className="text-xl font-semibold text-gray-900">Timezone Settings</h2>
<p className="text-gray-600">Configure the timezone used for displaying timestamps</p>
<h3 className="font-semibold text-gray-900">Registration Tokens</h3>
<p className="text-sm text-gray-600 mt-1">Manage agent registration tokens</p>
</Link>
<Link
to="/settings/rate-limiting"
className="block p-6 bg-white border border-gray-200 rounded-lg hover:border-green-300 hover:shadow-sm transition-all"
>
<div className="flex items-center justify-between mb-4">
<Activity className="w-8 h-8 text-green-600" />
<ArrowRight className="w-5 h-5 text-gray-400" />
</div>
<h3 className="font-semibold text-gray-900">Rate Limiting</h3>
<p className="text-sm text-gray-600 mt-1">Configure API rate limits</p>
</Link>
<div className="p-6 bg-gray-50 border border-gray-200 rounded-lg opacity-60">
<div className="flex items-center justify-between mb-4">
<Server className="w-8 h-8 text-gray-400" />
<ArrowRight className="w-5 h-5 text-gray-300" />
</div>
<h3 className="font-semibold text-gray-500">System Configuration</h3>
<p className="text-sm text-gray-400 mt-1">Coming soon</p>
</div>
<div className="space-y-4">
<div className="p-6 bg-gray-50 border border-gray-200 rounded-lg opacity-60">
<div className="flex items-center justify-between mb-4">
<SettingsIcon className="w-8 h-8 text-gray-400" />
<ArrowRight className="w-5 h-5 text-gray-300" />
</div>
<h3 className="font-semibold text-gray-500">Agent Management</h3>
<p className="text-sm text-gray-400 mt-1">Coming soon</p>
</div>
</div>
{/* Overview Statistics */}
<div className="grid grid-cols-1 md:grid-cols-2 gap-6 mb-8">
{/* Token Overview */}
<div className="bg-white border border-gray-200 rounded-lg p-6">
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold text-gray-900">Token Overview</h2>
<Link
to="/settings/tokens"
className="text-blue-600 hover:text-blue-800 text-sm font-medium"
>
Manage all →
</Link>
</div>
{tokenStats ? (
<div className="grid grid-cols-2 gap-4">
<div>
<p className="text-2xl font-bold text-gray-900">{tokenStats.total_tokens}</p>
<p className="text-sm text-gray-600">Total Tokens</p>
</div>
<div>
<p className="text-2xl font-bold text-green-600">{tokenStats.active_tokens}</p>
<p className="text-sm text-gray-600">Active</p>
</div>
<div>
<p className="text-2xl font-bold text-blue-600">{tokenStats.used_tokens}</p>
<p className="text-sm text-gray-600">Used</p>
</div>
<div>
<p className="text-2xl font-bold text-gray-600">{tokenStats.expired_tokens}</p>
<p className="text-sm text-gray-600">Expired</p>
</div>
</div>
) : (
<div className="text-center py-4">
<div className="animate-spin rounded-full h-6 w-6 border-b-2 border-blue-600 mx-auto"></div>
<p className="text-sm text-gray-500 mt-2">Loading token statistics...</p>
</div>
)}
</div>
{/* Rate Limiting Overview */}
<div className="bg-white border border-gray-200 rounded-lg p-6">
<div className="flex items-center justify-between mb-4">
<h2 className="text-lg font-semibold text-gray-900">Rate Limiting Status</h2>
<Link
to="/settings/rate-limiting"
className="text-blue-600 hover:text-blue-800 text-sm font-medium"
>
Configure →
</Link>
</div>
{rateLimitSummary ? (
<div className="grid grid-cols-2 gap-4">
<div>
<p className="text-2xl font-bold text-gray-900">{rateLimitSummary.active_endpoints}</p>
<p className="text-sm text-gray-600">Active Endpoints</p>
</div>
<div>
<p className="text-2xl font-bold text-green-600">
{rateLimitSummary.total_requests_per_minute}
</p>
<p className="text-sm text-gray-600">Requests/Min</p>
</div>
<div>
<p className="text-2xl font-bold text-blue-600">
{Math.round(rateLimitSummary.average_utilization)}%
</p>
<p className="text-sm text-gray-600">Avg Utilization</p>
</div>
<div>
<div className="flex items-center gap-2">
<CheckCircle className="w-5 h-5 text-green-600" />
<p className="text-lg font-bold text-green-600">Enabled</p>
</div>
<p className="text-sm text-gray-600">System Protected</p>
</div>
</div>
) : (
<div className="text-center py-4">
<div className="animate-spin rounded-full h-6 w-6 border-b-2 border-blue-600 mx-auto"></div>
<p className="text-sm text-gray-500 mt-2">Loading rate limit status...</p>
</div>
)}
</div>
</div>
{/* Account Settings */}
<div className="bg-white border border-gray-200 rounded-lg p-6 mb-8">
<h2 className="text-xl font-semibold text-gray-900 mb-6 pb-2 border-b border-gray-200">Account Settings</h2>
<div className="space-y-8">
{/* Display Preferences */}
<div>
<label htmlFor="timezone" className="block text-sm font-medium text-gray-700 mb-2">
Display Timezone
</label>
<div className="relative">
<h3 className="text-lg font-medium text-gray-900 mb-4">Display Preferences</h3>
<div className="mb-6">
<label className="block text-sm font-medium text-gray-700 mb-2">
Timezone
<span className="ml-1 text-xs text-gray-500">(Note: Changes apply to current session only)</span>
</label>
<select
id="timezone"
value={selectedTimezone}
onChange={handleTimezoneChange}
disabled={isLoadingTimezones || isLoadingCurrentTimezone || updateTimezone.isPending}
className="w-full px-4 py-2 bg-white border border-gray-300 rounded-lg text-gray-900 focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent appearance-none cursor-pointer disabled:opacity-50 disabled:cursor-not-allowed"
disabled={isLoadingTimezones || updateTimezone.isPending}
className="w-full md:w-64 px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500"
>
{isLoadingTimezones ? (
<option>Loading timezones...</option>
<option>Loading...</option>
) : (
timezones?.map((tz) => (
<option key={tz.value} value={tz.value}>
{tz.label}
</option>
<option key={tz.value} value={tz.value}>{tz.label}</option>
))
)}
</select>
{updateTimezone.isPending && (
<p className="mt-2 text-sm text-blue-600">Updating timezone...</p>
)}
{updateTimezone.isSuccess && (
<p className="mt-2 text-sm text-green-600">Timezone updated successfully</p>
)}
{updateTimezone.isError && (
<p className="mt-2 text-sm text-red-600">Failed to update timezone</p>
)}
</div>
</div>
{/* Custom dropdown arrow */}
<div className="absolute inset-y-0 right-0 flex items-center pr-3 pointer-events-none">
<svg className="w-4 h-4 text-gray-400" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path strokeLinecap="round" strokeLinejoin="round" strokeWidth={2} d="M19 9l-7 7-7-7" />
</svg>
{/* Dashboard Behavior */}
<div>
<h3 className="text-lg font-medium text-gray-900 mb-4">Dashboard Behavior</h3>
<div className="space-y-4">
<div className="flex items-center justify-between">
<div>
<div className="font-medium text-gray-900">Auto-refresh</div>
<div className="text-sm text-gray-600">Automatically refresh dashboard data</div>
</div>
<button
onClick={() => setAutoRefresh(!autoRefresh)}
className={`relative inline-flex h-6 w-11 flex-shrink-0 cursor-pointer rounded-full border-2 border-transparent transition-colors ${
autoRefresh ? 'bg-blue-600' : 'bg-gray-200'
}`}
>
<span className={`translate-x-${autoRefresh ? '5' : '0'} inline-block h-5 w-5 transform rounded-full bg-white transition`} />
</button>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">Refresh Interval</label>
<select
value={refreshInterval}
onChange={(e) => setRefreshInterval(Number(e.target.value))}
disabled={!autoRefresh}
className="w-full md:w-64 px-3 py-2 border border-gray-300 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500 disabled:opacity-50"
>
<option value={10000}>10 seconds</option>
<option value={30000}>30 seconds</option>
<option value={60000}>1 minute</option>
<option value={300000}>5 minutes</option>
<option value={600000}>10 minutes</option>
</select>
</div>
</div>
{updateTimezone.isPending && (
<p className="mt-2 text-sm text-yellow-600">Updating timezone...</p>
)}
{updateTimezone.isSuccess && (
<p className="mt-2 text-sm text-green-600">Timezone updated successfully!</p>
)}
{updateTimezone.isError && (
<p className="mt-2 text-sm text-red-600">
Failed to update timezone. Please try again.
</p>
)}
</div>
<div className="pt-4 border-t border-gray-200">
<p className="text-sm text-gray-600">
This setting affects how timestamps are displayed throughout the dashboard, including agent
last check-in times, scan times, and update timestamps.
</p>
</div>
</div>
</div>
{/* Dashboard Settings */}
<div className="bg-white rounded-lg shadow-sm border border-gray-200 p-6">
<div className="flex items-center gap-3 mb-6">
<div className="p-2 bg-gray-100 rounded-lg">
<Clock className="w-5 h-5 text-gray-600" />
{/* Implementation Status */}
<div className="bg-yellow-50 border border-yellow-200 rounded-lg p-6">
<h2 className="text-lg font-semibold text-yellow-800 mb-4">Implementation Status</h2>
<div className="grid grid-cols-1 md:grid-cols-2 gap-6">
<div>
<h3 className="font-medium text-yellow-800 mb-3">✅ Implemented Features</h3>
<ul className="space-y-1 text-sm text-yellow-700">
<li>• Registration token management (full CRUD)</li>
<li>• API rate limiting configuration</li>
<li>• Real-time usage monitoring</li>
<li>• User preferences (timezone, dashboard)</li>
</ul>
</div>
<div>
<h2 className="text-xl font-semibold text-gray-900">Dashboard Settings</h2>
<p className="text-gray-600">Configure how the dashboard behaves and displays information</p>
<h3 className="font-medium text-yellow-800 mb-3">🚧 Planned Features</h3>
<ul className="space-y-1 text-sm text-yellow-700">
<li>• System configuration management</li>
<li>• Agent management settings</li>
<li>• Integration with third-party services</li>
<li>• Persistent settings storage</li>
</ul>
</div>
</div>
<div className="space-y-6">
{/* Auto Refresh */}
<div className="flex items-center justify-between">
<div>
<h3 className="text-sm font-medium text-gray-900">Auto Refresh</h3>
<p className="text-sm text-gray-600">
Automatically refresh dashboard data at regular intervals
</p>
</div>
<button
onClick={() => setAutoRefresh(!autoRefresh)}
className={`relative inline-flex h-6 w-11 flex-shrink-0 cursor-pointer rounded-full border-2 border-transparent transition-colors duration-200 ease-in-out focus:outline-none focus:ring-2 focus:ring-primary-500 focus:ring-offset-2 ${
autoRefresh ? 'bg-primary-600' : 'bg-gray-200'
}`}
>
<span
className={`pointer-events-none inline-block h-5 w-5 transform rounded-full bg-white shadow ring-0 transition duration-200 ease-in-out ${
autoRefresh ? 'translate-x-5' : 'translate-x-0'
}`}
/>
</button>
</div>
{/* Refresh Interval */}
<div>
<h3 className="text-sm font-medium text-gray-900 mb-3">Refresh Interval</h3>
<select
value={refreshInterval}
onChange={(e) => setRefreshInterval(Number(e.target.value))}
disabled={!autoRefresh}
className="w-full px-4 py-2 bg-white border border-gray-300 rounded-lg text-gray-900 focus:outline-none focus:ring-2 focus:ring-primary-500 focus:border-transparent disabled:opacity-50 disabled:cursor-not-allowed"
>
<option value={10000}>10 seconds</option>
<option value={30000}>30 seconds</option>
<option value={60000}>1 minute</option>
<option value={300000}>5 minutes</option>
<option value={600000}>10 minutes</option>
</select>
<p className="mt-1 text-xs text-gray-500">
How often to refresh dashboard data when auto-refresh is enabled
</p>
</div>
</div>
</div>
{/* Future Settings Sections */}
<div className="bg-white rounded-lg shadow-sm border border-gray-200 p-6 opacity-60">
<div className="flex items-center gap-3 mb-4">
<div className="p-2 bg-gray-100 rounded-lg">
<Clock className="w-5 h-5 text-gray-400" />
</div>
<div>
<h2 className="text-xl font-semibold text-gray-400">Additional Settings</h2>
<p className="text-gray-500">More configuration options coming soon</p>
</div>
</div>
<div className="space-y-3 text-sm text-gray-500">
<div>• Notification preferences</div>
<div>• Agent monitoring settings</div>
<div>• Data retention policies</div>
<div>• API access tokens</div>
</div>
<p className="mt-4 text-xs text-yellow-600">
This settings page reflects the current state of the RedFlag backend API.
</p>
</div>
</div>
);

524
aggregator-web/src/pages/TokenManagement.tsx Normal file
View File

@@ -0,0 +1,524 @@
import React, { useState } from 'react';
import {
Shield,
Plus,
Search,
Filter,
RefreshCw,
Download,
Trash2,
Copy,
Eye,
EyeOff,
AlertTriangle,
CheckCircle,
Clock,
Users
} from 'lucide-react';
import {
useRegistrationTokens,
useCreateRegistrationToken,
useRevokeRegistrationToken,
useRegistrationTokenStats,
useCleanupRegistrationTokens
} from '../hooks/useRegistrationTokens';
import { RegistrationToken, CreateRegistrationTokenRequest } from '@/types';
import { formatDateTime } from '@/lib/utils';
const TokenManagement: React.FC = () => {
// Filters and search
const [searchTerm, setSearchTerm] = useState('');
const [statusFilter, setStatusFilter] = useState<'all' | 'active' | 'used' | 'expired' | 'revoked'>('all');
const [showCreateForm, setShowCreateForm] = useState(false);
const [showToken, setShowToken] = useState<Record<string, boolean>>({});
// Pagination
const [currentPage, setCurrentPage] = useState(1);
const pageSize = 50;
// Token management
const { data: tokensData, isLoading, refetch } = useRegistrationTokens({
page: currentPage,
page_size: pageSize,
is_active: statusFilter === 'all' ? undefined : statusFilter === 'active',
label: searchTerm || undefined,
});
const { data: stats, isLoading: isLoadingStats } = useRegistrationTokenStats();
const createToken = useCreateRegistrationToken();
const revokeToken = useRevokeRegistrationToken();
const cleanupTokens = useCleanupRegistrationTokens();
// Reset page when filters change
React.useEffect(() => {
setCurrentPage(1);
}, [searchTerm, statusFilter]);
// Form state
const [formData, setFormData] = useState<CreateRegistrationTokenRequest>({
label: '',
max_seats: 10,
expires_at: '',
});
const handleCreateToken = (e: React.FormEvent) => {
e.preventDefault();
createToken.mutate(formData, {
onSuccess: () => {
setFormData({ label: '', max_seats: 10, expires_at: '' });
setShowCreateForm(false);
refetch();
},
});
};
const handleRevokeToken = (tokenId: string, tokenLabel: string) => {
if (confirm(`Revoke token "${tokenLabel}"? Agents using it will need to re-register.`)) {
revokeToken.mutate(tokenId, { onSuccess: () => refetch() });
}
};
const handleCleanup = () => {
if (confirm('Clean up all expired tokens? This cannot be undone.')) {
cleanupTokens.mutate(undefined, { onSuccess: () => refetch() });
}
};
const copyToClipboard = async (text: string) => {
await navigator.clipboard.writeText(text);
// Show success feedback
};
const copyInstallCommand = async (token: string) => {
const command = `curl -sSL https://get.redflag.dev | bash -s -- ${token}`;
await navigator.clipboard.writeText(command);
};
const generateInstallCommand = (token: string) => {
return `curl -sSL https://get.redflag.dev | bash -s -- ${token}`;
};
const getStatusColor = (token: RegistrationToken) => {
if (!token.is_active) return 'text-gray-500';
if (token.expires_at && new Date(token.expires_at) < new Date()) return 'text-red-600';
if (token.max_seats && token.current_seats >= token.max_seats) return 'text-yellow-600';
return 'text-green-600';
};
const getStatusText = (token: RegistrationToken) => {
if (!token.is_active) return 'Revoked';
if (token.expires_at && new Date(token.expires_at) < new Date()) return 'Expired';
if (token.max_seats && token.current_seats >= token.max_seats) return 'Full';
return 'Active';
};
const filteredTokens = tokensData?.tokens || [];
return (
<div className="max-w-7xl mx-auto px-6 py-8">
{/* Header */}
<div className="mb-8">
<div className="flex items-center justify-between">
<div>
<h1 className="text-3xl font-bold text-gray-900">Registration Tokens</h1>
<p className="mt-2 text-gray-600">Manage agent registration tokens and monitor their usage</p>
</div>
<div className="flex gap-3">
<button
onClick={handleCleanup}
disabled={cleanupTokens.isPending}
className="inline-flex items-center gap-2 px-4 py-2 bg-orange-600 text-white rounded-lg hover:bg-orange-700 disabled:opacity-50"
>
<RefreshCw className={`w-4 h-4 ${cleanupTokens.isPending ? 'animate-spin' : ''}`} />
Cleanup Expired
</button>
<button
onClick={() => refetch()}
className="inline-flex items-center gap-2 px-4 py-2 border border-gray-300 text-gray-700 rounded-lg hover:bg-gray-50"
>
<RefreshCw className="w-4 h-4" />
Refresh
</button>
<button
onClick={() => setShowCreateForm(!showCreateForm)}
className="inline-flex items-center gap-2 px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700"
>
<Plus className="w-4 h-4" />
Create Token
</button>
</div>
</div>
</div>
{/* Statistics Cards */}
{stats && (
<div className="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-5 gap-4 mb-8">
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Total Tokens</p>
<p className="text-2xl font-bold text-gray-900">{stats.total_tokens}</p>
</div>
<Shield className="w-8 h-8 text-blue-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Active</p>
<p className="text-2xl font-bold text-green-600">{stats.active_tokens}</p>
</div>
<CheckCircle className="w-8 h-8 text-green-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Used</p>
<p className="text-2xl font-bold text-blue-600">{stats.used_tokens}</p>
</div>
<Users className="w-8 h-8 text-blue-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Expired</p>
<p className="text-2xl font-bold text-gray-600">{stats.expired_tokens}</p>
</div>
<Clock className="w-8 h-8 text-gray-600" />
</div>
</div>
<div className="bg-white rounded-lg border border-gray-200 p-4">
<div className="flex items-center justify-between">
<div>
<p className="text-sm text-gray-600">Seats Used</p>
<p className="text-2xl font-bold text-purple-600">
{stats.total_seats_used}/{stats.total_seats_available || '∞'}
</p>
</div>
<Users className="w-8 h-8 text-purple-600" />
</div>
</div>
</div>
)}
{/* Create Token Form */}
{showCreateForm && (
<div className="bg-white rounded-lg border border-gray-200 p-6 mb-8">
<h3 className="text-lg font-semibold text-gray-900 mb-4">Create New Registration Token</h3>
<form onSubmit={handleCreateToken} className="space-y-4">
<div className="grid grid-cols-1 md:grid-cols-3 gap-4">
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">Label *</label>
<input
type="text"
required
value={formData.label}
onChange={(e) => setFormData({ ...formData, label: e.target.value })}
placeholder="e.g., Production Team"
className="w-full px-3 py-2 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">Max Seats</label>
<input
type="number"
min="1"
value={formData.max_seats}
onChange={(e) => setFormData({ ...formData, max_seats: e.target.value ? parseInt(e.target.value) : undefined })}
placeholder="Leave empty for unlimited"
className="w-full px-3 py-2 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
</div>
<div>
<label className="block text-sm font-medium text-gray-700 mb-2">Expiration Date</label>
<input
type="datetime-local"
value={formData.expires_at}
onChange={(e) => setFormData({ ...formData, expires_at: e.target.value })}
min={new Date().toISOString().slice(0, 16)}
className="w-full px-3 py-2 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
</div>
</div>
<div className="flex gap-3">
<button
type="submit"
disabled={createToken.isPending}
className="px-4 py-2 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50"
>
{createToken.isPending ? 'Creating...' : 'Create Token'}
</button>
<button
type="button"
onClick={() => setShowCreateForm(false)}
className="px-4 py-2 bg-gray-200 text-gray-800 rounded-lg hover:bg-gray-300"
>
Cancel
</button>
</div>
</form>
</div>
)}
{/* Filters and Search */}
<div className="bg-white rounded-lg border border-gray-200 p-6 mb-8">
<div className="flex flex-col lg:flex-row gap-4">
<div className="flex-1">
<div className="relative">
<Search className="absolute left-3 top-1/2 transform -translate-y-1/2 text-gray-400 w-4 h-4" />
<input
type="text"
placeholder="Search by label..."
value={searchTerm}
onChange={(e) => setSearchTerm(e.target.value)}
className="w-full pl-10 pr-4 py-2 border border-gray-300 rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
/>
</div>
</div>
<div className="flex gap-2">
<button
onClick={() => setStatusFilter('all')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'all'
? 'bg-gray-100 text-gray-800 border border-gray-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
All
</button>
<button
onClick={() => setStatusFilter('active')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'active'
? 'bg-green-100 text-green-800 border border-green-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
Active
</button>
<button
onClick={() => setStatusFilter('used')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'used'
? 'bg-blue-100 text-blue-800 border border-blue-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
Used
</button>
<button
onClick={() => setStatusFilter('expired')}
className={`px-4 py-2 rounded-lg transition-colors ${
statusFilter === 'expired'
? 'bg-red-100 text-red-800 border border-red-300'
: 'bg-white text-gray-600 border border-gray-300 hover:bg-gray-50'
}`}
>
Expired
</button>
</div>
</div>
</div>
{/* Tokens List */}
<div className="bg-white rounded-lg border border-gray-200">
{isLoading ? (
<div className="p-12 text-center">
<div className="inline-block animate-spin rounded-full h-8 w-8 border-b-2 border-blue-600"></div>
<p className="mt-2 text-gray-600">Loading tokens...</p>
</div>
) : filteredTokens.length === 0 ? (
<div className="p-12 text-center">
<Shield className="w-16 h-16 text-gray-400 mx-auto mb-4" />
<h3 className="text-lg font-medium text-gray-900 mb-2">No tokens found</h3>
<p className="text-gray-600">
{searchTerm || statusFilter !== 'all'
? 'Try adjusting your search or filter criteria'
: 'Create your first token to begin registering agents'}
</p>
</div>
) : (
<div className="overflow-x-auto">
<table className="min-w-full divide-y divide-gray-200">
<thead className="bg-gray-50">
<tr>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Token
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Label
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Status
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Seats
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Created
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Expires
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Last Used
</th>
<th className="px-6 py-3 text-left text-xs font-medium text-gray-500 uppercase tracking-wider">
Actions
</th>
</tr>
</thead>
<tbody className="bg-white divide-y divide-gray-200">
{filteredTokens.map((token) => (
<tr key={token.id} className="hover:bg-gray-50">
<td className="px-6 py-4 whitespace-nowrap">
<div className="flex items-center gap-3">
<div className="font-mono text-sm bg-gray-100 px-3 py-2 rounded">
{showToken[token.id] ? token.token : '•••••••••••••••••'}
</div>
<button
onClick={() => setShowToken({ ...showToken, [token.id]: !showToken[token.id] })}
className="text-gray-400 hover:text-gray-600"
>
{showToken[token.id] ? <EyeOff className="w-4 h-4" /> : <Eye className="w-4 h-4" />}
</button>
</div>
</td>
<td className="px-6 py-4 whitespace-nowrap">
<div className="text-sm font-medium text-gray-900">{token.label}</div>
</td>
<td className="px-6 py-4 whitespace-nowrap">
<div className={`flex items-center gap-1 px-3 py-1 rounded-full text-xs font-medium ${getStatusColor(token)}`}>
{getStatusText(token)}
</div>
</td>
<td className="px-6 py-4 whitespace-nowrap">
<div className="text-sm text-gray-900">
{token.current_seats}
{token.max_seats && ` / ${token.max_seats}`}
</div>
{token.max_seats && (
<div className="w-full bg-gray-200 rounded-full h-2 mt-1">
<div
className="bg-blue-600 h-2 rounded-full"
style={{ width: `${Math.min((token.current_seats / token.max_seats) * 100, 100)}%` }}
></div>
</div>
)}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{formatDateTime(token.created_at)}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{formatDateTime(token.expires_at) || 'Never'}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm text-gray-500">
{formatDateTime(token.last_used_at) || 'Never'}
</td>
<td className="px-6 py-4 whitespace-nowrap text-sm font-medium">
<div className="flex items-center gap-2">
<button
onClick={() => copyToClipboard(token.token)}
className="text-blue-600 hover:text-blue-800"
title="Copy token"
>
<Copy className="w-4 h-4" />
</button>
<button
onClick={() => copyInstallCommand(token.token)}
className="text-blue-600 hover:text-blue-800"
title="Copy install command"
>
<Download className="w-4 h-4" />
</button>
{token.is_active && (
<button
onClick={() => handleRevokeToken(token.id, token.label)}
disabled={revokeToken.isPending}
className="text-red-600 hover:text-red-800 disabled:opacity-50"
title="Revoke token"
>
<Trash2 className="w-4 h-4" />
</button>
)}
</div>
</td>
</tr>
))}
</tbody>
</table>
</div>
)}
</div>
{/* Pagination */}
{tokensData && tokensData.total > pageSize && (
<div className="mt-6 flex items-center justify-between">
<div className="text-sm text-gray-700">
Showing {((currentPage - 1) * pageSize) + 1}-{Math.min(currentPage * pageSize, tokensData.total)} of {tokensData.total} tokens
</div>
<div className="flex items-center gap-2">
<button
onClick={() => setCurrentPage(Math.max(1, currentPage - 1))}
disabled={currentPage === 1}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Previous
</button>
<div className="flex items-center gap-1">
{Array.from({ length: Math.min(5, Math.ceil(tokensData.total / pageSize)) }, (_, i) => {
const totalPages = Math.ceil(tokensData.total / pageSize);
let pageNum;
if (totalPages <= 5) {
pageNum = i + 1;
} else if (currentPage <= 3) {
pageNum = i + 1;
} else if (currentPage >= totalPages - 2) {
pageNum = totalPages - 4 + i;
} else {
pageNum = currentPage - 2 + i;
}
return (
<button
key={pageNum}
onClick={() => setCurrentPage(pageNum)}
className={`px-3 py-1 border rounded text-sm ${
currentPage === pageNum
? 'bg-blue-600 text-white border-blue-600'
: 'border-gray-300 hover:bg-gray-50'
}`}
>
{pageNum}
</button>
);
})}
</div>
<button
onClick={() => setCurrentPage(Math.min(Math.ceil(tokensData.total / pageSize), currentPage + 1))}
disabled={currentPage >= Math.ceil(tokensData.total / pageSize)}
className="px-3 py-1 border border-gray-300 rounded text-sm disabled:opacity-50 disabled:cursor-not-allowed hover:bg-gray-50"
>
Next
</button>
</div>
</div>
)}
</div>
);
};
export default TokenManagement;

12
aggregator-web/src/pages/Updates.tsx
View File

@@ -419,14 +419,14 @@ const Updates: React.FC = () => {
<div>
<p className="text-sm text-gray-600">Discovered</p>
<p className="text-sm font-medium text-gray-900">
{formatRelativeTime(selectedUpdate.created_at)}
{formatRelativeTime(selectedUpdate.last_discovered_at)}
</p>
</div>
<div>
<p className="text-sm text-gray-600">Last Updated</p>
<p className="text-sm font-medium text-gray-900">
{formatRelativeTime(selectedUpdate.updated_at)}
{formatRelativeTime(selectedUpdate.last_updated_at)}
</p>
</div>
</div>
@@ -521,7 +521,7 @@ const Updates: React.FC = () => {
onClick={() => {
// This would need a way to find the associated command ID
// For now, we'll show a message indicating this needs to be implemented
toast.info('Retry functionality will be available in the command history view');
toast('Retry functionality will be available in the command history view', { icon: 'ℹ️' });
}}
className="w-full btn btn-warning"
>
@@ -1295,11 +1295,11 @@ const Updates: React.FC = () => {
</th>
<th className="table-header">
<button
onClick={() => handleSort('created_at')}
onClick={() => handleSort('last_discovered_at')}
className="flex items-center hover:text-primary-600 font-medium"
>
Discovered
{renderSortIcon('created_at')}
{renderSortIcon('last_discovered_at')}
</button>
</th>
<th className="table-header">Actions</th>
@@ -1376,7 +1376,7 @@ const Updates: React.FC = () => {
</td>
<td className="table-cell">
<div className="text-sm text-gray-900">
{formatRelativeTime(update.created_at)}
{formatRelativeTime(update.last_discovered_at)}
</div>
</td>
<td className="table-cell">

77
aggregator-web/src/types/index.ts
View File

@@ -47,8 +47,9 @@ export interface UpdatePackage {
available_version: string;
severity: 'low' | 'medium' | 'high' | 'critical';
status: 'pending' | 'approved' | 'scheduled' | 'installing' | 'installed' | 'failed' | 'checking_dependencies' | 'pending_dependencies';
created_at: string;
updated_at: string;
// Timestamp fields - matching backend API response
last_discovered_at: string; // When package was first discovered
last_updated_at: string; // When package status was last updated
approved_at: string | null;
scheduled_at: string | null;
installed_at: string | null;
@@ -285,4 +286,76 @@ export interface ApiError {
message: string;
code?: string;
details?: any;
}
// Registration Token types
export interface RegistrationToken {
id: string;
token: string;
label: string;
expires_at: string | null;
max_seats: number | null;
current_seats: number;
is_active: boolean;
created_at: string;
updated_at: string;
last_used_at: string | null;
metadata: Record<string, any>;
}
export interface CreateRegistrationTokenRequest {
label: string;
expires_at?: string;
max_seats?: number;
metadata?: Record<string, any>;
}
export interface RegistrationTokenStats {
total_tokens: number;
active_tokens: number;
used_tokens: number;
expired_tokens: number;
revoked_tokens: number;
total_seats_used: number;
total_seats_available: number;
}
// Rate Limiting types
export interface RateLimitConfig {
endpoint: string;
requests_per_minute: number;
window_minutes: number;
max_requests: number;
burst_allowance: number;
metadata: Record<string, any>;
}
export interface RateLimitStats {
endpoint: string;
current_requests: number;
limit: number;
window_start: string;
window_end: string;
blocked_requests: number;
top_clients: Array<{
identifier: string;
request_count: number;
}>;
}
export interface RateLimitUsage {
endpoint: string;
limit: number;
current: number;
remaining: number;
reset_time: string;
window_minutes: number;
}
export interface RateLimitSummary {
total_endpoints: number;
active_endpoints: number;
total_requests_per_minute: number;
most_active_endpoint: string;
average_utilization: number;
}