Add docs and project files - force for Culurien
This commit is contained in:
Fimeg
368
docs/days/October/README_DETAILED.bak
Normal file
368
docs/days/October/README_DETAILED.bak
Normal file
@@ -0,0 +1,368 @@
|
||||
# 🚩 RedFlag (Aggregator)
|
||||
|
||||
**"From each according to their updates, to each according to their needs"**
|
||||
|
||||
> 🚧 **IN ACTIVE DEVELOPMENT - NOT PRODUCTION READY**
|
||||
> Alpha software - use at your own risk. Breaking changes expected.
|
||||
|
||||
A self-hosted, cross-platform update management platform that provides centralized visibility and control over system updates across your entire infrastructure.
|
||||
|
||||
## What is RedFlag?
|
||||
|
||||
RedFlag is an open-source update management dashboard that gives you a **single pane of glass** for:
|
||||
|
||||
- **Windows Updates** (coming soon)
|
||||
- **Linux packages** (apt, yum/dnf - MVP has apt)
|
||||
- **Winget applications** (coming soon)
|
||||
- **Docker containers** ✅
|
||||
|
||||
Think of it as your own self-hosted RMM (Remote Monitoring & Management) for updates, but:
|
||||
- ✅ **Open source** (AGPLv3)
|
||||
- ✅ **Self-hosted** (your data, your infrastructure)
|
||||
- ✅ **Beautiful** (modern React dashboard)
|
||||
- ✅ **Cross-platform** (Go agents + web interface)
|
||||
|
||||
## Current Status: Session 4 Complete (October 13, 2025)
|
||||
|
||||
⚠️ **ALPHA SOFTWARE - Development in Progress**
|
||||
|
||||
🎉 **✅ What's Working Now:**
|
||||
- ✅ **Server backend** (Go + Gin + PostgreSQL) - Production ready
|
||||
- ✅ **Linux agent** with APT scanner + local CLI features
|
||||
- ✅ **Docker scanner** with real Registry API v2 integration
|
||||
- ✅ **Web dashboard** (React + TypeScript + TailwindCSS) - Full UI
|
||||
- ✅ **Agent registration** and check-in loop
|
||||
- ✅ **Update discovery** and reporting
|
||||
- ✅ **Update approval** workflow (web UI + API)
|
||||
- ✅ **REST API** for all operations
|
||||
- ✅ **Local CLI tools** (--scan, --status, --list-updates, --export)
|
||||
|
||||
🚧 **Current Limitations:**
|
||||
- ❌ No actual update installation yet (just discovery and approval)
|
||||
- ❌ No CVE data enrichment from security advisories
|
||||
- ❌ No Windows agent (planned)
|
||||
- ❌ No rate limiting on API endpoints (security concern)
|
||||
- ❌ Docker deployment not ready (needs networking config)
|
||||
- ❌ No real-time WebSocket updates (polling only)
|
||||
|
||||
🔜 **Next Development Session:**
|
||||
- Real-time updates with WebSocket or polling
|
||||
- Update installation execution (APT packages first)
|
||||
- Rate limiting and security hardening
|
||||
- Docker Compose deployment with proper networking
|
||||
- Windows agent foundation
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Web Dashboard │ ✅ React + TypeScript + TailwindCSS
|
||||
└────────┬────────┘
|
||||
│ HTTPS
|
||||
┌────────▼────────┐
|
||||
│ Server (Go) │ ✅ Production Ready
|
||||
│ + PostgreSQL │
|
||||
└────────┬────────┘
|
||||
│ Pull-based (agents check in every 5 min)
|
||||
┌────┴────┬────────┐
|
||||
│ │ │
|
||||
┌───▼──┐ ┌──▼──┐ ┌──▼───┐
|
||||
│Linux │ │Linux│ │Linux │
|
||||
│Agent │ │Agent│ │Agent │
|
||||
└──────┘ └─────┘ └──────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
⚠️ **BEFORE YOU BEGIN**: Read [SECURITY.md](SECURITY.md) and change your JWT secret!
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Go 1.25+
|
||||
- Docker & Docker Compose
|
||||
- PostgreSQL 16+ (provided via Docker Compose)
|
||||
- Linux system (for agent testing)
|
||||
|
||||
### 1. Start the Database
|
||||
|
||||
```bash
|
||||
make db-up
|
||||
```
|
||||
|
||||
This starts PostgreSQL in Docker.
|
||||
|
||||
### 2. Start the Server
|
||||
|
||||
```bash
|
||||
cd aggregator-server
|
||||
cp .env.example .env
|
||||
# Edit .env if needed (defaults are fine for local development)
|
||||
go run cmd/server/main.go
|
||||
```
|
||||
|
||||
The server will:
|
||||
- Connect to PostgreSQL
|
||||
- Run database migrations automatically
|
||||
- Start listening on `:8080`
|
||||
|
||||
You should see:
|
||||
```
|
||||
✓ Executed migration: 001_initial_schema.up.sql
|
||||
🚩 RedFlag Aggregator Server starting on :8080
|
||||
```
|
||||
|
||||
### 3. Register an Agent
|
||||
|
||||
On the machine you want to monitor:
|
||||
|
||||
```bash
|
||||
cd aggregator-agent
|
||||
go build -o aggregator-agent cmd/agent/main.go
|
||||
|
||||
# Register with server
|
||||
sudo ./aggregator-agent -register -server http://YOUR_SERVER:8080
|
||||
```
|
||||
|
||||
You should see:
|
||||
```
|
||||
✓ Agent registered successfully!
|
||||
Agent ID: 550e8400-e29b-41d4-a716-446655440000
|
||||
```
|
||||
|
||||
### 4. Run the Agent
|
||||
|
||||
```bash
|
||||
sudo ./aggregator-agent
|
||||
```
|
||||
|
||||
The agent will:
|
||||
- Check in with the server every 5 minutes
|
||||
- Scan for APT updates
|
||||
- Scan for Docker image updates
|
||||
- Report findings to the server
|
||||
|
||||
### 5. Access the Web Dashboard
|
||||
|
||||
```bash
|
||||
cd aggregator-web
|
||||
yarn install
|
||||
yarn dev
|
||||
```
|
||||
|
||||
Visit http://localhost:3000 and login with your JWT token.
|
||||
|
||||
## API Usage
|
||||
|
||||
### List All Agents
|
||||
|
||||
```bash
|
||||
curl http://localhost:8080/api/v1/agents
|
||||
```
|
||||
|
||||
### Trigger Update Scan
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/api/v1/agents/{agent-id}/scan
|
||||
```
|
||||
|
||||
### List All Updates
|
||||
|
||||
```bash
|
||||
# All updates
|
||||
curl http://localhost:8080/api/v1/updates
|
||||
|
||||
# Filter by severity
|
||||
curl http://localhost:8080/api/v1/updates?severity=critical
|
||||
|
||||
# Filter by status
|
||||
curl http://localhost:8080/api/v1/updates?status=pending
|
||||
|
||||
# Filter by package type
|
||||
curl http://localhost:8080/api/v1/updates?package_type=apt
|
||||
```
|
||||
|
||||
### Approve an Update
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/api/v1/updates/{update-id}/approve
|
||||
```
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
RedFlag/
|
||||
├── aggregator-server/ # Go server (Gin + PostgreSQL)
|
||||
│ ├── cmd/server/ # Main entry point
|
||||
│ ├── internal/
|
||||
│ │ ├── api/ # HTTP handlers & middleware
|
||||
│ │ ├── database/ # Database layer & migrations
|
||||
│ │ ├── models/ # Data models
|
||||
│ │ └── config/ # Configuration
|
||||
│ └── go.mod
|
||||
|
||||
├── aggregator-agent/ # Go agent
|
||||
│ ├── cmd/agent/ # Main entry point
|
||||
│ ├── internal/
|
||||
│ │ ├── client/ # API client
|
||||
│ │ ├── scanner/ # Update scanners (APT, Docker)
|
||||
│ │ └── config/ # Configuration
|
||||
│ └── go.mod
|
||||
|
||||
├── aggregator-web/ # React dashboard ✅
|
||||
├── docker-compose.yml # PostgreSQL for local dev
|
||||
├── Makefile # Common tasks
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
**Key Tables:**
|
||||
- `agents` - Registered agents
|
||||
- `update_packages` - Discovered updates
|
||||
- `agent_commands` - Command queue for agents
|
||||
- `update_logs` - Execution logs
|
||||
- `agent_tags` - Agent tagging/grouping
|
||||
|
||||
See `aggregator-server/internal/database/migrations/001_initial_schema.up.sql` for full schema.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Server (.env)
|
||||
|
||||
```bash
|
||||
SERVER_PORT=8080
|
||||
DATABASE_URL=postgres://aggregator:aggregator@localhost:5432/aggregator?sslmode=disable
|
||||
JWT_SECRET=change-me-in-production
|
||||
CHECK_IN_INTERVAL=300 # seconds
|
||||
OFFLINE_THRESHOLD=600 # seconds
|
||||
```
|
||||
|
||||
### Agent (/etc/aggregator/config.json)
|
||||
|
||||
Auto-generated on registration:
|
||||
|
||||
```json
|
||||
{
|
||||
"server_url": "http://localhost:8080",
|
||||
"agent_id": "uuid",
|
||||
"token": "jwt-token",
|
||||
"check_in_interval": 300
|
||||
}
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Makefile Commands
|
||||
|
||||
```bash
|
||||
make help # Show all commands
|
||||
make db-up # Start PostgreSQL
|
||||
make db-down # Stop PostgreSQL
|
||||
make server # Run server (with auto-reload)
|
||||
make agent # Run agent
|
||||
make build-server # Build server binary
|
||||
make build-agent # Build agent binary
|
||||
make test # Run tests
|
||||
make clean # Clean build artifacts
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
cd aggregator-server && go test ./...
|
||||
cd aggregator-agent && go test ./...
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- **Agent Authentication**: JWT tokens with 24h expiry
|
||||
- **Pull-based Model**: Agents poll server (firewall-friendly)
|
||||
- **Command Validation**: Whitelisted commands only
|
||||
- **TLS Required**: Production deployments must use HTTPS
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1: MVP (✅ Current)
|
||||
- [x] Server backend with PostgreSQL
|
||||
- [x] Agent registration & check-in
|
||||
- [x] Linux APT scanner
|
||||
- [x] Docker scanner
|
||||
- [x] Update approval workflow
|
||||
|
||||
### Phase 2: Feature Complete (Next)
|
||||
- [x] Web dashboard ✅ (React + TypeScript + TailwindCSS)
|
||||
- [ ] Windows agent (Windows Update + Winget)
|
||||
- [ ] Update installation execution
|
||||
- [ ] Maintenance windows
|
||||
- [ ] YUM/DNF scanner
|
||||
- [ ] Rollback capability
|
||||
- [ ] Real-time updates (WebSocket or polling)
|
||||
- [ ] Docker deployment with proper networking
|
||||
|
||||
### Phase 3: AI Integration
|
||||
- [ ] Natural language queries
|
||||
- [ ] Intelligent scheduling
|
||||
- [ ] Failure analysis
|
||||
- [ ] AI chat sidebar in UI
|
||||
|
||||
### Phase 4: Enterprise Features
|
||||
- [ ] Multi-tenancy
|
||||
- [ ] RBAC
|
||||
- [ ] SSO integration
|
||||
- [ ] Compliance reporting
|
||||
- [ ] Prometheus metrics
|
||||
|
||||
## Contributing
|
||||
|
||||
We welcome contributions! Areas that need help:
|
||||
|
||||
- **Windows agent** - Windows Update API integration
|
||||
- **Package managers** - snap, flatpak, chocolatey, brew
|
||||
- **Web dashboard** - React frontend
|
||||
- **Documentation** - Installation guides, troubleshooting
|
||||
- **Testing** - Unit tests, integration tests
|
||||
|
||||
## License
|
||||
|
||||
**AGPLv3** - This ensures:
|
||||
- Modifications must stay open source
|
||||
- No proprietary SaaS forks without contribution
|
||||
- Commercial use allowed with attribution
|
||||
- Forces cloud providers to contribute back
|
||||
|
||||
For commercial licensing options (if AGPL doesn't work for you), contact the project maintainers.
|
||||
|
||||
## Why "RedFlag"?
|
||||
|
||||
The project embraces a tongue-in-cheek communist theming:
|
||||
- **Updates are the "means of production"** (they produce secure systems)
|
||||
- **Commercial RMMs are "capitalist tools"** (expensive, SaaS-only)
|
||||
- **RedFlag "seizes" control** back to the user (self-hosted, free)
|
||||
|
||||
But ultimately, it's a serious tool with a playful brand. The core mission is providing enterprise-grade update management to everyone, not just those who can afford expensive RMMs.
|
||||
|
||||
## Documentation
|
||||
|
||||
- 🏠 **Website**: Open `docs/index.html` in your browser for a fun intro!
|
||||
- 📖 **Getting Started**: `docs/getting-started.html` - Complete setup guide
|
||||
- 🔐 **Security Guide**: `SECURITY.md` - READ THIS BEFORE DEPLOYING
|
||||
- 💬 **Discussions**: GitHub Discussions
|
||||
- 🐛 **Bug Reports**: GitHub Issues
|
||||
- 🚀 **Feature Requests**: GitHub Issues
|
||||
|
||||
## Acknowledgments
|
||||
|
||||
Built with:
|
||||
- **Go** - Server & agent
|
||||
- **Gin** - HTTP framework
|
||||
- **PostgreSQL** - Database
|
||||
- **Docker** - For development & deployment
|
||||
- **React** (completed) - Web dashboard
|
||||
|
||||
Inspired by: ConnectWise Automate, Grafana, Wazuh, and the self-hosting community.
|
||||
|
||||
---
|
||||
|
||||
**Built with ❤️ for the self-hosting community**
|
||||
|
||||
🚩 **Seize the means of production!**
|
||||
Reference in New Issue
Block a user