Redflag/docs/3_BACKLOG/P0-009_Storage-Scanner-Reports-to-Wrong-Table.md

P0-009: Storage Scanner Reports Disk Info to Updates Table Instead of System Info

Priority: P0 (Critical - Data Architecture Issue) Date Identified: 2025-12-17 Date Created: 2025-12-17 Status: Open (Investigation Complete, Fix Needed) Created By: Casey & Claude

Problem Description

The storage scanner (Disk Usage Reporter) is reporting disk/partition information to the updates table instead of populating system_info. This causes:

• Disk information not appearing in Agent Storage UI tab
• Disk data stored in wrong table (treated as updatable packages)
• Hourly delay for disk info to appear (waiting for system info report)
• Inappropriate severity tracking for static system information

Current Behavior

Agent Side:

• Storage scanner runs and collects detailed disk info (mountpoints, devices, types, filesystems)
• Converts to UpdateReportItem format with severity levels
• Reports via /api/v1/agents/:id/updates endpoint
• Stored in update_packages table with package_type='storage'

Server Side:

• Storage metrics endpoint reads from system_info structure (not updates table)
• UI expects agent.system_info.disk_info array
• Disk data is in wrong place, so UI shows empty

Root Cause

In aggregator-agent/internal/orchestrator/storage_scanner.go:

func (s *StorageScanner) Scan() ([]client.UpdateReportItem, error) {
    // Converts StorageMetric to UpdateReportItem
    // Stores disk info in updates table, not system_info
}

The storage scanner implements legacy interface:

• Uses Scan() → UpdateReportItem
• Should use ScanTyped() → TypedScannerResult with StorageData
• System info reporters (system, filesystem) already use proper interface

What I've Investigated

Agent Code:

• ✅ Storage scanner collects comprehensive disk data
• ✅ Data includes: mountpoints, devices, disk_type, filesystem, severity
• ❌ Reports via legacy conversion to updates

Server Code:

• ✅ Has /api/v1/agents/:id/metrics/storage endpoint (reads from system_info)
• ✅ Has proper TypedScannerResult infrastructure
• ❌ Never receives disk data because it's in wrong table

Database Schema:

• update_packages table stores disk info (package_type='storage')
• agent_specs table has metadata JSONB field
• No dedicated system_info table - it's embedded in agent response

UI Code:

• AgentStorage.tsx reads from agent.system_info.disk_info
• Has both overview bars AND detailed table implemented
• Works correctly when data is in right place

What Needs to be Fixed

Option 1: Store in AgentSpecs.metadata (Proper)

1. Modify storage scanner to return TypedScannerResult
2. Call client.ReportSystemInfo() with disk_info populated
3. Update agent_specs.metadata or add system_info column
4. Remove legacy Scan() method from storage scanner

Pros:

• Data in correct semantic location
• No hourly delay for disk info
• Aligns with system/filesystem scanners
• Works immediately with existing UI

Cons:

• Requires database schema change (add system_info column or use metadata)
• Breaking change for existing disk usage report structure
• Need migration for existing storage data

Option 2: Make Metrics READ from Updates (Workaround)

1. Keep storage scanner reporting to updates table
2. Modify GetAgentStorageMetrics() to read from updates
3. Transform update_packages rows into storage metrics format

Pros:

• No agent code changes
• Works with current data flow
• Quick fix

Cons:

• Semantic wrongness (system info in updates)
• Performance issues (querying updates table for system info)
• Still has severity tracking (inappropriate for static info)

Option 3: Dual Write (Temporary Bridge)

1. Storage scanner reports BOTH to system_info AND updates (for backward compatibility)
2. After migration, remove updates reporting

Pros:

• Backward compatible
• Smooth transition

Cons:

• Data duplication
• Temporary hack
• Still need Option 1 eventually

Recommended Fix: Option 1

Implement proper typed scanning for storage:

1. In storage_scanner.go:

   • Remove Scan() legacy method
   • Implement ScanTyped() returning TypedScannerResult
   • Populate TypedScannerResult.StorageData with disks

2. In metrics handler or agent check-in:

   • When storage scanner runs, collect TypedScannerResult
   • Call client.ReportSystemInfo() with report.SystemInfo.DiskInfo populated
   • This updates agent's system_info in real-time

3. In database:

   • Add system_info JSONB column to agent_specs table
   • Or reuse existing metadata field

4. In UI:

   • No changes needed (already reads from system_info)

Files to Modify

Agent:

• /home/casey/Projects/RedFlag/aggregator-agent/internal/orchestrator/storage_scanner.go

Server:

• /home/casey/Projects/RedFlag/aggregator-server/internal/api/handlers/metrics.go
• Database schema (add system_info column)

Migration:

• Create migration to add system_info column
• Optional: migrate existing storage update_reports to system_info

Testing After Fix

1. Install agent with fixed storage scanner
2. Navigate to Agent → Storage tab
3. Should immediately see:
   • Overview disk usage bars
   • Detailed partition table with all disks
   • Device names, types, filesystems, mountpoints
   • Severity indicators
4. No waiting for hourly system info report
5. Data should persist correctly

Related Issues

• P0-007: Install script path variables (fixed)
• P0-008: Migration false positives (fixed)
• P0-009: This issue (storage scanner wrong table)

Notes for Implementer

• Look at how system scanner implements ScanTyped() for reference
• The agent already has reportSystemInfo() method - just need to populate disk_info
• Storage scanner is the ONLY scanner still using legacy Scan() interface
• Remove legacy Scan() method entirely once ScanTyped() is implemented