8.9 KiB
8.9 KiB
Subsystem UI Testing Checklist (v0.1.20)
Phase 1 implementation added granular subsystem controls to AgentScanners and subsystem-specific labels to ChatTimeline. All of this needs testing before we ship.
Prerequisites
- Server running with migration 015 applied
- At least one agent registered (preferably with different subsystems available)
- Fresh browser session (clear cache if needed)
AgentScanners Component Tests
Initial State & Data Loading
- Component loads without errors
- Loading spinner shows while fetching subsystems
- All 4 subsystems appear in table (updates, storage, system, docker)
- Correct icons display for each subsystem:
- Package icon for "updates"
- HardDrive icon for "storage"
- Cpu icon for "system"
- Container icon for "docker"
- Summary stats show correct counts (Enabled: X/4, Auto-Run: Y)
- Refresh button works and triggers re-fetch
Enable/Disable Toggle
- Click ON button → changes to OFF, subsystem disabled
- Click OFF button → changes to ON, subsystem enabled
- Toast notification appears on toggle
- Table updates immediately after toggle
- Auto-Run button becomes disabled when subsystem is OFF
- Interval dropdown becomes "-" when subsystem is OFF
- Scan button becomes disabled and grayed when subsystem is OFF
- Test with all 4 subsystems
Auto-Run Toggle
- Click MANUAL → changes to AUTO
- Click AUTO → changes to MANUAL
- Toast notification appears
- Next Run column populates when AUTO enabled
- Next Run column shows "-" when MANUAL
- Can't toggle when subsystem is disabled (button grayed out)
- Test enabling auto-run on disabled subsystem (should stay grayed)
Interval Dropdown
- Dropdown shows when subsystem enabled
- All 7 options present: 5min, 15min, 30min, 1hr, 4hr, 12hr, 24hr
- Selecting new interval updates immediately
- Toast shows "Interval updated to X minutes"
- Next Run time recalculates if auto-run enabled
- Dropdown disabled/hidden when subsystem disabled
- Test rapid changes (click multiple intervals quickly)
- Test with slow network (ensure no duplicate requests)
Manual Scan Trigger
- Scan button works when subsystem enabled
- Toast shows "X Scanner triggered"
- Button disabled when subsystem disabled
- Last Run updates after scan completes (may take time)
- Can trigger multiple scans in succession
- Test triggering scan while auto-run active
- Verify scan creates command in agent_commands table
Real-time Updates
- Auto-refresh every 30s updates all fields
- Last Run times update correctly
- Next Run times update correctly
- Status changes reflect immediately
- Enabled/disabled state persists across refreshes
- Changes made in one browser tab appear in another (after refresh)
Error Handling
- Network error shows toast notification
- Invalid interval (manually edited in browser) handled gracefully
- 404 on subsystem endpoint shows proper error
- 500 server error shows proper error
- Rate limit exceeded shows proper error
- Offline agent scenario (what should happen?)
Edge Cases
- Agent with no subsystems (newly registered)
- Agent with subsystems but all disabled
- Agent with subsystems all on auto-run
- Subsystem that never ran (Last Run: -, Next Run: -)
- Subsystem with next_run_at in the past (overdue)
- Very long subsystem names (custom subsystems in future)
- Many subsystems (pagination? scrolling?)
ChatTimeline Component Tests
Subsystem Label Display
- scan_updates shows "Package Update Scanner"
- scan_storage shows "Disk Usage Reporter"
- scan_system shows "System Metrics Scanner"
- scan_docker shows "Docker Image Scanner"
- Legacy scan_updates (old format) still works
- Labels show in all status states (initiated/completed/failed)
Subsystem Icons
- scan_updates shows Package icon
- scan_storage shows HardDrive icon
- scan_system shows Cpu icon
- scan_docker shows Container icon
- Icons match AgentScanners component
Timeline Details Parsing
scan_updates (existing - should still work)
- Total updates count parsed
- Available scanners list parsed
- Scanner failures parsed
- Update details extracted correctly
scan_storage (new)
- Mount point extracted
- Disk usage percentage shown
- Total size displayed
- Available space shown
- Multiple disk entries parsed correctly
scan_system (new)
- CPU info extracted
- Memory usage shown
- Process count displayed
- Uptime parsed correctly
- Load average shown (if present)
scan_docker (new)
- Container count shown
- Image count shown
- Updates available count shown
- Running containers count shown
Status Badges & Colors
- SUCCESS badge green for completed scans
- FAILED badge red for failed scans
- RUNNING badge blue + spinner for running scans
- PENDING badge amber for pending scans
- Correct colors for each subsystem scan type
Timeline Filtering & Search
- Search for "Storage" finds storage scans
- Search for "System" finds system scans
- Search for "Docker" finds docker scans
- Filter by status works with new scan types
- Date dividers work correctly
- Pagination works with mixed scan types
Real-time Updates
- New scan entries appear when triggered from AgentScanners
- Status changes reflect (pending → running → completed)
- Duration updates when scan completes
- Auto-refresh (30s) picks up new scans
Error Handling
- Malformed stdout doesn't break timeline
- Missing fields show gracefully (with "-")
- Unknown scan type shows generic icon/label
- Very long stdout truncates properly
- Stderr with subsystem scans displays correctly
Integration Tests (Cross-Component)
Trigger from AgentScanners → See in ChatTimeline
- Trigger storage scan → appears in timeline with correct label
- Trigger system scan → appears in timeline with correct label
- Trigger docker scan → appears in timeline with correct label
- Trigger updates scan → appears in timeline with correct label
- Status progresses correctly (pending → running → completed)
- Duration appears in timeline after completion
API Endpoint Verification
Test these via browser DevTools Network tab:
- GET /api/v1/agents/:id/subsystems → 200 with array
- POST /api/v1/agents/:id/subsystems/:subsystem/enable → 200
- POST /api/v1/agents/:id/subsystems/:subsystem/disable → 200
- POST /api/v1/agents/:id/subsystems/:subsystem/auto-run → 200
- POST /api/v1/agents/:id/subsystems/:subsystem/interval → 200
- POST /api/v1/agents/:id/subsystems/:subsystem/trigger → 200
- GET /api/v1/agents/:id/subsystems/:subsystem/stats → 200
- GET /api/v1/logs (verify subsystem logs appear)
Database Verification
After manual testing, verify in PostgreSQL:
- agent_subsystems table populated for each agent
- enabled/disabled state matches UI
- interval_minutes matches UI dropdown
- auto_run matches UI toggle
- last_run_at updates after scan
- next_run_at calculates correctly
- Trigger creates command in agent_commands with correct type (scan_storage, etc.)
Performance Tests
- Page loads quickly with 10+ subsystems
- Interval changes don't cause UI lag
- Rapid toggling doesn't queue up requests
- Auto-refresh doesn't cause memory leaks (leave open 30+ min)
- Timeline with 100+ mixed scan entries renders smoothly
- Expanding timeline entries with large stdout doesn't freeze
Browser Compatibility
Test in at least:
- Chrome/Chromium (latest)
- Firefox (latest)
- Safari (if available)
- Edge (latest)
- Mobile Chrome (responsive design)
- Mobile Safari (responsive design)
Accessibility
- Keyboard navigation works (tab through controls)
- Screen reader announces status changes
- Color contrast meets WCAG AA (green/red/blue badges)
- Focus indicators visible
- Buttons have proper aria-labels
Regression Tests (Existing Features)
Make sure we didn't break anything:
- Legacy agent commands still work (scan, update, reboot, heartbeat)
- Update approval flow unchanged
- Docker update flow unchanged
- Agent registration unchanged
- Command retry works
- Command cancel works
- Admin token management unchanged
- Rate limiting still works
Known Issues to Document
If any of the above fail, document here instead of blocking release:
- (none yet)
Sign-off
- All critical tests passing
- Known issues documented
- Screenshots captured for docs
- Ready for production testing
Tester: ___________ Date: ___________ Version: v0.1.20 Branch: feature/agent-subsystems-logging