Fimeg/Redflag
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
484a7f77ceb77964b6d1f2e267b181753758cc1d
Redflag/docs/3_BACKLOG/P5-002_Development-Workflow-Documentation.md

16 KiB
Raw Blame History

P5-002: Development Workflow Documentation

Priority: P5 (Process/Documentation) Source Reference: From analysis of DEVELOPMENT_TODOS.md and codebase development patterns Date Identified: 2025-11-12

Problem Description

RedFlag lacks comprehensive development workflow documentation. Current development processes are undocumented, leading to inconsistent practices, onboarding difficulties, and potential quality issues. New developers lack guidance for contributing effectively.

Impact

  • Onboarding Difficulty: New contributors lack development guidance
  • Inconsistent Processes: Different developers use different approaches
  • Quality Variations: No standardized code review or testing procedures
  • Knowledge Loss: Development practices exist only in team members' heads
  • Collaboration Issues: No shared understanding of development workflows

Current State Analysis

Existing Documentation Gaps

  • No step-by-step development setup guide
  • No code contribution guidelines
  • No pull request process documentation
  • No testing requirements documentation
  • No release process guidelines
  • No debugging and troubleshooting guides

Informal Practices Observed

  • Docker-based development environment
  • Multi-component architecture (server, agent, web)
  • Go backend with React frontend
  • PostgreSQL database with migrations
  • Cross-platform agent builds

Proposed Solution

Create comprehensive development workflow documentation:

1. Development Setup Guide

# RedFlag Development Setup

## Prerequisites
- Docker and Docker Compose
- Go 1.21+ (for local development)
- Node.js 18+ (for frontend development)
- PostgreSQL client tools (optional)

## Quick Start (Docker Environment)
```bash
# Clone repository
git clone https://github.com/redflag/redflag.git
cd redflag

# Start development environment
docker-compose up -d

# Initialize database
docker-compose exec server ./redflag-server migrate

# Access services
# Web UI: http://localhost:3000
# API: http://localhost:8080
# Database: localhost:5432

Local Development Setup

# Install dependencies
make install-deps

# Setup database
make setup-db

# Build components
make build

# Run tests
make test

# Start development servers
make dev

Development Workflow

  1. Create feature branch: git checkout -b feature/your-feature
  2. Make changes: Edit code, add tests
  3. Run tests: make test-all
  4. Lint code: make lint
  5. Commit changes: Follow commit message format
  6. Push and create PR: Submit for code review

### 2. Code Contribution Guidelines
```markdown
# Code Contribution Guidelines

## Coding Standards

### Go Code Style
- Follow standard Go formatting (`gofmt`)
- Use meaningful variable and function names
- Add comments for public functions and complex logic
- Handle errors explicitly
- Use `golint` and `go vet` for static analysis

### TypeScript/React Code Style
- Use Prettier for formatting
- Follow TypeScript best practices
- Use functional components with hooks
- Add JSDoc comments for complex logic
- Use ESLint for static analysis

### File Organization

RedFlag/ ├── aggregator-server/ # Go server │ ├── cmd/ # Main applications │ ├── internal/ # Internal packages │ │ ├── api/ # API handlers │ │ ├── database/ # Database operations │ │ ├── models/ # Data models │ │ └── services/ # Business logic │ └── migrations/ # Database migrations ├── aggregator-agent/ # Go agent │ ├── cmd/ # Agent commands │ ├── internal/ # Internal packages │ │ ├── scanner/ # Update scanners │ │ ├── installer/ # Package installers │ │ └── orchestrator/ # Command orchestration │ └── pkg/ # Public packages ├── aggregator-web/ # React frontend │ ├── src/ │ │ ├── components/ # Reusable components │ │ ├── pages/ # Page components │ │ ├── lib/ # Utilities │ │ └── types/ # TypeScript types │ └── public/ # Static assets └── docs/ # Documentation


## Testing Requirements

### Unit Tests
- All new code must have unit tests
- Test coverage should not decrease
- Use table-driven tests for multiple scenarios
- Mock external dependencies

### Integration Tests
- API endpoints must have integration tests
- Database operations must be tested
- Agent-server communication should be tested
- Use test database for integration tests

### Test Organization
```go
// Example test structure
func TestFunctionName(t *testing.T) {
    tests := []struct {
        name     string
        input    InputType
        expected OutputType
        wantErr  bool
    }{
        {
            name:     "Valid input",
            input:    validInput,
            expected: expectedOutput,
            wantErr:  false,
        },
        {
            name:     "Invalid input",
            input:    invalidInput,
            expected: OutputType{},
            wantErr:  true,
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            result, err := FunctionName(tt.input)
            if tt.wantErr {
                assert.Error(t, err)
                return
            }
            assert.NoError(t, err)
            assert.Equal(t, tt.expected, result)
        })
    }
}

Code Review Process

Before Submitting PR

  1. Self-review: Review your own code changes
  2. Testing: Ensure all tests pass
  3. Documentation: Update relevant documentation
  4. Style: Run linting and formatting tools

PR Requirements

  • Clear description of changes
  • Link to related issues
  • Tests for new functionality
  • Documentation updates
  • Screenshots for UI changes

Review Guidelines

  • Review code logic and design
  • Check for potential security issues
  • Verify test coverage
  • Ensure documentation is accurate
  • Check for performance implications

### 3. Pull Request Process
```markdown
# Pull Request Process

## PR Template
```markdown
## Description
Brief description of changes made

## Type of Change
- [ ] Bug fix
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update

## Testing
- [ ] Unit tests pass
- [ ] Integration tests pass
- [ ] Manual testing completed
- [ ] Cross-platform testing (if applicable)

## Checklist
- [ ] Code follows style guidelines
- [ ] Self-review completed
- [ ] Documentation updated
- [ ] Tests added/updated
- [ ] Database migrations included (if needed)
- [ ] Security considerations addressed

## Related Issues
Fixes #123
Related to #456

Review Process

  1. Automated Checks

    • CI/CD pipeline runs tests
    • Code quality checks
    • Security scans

  2. Peer Review

    • At least one developer approval required
    • Reviewer checks code quality and logic
    • Security review for sensitive changes

  3. Merge Process

    • Address all reviewer comments
    • Ensure CI/CD checks pass
    • Merge with squash or rebase

Release Process

  1. Prepare Release

    • Update version numbers
    • Update CHANGELOG.md
    • Tag release commit

  2. Build and Test

    • Build all components
    • Run full test suite
    • Perform manual testing

  3. Deploy

    • Deploy to staging environment
    • Perform smoke tests
    • Deploy to production

### 4. Debugging and Troubleshooting Guide
```markdown
# Debugging and Troubleshooting Guide

## Common Development Issues

### Database Connection Issues
```bash
# Check database connectivity
docker-compose exec server ping postgres

# Reset database
docker-compose down -v
docker-compose up -d
docker-compose exec server ./redflag-server migrate

Agent Connection Problems

# Check agent logs
sudo journalctl -u redflag-agent -f

# Test agent connectivity
./redflag-agent --server http://localhost:8080 --check

# Verify agent registration
curl -H "Authorization: Bearer $TOKEN" \
  http://localhost:8080/api/v1/agents

Build Issues

# Clean build
make clean
make build

# Check Go version
go version

# Check dependencies
go mod tidy
go mod verify

Debugging Tools

Server Debugging

# Enable debug logging
export LOG_LEVEL=debug

# Run server with debugger
dlv debug ./cmd/server

# Profile server performance
go tool pprof http://localhost:8080/debug/pprof/profile

Agent Debugging

# Run agent in debug mode
./redflag-agent --debug --server http://localhost:8080

# Test specific scanner
./redflag-agent --scan-only dnf --debug

# Check agent configuration
./redflag-agent --config-check

Frontend Debugging

# Start development server
cd aggregator-web
npm run dev

# Run tests with coverage
npm run test:coverage

# Check for linting issues
npm run lint

Performance Debugging

Database Performance

-- Check slow queries
SELECT query, mean_time, calls
FROM pg_stat_statements
ORDER BY mean_time DESC
LIMIT 10;

-- Check database size
SELECT pg_size_pretty(pg_database_size('aggregator'));

-- Check table sizes
SELECT schemaname, tablename,
       pg_size_pretty(pg_total_relation_size(schemaname||'.'||tablename)) as size
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY pg_total_relation_size(schemaname||'.'||tablename) DESC;

Agent Performance

# Monitor agent resource usage
top -p $(pgrep redflag-agent)

# Check agent memory usage
ps aux | grep redflag-agent

# Profile agent performance
go tool pprof http://localhost:8081/debug/pprof/profile

Log Analysis

Server Logs

# View server logs
docker-compose logs -f server

# Filter logs by level
docker-compose logs server | grep ERROR

# Analyze log patterns
docker-compose logs server | grep "rate limit"

Agent Logs

# View agent logs
sudo journalctl -u redflag-agent -f

# Filter by log level
sudo journalctl -u redflag-agent | grep ERROR

# Check specific time period
sudo journalctl -u redflag-agent --since "2025-01-01" --until "2025-01-02"

Environment-Specific Issues

Development Environment

# Reset development environment
make dev-reset

# Check service status
docker-compose ps

# Access development database
docker-compose exec postgres psql -U aggregator -d aggregator

Production Environment

# Check service health
curl -f http://localhost:8080/health || echo "Health check failed"

# Monitor system resources
htop
iostat -x 1

# Check disk space
df -h


### 5. Release and Deployment Guide
```markdown
# Release and Deployment Guide

## Version Management

### Semantic Versioning
- Major version: Breaking changes
- Minor version: New features (backward compatible)
- Patch version: Bug fixes (backward compatible)

### Version Number Format

vX.Y.Z X = Major version Y = Minor version Z = Patch version


### Version Bump Checklist
1. **Update version numbers**
   - `aggregator-server/internal/version/version.go`
   - `aggregator-agent/cmd/agent/version.go`
   - `aggregator-web/package.json`

2. **Update CHANGELOG.md**
   - Add new version section
   - Document all changes
   - Credit contributors

3. **Tag release**
   ```bash
   git tag -a v0.2.0 -m "Release v0.2.0"
   git push origin v0.2.0

Build Process

Automated Builds

# Build all components
make build-all

# Build specific component
make build-server
make build-agent
make build-web

# Build for all platforms
make build-cross-platform

Release Builds

# Create release artifacts
make release

# Verify builds
make verify-release

Deployment Process

Staging Deployment

  1. Prepare staging environment

    # Deploy to staging
make deploy-staging

  2. Run smoke tests

    make test-staging

  3. Manual verification

    • Check web UI functionality
    • Verify API endpoints
    • Test agent registration

Production Deployment

  1. Pre-deployment checklist

    • All tests passing
    • Documentation updated
    • Security review completed
    • Performance tests passed
    • Backup created

  2. Deploy to production

    # Deploy to production
make deploy-production

  3. Post-deployment verification

    # Health checks
make verify-production

Rollback Procedures

Quick Rollback

# Rollback to previous version
make rollback-to v0.1.23

Full Rollback

  1. Stop current deployment
  2. Restore from backup
  3. Deploy previous version
  4. Verify functionality
  5. Communicate rollback

Monitoring After Deployment

Health Checks

# Check service health
curl -f http://localhost:8080/health

# Check database connectivity
docker-compose exec server ./redflag-server health-check

# Monitor agent check-ins
curl -H "Authorization: Bearer $TOKEN" \
  http://localhost:8080/api/v1/agents | jq '. | length'

Performance Monitoring

# Monitor response times
curl -w "@curl-format.txt" http://localhost:8080/api/v1/agents

# Check error rates
docker-compose logs server | grep ERROR | wc -l

Communication

Release Announcement Template

## Release v0.2.0

### New Features
- Feature 1 description
- Feature 2 description

### Bug Fixes
- Bug fix 1 description
- Bug fix 2 description

### Breaking Changes
- Breaking change 1 description

### Upgrade Instructions
1. Backup your installation
2. Follow upgrade guide
3. Verify functionality

### Known Issues
- Any known issues or limitations


## Definition of Done

- [ ] Development setup guide created and tested
- [ ] Code contribution guidelines documented
- [ ] Pull request process defined
- [ ] Debugging guide created
- [ ] Release and deployment guide documented
- [ ] Developer onboarding checklist created
- [ ] Code review checklist developed
- [ ] Makefile targets for all documented processes

## Implementation Details

### Documentation Structure

docs/ ├── development/ │ ├── setup.md │ ├── contributing.md │ ├── pull-request-process.md │ ├── debugging.md │ ├── release-process.md │ └── onboarding.md ├── templates/ │ ├── pull-request-template.md │ ├── release-announcement.md │ └── bug-report.md └── scripts/ ├── setup-dev.sh ├── test-all.sh └── release.sh


### Makefile Targets
```makefile
.PHONY: install-deps setup-db build test lint dev clean release

install-deps:
	# Install development dependencies

setup-db:
	# Setup development database

build:
	# Build all components

test:
	# Run all tests

lint:
	# Run code quality checks

dev:
	# Start development environment

clean:
	# Clean build artifacts

release:
	# Create release artifacts

Prerequisites

  • Development environment standards established
  • CI/CD pipeline in place
  • Code review process defined
  • Documentation templates created
  • Team agreement on processes

Effort Estimate

Complexity: Medium Effort: 1-2 weeks

  • Week 1: Create core development documentation
  • Week 2: Review, test, and refine processes

Success Metrics

  • New developer onboarding time reduced
  • Consistent code quality across contributions
  • Faster PR review and merge process
  • Fewer deployment issues
  • Better team collaboration
  • Improved development productivity

Monitoring

Track these metrics after implementation:

  • Developer onboarding time
  • Code review turnaround time
  • PR merge time
  • Deployment success rate
  • Developer satisfaction surveys
  • Documentation usage analytics
Reference in New Issue View Git Blame Copy Permalink