community-ade/docs/redis-schema.md

Community ADE Approval System - Redis Schema

Overview

This document defines all Redis key patterns used by the Approval System. All keys use the ade: prefix for namespacing.

---

Key Naming Convention

ade:{category}:{subcategory}:{identifier}:{attribute}

Segment	Description	Examples
ade	Global namespace prefix	-
category	High-level component	lock, approval, task, session
subcategory	Specific entity type	task, resource, agent, user
identifier	Unique entity ID	UUID or slug
attribute	Property/attribute	data, queue, index

---

Lock Keys

Primary Lock Storage

Task Lock (Exclusive)

Key: ade:lock:task:{task_id}
Type: Hash
TTL: 30 seconds (renewable)

Fields:
  holder_agent_id    (string)    UUID of agent holding the lock
  acquired_at        (string)    ISO 8601 timestamp
  expires_at         (string)    ISO 8601 timestamp
  purpose            (string)    Human-readable purpose
  heartbeat_count    (integer)   Number of heartbeats received
  queue_length       (integer)   Number of waiters in queue

Example:
  HSET ade:lock:task:550e8400-e29b-41d4-a716-446655440000 \
    holder_agent_id agent-123 \
    acquired_at "2026-03-18T15:30:00Z" \
    expires_at "2026-03-18T15:30:30Z" \
    purpose "Applying database migration"

Resource Lock (Shared/Exclusive)

Key: ade:lock:resource:{resource_type}:{resource_id}
Type: Hash
TTL: 30 seconds (renewable)

Fields:
  mode               (string)    "exclusive" or "shared"
  holders            (JSON)      Array of {agent_id, acquired_at}
  exclusive_holder   (string)    Agent ID (if exclusive mode)
  acquired_at        (string)    ISO 8601 timestamp
  expires_at         (string)    ISO 8601 timestamp

Example:
  HSET ade:lock:resource:database:prod-db-01 \
    mode "exclusive" \
    exclusive_holder agent-456 \
    acquired_at "2026-03-18T15:30:00Z" \
    expires_at "2026-03-18T15:30:30Z"

Agent Capacity Lock

Key: ade:lock:agent:{agent_id}:capacity
Type: Hash
TTL: None (persistent, cleaned up on agent deregistration)

Fields:
  max_tasks          (integer)   Maximum concurrent tasks
  active_tasks       (integer)   Currently executing tasks
  queued_tasks       (integer)   Tasks waiting for capacity
  last_heartbeat     (string)    ISO 8601 timestamp
  status             (string)    "active", "draining", "offline"

Example:
  HSET ade:lock:agent:agent-123:capacity \
    max_tasks 10 \
    active_tasks 3 \
    queued_tasks 1 \
    last_heartbeat "2026-03-18T15:30:00Z" \
    status "active"

Lock Queue Keys

Lock Wait Queue (Ordered list of waiting agents)

Key: ade:lock:task:{task_id}:queue
Type: Sorted Set (ZSET)
TTL: 5 minutes (cleaned up when lock released)

Score: Unix timestamp (millisecond precision for FIFO ordering)
Value: JSON object

Value Format:
  {
    "agent_id": "agent-uuid",
    "mode": "exclusive",
    "priority": 100,
    "requested_at": "2026-03-18T15:30:00Z",
    "max_wait_seconds": 60
  }

Example:
  ZADD ade:lock:task:550e8400-e29b-41d4-a716-446655440000:queue \
    1710775800000 '{"agent_id":"agent-789","mode":"exclusive","priority":100,...}'

Lock Notification Channel

Key: ade:lock:task:{task_id}:channel
Type: Pub/Sub Channel

Events:
  "acquired:{agent_id}"      - Lock acquired
  "released:{agent_id}"      - Lock released
  "expired"                  - Lock expired
  "queued:{agent_id}"        - Agent added to queue
  "promoted:{agent_id}"      - Agent promoted from queue

Lock Index Keys

Active Locks by Agent (Reverse index)

Key: ade:lock:index:agent:{agent_id}
Type: Set
TTL: Matches individual lock TTLs

Members: Lock key references
  ade:lock:task:{task_id}
  ade:lock:resource:{type}:{id}

Purpose: Quick lookup of all locks held by an agent

Active Locks by Resource Type

Key: ade:lock:index:resource:{resource_type}
Type: Set
TTL: Matches individual lock TTLs

Members: Resource lock keys
  ade:lock:resource:database:prod-db-01
  ade:lock:resource:service:api-gateway

Global Lock Registry

Key: ade:lock:registry
Type: Sorted Set
TTL: None

Score: Expiration timestamp
Value: Lock key

Purpose: Background cleanup of expired locks
Example:
  ZADD ade:lock:registry 1710775830 "ade:lock:task:550e8400-..."

Deadlock Detection Keys

Wait-For Graph Edge

Key: ade:lock:waitfor:{agent_id}
Type: Set
TTL: 5 minutes

Members: Lock keys the agent is waiting for
  ade:lock:task:{task_id}
  ade:lock:resource:{type}:{id}

Purpose: Build wait-for graph for deadlock detection

Deadlock Detection Timestamp

Key: ade:lock:deadlock:check:{agent_id}
Type: String
TTL: 30 seconds

Value: ISO 8601 timestamp of last deadlock check

Purpose: Rate limit deadlock detection attempts

---

Approval Keys

Approval Request Keys

Approval Request Data

Key: ade:approval:request:{approval_id}
Type: Hash
TTL: 30 days (archived after completion)

Fields:
  task_id            (string)    UUID of associated task
  reviewer_id        (string)    User ID of assigned reviewer
  reviewer_name      (string)    Display name
  status             (string)    "PENDING", "APPROVED", "REJECTED", "DELEGATED"
  priority           (string)    "LOW", "NORMAL", "HIGH", "URGENT"
  delegated_to       (string)    User ID (if delegated)
  delegation_chain   (JSON)      Array of user IDs in delegation chain
  created_at         (string)    ISO 8601 timestamp
  due_at             (string)    ISO 8601 timestamp
  responded_at       (string)    ISO 8601 timestamp
  response_action    (string)    "approve", "reject", "request_changes"
  response_reason    (string)    Free text explanation
  reviewed_by        (string)    Final responding user ID

Example:
  HSET ade:approval:request:app-123 \
    task_id "task-456" \
    reviewer_id "user-789" \
    status "PENDING" \
    priority "HIGH" \
    created_at "2026-03-18T15:30:00Z" \
    due_at "2026-03-20T15:30:00Z"

Approval Queue Keys

User Approval Queue (Pending approvals for a user)

Key: ade:approval:queue:user:{user_id}
Type: Sorted Set
TTL: None (entries expire based on approval TTL)

Score: Priority score (higher = more urgent)
  Calculated as: (risk_score * 10) + priority_bonus
  priority_bonus: URGENT=1000, HIGH=500, NORMAL=100, LOW=0

Value: approval_id

Example:
  ZADD ade:approval:queue:user:user-789 850 "app-123"
  ZADD ade:approval:queue:user:user-789 450 "app-124"

Task Approval Index (All approvals for a task)

Key: ade:approval:index:task:{task_id}
Type: Set
TTL: Matches approval data TTL

Members: approval_ids
  app-123
  app-124
  app-125

Global Approval Queue (All pending approvals)

Key: ade:approval:queue:global
Type: Sorted Set
TTL: None

Score: Due timestamp (Unix seconds)
Value: approval_id

Purpose: Background worker for escalation/timeout handling

Approval Statistics Keys

User Approval Stats

Key: ade:approval:stats:user:{user_id}
Type: Hash
TTL: None (rolling window)

Fields:
  pending_count      (integer)   Current pending approvals
  approved_today     (integer)   Approvals given today
  rejected_today     (integer)   Rejections given today
  avg_response_time  (float)     Average response time in seconds
  last_action_at     (string)    ISO 8601 timestamp

Note: Daily counters reset at midnight UTC via background job

Task Approval Stats

Key: ade:approval:stats:task:{task_id}
Type: Hash
TTL: 30 days

Fields:
  required_count     (integer)   Required approvals
  approved_count     (integer)   Current approvals
  rejected_count     (integer)   Current rejections
  pending_count      (integer)   Awaiting response
  quorum_reached     (boolean)   Whether minimum approvals met

Delegation Keys

User Delegation Policy

Key: ade:approval:delegation:{user_id}:{policy_id}
Type: Hash
TTL: Based on policy expiration

Fields:
  owner_id           (string)    Policy owner
  delegate_to        (string)    Delegated reviewer
  conditions         (JSON)      Matching conditions
  cascade            (boolean)   Allow further delegation
  active             (boolean)   Policy enabled/disabled
  created_at         (string)    ISO 8601 timestamp
  expires_at         (string)    ISO 8601 timestamp

Example:
  HSET ade:approval:delegation:user-123:policy-456 \
    owner_id "user-123" \
    delegate_to "user-789" \
    conditions '{"task_types":["infrastructure"],"risk_above":50}' \
    cascade "true" \
    active "true"

Delegation Policy Index

Key: ade:approval:delegation:index:{user_id}
Type: Set
TTL: None

Members: policy_ids for the user
  policy-456
  policy-789

---

Task Keys

Task Data Keys

Task State

Key: ade:task:{task_id}:state
Type: String
TTL: 90 days

Value: Current state
  DRAFT, SUBMITTED, REVIEWING, APPROVED, APPLYING, COMPLETED, REJECTED, CANCELLED

Example:
  SET ade:task:task-123:state "REVIEWING"

Task Data (Full object)

Key: ade:task:{task_id}:data
Type: JSON (RedisJSON module) or String (serialized JSON)
TTL: 90 days

Value: Complete task object including config, metadata, execution results

Note: For Redis versions without JSON module, store as serialized string

Task Configuration (Immutable)

Key: ade:task:{task_id}:config
Type: Hash
TTL: 90 days

Fields:
  type               (string)    Task type
  version            (string)    Config version
  description        (string)    Human-readable description
  parameters         (JSON)      Task parameters
  resources          (JSON)      Array of resource references
  rollback_strategy  (string)    "automatic", "manual", "none"
  timeout_seconds    (integer)   Execution timeout
  priority           (integer)   0-100 priority score

Task Metadata

Key: ade:task:{task_id}:metadata
Type: Hash
TTL: 90 days

Fields:
  author_id          (string)    Creating user
  author_name        (string)    Display name
  team               (string)    Team/organization
  ticket_ref         (string)    External ticket reference
  tags               (JSON)      Array of string tags
  created_at         (string)    ISO 8601 timestamp
  updated_at         (string)    ISO 8601 timestamp
  submitted_at       (string)    ISO 8601 timestamp
  approved_at        (string)    ISO 8601 timestamp
  applying_at        (string)    ISO 8601 timestamp
  completed_at       (string)    ISO 8601 timestamp

Task State Index Keys

Tasks by State

Key: ade:task:index:state:{state}
Type: Sorted Set
TTL: None (members removed on state change)

Score: created_at timestamp (Unix seconds)
Value: task_id

Example Keys:
  ade:task:index:state:DRAFT
  ade:task:index:state:REVIEWING
  ade:task:index:state:APPROVED

Tasks by Author

Key: ade:task:index:author:{user_id}
Type: Sorted Set
TTL: 90 days

Score: created_at timestamp
Value: task_id

Tasks by Resource

Key: ade:task:index:resource:{resource_type}:{resource_id}
Type: Sorted Set
TTL: 90 days

Score: created_at timestamp
Value: task_id

Example:
  ade:task:index:resource:database:prod-db-01

Tasks by Tag

Key: ade:task:index:tag:{tag_name}
Type: Sorted Set
TTL: 90 days

Score: created_at timestamp
Value: task_id

Task Execution Keys

Task Execution Status

Key: ade:task:{task_id}:execution
Type: Hash
TTL: 90 days

Fields:
  started_at         (string)    ISO 8601 timestamp
  completed_at       (string)    ISO 8601 timestamp
  agent_id           (string)    Executing agent
  result             (string)    "success", "failure", "timeout", "cancelled"
  output             (string)    Execution output (truncated)
  output_key         (string)    Key to full output in S3/blob storage
  error              (string)    Error message (if failed)
  error_details      (JSON)      Structured error information
  retry_count        (integer)   Number of retry attempts

Task Preview Results

Key: ade:task:{task_id}:preview
Type: JSON/String
TTL: 7 days

Value: Preview result object with changes, warnings, errors

Task Risk Assessment

Key: ade:task:{task_id}:risk
Type: Hash
TTL: 90 days

Fields:
  score              (integer)   0-100 risk score
  level              (string)    "LOW", "MEDIUM", "HIGH", "CRITICAL"
  factors            (JSON)      Array of risk factors
  auto_approvable    (boolean)   Can skip human review
  assessed_at        (string)    ISO 8601 timestamp
  assessed_by        (string)    Algorithm version

---

Session Keys

User Session

Key: ade:session:{session_id}
Type: Hash
TTL: 24 hours

Fields:
  user_id            (string)    Authenticated user
  user_name          (string)    Display name
  roles              (JSON)      Array of role strings
  permissions        (JSON)      Array of permission strings
  created_at         (string)    ISO 8601 timestamp
  last_active        (string)    ISO 8601 timestamp
  ip_address         (string)    Client IP
  user_agent         (string)    Client user agent

User Active Sessions

Key: ade:session:index:user:{user_id}
Type: Set
TTL: 24 hours

Members: session_ids

---

Rate Limiting Keys

API Rate Limit

Key: ade:ratelimit:{endpoint}:{user_id}
Type: String (counter) or Redis Cell (if available)
TTL: 1 minute (sliding window)

Value: Request count

Example:
  ade:ratelimit:tasks:create:user-123
  ade:ratelimit:approvals:respond:user-456

Lock Acquisition Rate Limit (per agent)

Key: ade:ratelimit:lock:acquire:{agent_id}
Type: String (counter)
TTL: 1 minute

Value: Lock acquisition attempts

Purpose: Prevent lock starvation attacks

---

Event Keys

Event Stream (Redis Streams)

Key: ade:events:{event_type}
Type: Stream
TTL: 7 days (MAXLEN ~10000)

Event Types:
  ade:events:task
  ade:events:approval
  ade:events:lock

Entry Fields:
  event              (string)    Event name
  timestamp          (string)    ISO 8601 timestamp
  payload            (JSON)      Event data
  source             (string)    Service/agent that generated event

Example:
  XADD ade:events:task * \
    event "task:state_changed" \
    timestamp "2026-03-18T15:30:00Z" \
    payload '{"task_id":"...","from":"DRAFT","to":"SUBMITTED"}' \
    source "api-server-01"

Event Consumer Groups

Key: ade:events:{event_type}:consumers
Type: Stream Consumer Group

Groups:
  notification-service
  audit-logger
  webhook-dispatcher
  analytics-pipeline

---

Background Job Keys

Job Queue

Key: ade:job:queue:{queue_name}
Type: List or Sorted Set
TTL: None

Queues:
  ade:job:queue:lock_cleanup      - Expired lock cleanup
  ade:job:queue:approval_timeout  - Approval escalation
  ade:job:queue:task_timeout      - Task execution timeout
  ade:job:queue:deadlock_detect   - Deadlock detection
  ade:job:queue:archive           - Old data archival

Scheduled Jobs

Key: ade:job:scheduled
Type: Sorted Set
TTL: None

Score: Execution timestamp (Unix seconds)
Value: JSON job description

Example:
  ZADD ade:job:scheduled 1710776400 \
    '{"type":"lock_cleanup","target":"ade:lock:task:123"}'

Job Locks (prevent duplicate job execution)

Key: ade:job:lock:{job_id}
Type: String
TTL: Job execution timeout

Value: Worker instance ID

---

Configuration Keys

System Configuration

Key: ade:config:{config_name}
Type: String or Hash
TTL: None

Configs:
  ade:config:lock:default_ttl          (integer, seconds)
  ade:config:lock:max_ttl              (integer, seconds)
  ade:config:lock:heartbeat_interval   (integer, seconds)
  ade:config:approval:default_timeout  (integer, seconds)
  ade:config:approval:max_timeout      (integer, seconds)
  ade:config:task:default_timeout      (integer, seconds)
  ade:config:risk:thresholds           (JSON)

Feature Flags

Key: ade:feature:{flag_name}
Type: String
TTL: None

Value: "enabled" or "disabled"

Examples:
  ade:feature:auto_approve_low_risk
  ade:feature:deadlock_detection
  ade:feature:batch_approvals

---

Key Lifecycle Summary

Key Pattern	Type	Default TTL	Cleanup Strategy
ade:lock:* (active)	Hash	30s	Heartbeat extends, expires auto-release
ade:lock:*:queue	ZSET	5m	Cleared on lock release
ade:lock:registry	ZSET	None	Background job cleans expired
ade:approval:request:*	Hash	30d	Archived, then deleted
ade:approval:queue:*	ZSET	None	Entries removed on status change
ade:task:*:state	String	90d	Archived to cold storage
ade:task:*:data	JSON	90d	Archived to cold storage
ade:task:index:*	ZSET	90d	Cleared on task deletion
ade:session:*	Hash	24h	Auto-expire
ade:events:*	Stream	7d	MAXLEN eviction
ade:ratelimit:*	String	1m	Auto-expire

---

Redis Commands Reference

Lock Operations

# Acquire lock (with NX - only if not exists)
HSET ade:lock:task:123 \
  holder_agent_id agent-001 \
  acquired_at "2026-03-18T15:30:00Z" \
  expires_at "2026-03-18T15:30:30Z" \
  NX

# Extend lock TTL
HEXPIRE ade:lock:task:123 30

# Check lock
HGETALL ade:lock:task:123

# Release lock (use Lua for atomic check-and-delete)
# Lua script:
# if redis.call('hget', KEYS[1], 'holder_agent_id') == ARGV[1] then
#   return redis.call('del', KEYS[1])
# end
# return 0

# Add to queue
ZADD ade:lock:task:123:queue 1710775800000 '{"agent_id":"agent-002",...}'

# Get next waiter
ZPOPMIN ade:lock:task:123:queue 1

Approval Operations

# Create approval request
HSET ade:approval:request:app-123 \
  task_id task-456 \
  reviewer_id user-789 \
  status PENDING

# Add to user queue
ZADD ade:approval:queue:user:user-789 850 app-123

# Record response
HSET ade:approval:request:app-123 \
  status APPROVED \
  responded_at "2026-03-18T16:00:00Z" \
  response_action approve

# Remove from queue
ZREM ade:approval:queue:user:user-789 app-123

Task Operations

# Create task
SET ade:task:task-123:state DRAFT
HSET ade:task:task-123:metadata \
  author_id user-001 \
  created_at "2026-03-18T15:00:00Z"

# Update state (atomic)
SET ade:task:task-123:state REVIEWING
ZREM ade:task:index:state:DRAFT task-123
ZADD ade:task:index:state:REVIEWING 1710774000 task-123

# Get task with all data
HMGET ade:task:task-123:metadata author_id created_at
GET ade:task:task-123:state

---

Cluster Mode Considerations

When using Redis Cluster, ensure related keys are on the same hash slot using hash tags:

ade:{task:123}:state        → hash slot for "task:123"
ade:{task:123}:data         → same slot
ade:{task:123}:execution    → same slot
ade:lock:task:{task:123}    → hash slot for "task:123"
ade:approval:index:task:{task:123} → hash slot for "task:123"

This enables multi-key operations (transactions, Lua scripts) on related data.

---

Migration Notes

From v1 to v2

• Renamed lock:* to ade:lock:* for namespacing
• Changed approval status from integers to strings
• Added JSON support for complex fields (requires RedisJSON or serialization)

Backup Strategy

# Daily RDB snapshot
# Real-time AOF for point-in-time recovery
# Cross-region replication for disaster recovery