Sentinel Grid Monitoring

Platform Sentinels (8)

Monitor core infrastructure:

• PostgreSQL, Redis, Supabase Auth

• DO App Platform, DO Spaces

• DNS/SSL, API Health, WebSocket

Vendor Sentinels (6)

Monitor integration health:

• Acronis, ConnectWise, Datto

• HaloPSA, NinjaOne, Defender

AI Sentinels (5)

Monitor AI provider availability:

• Claude, OpenAI, Gemini, Perplexity, Grok

Intelligence Sentinels (7)

Monitor the brain's own systems:

• Brain Decisions, Signal Processing, Evidence Chain

• Approval Gates, Learning Loop, Control Graph, OODA Cycle

Fleet Sentinels (4)

Monitor the edge agent fleet:

• Edge Heartbeat, Auto-Update, Tool Dispatch, Telemetry

How Probes Work

Each sentinel runs probes on a configurable interval:

1. **Probe**: Execute a health check (API call, query, ping)

2. **Assess**: Compare result against expected thresholds

3. **Record**: Store probe result in the sentinel buffer

4. **Flush**: Periodically flush buffer to database for history

Probe Intervals

|------|-----------------|

Probe Retention

Probe history is retained for:

• Last 24 hours: Full resolution

• Last 7 days: 5-minute aggregates

• Last 30 days: Hourly aggregates

Automatic Remediation

When a sentinel detects an issue, the self-healing cycle activates:

1. **Detect**: Probe fails or returns degraded status

2. **Diagnose**: Sentinel determines the issue type

3. **Heal**: Execute auto-remediation if available:

- Restart failed services

- Retry failed connections

- Clear stale caches

- Re-establish WebSocket connections

4. **Escalate**: If healing fails after 3 attempts, escalate to the brain

Healing Actions by Tier

• **Platform**: Service restart, connection pool refresh

• **Vendor**: Token refresh, retry with backoff

• **AI**: Provider fallback, model swap

• **Intelligence**: Brain cycle restart, buffer flush

• **Fleet**: Agent reconnect, heartbeat reset

Sentinel Status Colors

• **GREEN**: All probes passing, no issues

• **YELLOW**: Degraded performance or transient errors

• **RED**: Critical failure, service down

Grid Status Page

View the sentinel grid at **Brain > Observatory**:

• Grid layout shows all 30 sentinels organized by tier

• Each sentinel shows current status, last probe time, and trend

• Click any sentinel for detailed probe history

Status Aggregation

The overall system health is computed from sentinel statuses:

• All GREEN = System Healthy

• Any YELLOW = System Degraded

• Any RED = System Alert (notifications sent)

When Sentinels Go Dormant

A sentinel enters dormant mode when:

• The monitored service is not configured (e.g., no Grok API key)

• The integration is disabled by the tenant

• The feature is not available in the current plan

Dormant Behavior

• Dormant sentinels are shown as gray in the grid

• They do not run probes or consume resources

• They automatically activate when the dependency is configured

Activating Dormant Sentinels

1. Configure the missing dependency (API key, integration, etc.)

2. The sentinel detects the configuration change

3. First probe runs within one interval period

4. Sentinel transitions from dormant to active (GREEN/YELLOW/RED)