Blog Details

Incident categories

We classify incidents by impact and failure mode so operators can react fast:

• Quality regressions and hallucinations
• Retrieval outages or stale indexes
• Tool failures and permission errors
• Latency spikes and budget overruns

First-response checklist

1. Freeze deployments and capture traces.
2. Compare against the last passing eval baseline.
3. Switch to a safer routing policy or fallback model.
4. Communicate impact and recovery window.

Post-incident follow-up

Every incident ends with an updated runbook, new eval cases, and tighter alert thresholds.