9 articles tagged with Incident Response.
A single Availability Zone in AWS's Spain region degraded for roughly 2.5 hours, and most "multi-AZ" architectures would have failed anyway.
A slow database made a backend service look dead, and the health check that was supposed to protect it killed it faster. The lesson has nothing to do with AI.
A background cleanup job saturated the database behind GitHub's permission checks, and the safeguard pacing it watched only replica lag. Throttle on more than one signal.
ChatGPT, Claude, and Grok degraded together when Azure East US failed. Gemini stayed up. Multi-provider failover does not help when your providers share a substrate.
Both page the right person at 3am and both integrate with everything. The real differences show up in pricing structure, workflow depth, and who already owns the ecosystem around you.
Static keys leak. The question isn't if but how fast you notice and how clean your response runbook is when the pager goes off.
Our best engineer quit citing on-call. We rebuilt the whole thing: saner rotations, runbooks that actually help at 3am, and escalation that doesn't punish asking for help.
Most post-mortems produce a document and no follow-through. The blameless template, the facilitation rules, and the action-item discipline that actually convert incidents into engineering improvements.
AI agents for incident triage sound great in demos. We've tried it in production. The patterns that earn their keep, the ones that backfire, and where humans still beat agents.
