You've tested your rollback procedure. You've run game days. You've validated that your alerting fires within thirty seconds of a threshold breach. What you probably haven't tested is what happens when the engineer who gets paged doesn't know what they're looking at, can't reach the person who does, and the escalation path in your runbook points to a Slack handle that hasn't been active in four months.
Escalation is the connective tissue of incident response, and it's almost universally undertested.
The Gap Between the Diagram and the 3am Reality
Every incident response process looks clean on a wiki page. There's a primary on-call, a secondary, a team lead, an escalation manager. The chain of custody is clear. What the diagram doesn't capture is that your secondary is traveling internationally this week, your team lead's phone is on Do Not Disturb because they just had a baby, and the escalation manager listed is someone who moved to a different org six months ago and never got removed from the rotation.
This isn't a hypothetical. It's the pattern that shows up in postmortems with uncomfortable regularity — not as the root cause, but as the reason a recoverable incident became a long one. The technical failure happened fast. The organizational failure happened in the thirty minutes of confusion before anyone had clear ownership.
The problem is structural. Teams invest heavily in the detection layer — alerting, dashboards, SLO burn rate calculations — and comparatively little in the human routing layer that determines who acts on what the detection layer finds. You can have world-class observability and still lose forty-five minutes because nobody was sure who had the authority to make a call.
Escalation Paths Rot Faster Than Code
Code at least has a compiler to tell you when something's broken. Escalation paths have nothing. They sit in a runbook or a PagerDuty schedule, updated when someone remembers, which is usually never.
The rot happens in predictable ways. People change teams. Contractors finish engagements. On-call rotations get restructured but the escalation policy doesn't follow. A service gets transferred to a new team and the old team's contacts stay in the runbook because nobody thought to update it during the handoff. Six months later, someone pages the old team at 2am and gets silence.
The deeper issue is that escalation paths are treated as configuration rather than as live operational state. Configuration gets reviewed. Live operational state needs to be continuously validated — which means actually exercising it, not just reading it.
The teams that get this right tend to do a few things differently. They run escalation drills that are specifically designed to fail — they page the secondary deliberately, they simulate the primary being unreachable, they test what happens when the first two rungs of the ladder don't respond. They also treat the escalation path as part of the postmortem review: after every significant incident, one of the questions is whether the right people got looped in at the right time, and if not, why not.
The Ownership Problem Hiding Inside the Escalation Problem
There's a subtler failure mode worth naming. Even when escalation paths are technically correct — the right people get paged, they respond — incidents still stall because nobody has clear authority to make decisions.
This shows up as the "waiting for approval" pattern. An engineer has diagnosed the problem and knows the fix, but the fix involves a database migration or a config change in a system they don't own, and they're not sure if they're allowed to do it without sign-off. So they wait. They page someone. That person pages someone else. Meanwhile the incident clock runs.
The fix here isn't purely technical. It's a combination of pre-authorized playbooks — explicit documentation that says "in this scenario, the on-call engineer is authorized to take these actions without additional approval" — and a culture that treats a wrong decision made quickly as more recoverable than no decision made slowly. Both require organizational buy-in that's harder to build than a monitoring dashboard.
I'd argue this is where most incident response improvement programs stall out. Teams get good at detection. They get decent at diagnosis. They stay mediocre at decision-making under uncertainty because that requires trusting engineers with authority, and authority is uncomfortable to delegate in advance.
What to Actually Test Before You Need It
The practical version of this is straightforward, even if executing it takes discipline. Once a quarter, run a drill that specifically targets your escalation path. Don't announce it. Page the primary. If they respond, great — then simulate them being unavailable and see what happens next. Document every place the chain broke or slowed down.
Review your escalation policies the same way you review your runbooks: with the assumption that they're wrong until proven otherwise. Check that every contact is current. Verify that every person listed knows they're listed and knows what's expected of them.
And write down, explicitly, what your on-call engineers are authorized to do without asking. The absence of that document is itself a reliability risk — one that won't show up in your error budget until the night you need it.
The escalation path you've never tested is the one you'll be debugging in production. Better to find out now.