# On-Call Without Burnout: Rotations, Runbooks, and Escalation

URL: https://www.devopsness.com/blog/on-call-without-burnout-rotations-runbooks
Author: Kiril Urbonas
Published: 2026-07-07
Updated: 2026-11-15
Categories: DevOps
Tags: SRE, On-Call, Reliability, Incident Response, Operations

Summary: When strong engineers quit citing on-call, the fix is rarely a bigger stipend. Saner rotations, runbooks that help at 3am, and escalation that doesn't punish asking for help.

---
# On-Call Without Burnout: Rotations, Runbooks, and Escalation

To run on-call without burnout, delete or auto-remediate every page that needs no human action, then design the rotation: at least eight engineers per single-site rotation, no back-to-back weeks, and a recovery day after night pages. Write runbooks with copy-paste commands, and auto-escalate on no-ack so asking for help is normal. The stipend matters far less.

> [!VERIFIED]
> Prometheus 3.15 alerting-rule docs, kube-state-metrics 2.20, PagerDuty escalation policy docs, against prometheus.io, docs.pagerduty.com and the Google SRE book chapter "Being On-Call". Checked October 2026.

The way on-call usually breaks is quiet. Someone dependable gets paged dozens of times in a week-long shift, mostly at night, mostly for alerts that resolved themselves before they finished logging in. Then the resignation email cites on-call. A system like that punishes people for being reliable enough to trust with the pager.

Fixing it isn't about paying a bigger stipend. It's rotation design, alert hygiene, runbooks, and an escalation culture that treats asking for help as normal.

## Cut the pages before you touch the schedule

No rotation is humane if the pager never stops. Google's SRE book puts a number on "too much": an incident takes about six hours end to end, including root cause and follow-up, so a 12-hour shift should see at most two, and the median should be zero pages per day. Before rescheduling anything, audit a month of alerts and sort them by "did a human do anything useful in response." In noisy rotations a large share of pages fail that test: the alert self-resolved, or the fix was "restart and it comes back." Those aren't incidents, they're noise trained to wake people.

Kill them or auto-remediate them. A pod that OOMs and gets rescheduled by Kubernetes does not need a human at 3am; it needs a ticket in the morning and a memory-limit review, as in [fixing OOMKilled pods](/blog/fix-oomkilled).

```yaml
# prometheus/rules/oncall.yml
# page only if it's still broken after the system tried to self-heal
groups:
  - name: oncall
    rules:
      - alert: PodCrashLoopingPersistent
        expr: |
          rate(kube_pod_container_status_restarts_total[15m]) > 0
          and on (namespace, pod, container)
          kube_pod_container_status_ready == 0
        for: 15m          # give auto-healing a chance first
        labels:
          severity: page
```

The `for: 15m` is deliberate. Prometheus keeps the alert in a pending state until the condition has held for the whole window, so if it heals itself in that window, nobody wakes up. Removing self-resolving pages usually does more for burnout than any schedule tweak. Pairing paging with SLO burn rates, as in [burn-rate alerting](/blog/burn-rate-alerting-slo-discipline), cuts the rest.

## Rotation math that respects sleep

Once the noise is gone, redesign the schedule. The SRE book's guidance is a useful floor: on-call should take no more than 25% of an engineer's time, which means at least eight engineers for a single-site rotation (about one week in four), or at least six per site for a two-site rotation. A few rules we'd hold on top of that:

Follow-the-sun where headcount allows. With engineers in Berlin and Austin, a night page in Europe becomes a daytime page in the Americas. Where a team is single-region and can't do this, split the week so nobody carries a full seven days of nights.

Never schedule back-to-back weeks, and enforce a recovery day after any shift with a nighttime page. Encode it in the scheduler rather than trusting goodwill.

Cap the blast radius: primary handles the page, but if they've been up past 2am on an incident, the policy explicitly says hand off to secondary and sleep. Track "sleep-hours interrupted per shift" as a real metric alongside MTTR, because the human cost stays invisible until you count it.

## Runbooks written for 3am, not for a demo

A runbook you can only follow while fully awake is useless during the incident it exists for. Architecture essays don't help here. Use a strict shape: symptom at the top, exact commands in the middle, escalation at the bottom.

```markdown
## Alert: CheckoutLatencyBurn

### You'll see
p99 on /checkout > 800ms, burn-rate alert firing.

### Check first (copy-paste)
1. Is it the DB pool?
   kubectl exec -n payments deploy/pgbouncer -- \
     psql -c "SHOW POOLS;" | grep cl_waiting
2. Any recent deploy? (last 30m)
   kubectl rollout history deploy/checkout -n payments

### Most likely fix
If a deploy in the last 30m: roll it back.
   kubectl rollout undo deploy/checkout -n payments

### If that doesn't work
Escalate to #payments-oncall secondary. Do not keep debugging alone past 20 min.
```

Copy-pasteable commands with real namespaces. No "investigate the database," which is not an instruction anyone half-asleep can act on. The bottom line matters as much as the top: explicit permission to escalate at 20 minutes.

## Escalation as a feature, not a failure

The cultural rot in a bad rotation is that escalating feels like admitting you weren't good enough. So people grind away alone at 3am on things a specialist could fix in five minutes. Make escalation the expected move, not the last resort.

```yaml
# pagerduty/escalation-policy.yml (illustrative shape)
escalation_policy:
  - level: 1
    target: primary_oncall
    timeout: 10m           # no ack in 10m -> auto-escalate
  - level: 2
    target: secondary_oncall
    timeout: 10m
  - level: 3
    target: eng_manager     # yes, the manager gets paged
```

PagerDuty's default escalation timeout is 30 minutes, which is a long time to be stuck alone at night; we'd shorten it to 10. Two things here change behavior. Auto-escalation on no-ack means a primary who sleeps through a page isn't a catastrophe, because the system routes onward. And putting the manager at level 3 signals that a bad night is an organizational problem, not one engineer's failure. Never treat "how often did you escalate" as a negative metric. Escalating early is good judgment. If you're still picking a tool, [PagerDuty vs Opsgenie](/blog/pagerduty-vs-opsgenie) compares the options.

## Measure the human cost

MTTR alone will happily hide a team burning out. Add metrics that see the people: pages per shift, percentage of pages at night, sleep-hours interrupted, and how many shifts hit the recovery-day rule. When any of those trend the wrong way, it's a planning topic, same as a reliability regression. The SRE book recommends exactly this for overloaded teams: set a measurable goal, such as pages per shift, in quarterly planning.

## Questions people ask

### How many pages per on-call shift is too many?

Google's SRE book sets the bar at no more than two incidents per 12-hour shift, because each one takes about six hours to handle properly, including root cause and follow-up. The median should be zero pages a day. If your rotation regularly exceeds that, the fix is alert hygiene and engineering work on the noisiest services, not a bigger team.

### How many engineers do you need for an on-call rotation?

The SRE book recommends at least eight engineers for a single-site rotation, so each person is on call about one week a month, or at least six per site when two sites share a follow-the-sun rotation. Smaller teams can work with split shifts and strict recovery days, but they burn out faster and should page on fewer things.

### What should an on-call runbook contain?

Four things, in order: what the alert looks like, copy-paste diagnostic commands with real names and namespaces, the most likely fix, and when and whom to escalate. Leave out architecture background; link to it instead. A runbook is good when someone woken at 3am can follow it without thinking.

## The call we'd make

Start with alert hygiene, not the schedule. A humane rotation on top of noisy alerts is still misery, and many of your pages probably shouldn't exist. Then fix runbooks so they work half-asleep, and make escalation the celebrated default instead of the walk of shame. Fewer pages means the incidents that do fire get real attention, and the engineers you rely on most stay.

## Sources

- [Google SRE book: Being On-Call](https://sre.google/sre-book/being-on-call/)
- [Prometheus docs: Alerting rules](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/)
- [PagerDuty docs: Escalation policies](https://docs.pagerduty.com/incident-management/schedules/escalation-policies)

