# Blameless Post-Mortems That Drive Change: Template and Facilitation URL: https://www.devopsness.com/blog/incident-post-mortems-that-drive-change Author: Kiril Urbonas Published: 2026-06-13 Updated: 2026-10-31 Categories: DevOps Tags: Post Mortems, Incident Response, SRE, Reliability, Operations Summary: Most post-mortems produce a document and no follow-through. A copy-paste blameless template, a filled-in example, the facilitation rules, and the action-item discipline that turn incidents into engineering improvements. --- # Blameless Post-Mortems That Drive Change: Template and Facilitation A post-mortem drives change when it is one page long, builds its timeline from logs, names several contributing factors instead of a culprit, and ends with three to five action items that each have an owner, a due date and a ticket. Then someone has to chase those items weekly. The facilitation rules matter more than the template. > [!VERIFIED] > Template and process checked against the Google SRE Book chapter on postmortem culture, the PagerDuty Incident Response postmortem process and template pages, and the AWS CloudWatch Logs Insights query syntax docs. Checked October 2026. ## Post-mortems exist to change the system Not to assign blame, which teaches people to hide incidents, but to find changes that prevent or limit the next one. The Google SRE Book defines a blameless post-mortem as one that focuses on contributing causes without indicting any individual or team, and suggests agreeing the triggers in advance: user-visible downtime past a threshold, any data loss, on-call intervention such as a rollback, or an incident found by a human instead of monitoring. Below those triggers, a three-line incident note is enough. ## The blameless postmortem template we use Paste this into your wiki and delete the bracketed guidance as you fill it in. Anything longer than a page doesn't get read. ```markdown # Postmortem: Status: Draft | In review | Final Severity: SEV-1 | SEV-2 Incident window (UTC): to Owner: Incident commander: Scribe: Review meeting: ## Summary [2-3 sentences: what broke, for whom, for how long. Write it last.] ## Impact [Numbers: failed requests, users affected, SLO budget, support tickets.] ## Timeline (UTC, from monitoring/logs, not memory) [One line per event or decision, each linked to a graph, log or chat message.] ## Contributing factors [Several, not one.] ## Detection [Alert, dashboard or customer? Time from start to detection, and why.] ## Resolution [What stopped the impact; what is temporary.] ## What went well ## What slowed us down [Every item maps to an action item or an explicit decision not to act.] ## Where we got lucky [Anything that limited impact by chance rather than design.] ## Action items (3-5 max) | Action | Type | Owner | Due | Ticket | |---|---|---|---|---| | | Prevent / Detect / Mitigate / Process | | | | ## Messaging - Internal summary: - External / status page message: ``` **Contributing factors** is plural because real incidents never have one cause. If **Detection** keeps saying "customer," the fix is usually [burn-rate alerting on your SLOs](/blog/burn-rate-alerting-slo-discipline). The incident commander names the **owner** at the end of the incident call, as in PagerDuty's process; the owner writes and presents but does not fix every item. ## A filled-in example (fictional incident) Invented to show the level of detail; the company, numbers and names are not real. A config change cut the checkout database pool from 50 to 5 connections per instance. ```markdown # Postmortem: Checkout errors after connection pool config change Status: Final Severity: SEV-2 Incident window (UTC): 2026-10-14 13:02 to 13:49 Owner: Priya N. Incident commander: Tom R. Scribe: Ana L. ## Impact - 18,400 failed checkout requests (31% of attempts in the window) - 62% of the monthly checkout error budget consumed - 140 support tickets ## Timeline (UTC) - 13:02 Config PR merged and auto-deployed (link) - 13:09 Burn-rate page fired for checkout availability (link) - 13:15 SEV-2 declared, Tom R. as IC - 13:31 Pool exhaustion identified from app logs: "timeout acquiring connection" (link) - 13:44 Config reverted and redeployed - 13:49 Error rate back under SLO; incident closed 14:20 ## Contributing factors - The pool size lived in a shared config file with no per-service review owner - Staging load was too low to exhaust a pool of 5 - The deploy pipeline had no automatic rollback on SLI regression ## Action items | Action | Type | Owner | Due | Ticket | |---|---|---|---|---| | Add CODEOWNERS for checkout config keys | Prevent | Priya N. | 2026-10-21 | OPS-412 | | Alert on pool wait time p99 over 100 ms | Detect | Ana L. | 2026-10-24 | OPS-413 | | Canary with auto-rollback on checkout SLI | Mitigate | Tom R. | 2026-11-07 | OPS-414 | ``` Notice what is missing: no "engineer X merged a bad change." The merge was normal; the system let a risky change ship without the right reviewer or a canary. ## Timeline from logs, not memory Ban "I think it was around 2ish." Every entry cites a graph, deploy log or Slack timestamp, because memory reorders events and flatters response times. On AWS we paste raw CloudWatch Logs Insights results in before writing prose: ``` # CloudWatch Logs Insights, deploy and alert log groups, incident window filter @message like /deploy|rollback|alert/ | sort @timestamp asc | fields @timestamp, @message ``` ## Facilitation is where blame gets designed out The rules our facilitator enforces: - **Names appear only in the timeline, and even there roles beat names.** "The on-call engineer restarted the pod," not "Dave restarted the pod." - **Ban "should have."** "The engineer should have caught it in review" is blame wearing a lab coat. Reframe to the system: "our review process didn't flag config changes, so a reviewer had no easy way to catch it." - **Ask "what made this the reasonable thing to do at the time?"** If someone ran a risky command, ask why the system made it look safe; often it is a CLI with no confirmation prompt. - **The author is not the responder, and the moderator was not in the incident**, so someone can interrupt blame and refocus on systems. Five-whys works against all of this: it forces one linear chain that stops at "human error" and "be more careful." Probe people, process, tools and environment in parallel, then keep each factor whose absence would have prevented the incident. A config typo outage typically yields three: no schema validation, no canary, no deploy confirmation. ## The review meeting, on a deadline PagerDuty schedules the review within 3 calendar days for a SEV-1 and 5 business days for a SEV-2, shares the draft about 24 hours ahead, and keeps the meeting to 15 to 30 minutes. Book it before writing, so the deadline drives the draft. The agenda: 1. Recap the timeline and confirm everyone agrees with it. 2. Recap the important points and anything unusual. 3. Discuss how the problem could have been caught earlier: canary, tests, load testing. 4. Discuss customer impact. 5. Review the action items and decide whether more are needed. ## Action items: the make-or-break The pattern that fails: 10 items, all "high," owned by "the platform team," none done six months later. What works: - **3-5 action items max**, each **owned by a named person**, not a team, with **a specific due date**, not "Q3." - **Small enough to finish in 2 weeks**: "add a config validation step," not "rewrite the pipeline." - **A ticket in the same tracker as feature work**, written by the person who'll do it. "Improve monitoring" goes nowhere; "add a Prometheus alert for queue depth > 5000" ships. - **Typed as Prevent, Detect, Mitigate or Process**, so a quarter of post-mortems shows whether you only ever add alerts. We review open items in a 15-minute weekly slot: stuck for two weeks gets escalated, pushed three times gets closed as "won't do." The metric that matters is the percentage done by their due date. Publish every post-mortem org-wide, and have a senior engineer write up the quarterly trend. [Game days](/blog/chaos-engineering-game-days-platform-teams) let you rehearse the whole loop without real pressure. ## Questions people ask ### What should a blameless postmortem template include? Severity, owner and incident window, then summary, measured impact, a UTC timeline from logs, contributing factors (plural), detection, resolution, what went well and what slowed you down, where you got lucky, 3-5 typed action items with owner, due date and ticket, and messaging links. Keep it to one page. ### How soon after an incident should a postmortem be done? PagerDuty's process schedules the review meeting within 3 calendar days of a SEV-1 and within 5 business days of a SEV-2, with the draft shared about 24 hours before and a 15 to 30 minute meeting. Book the meeting first, then write. ### How do you keep a postmortem blameless in practice? Use roles instead of names, ban "should have," and ask what made each action reasonable at the time. Have someone outside the incident facilitate with authority to reframe language live, and have someone other than the responder write the doc. ## The decision, concretely - **Timeline written from memory?** Rebuild it from logs before the meeting. - **More than five action items, or any without an owner and date?** Reject them in the meeting. - **Incidents repeating?** Check action item completion first; it's usually the cause. ## The call we'd make Fix the facilitation before you polish the template: a perfect template run by someone who lets "should have" into the room still teaches people to hide. Train one good facilitator, give them authority to reframe language live, and measure action-item completion rather than how many post-mortems you wrote. ## Sources - [Google SRE Book: Postmortem Culture: Learning from Failure](https://sre.google/sre-book/postmortem-culture/) - [PagerDuty Incident Response: Postmortem Process](https://response.pagerduty.com/after/post_mortem_process/) - [PagerDuty Incident Response: Postmortem Template](https://response.pagerduty.com/after/post_mortem_template/) - [AWS docs: CloudWatch Logs Insights query syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html)