A history of the 5 whys and root cause analysis from papers

Some critiques of the 5 whys:

From John Allspaw: https://www.oreilly.com/radar/the-infinite-hows/

From Alan J Card: https://qualitysafety.bmj.com/content/26/8/671

James Reason and the Swiss Cheese Model: 

https://pmc.ncbi.nlm.nih.gov/articles/PMC8514562/

James Reason’s book Human Error: https://bookshop.org/p/books/human-error/9e06d8a100a07537?ean=9780521314190&next=t

And a classic from Sidney Dekker (et al.) on the implication of complexity within safety investigations:

https://www.sciencedirect.com/science/article/abs/pii/S0925753511000105?via%3Dihub

We always recommend the Howie Guide: https://howie-guide.pagerduty.com/

STAMP is starting to get popular: https://functionalsafetyengineer.com/introduction-to-stamp/

Google’s STAMP paper: https://www.usenix.org/publications/loginonline/evolution-sre-google

Google’s STAMP discussion on ProdCast: https://sre.google/prodcast/#season4-episode7

And presentation at SRECon: https://www.usenix.org/conference/srecon25americas/presentation/klein

Nancy Leveson’s google scholar is always worth browsing: https://scholar.google.com/citations?user=78y4sEcAAAAJ&hl=en

Allspaw’s LinkedIn post that we quoted: https://www.linkedin.com/posts/jallspaw_important-reminders-about-learning-effectively-activity-7378775591447183360-c_eD

Lorin’s Law: https://surfingcomplexity.blog/2017/06/24/a-conjecture-on-why-reliable-systems-fail/

Want to talk more about this subject? We’re doing a live event co-sponsored by RISF and you can sign up for it here: https://resilienceinsoftware.org/networks/events/146485

More robustness than resilience, but worth repeating that you should always check your earthquake go-bag: https://www.earthquakeauthority.com/blog/2019/how-to-make-an-earthquake-emergency-kit

Clint did ASA 103: https://americansailing.com/learn-to-sail/certifications/asa-103-coastal-cruising/

Since this is a science podcast, there is a scientific reason people get emotional on airplanes: https://www.cntraveler.com/story/why-do-we-always-cry-on-planes

52 Hertz Whale documentary: https://en.wikipedia.org/wiki/The_Loneliest_Whale:_The_Search_for_52

And Leslie Jamison wrote 52 Blue as a chapter in one of her essay collections (you can read it excerpted here: https://slate.com/technology/2014/08/52-blue-the-loneliest-whale-in-the-world.html )

Colette was wrong, Jamison referenced a famous Kathryn Schulz piece in one of her own essays, which was the source of confusion - The Big One: ​​https://www.newyorker.com/magazine/2015/07/20/the-really-big-one about a cataclysmic earthquake on the west coast. 

In case you’re curious, Colette uses scholar.google.com and paperpile.com shamelessly live.

We reference A Tale of Two Stories: Contrasting View of Patient Safety by Richard Cook and Dave Woods: https://www.researchgate.net/publication/245102691_A_Tale_of_Two_Stories_Contrasting_Views_of_Patient_Safety?enrichId=rgreq-a699511fb5bc518bf1584a0a6613d8d0-XXX&enrichSource=Y292ZXJQYWdlOzI0NTEwMjY5MTtBUzoyMDYyMjM2NjExMTMzNDdAMTQyNjE3ODk2MDQ4NA%3D%3D&el=1_x_2&_esc=publicationCoverPdf

The Beaumaiden report (that dives into a deeper, second story) is here: https://dmaib.com/reports/2021/beaumaiden-grounding-on-18-october-2021

We will continue to point to DORA’s organizational model page: https://dora.dev/capabilities/generative-organizational-culture/

Some Wikipedia on double loop learning: https://en.wikipedia.org/wiki/Double-loop_learning

Colette mentioned Mads Møller’s Lund HFSS thesis on deaths and accountability: https://lup.lub.lu.se/student-papers/search/publication/9106422

And Bram Couteaux’s Lund HFSS thesis on the drunk flight attendants/pilots court: https://lup.lub.lu.se/student-papers/search/publication/9111661

J Paul Reed wrote about being ‘Blame Aware’ - https://medium.com/@jpaulreed/why-blameless-postmortems-might-feel-wrong-cbeee00d51b2

Lorikeets are pretty: https://en.wikipedia.org/wiki/Rainbow_lorikeet

You think Colette’s kidding about the kangaroo? https://www.youtube.com/watch?v=DQjHVRHXbc8 

The Mackinac Bridge is long: https://en.wikipedia.org/wiki/Mackinac_Bridge

Michelle’s Blog post:

https://resilienceinsoftware.org/news/1288714

DORA has some good writing on Westrum’s cultural models if you’re wondering about it: https://dora.dev/capabilities/generative-organizational-culture/

The link to our TiF live event with Michelle where we will be discussing the blog post! https://resilienceinsoftware.org/networks/events/143194

Please ask us questions! You can go to thisisfinepod.com to get the link to our anonymous google form!

Corn sweat is a real thing: https://www.scientificamerican.com/article/humidity-from-corn-sweat-intensifies-extreme-heat-wave-in-midwest-u-s/

Also, plugging Tajin here, because: https://en.wikipedia.org/wiki/Taj%C3%ADn_seasoning

Wikipedia tells me Tajin is Mexican. I dunno, Clint.

Beaumaiden report, for those that didn’t listen to the prior episode where we mentioned it: https://dmaib.com/reports/2021/beaumaiden-grounding-on-18-october-2021

John Allspaw’s talk at Spotify that we referenced: https://www.youtube.com/watch?v=M8mYPyRG1fQ

Lorin’s Law is always a good plug: https://surfingcomplexity.blog/2017/06/24/a-conjecture-on-why-reliable-systems-fail/

Clint’s book recommendation: https://bookshop.org/p/books/the-15-commitments-of-conscious-leadership-a-new-paradigm-for-sustainable-success-diana-chapman/14574335?ean=9780990976905&next=t

Send us questions! at thisisfinepod.com or find us on LinkedIn here: https://www.linkedin.com/company/this-is-fine-a-podcast-about-software-and-resilience-engineering/

You can come to the Lund panelist event for RISF by signing up here: https://resilienceinsoftware.org/networks/events/133948

A huge thanks to our panelists:

⁠John Allspaw⁠

⁠Jed Needle⁠

⁠Chad Todd⁠

RISF and TiF will host a live follow up to this episode on July 31st! You can sign up here: ⁠https://resilienceinsoftware.org/networks/events/133948⁠

If you’re interested in Lund’s Masters of Science program in Human Factors and Systems Safety, or any of their learning labs, you can check out more info here: ⁠https://www.humanfactors.lth.se/⁠

⁠Adaptive Capacity Labs⁠ is how Jed was introduced to some of the concepts of LFI & Resilience Engineering, which eventually landed him at Lund.

John mentioned SciShow Tangents, a podcast by Hank Green and Ceri Riley: ⁠https://www.youtube.com/c/scishowtangents⁠

As well as Conway’s Law: https://en.wikipedia.org/wiki/Conway%27s_law

And Dunbar’s Number: ⁠https://en.wikipedia.org/wiki/Dunbar%27s_number⁠ 

And the Theory of Graceful Extensibility, which you can read about here: ⁠https://infoscience.epfl.ch/server/api/core/bitstreams/87cfe245-c138-43cb-87c9-4062dc1a0519/content⁠

Lund theses list: https://www.humanfactors.lth.se/ny-sajt/msc-programme/msc-theses/

Our panel’s select theses that they love:

Colette’s pick: ⁠https://lup.lub.lu.se/student-papers/search/publication/9106422⁠

Chad’s pick: ⁠https://lup.lub.lu.se/student-papers/search/publication/9009930⁠

John’s picks were all of the software theses, I’m probably missing some but this is my attempt:

John’s (was the first): ⁠https://lup.lub.lu.se/student-papers/search/publication/8084520⁠ 

J Paul Reed: ⁠https://lup.lub.lu.se/student-papers/search/publication/8966930⁠ 

Chad’s thesis on handovers in software: ⁠https://lup.lub.lu.se/student-papers/search/publication/9076274⁠ 

Michael Wettick: ⁠https://lup.lub.lu.se/student-papers/search/publication/9150096⁠ 

Colette’s thesis on QRA: ⁠https://lup.lub.lu.se/student-papers/search/publication/9148570⁠

Jessica De Vita: ⁠https://lup.lub.lu.se/student-papers/search/publication/9149521⁠ 

Dr. Raymer’s I want to Treat the Patient and Not the Alarm: ⁠https://lup.lub.lu.se/student-papers/search/publication/2861164⁠ 

Dave Wood’s Talk at SRECon 25 was on Complexification and SRE: https://www.youtube.com/watch?v=lmBvUJnGUX4

Jens Rasmussen’s model - Is really well explained by Richard Cook’s talk at Velocity: https://www.youtube.com/watch?v=PGLYEDpNu60&t=3s

Lorin’s blog also has a good summary: https://surfingcomplexity.blog/2021/05/31/transgressing-the-boundaries-rasmussen-and-woods/

And finally, Jens Rasmussen’s original paper on the subject: Risk Management in a Dynamic Society https://linkinghub.elsevier.com/retrieve/pii/S0925753597000520

SRECon 25 talk on Incident Metrics that Matter that was awesome -  https://www.youtube.com/watch?v=QrR2SvpWvdg

Want to read about how things are getting a bit fash-y in tech these days?

https://www.newyorker.com/culture/infinite-scroll/techno-fascism-comes-to-america-elon-musk

https://www.theguardian.com/technology/ng-interactive/2025/jan/29/silicon-valley-rightwing-technofascism

Perrow/Normal Accidents: https://bookshop.org/p/books/normal-accidents-living-with-high-risk-technologies-updated-edition-revised-charles-perrow/10369279?ean=9780691004129

High Reliability Organizations (HROs):

Started (ish) with “A Rejoinder to Perrow” https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-5973.1994.tb00047.x

And you can find Rochlin & La Porte behind a lot of the early writing on HROs, including https://www.jstor.org/stable/44637690?seq=1 and https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-5973.1996.tb00078.x

As well as Weick and Sutcliffe: https://bookshop.org/p/books/managing-the-unexpected-sustained-performance-in-a-complex-world-kathleen-m-sutcliffe/11267666?ean=9781118862414&

https://journals.sagepub.com/doi/10.2307/41165243

You can register for the After-the-Episode chat with Andrew at https://resilienceinsoftware.org/networks/events/129997

Tickets are free for members, $10 for non-members. You can join the Foundation at https://resilienceinsoftware.org/signup 

Zuul is what Volvo uses for their CI, and it’s part of the OpenInfra Foundation, it’s rad.

You can find Andrew on LinkedIn here.

Michael Wettick’s Lund thesis is great, and Laura Maguire’s paper on the Costs of Coordination that is a shortened version of her dissertation is worth a read!

Clint’s SRECon talk that he mentioned a couple times: https://www.youtube.com/watch?v=k4UaDDkLOhw

Lorin wrote a great article on incidents and improvisation: https://surfingcomplexity.blog/2023/06/11/when-theres-no-plan-for-this-scenario-youve-got-to-improvise/

Incident.io and the people who work there have hilarious LinkedIn posts about how people use incidents in their org.

We talked about BlackRock3 who do incident command training: https://www.blackrock3.com/

Brent Chapman has also done great incident command training and has done some talks on why IT incident management can learn from fire/emergency response management processes.

We have a LinkedIn! https://www.linkedin.com/company/this-is-fine-a-podcast-about-software-and-resilience-engineering/

And you can ask us questions here: https://forms.gle/rggrbGG6aFVrgZsv9

The O’Reilly book on Chaos Engineering by Casey and Nora Jones is here: https://www.oreilly.com/library/view/chaos-engineering/9781492043850/

Some of the Netflix posts introducing Chaos Monkey and Simian Army are here and here.

You can see Lorin Hochstein talking about Chaos Engineering at Netflix here.

The Void is an awesome collection of information on incidents throughout tech and you can find it here.

Casey mentioned Rasmussen’s model. Lorin has a great summary of that on his blog, but you can read the original paper by Rasmussen introducing this model here.

A report on the Netflix outage during Christmas of 2012.

A reminder - you can ask us questions for the podcast at www.thisisfinepod.com

Clint wrote the Socio-Technical Reality Engineer as a blog post it’s a good read.

The Burnout book by the Nagoski sisters is A+++ reading.

Those Found Responsible Have Been Sacked is by the late, great, Dr. Richard Cook and Chris Nemeth

The Perverse Incentives of Reliability by Katie Wilde from Snyk at this year’s SRECon was just an incredible talk.

Colette mentioned the Beaumaiden report from DMAIB. She gave a talk for the DORA community on resilience engineering that you can see here.