If you've ever debugged a program, looked for lost socks or tried to figure out why red spots are developing on your skin, then Bayes' rule was almost certainly used to help you on your journey. Even if you don't know anything about it. Humans have evolved to solve problems but along the way, we as a species sometimes fall for traps or fail to consider all the evidence when figuring things out.

In this episode, Wolf explains what Bayes' rule is, how we use it and how we could use it better to solve our mysteries.

One sentence
Bayes' Rule is the formula that tells you how to update what you believe when you get new evidence — it combines what was already true with what you just learned.

The math
The probability of A given B equals the probability of B given A, times the probability of A, divided by the probability of B

P(A | B) = P(B | A) * P(A) / P(B)

Key concepts

• Bayes' Rule — the formula for updating what you believe when you get new evidence
• Representativeness heuristic — substituting "how well does this match?" for "how likely is this?" (ignoring base rates)
• Base rate neglect — the tendency to ignore population-level frequencies when evaluating specific cases
• Prior / likelihood / posterior — what you believed before, how likely the evidence is, what you should believe now
• System 1 / System 2 — Kahneman's framework for fast intuitive thinking vs. slow deliberate reasoning

The Tom W problem
From Kahneman's Thinking, Fast and Slow, Chapter 14. A personality description that tricks you into ignoring base rates. The Sin of Representativeness — Unearned Wisdom

The cab problem
Also from Kahneman. A witness, a hit-and-run, and the surprising math of why 80% reliability doesn't mean 80% probability. Kahneman's Bayesian inference example

Books:

• Daniel Kahneman, Thinking, Fast and Slow (2011) — the Tom W problem, the cab problem, System 1/System 2, representativeness
• Sharon Bertsch McGrayne, The Theory That Wouldn't Die (2011) — the history of Bayes' theorem from its discovery through the frequentist wars to its modern resurgence
• Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid (1979) — a Pulitzer-winning exploration of how self-reference and formal systems connect mathematics, art, and music
• Ernest Nagel and James R. Newman, Gödel's Proof (1958) — a concise, accessible walkthrough of Gödel's Incompleteness Theorems for non-mathematicians

Historical:

• Thomas Bayes (1701–1761) — Presbyterian minister who first derived the theorem; never published it. Richard Price submitted it posthumously.
• Pierre-Simon Laplace — independently derived and generalized Bayes' work; arguably did the heavier mathematical lifting

Tools (if you want to go deeper):

• PyMC — Python library for Bayesian statistical modeling
• Bayes' theorem — Wikipedia
• Thinking, Fast and Slow — Wikipedia

Hosts:
Jim McQuillan can be reached at [email protected]
Wolf can be reached at [email protected]

Follow us on Mastodon: @[email protected]

If you have feedback for us, please send it to [email protected]

Checkout our webpage at http://RuntimeArguments.fm

Theme music:
Dawn by nuer self, from the album Digital Sky