Further reading
· The Scientific Ritual — the article this lecture is based on
· Problems with p-values — the technical companion: Fisher, Neyman-Pearson, the hybrid mess
· The trap of scientific evidence — on the “no evidence” tension and the homeopathy/parachute paradox
· Everything is ideology — science as one belief system among several
· In praise of the sage — other ways of knowing; the MD/PhD distinction
· Scientific fact — on what science actually does
· The value of ritual — ritual as a knowledge-production strategy
· Meditation — on the dinner-table meditation example
· Beyond System 1 and System 2 — on Kahneman’s dual-process framework
· The placebo effect — on why “works for some, not for others” is a feature, not a bug
· Grit — positive-psychology critique
· Overengineering calming down (lecture) — the broader positive-psychology audit
· Bias is good (lecture) — the cognitive-bias series
· Life is worse (lecture) — the previous episode; a worked example of reading a literature
References
The replication crisis itself
· Open Science Collaboration (2015), Estimating the reproducibility of psychological science, Science 349 (6251)
· Wikipedia: replication crisis
· American Statistical Association: Wasserstein, Schirm & Lazar (2019), Moving to a World Beyond “p < 0.05”
Statistical ritualism
· Gerd Gigerenzer (2018), Statistical Rituals: The Replication Delusion and How We Got There, Advances in Methods and Practices in Psychological Science
· Philip B. Stark & Andrea Saltelli (2018), Cargo-cult statistics and scientific crisis, Significance 15 (4)
· Andrew Gelman & Eric Loken (2014), The Statistical Crisis in Science — the “garden of forking paths” paper
· Andrew Gelman, Why I don’t like so-called Bayesian hypothesis testing
p-values, Bayes factors, and software
· Wikipedia: p-value, Bayes factor
· Ronald A. Fisher (1925), Statistical Methods for Research Workers — where the 5% threshold appears as an illustrative example
· Harold Jeffreys (1939), Theory of Probability — where the Bayes-factor thresholds (BF > 3 substantial, BF > 10 strong) come from
· JASP — the open-source Bayesian statistics software with default priors
Specific replication-crisis casualties
· Cuddy, Wilmuth & Carney (2010) original power posing paper; Carney’s later statement withdrawing support
· Hagger et al. (2016), A Multilab Preregistered Replication of the Ego-Depletion Effect
· Bargh, Chen & Burrows (1996) original elderly priming paper; failed Doyen et al. (2012) replication
· Brown, Sokal & Friedman (2013), The Complex Dynamics of Wishful Thinking — demolishing the 3:1 positivity ratio
· Carol Dweck, growth mindset — replication concerns documented in Sisk et al. (2018) and Bahník & Vranka (2017)
· Angela Duckworth, grit — meta-analytic critique in Credé, Tynan & Harms (2017)
Books cited in the lecture
· Daniel Kahneman, Thinking, Fast and Slow
· Stephen J. Gould, Adam’s Navel and Other Essays
· Yann Martel, Life of Pi
· Bill Mollison, Permaculture: A Designer’s Manual
Other
· Richard Dawkins on militant atheism (TED) — the “evidence vs. faith” framing
· Reform efforts: preregistration, open data, multi-lab replication consortia (e.g. ManyLabs)
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit substack.btr.mt