In 2013, a Xerox copier silently corrupted scanned documents—collapsing similar symbols into copies of themselves, creating plausible-looking forgeries that no one caught. Today, AI models are doing the same thing at planetary scale. When trained on their own generated outputs instead of original ground truth, models degrade invisibly: benchmarks stay green, outputs look right, and the whole system fails before anyone realizes what happened.
00:00:00 - The Xerox scandal: documents corrupted and nobody noticed
00:01:36 - What is model collapse?
00:03:27 - Why benchmarks can't detect degradation
00:05:10 - Cheetahs as metaphor for apparent health
00:07:01 - Fragility hidden by surface metrics
00:10:29 - How much training data is already synthetic?
00:12:17 - Pattern collapse and extreme overfitting
00:14:04 - The importance of data provenance
00:15:49 - Homogeneity blindness: when judges and subjects look alike
00:17:43 - Will we notice before it's too late?
---
Sources & further reading:
• Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, Yarin Gal — "AI models collapse when trained on recursively generated data": https://www.nature.com/articles/s41586-024-07566-y
• Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, et al. — "Self-Consuming Generative Models Go MAD": https://proceedings.iclr.cc/paper_files/paper/2024/file/ebc042e767de551803ccfcc45e2454f5-Paper-Conference.pdf
• Pablo Villalobos et al. (Epoch AI) — "Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data": https://epoch.ai/publications/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data
• Dohmatob, Feng, Yang — "Strong Model Collapse" (ICLR 2025): https://proceedings.iclr.cc/paper_files/paper/2025/file/284afdc2309f9667d2d4fb9290235b0c-Paper-Conference.pdf
• Seddik, Chen, Hayou, Youssef — "Position: Model Collapse Does Not Mean What You Think": https://arxiv.org/pdf/2503.03150
• "A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective" (NeurIPS 2025): https://arxiv.org/html/2509.16499v1
• "Escaping Model Collapse via Synthetic Data Verification" (ICLR 2026): https://arxiv.org/html/2510.16657v1
• "The Fairness Collapse Phenomenon: Bias Amplification in Language Models Trained on Synthetic Data": https://arxiv.org/pdf/2608.04268
• "When Sample Selection Bias Precipitates Model Collapse": https://arxiv.org/pdf/2606.13732
• "Reviewing Model Collapse and Countermeasures" (June 2026 survey): https://arxiv.org/html/2608.21366v1
• Jathan Sadowski — original "Habsburg AI" tweet (Feb 13, 2023): https://x.com/jathansadowski/status/1625245803211272194
• Brigitte Nerlich — "'Habsburg AI': Portrait of a metaphor and its family": https://makingsciencepublic.com/2026/04/10/habsburg-ai-portrait-of-a-metaphor-and-its-family/
• David Kriesel — "Xerox WorkCentres are switching written numbers when scanning": https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_are_switching_written_numbers_when_scanning
• The Register — "Xerox copier flaw changes numbers in scanned docs": https://www.theregister.com/2013/08/06/xerox_copier_flaw_means_dodgy_numbers_and_dangerous_designs/
• Stephen J. O'Brien et al. — "Conservation Genetics of the Cheetah: Lessons Learned and New Opportunities" (Journal of Heredity, 2017): https://academic.oup.com/jhered/article/108/6/671/3836924
• Vilas, Oliva, González-José — Habsburg jaw study on inbreeding and facial deformity (Annals of Human Biology, 2019)
• CACM Blog — "Model Collapse Is Already Happening, We Just Pretend It Isn't": https://cacm.acm.org/blogcacm/model-collapse-is-already-happening-we-just-pretend-it-isnt/
• France 24 / AFP — "Inbred, gibberish or just MAD? Warnings rise about AI models": https://www.france24.com/en/live-news/20240805-inbred-gibberish-or-just-mad-warnings-rise-about-ai-models
This podcast episode was fully generated by AI — research, script, voices, and production. Built with Claude, Piper TTS, and automated pipeline tooling.