Doom Debates!

Eneasz Brodski and Steven Zuber host the Bayesian Conspiracy podcast, which has been running for nine years and covers rationalist topics from AI safety to social dynamics. They're both OG rationalists who've been in the community since the early LessWrong days around 2007-2010. I've been listening to their show since the beginning, and finally got to meet my podcast heroes!

In this episode, we get deep into the personal side of having a high P(Doom) — how do you actually live a good life when you think there's a 50% chance civilization ends by 2040? We also debate whether spreading doom awareness helps humanity or just makes people miserable, with Eneasz pushing back on my fearmongering approach.

We also cover my Doom Train framework for systematically walking through AI risk arguments, why most guests never change their minds during debates, the sorry state of discourse on tech Twitter, and how rationalists can communicate better with normies. Plus some great stories from the early LessWrong era, including my time sitting next to Eliezer while he wrote Harry Potter and the Methods of Rationality.

* 00:00 - Opening and introductions

* 00:43 - Origin stories: How we all got into rationalism and LessWrong

* 03:42 - Liron's incredible story: Sitting next to Eliezer while he wrote HPMOR

* 06:19 - AI awakening moments: ChatGPT, AlphaGo, and move 37

* 13:48 - Do AIs really "understand" meaning? Symbol grounding and consciousness

* 26:21 - Liron's 50% P(Doom) by 2040 and the Doom Debates mission

* 29:05 - The fear mongering debate: Does spreading doom awareness hurt people?

* 34:43 - "Would you give 95% of people 95% P(Doom)?" - The recoil problem

* 42:02 - How to live a good life with high P(Doom)

* 45:55 - Economic disruption predictions and Liron's failed unemployment forecast

* 57:19 - The Doom Debates project: 30,000 watch hours and growing

* 58:43 - The Doom Train framework: Mapping the stops where people get off

* 1:03:19 - Why guests never change their minds (and the one who did)

* 1:07:08 - Communication advice: "Zooming out" for normies

* 1:09:39 - The sorry state of arguments on tech Twitter

* 1:24:11 - Do guests get mad? The hologram effect of debates

* 1:30:11 - Show recommendations and final thoughts

Show Notes

The Bayesian Conspiracy — https://www.thebayesianconspiracy.com

Doom Debates episode with Mike Israetel — https://www.youtube.com/watch?v=RaDWSPMdM4o

Doom Debates episode with David Duvenaud — https://www.youtube.com/watch?v=mb9w7lFIHRM

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Liam Robins is a math major at George Washington University who's diving deep into AI policy and rationalist thinking.

In Part 1, we explored how AI is transforming college life. Now in Part 2, we ride the Doom Train together to see if we can reconcile our P(Doom) estimates. 🚂

Liam starts with a P(Doom) of just 3%, but as we go through the stops on the Doom Train, something interesting happens: he actually updates his beliefs in realtime!

We get into heated philosophical territory around moral realism, psychopaths, and whether intelligence naturally yields moral goodness.

By the end, Liam's P(Doom) jumps from 3% to 8% - one of the biggest belief updates I've ever witnessed on the show. We also explore his "Bayes factors" approach to forecasting, debate the reliability of superforecasters vs. AI insiders, and discuss why most AI policies should be Pareto optimal regardless of your P(Doom).

This is rationality in action: watching someone systematically examine their beliefs, engage with counterarguments, and update accordingly.

0:00 - Opening

0:42 - What’s Your P(Doom)™

01:18 - Stop 1: AGI timing (15% chance it's not coming soon)

01:29 - Stop 2: Intelligence limits (1% chance AI can't exceed humans)

01:38 - Stop 3: Physical threat assessment (1% chance AI won't be dangerous)

02:14 - Stop 4: Intelligence yields moral goodness - the big debate begins

04:42 - Moral realism vs. evolutionary explanations for morality

06:43 - The psychopath problem: smart but immoral humans exist

08:50 - Game theory and why psychopaths persist in populations

10:21 - Liam's first major update: 30% down to 15-20% on moral goodness

12:05 - Stop 5: Safe AI development process (20%)

14:28 - Stop 6: Manageable capability growth (20%)

15:38 - Stop 7: AI conquest intentions - breaking down into subcategories

17:03 - Alignment by default vs. deliberate alignment efforts

19:07 - Stop 8: Super alignment tractability (20%)

20:49 - Stop 9: Post-alignment peace (80% - surprisingly optimistic)

23:53 - Stop 10: Unaligned ASI mercy (1% - "just cope")

25:47 - Stop 11: Epistemological concerns about doom predictions

27:57 - Bayes factors analysis: Why Liam goes from 38% to 3%

30:21 - Bayes factor 1: Historical precedent of doom predictions failing

33:08 - Bayes factor 2: Superforecasters think we'll be fine

39:23 - Bayes factor 3: AI insiders and government officials seem unconcerned

45:49 - Challenging the insider knowledge argument with concrete examples

48:47 - The privilege access epistemology debate

56:02 - Major update: Liam revises base factors, P(Doom) jumps to 8%

58:18 - Odds ratios vs. percentages: Why 3% to 8% is actually huge

59:14 - AI policy discussion: Pareto optimal solutions across all P(Doom) levels

1:01:59 - Why there's low-hanging fruit in AI policy regardless of your beliefs

1:04:06 - Liam's future career plans in AI policy

1:05:02 - Wrap-up and reflection on rationalist belief updating

Show Notes

* Liam Robins on Substack -

* Liam’s Doom Train post -

* Liam’s Twitter - @liamhrobins

Anthropic's "Alignment Faking in Large Language Models" - The paper that updated Liam's beliefs on alignment by default

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Amjad Masad is the founder and CEO of Replit, a full-featured AI-powered software development platform whose revenue reportedly just shot up from $10M/yr to $100M/yr+.

Last week, he went on Joe Rogan to share his vision that "everyone will become an entrepreneur" as AI automates away traditional jobs.

In this episode, I break down why Amjad's optimistic predictions rely on abstract hand-waving rather than concrete reasoning. While Replit is genuinely impressive, his claims about AI limitations—that they can only "remix" and do "statistics" but can't "generalize" or create "paradigm shifts"—fall apart when applied to specific examples.

We explore the entrepreneurial bias problem, why most people can't actually become successful entrepreneurs, and how Amjad's own success stories (like quality assurance automation) actually undermine his thesis. Plus: Roger Penrose's dubious consciousness theories, the "Duplo vs. Lego" problem in abstract thinking, and why Joe Rogan invited an AI doomer the very next day.

00:00 - Opening and introduction to Amjad Masad

03:15 - "Everyone will become an entrepreneur" - the core claim

08:45 - Entrepreneurial bias: Why successful people think everyone can do what they do

15:20 - The brainstorming challenge: Human vs. AI idea generation

22:10 - "Statistical machines" and the remixing framework

28:30 - The abstraction problem: Duplos vs. Legos in reasoning

35:50 - Quantum mechanics and paradigm shifts: Why bring up Heisenberg?

42:15 - Roger Penrose, Gödel's theorem, and consciousness theories

52:30 - Creativity definitions and the moving goalposts

58:45 - The consciousness non-sequitur and Silicon Valley "hubris"

01:07:20 - Ahmad George success story: The best case for Replit

01:12:40 - Job automation and the 50% reskilling assumption

01:18:15 - Quality assurance jobs: Accidentally undermining your own thesis

01:23:30 - Online learning and the contradiction in AI capabilities

01:29:45 - Superintelligence definitions and learning in new environments

01:35:20 - Self-play limitations and literature vs. programming

01:41:10 - Marketing creativity and the Think Different campaign

01:45:45 - Human-machine collaboration and the prompting bottleneck

01:50:30 - Final analysis: Why this reasoning fails at specificity

01:58:45 - Joe Rogan's real opinion: The Roman Yampolskiy follow-up

02:02:30 - Closing thoughts

Show Notes

Source video: Amjad Masad on Joe Rogan - July 2, 2025

Roman Yampolskiy on Joe Rogan - https://www.youtube.com/watch?v=j2i9D24KQ5k

Replit - https://replit.com

Amjad’s Twitter - https://x.com/amasad

Doom Debates episode where I react to Emmett Shear’s Softmax - https://www.youtube.com/watch?v=CBN1E1fvh2g

Doom Debates episode where I react to Roger Penrose - https://www.youtube.com/watch?v=CBN1E1fvh2g

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Liam Robins is a math major at George Washington University who recently had his own "AGI awakening" after reading Leopold Aschenbrenner's Situational Awareness. I met him at my Manifest 2025 talk about stops on the Doom Train.

In this episode, Liam confirms what many of us suspected: pretty much everyone in college is cheating with AI now, and they're completely shameless about it.

We dive into what college looks like today: how many students are still "rawdogging" lectures, how professors are coping with widespread cheating, how the social life has changed, and what students think they’ll do when they graduate.

* 00:00 - Opening

* 00:50 - Introducing Liam Robins

* 05:27 - The reality of college today: Do they still have lectures?

* 07:20 - The rise of AI-enabled cheating in assignments

* 14:00 - College as a credentialing regime vs. actual learning

* 19:50 - "Everyone is cheating their way through college" - the epidemic

* 26:00 - College social life: "It's just pure social life"

* 31:00 - Dating apps, social media, and Gen Z behavior

* 36:21 - Do students understand the singularity is near?

Show Notes

Guest:

* Liam Robins on Substack - https://thelimestack.substack.com/

* Liam's Doom Train post - https://thelimestack.substack.com/p/my-pdoom-is-276-heres-why

* Liam’s Twitter - @liamrobins

Key References:

* Leopold Aschenbrenner - "Situational Awareness"

* Bryan Caplan - "The Case Against Education"

* Scott Alexander - Astral Codex Ten

* Jeffrey Ding - ChinAI Newsletter

* New York Magazine - "Everyone Is Cheating Their Way Through College"

Events & Communities:

* Manifest Conference

* LessWrong

* Eliezer Yudkowsky - "Harry Potter and the Methods of Rationality"

Previous Episodes:

* Doom Debates Live at Manifest 2025 - https://www.youtube.com/watch?v=detjIyxWG8M

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Carl Feynman got his Master’s in Computer Science and B.S. in Philosophy from MIT, followed by a four-decade career in AI engineering.

He’s known Eliezer Yudkowsky since the ‘90s, and witnessed Eliezer’s AI doom argument taking shape before most of us were paying any attention!

He agreed to come on the show because he supports Doom Debates’s mission of raising awareness of imminent existential risk from superintelligent AI.

00:00 - Teaser

00:34 - Carl Feynman’s Background

02:40 - Early Concerns About AI Doom

03:46 - Eliezer Yudkowsky and the Early AGI Community

05:10 - Accelerationist vs. Doomer Perspectives

06:03 - Mainline Doom Scenarios: Gradual Disempowerment vs. Foom

07:47 - Timeline to Doom: Point of No Return

08:45 - What’s Your P(Doom)™

09:44 - Public Perception and Political Awareness of AI Risk

11:09 - AI Morality, Alignment, and Chatbots Today

13:05 - The Alignment Problem and Competing Values

15:03 - Can AI Truly Understand and Value Morality?

16:43 - Multiple Competing AIs and Resource Competition

18:42 - Alignment: Wanting vs. Being Able to Help Humanity

19:24 - Scenarios of Doom and Odds of Success

19:53 - Mainline Good Scenario: Non-Doom Outcomes

20:27 - Heaven, Utopia, and Post-Human Vision

22:19 - Gradual Disempowerment Paper and Economic Displacement

23:31 - How Humans Get Edged Out by AIs

25:07 - Can We Gaslight Superintelligent AIs?

26:38 - AI Persuasion & Social Influence as Doom Pathways

27:44 - Riding the Doom Train: Headroom Above Human Intelligence

29:46 - Orthogonality Thesis and AI Motivation

32:48 - Alignment Difficulties and Deception in AIs

34:46 - Elon Musk, Maximal Curiosity & Mike Israetel’s Arguments3

6:26 - Beauty and Value in a Post-Human Universe

38:12 - Multiple AIs Competing

39:31 - Space Colonization, Dyson Spheres & Hanson’s “Alien Descendants”

41:13 - What Counts as Doom vs. Not Doom?

43:29 - Post-Human Civilizations and Value Function

44:49 - Expertise, Rationality, and Doomer Credibility

46:09 - Communicating Doom: Missing Mood & Public Receptiveness

47:41 - Personal Preparation vs. Belief in Imminent Doom48:56 - Why Can't We Just Hit the Off Switch?

50:26 - The Treacherous Turn and Redundancy in AI

51:56 - Doom by Persuasion or Entertainment

53:43 - Differences with Eliezer Yudkowsky: Singleton vs. Multipolar Doom

55:22 - Why Carl Chose Doom Debates

56:18 - Liron’s Outro

Show Notes

Carl’s Twitter — https://x.com/carl_feynman

Carl’s LessWrong — https://www.lesswrong.com/users/carl-feynman

Gradual Disempowerment — https://gradual-disempowerment.ai

The Intelligence Curse — https://intelligence-curse.ai

AI 2027 — https://ai-2027.com

Alcor cryonics — https://www.alcor.org

The LessOnline Conference — https://less.online

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Richard Hanania is the President of the Center for the Study of Partisanship and Ideology. His work has been praised by Vice President JD Vance, Tyler Cowen, and Bryan Caplan among others.

In his influential newsletter, he’s written about why he finds AI doom arguments unconvincing. He was gracious enough to debate me on this topic. Let’s see if one of us can change the other’s P(Doom)!

0:00 Intro

1:53 Richard's politics

2:24 The state of political discourse

3:30 What's your P(Doom)?™

6:38 How to stop the doom train

8:27 Statement on AI risk

9:31 Intellectual influences

11:15 Base rates for AI doom

15:43 Intelligence as optimization power

31:26 AI capabilities progress

53:46 Why isn't AI yet a top blogger?

58:02 Diving into Richard's Doom Train

58:47 Diminishing Returns on Intelligence

1:06:36 Alignment will be relatively trivial

1:15:14 Power-seeking must be programmed

1:21:27 AI will simply be benevolent

1:27:17 Superintelligent AI will negotiate with humans

1:33:00 Super AIs will check and balance each other out

1:36:54 We're mistaken about the nature of intelligence

1:41:46 Summarizing Richard's AI doom position

1:43:22 Jobpocalypse and gradual disempowerment

1:49:46 Ad hominem attacks in AI discourse

Show Notes

Subscribe to Richard Hanania's Newsletter: https://richardhanania.com

Richard's blogpost laying out where he gets off the AI "doom train": https://www.richardhanania.com/p/ai-doomerism-as-science-fiction

Richard's interview with Steven Pinker: https://www.richardhanania.com/p/pinker-on-alignment-and-intelligence

Richard's interview with Robin Hanson: https://www.richardhanania.com/p/robin-hanson-says-youre-going-to

My Doom Debate with Robin Hanson: https://www.youtube.com/watch?v=dTQb6N3_zu8

My reaction to Steven Pinker's AI doom position, and why his arguments are shallow: https://www.youtube.com/watch?v=-tIq6kbrF-4

"The Betterness Explosion" by Robin Hanson: https://www.overcomingbias.com/p/the-betterness-explosionhtml

---

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/watch?v=9CUFbqh16Fg

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at https://DoomDebates.com and to https://youtube.com/@DoomDebates

Emmett Shear is the cofounder and ex-CEO of Twitch, ex-interim-CEO of OpenAI, and a former Y Combinator partner. He recently announced Softmax, a new company researching a novel solution to AI alignment.

In his recent interview, Emmett explained “organic alignment”, drawing comparisons to biological systems and advocating for AI to be raised in a community-like setting with humans.

Let’s go through his talk, point by point, to see if Emmett’s alignment plan makes sense…

00:00 Episode Highlights

00:36 Introducing Softmax and its Founders

01:33 Research Collaborators and Ken Stanley's Influence

02:16 Softmax's Mission and Organic Alignment

03:13 Critique of Organic Alignment

05:29 Emmett’s Perspective on AI Alignment

14:36 Human Morality and Cognitive Submodules

38:25 Top-Down vs. Emergent Morality in AI

44:56 Raising AI to Grow Up with Humanity

48:43 Softmax's Incremental Approach to AI Alignment

52:22 Convergence vs. Divergence in AI Learning

55:49 Multi-Agent Reinforcement Learning

01:12:28 The Importance of Storytelling in AI Development

01:16:34 Living With AI As It Grows

01:20:19 Species Giving Birth to Species

01:23:23 The Plan for AI's Adolescence

01:26:53 Emmett's Views on Superintelligence

01:31:00 The Future of AI Alignment

01:35:10 Final Thoughts and Criticisms

01:44:07 Conclusion and Call to Action

Show Notes

Emmett Shear’s interview on BuzzRobot with Sophia Aryan (source material) — https://www.youtube.com/watch?v=_3m2cpZqvdw

BuzzRobot’s YouTube channel — https://www.youtube.com/@BuzzRobot

BuzzRobot’s Twitter — https://x.com/buZZrobot/

SoftMax’s website — https://softmax.com

My Doom Debate with Ken Stanley (Softmax advisor) — https://www.youtube.com/watch?v=GdthPZwU1Co

My Doom Debate with Gil Mark on whether aligning AIs in groups is a more solvable problem — https://www.youtube.com/watch?v=72LnKW_jae8

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Prof. Scott Sumner is a well-known macroeconomist who spent more than 30 years teaching at Bentley University, and now holds an Emeritus Chair in monetary policy at George Mason University's Mercatus Center. He's best known for his blog, the Money Illusion, which sparked the idea of Market Monetarism and NGDP targeting.

I sat down with him at LessOnline 2025 to debate why his P(Doom) is pretty low. Where does he get off the Doom Train? 🚂

00:00 Episode Preview

00:34 Introducing Scott Sumner

05:20 Is AGI Coming Soon?

09:12 Potential of AI in Various Fields

36:49 Ethical Implications of Superintelligent AI

41:03 The Nazis as an Outlier in History

43:36 Intelligence and Morality: The Orthogonality Thesis

49:03 The Risk of Misaligned AI Goals

01:09:31 Recapping Scott’s Position

Show Notes

Scott’s current blog, The Pursuit of Happiness:

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

John Searle's "Chinese Room argument" has been called one of the most famous thought experiments of the 20th century. It's still frequently cited today to argue AI can never truly become intelligent.

People continue to treat the Chinese Room like a brilliant insight, but in my opinion, it's actively misleading and DUMB! Here’s why…

00:00 Intro

00:20 What is Searle's Chinese Room Argument?

01:43 John Searle (1984) on Why Computers Can't Understand

01:54 Why the "Chinese Room" Metaphor is Misleading

This mini-episode is taken from Liron's reaction to Sir Roger Penrose. Watch the full episode:

Show Notes

2008 Interview with John Searle: https://www.youtube.com/watch?v=3TnBjLmQawQ&t=253s

1984 Debate with John Searle: https://www.youtube.com/watch?v=6tzjcnPsZ_w

“Chinese Room” cartoon: https://miro.medium.com/v2/0*iTvDe5ebNPvg10AO.jpeg

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Let’s see where the attendees of Manifest 2025 get off the Doom Train, and whether I can convince them to stay on and ride with me to the end of the line!

00:00 Introduction to Doom Debates

03:21 What’s Your P(Doom)?™

05:03 🚂 “AGI Isn't Coming Soon”

08:37 🚂 “AI Can't Surpass Human Intelligence”

12:20 🚂 “AI Won't Be a Physical Threat”

13:39 🚂 “Intelligence Yields Moral Goodness”

17:21 🚂 “Safe AI Development Process”

17:38 🚂 “AI Capabilities Will Rise at a Manageable Pace”

20:12 🚂 “AI Won't Try to Conquer the Universe”

25:00 🚂 “Superalignment Is A Tractable Problem”

28:58 🚂 “Once We Solve Superalignment, We’ll Enjoy Peace”

31:51 🚂 “Unaligned ASI Will Spare Us”

36:40 🚂 “AI Doomerism Is Bad Epistemology”

40:11 Bonus 🚂: “Fine, P(Doom) is high… but that’s ok!”

42:45 Recapping the Debate

See also my previous episode explaining the Doom Train: https://lironshapira.substack.com/p/poking-holes-in-the-ai-doom-argument

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates