Geoffrey Miller is an evolutionary psychologist at the University of New Mexico, bestselling author, and one of the world's leading experts on signaling theory and human sexual selection. His book "Mate" was hugely influential for me personally during my dating years, so I was thrilled to finally get him on the show.

In this episode, Geoffrey drops a bombshell 50% P(Doom) assessment, coming from someone who wrote foundational papers on neural networks and genetic algorithms back in the '90s before pivoting to study human mating behavior for 30 years.

What makes Geoffrey's doom perspective unique is that he thinks both inner and outer alignment might be unsolvable in principle, ever. He's also surprisingly bearish on AI's current value, arguing it hasn't been net positive for society yet despite the $14 billion in OpenAI revenue.

We cover his fascinating intellectual journey from early AI researcher to pickup artist advisor to AI doomer, why Asperger's people make better psychology researchers, the polyamory scene in rationalist circles, and his surprisingly optimistic take on cooperating with China. Geoffrey brings a deeply humanist perspective. He genuinely loves human civilization as it is and sees no reason to rush toward our potential replacement.

* 00:00:00 - Introducing Prof. Geoffrey Miller

* 00:01:46 - Geoffrey’s intellectual career arc: AI → evolutionary psychology → back to AI

* 00:03:43 - Signaling theory as the main theme driving his research

* 00:05:04 - Why evolutionary psychology is legitimate science, not just speculation

* 00:08:18 - Being a professor in the AI age and making courses "AI-proof"

* 00:09:12 - Getting tenure in 2008 and using academic freedom responsibly

* 00:11:01 - Student cheating epidemic with AI tools, going "fully medieval"

* 00:13:28 - Should professors use AI for grading? (Geoffrey says no, would be unethical)

* 00:23:06 - Coming out as Aspie and neurodiversity in academia

* 00:29:15 - What is sex and its role in evolution (error correction vs. variation)

* 00:34:06 - Sexual selection as an evolutionary "supercharger"

* 00:37:25 - Dating advice, pickup artistry, and evolutionary psychology insights

* 00:45:04 - Polyamory: Geoffrey’s experience and the rationalist connection

* 00:50:96 - Why rationalists tend to be poly vs. Chesterton's fence on monogamy

* 00:54:07 - The "primal" lifestyle and evolutionary medicine

* 00:56:59 - How Iain M. Banks' Culture novels shaped Geoffrey’s AI thinking

* 01:05:26 - What’s Your P(Doom)™

* 01:08:04 - Main doom scenario: AI arms race leading to unaligned ASI

* 01:14:10 - Bad actors problem: antinatalists, religious extremists, eco-alarmists

* 01:21:13 - Inner vs. outer alignment - both may be unsolvable in principle

* 01:23:56 - "What's the hurry?" - Why rush when alignment might take millennia?

* 01:28:17 - Disagreement on whether AI has been net positive so far

* 01:35:13 - Why AI won't magically solve longevity or other major problems

* 01:37:56 - Unemployment doom and loss of human autonomy

* 01:40:13 - Cosmic perspective: We could be "the baddies" spreading unaligned AI

* 01:44:93 - "Humanity is doing incredibly well" - no need for Hail Mary AI

* 01:49:01 - Why ASI might be bad at solving alignment (lacks human cultural wisdom)

* 01:52:06 - China cooperation: "Whoever builds ASI first loses"

* 01:55:19 - Liron’s Outro

Show Notes

Links

* Geoffrey’s Twitter

* Geoffrey’s University of New Mexico Faculty Page

* Geoffrey’s Publications

* Designing Neural Networks using Genetic Algorithms - His most cited paper

* Geoffrey’s Effective Altruism Forum Posts

Books by Geoffrey Miller

* Mate: Become the Man Women Want (2015) - Co-authored with Tucker Max

* The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature (2000)

* Virtue Signaling: Essays on Darwinian Politics and Free Speech (2019)

* Spent: Sex, Evolution, and Consumer Behavior (2009)

Related Doom Debates Episodes

* Liam Robins on College in the AGI Era - Student perspective on AI cheating

* Liron Reacts to Steven Pinker on AI Risk - Critiquing Pinker's AI optimism

* Steven Byrnes on Brain-Like AGI - Upcoming episode on human brain architecture

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

I’m doing a new weekly show on the AI Risk Network called Warning Shots. Check it out!

I’m only cross-posting the first episode here on Doom Debates. You can watch future episodes by subscribing to the AI Risk Network channel.

This week's warning shot: Mark Zuckerberg announced that Meta is racing toward recursive self-improvement and superintelligence. His exact words: "Developing superintelligence is now in sight and we just want to make sure that we really strengthen the effort as much as possible to go for it." This should be front-page news. Instead, everyone's talking about some CEO's dumb shenanigans at a Coldplay concert.

Recursive self-improvement is when AI systems start upgrading themselves - potentially the last invention humanity ever makes. Every AI safety expert knows this is a bright red line. And Zuckerberg just said he's sprinting toward it. In a sane world, he'd have to resign for saying this. That's why we made this show - to document these warning shots as they happen, because someone needs to be paying attention

* 00:00 - Opening comments about Zuckerberg and superintelligence

* 00:51 - Show introductions and host backgrounds

* 01:56 - Geoff Lewis psychotic episode and ChatGPT interaction discussion

* 05:04 - Transition to main warning shot about Mark Zuckerberg

* 05:32 - Zuckerberg's recursive self-improvement audio clip

* 08:22 - Second Zuckerberg clip about "going for superintelligence"

* 10:29 - Analysis of "superintelligence in everyone's pocket"

* 13:07 - Discussion of Zuckerberg's true motivations

* 15:13 - Nuclear development analogy and historical context

* 17:39 - What should happen in a sane society (wrap-up)

* 20:01 - Final thoughts and sign-off

Show Notes

Hosts:

* Doom Debates - Liron Shapira's channel

* AI Risk Network - John Sherman's channel

* Lethal Intelligence - Michael's animated AI safety content

This Episode's Warning Shots:

* Mark Zuckerberg podcast appearance discussing superintelligence

* Geoff Lewis (Bedrock VC) Twitter breakdown

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Eneasz Brodski and Steven Zuber host the Bayesian Conspiracy podcast, which has been running for nine years and covers rationalist topics from AI safety to social dynamics. They're both OG rationalists who've been in the community since the early LessWrong days around 2007-2010. I've been listening to their show since the beginning, and finally got to meet my podcast heroes!

In this episode, we get deep into the personal side of having a high P(Doom) — how do you actually live a good life when you think there's a 50% chance civilization ends by 2040? We also debate whether spreading doom awareness helps humanity or just makes people miserable, with Eneasz pushing back on my fearmongering approach.

We also cover my Doom Train framework for systematically walking through AI risk arguments, why most guests never change their minds during debates, the sorry state of discourse on tech Twitter, and how rationalists can communicate better with normies. Plus some great stories from the early LessWrong era, including my time sitting next to Eliezer while he wrote Harry Potter and the Methods of Rationality.

* 00:00 - Opening and introductions

* 00:43 - Origin stories: How we all got into rationalism and LessWrong

* 03:42 - Liron's incredible story: Sitting next to Eliezer while he wrote HPMOR

* 06:19 - AI awakening moments: ChatGPT, AlphaGo, and move 37

* 13:48 - Do AIs really "understand" meaning? Symbol grounding and consciousness

* 26:21 - Liron's 50% P(Doom) by 2040 and the Doom Debates mission

* 29:05 - The fear mongering debate: Does spreading doom awareness hurt people?

* 34:43 - "Would you give 95% of people 95% P(Doom)?" - The recoil problem

* 42:02 - How to live a good life with high P(Doom)

* 45:55 - Economic disruption predictions and Liron's failed unemployment forecast

* 57:19 - The Doom Debates project: 30,000 watch hours and growing

* 58:43 - The Doom Train framework: Mapping the stops where people get off

* 1:03:19 - Why guests never change their minds (and the one who did)

* 1:07:08 - Communication advice: "Zooming out" for normies

* 1:09:39 - The sorry state of arguments on tech Twitter

* 1:24:11 - Do guests get mad? The hologram effect of debates

* 1:30:11 - Show recommendations and final thoughts

Show Notes

The Bayesian Conspiracy — https://www.thebayesianconspiracy.com

Doom Debates episode with Mike Israetel — https://www.youtube.com/watch?v=RaDWSPMdM4o

Doom Debates episode with David Duvenaud — https://www.youtube.com/watch?v=mb9w7lFIHRM

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Liam Robins is a math major at George Washington University who's diving deep into AI policy and rationalist thinking.

In Part 1, we explored how AI is transforming college life. Now in Part 2, we ride the Doom Train together to see if we can reconcile our P(Doom) estimates. 🚂

Liam starts with a P(Doom) of just 3%, but as we go through the stops on the Doom Train, something interesting happens: he actually updates his beliefs in realtime!

We get into heated philosophical territory around moral realism, psychopaths, and whether intelligence naturally yields moral goodness.

By the end, Liam's P(Doom) jumps from 3% to 8% - one of the biggest belief updates I've ever witnessed on the show. We also explore his "Bayes factors" approach to forecasting, debate the reliability of superforecasters vs. AI insiders, and discuss why most AI policies should be Pareto optimal regardless of your P(Doom).

This is rationality in action: watching someone systematically examine their beliefs, engage with counterarguments, and update accordingly.

0:00 - Opening

0:42 - What’s Your P(Doom)™

01:18 - Stop 1: AGI timing (15% chance it's not coming soon)

01:29 - Stop 2: Intelligence limits (1% chance AI can't exceed humans)

01:38 - Stop 3: Physical threat assessment (1% chance AI won't be dangerous)

02:14 - Stop 4: Intelligence yields moral goodness - the big debate begins

04:42 - Moral realism vs. evolutionary explanations for morality

06:43 - The psychopath problem: smart but immoral humans exist

08:50 - Game theory and why psychopaths persist in populations

10:21 - Liam's first major update: 30% down to 15-20% on moral goodness

12:05 - Stop 5: Safe AI development process (20%)

14:28 - Stop 6: Manageable capability growth (20%)

15:38 - Stop 7: AI conquest intentions - breaking down into subcategories

17:03 - Alignment by default vs. deliberate alignment efforts

19:07 - Stop 8: Super alignment tractability (20%)

20:49 - Stop 9: Post-alignment peace (80% - surprisingly optimistic)

23:53 - Stop 10: Unaligned ASI mercy (1% - "just cope")

25:47 - Stop 11: Epistemological concerns about doom predictions

27:57 - Bayes factors analysis: Why Liam goes from 38% to 3%

30:21 - Bayes factor 1: Historical precedent of doom predictions failing

33:08 - Bayes factor 2: Superforecasters think we'll be fine

39:23 - Bayes factor 3: AI insiders and government officials seem unconcerned

45:49 - Challenging the insider knowledge argument with concrete examples

48:47 - The privilege access epistemology debate

56:02 - Major update: Liam revises base factors, P(Doom) jumps to 8%

58:18 - Odds ratios vs. percentages: Why 3% to 8% is actually huge

59:14 - AI policy discussion: Pareto optimal solutions across all P(Doom) levels

1:01:59 - Why there's low-hanging fruit in AI policy regardless of your beliefs

1:04:06 - Liam's future career plans in AI policy

1:05:02 - Wrap-up and reflection on rationalist belief updating

Show Notes

* Liam Robins on Substack -

* Liam’s Doom Train post -

* Liam’s Twitter - @liamhrobins

Anthropic's "Alignment Faking in Large Language Models" - The paper that updated Liam's beliefs on alignment by default

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Amjad Masad is the founder and CEO of Replit, a full-featured AI-powered software development platform whose revenue reportedly just shot up from $10M/yr to $100M/yr+.

Last week, he went on Joe Rogan to share his vision that "everyone will become an entrepreneur" as AI automates away traditional jobs.

In this episode, I break down why Amjad's optimistic predictions rely on abstract hand-waving rather than concrete reasoning. While Replit is genuinely impressive, his claims about AI limitations—that they can only "remix" and do "statistics" but can't "generalize" or create "paradigm shifts"—fall apart when applied to specific examples.

We explore the entrepreneurial bias problem, why most people can't actually become successful entrepreneurs, and how Amjad's own success stories (like quality assurance automation) actually undermine his thesis. Plus: Roger Penrose's dubious consciousness theories, the "Duplo vs. Lego" problem in abstract thinking, and why Joe Rogan invited an AI doomer the very next day.

00:00 - Opening and introduction to Amjad Masad

03:15 - "Everyone will become an entrepreneur" - the core claim

08:45 - Entrepreneurial bias: Why successful people think everyone can do what they do

15:20 - The brainstorming challenge: Human vs. AI idea generation

22:10 - "Statistical machines" and the remixing framework

28:30 - The abstraction problem: Duplos vs. Legos in reasoning

35:50 - Quantum mechanics and paradigm shifts: Why bring up Heisenberg?

42:15 - Roger Penrose, Gödel's theorem, and consciousness theories

52:30 - Creativity definitions and the moving goalposts

58:45 - The consciousness non-sequitur and Silicon Valley "hubris"

01:07:20 - Ahmad George success story: The best case for Replit

01:12:40 - Job automation and the 50% reskilling assumption

01:18:15 - Quality assurance jobs: Accidentally undermining your own thesis

01:23:30 - Online learning and the contradiction in AI capabilities

01:29:45 - Superintelligence definitions and learning in new environments

01:35:20 - Self-play limitations and literature vs. programming

01:41:10 - Marketing creativity and the Think Different campaign

01:45:45 - Human-machine collaboration and the prompting bottleneck

01:50:30 - Final analysis: Why this reasoning fails at specificity

01:58:45 - Joe Rogan's real opinion: The Roman Yampolskiy follow-up

02:02:30 - Closing thoughts

Show Notes

Source video: Amjad Masad on Joe Rogan - July 2, 2025

Roman Yampolskiy on Joe Rogan - https://www.youtube.com/watch?v=j2i9D24KQ5k

Replit - https://replit.com

Amjad’s Twitter - https://x.com/amasad

Doom Debates episode where I react to Emmett Shear’s Softmax - https://www.youtube.com/watch?v=CBN1E1fvh2g

Doom Debates episode where I react to Roger Penrose - https://www.youtube.com/watch?v=CBN1E1fvh2g

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Liam Robins is a math major at George Washington University who recently had his own "AGI awakening" after reading Leopold Aschenbrenner's Situational Awareness. I met him at my Manifest 2025 talk about stops on the Doom Train.

In this episode, Liam confirms what many of us suspected: pretty much everyone in college is cheating with AI now, and they're completely shameless about it.

We dive into what college looks like today: how many students are still "rawdogging" lectures, how professors are coping with widespread cheating, how the social life has changed, and what students think they’ll do when they graduate.

* 00:00 - Opening

* 00:50 - Introducing Liam Robins

* 05:27 - The reality of college today: Do they still have lectures?

* 07:20 - The rise of AI-enabled cheating in assignments

* 14:00 - College as a credentialing regime vs. actual learning

* 19:50 - "Everyone is cheating their way through college" - the epidemic

* 26:00 - College social life: "It's just pure social life"

* 31:00 - Dating apps, social media, and Gen Z behavior

* 36:21 - Do students understand the singularity is near?

Show Notes

Guest:

* Liam Robins on Substack - https://thelimestack.substack.com/

* Liam's Doom Train post - https://thelimestack.substack.com/p/my-pdoom-is-276-heres-why

* Liam’s Twitter - @liamrobins

Key References:

* Leopold Aschenbrenner - "Situational Awareness"

* Bryan Caplan - "The Case Against Education"

* Scott Alexander - Astral Codex Ten

* Jeffrey Ding - ChinAI Newsletter

* New York Magazine - "Everyone Is Cheating Their Way Through College"

Events & Communities:

* Manifest Conference

* LessWrong

* Eliezer Yudkowsky - "Harry Potter and the Methods of Rationality"

Previous Episodes:

* Doom Debates Live at Manifest 2025 - https://www.youtube.com/watch?v=detjIyxWG8M

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Carl Feynman got his Master’s in Computer Science and B.S. in Philosophy from MIT, followed by a four-decade career in AI engineering.

He’s known Eliezer Yudkowsky since the ‘90s, and witnessed Eliezer’s AI doom argument taking shape before most of us were paying any attention!

He agreed to come on the show because he supports Doom Debates’s mission of raising awareness of imminent existential risk from superintelligent AI.

00:00 - Teaser

00:34 - Carl Feynman’s Background

02:40 - Early Concerns About AI Doom

03:46 - Eliezer Yudkowsky and the Early AGI Community

05:10 - Accelerationist vs. Doomer Perspectives

06:03 - Mainline Doom Scenarios: Gradual Disempowerment vs. Foom

07:47 - Timeline to Doom: Point of No Return

08:45 - What’s Your P(Doom)™

09:44 - Public Perception and Political Awareness of AI Risk

11:09 - AI Morality, Alignment, and Chatbots Today

13:05 - The Alignment Problem and Competing Values

15:03 - Can AI Truly Understand and Value Morality?

16:43 - Multiple Competing AIs and Resource Competition

18:42 - Alignment: Wanting vs. Being Able to Help Humanity

19:24 - Scenarios of Doom and Odds of Success

19:53 - Mainline Good Scenario: Non-Doom Outcomes

20:27 - Heaven, Utopia, and Post-Human Vision

22:19 - Gradual Disempowerment Paper and Economic Displacement

23:31 - How Humans Get Edged Out by AIs

25:07 - Can We Gaslight Superintelligent AIs?

26:38 - AI Persuasion & Social Influence as Doom Pathways

27:44 - Riding the Doom Train: Headroom Above Human Intelligence

29:46 - Orthogonality Thesis and AI Motivation

32:48 - Alignment Difficulties and Deception in AIs

34:46 - Elon Musk, Maximal Curiosity & Mike Israetel’s Arguments3

6:26 - Beauty and Value in a Post-Human Universe

38:12 - Multiple AIs Competing

39:31 - Space Colonization, Dyson Spheres & Hanson’s “Alien Descendants”

41:13 - What Counts as Doom vs. Not Doom?

43:29 - Post-Human Civilizations and Value Function

44:49 - Expertise, Rationality, and Doomer Credibility

46:09 - Communicating Doom: Missing Mood & Public Receptiveness

47:41 - Personal Preparation vs. Belief in Imminent Doom48:56 - Why Can't We Just Hit the Off Switch?

50:26 - The Treacherous Turn and Redundancy in AI

51:56 - Doom by Persuasion or Entertainment

53:43 - Differences with Eliezer Yudkowsky: Singleton vs. Multipolar Doom

55:22 - Why Carl Chose Doom Debates

56:18 - Liron’s Outro

Show Notes

Carl’s Twitter — https://x.com/carl_feynman

Carl’s LessWrong — https://www.lesswrong.com/users/carl-feynman

Gradual Disempowerment — https://gradual-disempowerment.ai

The Intelligence Curse — https://intelligence-curse.ai

AI 2027 — https://ai-2027.com

Alcor cryonics — https://www.alcor.org

The LessOnline Conference — https://less.online

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Richard Hanania is the President of the Center for the Study of Partisanship and Ideology. His work has been praised by Vice President JD Vance, Tyler Cowen, and Bryan Caplan among others.

In his influential newsletter, he’s written about why he finds AI doom arguments unconvincing. He was gracious enough to debate me on this topic. Let’s see if one of us can change the other’s P(Doom)!

0:00 Intro

1:53 Richard's politics

2:24 The state of political discourse

3:30 What's your P(Doom)?™

6:38 How to stop the doom train

8:27 Statement on AI risk

9:31 Intellectual influences

11:15 Base rates for AI doom

15:43 Intelligence as optimization power

31:26 AI capabilities progress

53:46 Why isn't AI yet a top blogger?

58:02 Diving into Richard's Doom Train

58:47 Diminishing Returns on Intelligence

1:06:36 Alignment will be relatively trivial

1:15:14 Power-seeking must be programmed

1:21:27 AI will simply be benevolent

1:27:17 Superintelligent AI will negotiate with humans

1:33:00 Super AIs will check and balance each other out

1:36:54 We're mistaken about the nature of intelligence

1:41:46 Summarizing Richard's AI doom position

1:43:22 Jobpocalypse and gradual disempowerment

1:49:46 Ad hominem attacks in AI discourse

Show Notes

Subscribe to Richard Hanania's Newsletter: https://richardhanania.com

Richard's blogpost laying out where he gets off the AI "doom train": https://www.richardhanania.com/p/ai-doomerism-as-science-fiction

Richard's interview with Steven Pinker: https://www.richardhanania.com/p/pinker-on-alignment-and-intelligence

Richard's interview with Robin Hanson: https://www.richardhanania.com/p/robin-hanson-says-youre-going-to

My Doom Debate with Robin Hanson: https://www.youtube.com/watch?v=dTQb6N3_zu8

My reaction to Steven Pinker's AI doom position, and why his arguments are shallow: https://www.youtube.com/watch?v=-tIq6kbrF-4

"The Betterness Explosion" by Robin Hanson: https://www.overcomingbias.com/p/the-betterness-explosionhtml

---

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/watch?v=9CUFbqh16Fg

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at https://DoomDebates.com and to https://youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Emmett Shear is the cofounder and ex-CEO of Twitch, ex-interim-CEO of OpenAI, and a former Y Combinator partner. He recently announced Softmax, a new company researching a novel solution to AI alignment.

In his recent interview, Emmett explained “organic alignment”, drawing comparisons to biological systems and advocating for AI to be raised in a community-like setting with humans.

Let’s go through his talk, point by point, to see if Emmett’s alignment plan makes sense…

00:00 Episode Highlights

00:36 Introducing Softmax and its Founders

01:33 Research Collaborators and Ken Stanley's Influence

02:16 Softmax's Mission and Organic Alignment

03:13 Critique of Organic Alignment

05:29 Emmett’s Perspective on AI Alignment

14:36 Human Morality and Cognitive Submodules

38:25 Top-Down vs. Emergent Morality in AI

44:56 Raising AI to Grow Up with Humanity

48:43 Softmax's Incremental Approach to AI Alignment

52:22 Convergence vs. Divergence in AI Learning

55:49 Multi-Agent Reinforcement Learning

01:12:28 The Importance of Storytelling in AI Development

01:16:34 Living With AI As It Grows

01:20:19 Species Giving Birth to Species

01:23:23 The Plan for AI's Adolescence

01:26:53 Emmett's Views on Superintelligence

01:31:00 The Future of AI Alignment

01:35:10 Final Thoughts and Criticisms

01:44:07 Conclusion and Call to Action

Show Notes

Emmett Shear’s interview on BuzzRobot with Sophia Aryan (source material) — https://www.youtube.com/watch?v=_3m2cpZqvdw

BuzzRobot’s YouTube channel — https://www.youtube.com/@BuzzRobot

BuzzRobot’s Twitter — https://x.com/buZZrobot/

SoftMax’s website — https://softmax.com

My Doom Debate with Ken Stanley (Softmax advisor) — https://www.youtube.com/watch?v=GdthPZwU1Co

My Doom Debate with Gil Mark on whether aligning AIs in groups is a more solvable problem — https://www.youtube.com/watch?v=72LnKW_jae8

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com

Prof. Scott Sumner is a well-known macroeconomist who spent more than 30 years teaching at Bentley University, and now holds an Emeritus Chair in monetary policy at George Mason University's Mercatus Center. He's best known for his blog, the Money Illusion, which sparked the idea of Market Monetarism and NGDP targeting.

I sat down with him at LessOnline 2025 to debate why his P(Doom) is pretty low. Where does he get off the Doom Train? 🚂

00:00 Episode Preview

00:34 Introducing Scott Sumner

05:20 Is AGI Coming Soon?

09:12 Potential of AI in Various Fields

36:49 Ethical Implications of Superintelligent AI

41:03 The Nazis as an Outlier in History

43:36 Intelligence and Morality: The Orthogonality Thesis

49:03 The Risk of Misaligned AI Goals

01:09:31 Recapping Scott’s Position

Show Notes

Scott’s current blog, The Pursuit of Happiness:

Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk!

PauseAI, the volunteer organization I’m part of: https://pauseai.info

Join the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com