Kevin Zhu walked away from a lucrative quant career to build Algoverse, one of the most productive mentorship programs for aspiring AI researchers. Can we agree on the best research path to a safe future? Let’s take a ride on the Doom Train!

00:00:00 — Cold Open

00:00:58 — Introducing Kevin Zhu

00:02:10 — From Citadel Quant to AI Researcher

00:09:14 — The Story of Founding AlgoVerse

00:12:53 — Discovering AI Safety: LessWrong & ARENA

00:17:22 — Emergent Misalignment Research

00:22:37 — Yudkowsky, MIRI & “Intellidynamics”

00:26:50 — What’s Your P(Doom)?™

00:29:37 — Kevin’s Timeline to AGI + AI 2027

00:30:42 — Would You Slow Down AI?

00:37:44 — Coming Out of the P(Doom) Closet

00:45:49 — Should We Shame AI Company Workers?

00:52:27 — OpenAI’s Superalignment Team Collapse

00:55:01 — Riding the Doom Train™

00:55:46 — First Stop: Instrumental Convergence

01:04:58 — Does Kevin Agree with the Orthogonality Thesis?

01:07:08 — “It’s Just Math.” Just Unplug It.

01:08:28 — “We Have a Safe Development Process”

01:11:49 — Group Dynamics & Laws Will Save Us

01:13:44 — Superintelligence Will Spare Us

01:14:25 — Is P(Doom) Just Bad Epistemology?

01:15:53 — China Will Race No Matter What

01:17:45 — Maybe Human Extinction Is Good?

01:20:36 — Wrap-Up

Links

Algoverse AI Research — https://algoverseairesearch.org/

Kevin Zhu on Instagram — https://www.instagram.com/kevinzhu.ai/reels/

Emergent Misalignment (Betley, Owain Evans et al.) — https://arxiv.org/abs/2502.17424

Emergent Misalignment via In-Context Learning (Algoverse follow-up, ACL 2025) — https://arxiv.org/abs/2510.11288

Agentic Misalignment (the blackmail result) — https://www.anthropic.com/research/agentic-misalignment

LessWrong — https://www.lesswrong.com/

PauseAI (global) — https://pauseai.info/

OpenAI Superalignment — disbanded (CNBC, May 2024) — https://www.cnbc.com/2024/05/17/openai-superalignment-sutskever-leike.html

Evan Hubinger — “Why I’m joining Anthropic” — https://www.alignmentforum.org/posts/7jn5aDadcMH6sFeJe/why-i-m-joining-anthropic

AI 2027 — https://ai-2027.com/

Bentham’s Bulldog / Matthew Adelstein on Doom Debates — https://doomdebates.com/p/benthams-bulldog-ai-doom-debate

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Liron explains the 80-year-old Erdős conjecture that a GPT model breakthrough, Scott Aaronson ponders "the last days of human relevance”, multiple live callers debate me, plus my 7-year-old son drops in to share his P(Doom)!

Timestamps

00:00:00 — Cold Open

00:00:42 — First Donation & “Why Is a Duck?”

00:02:54 — Can AI Draw “Colorless Green Ideas Sleep Furiously”?

00:08:26 — Doom Debates Sponsors Less Online — Be Our Intern

00:10:43 — Who We Still Need to Get on the Show

00:26:44 — Claude Opus 4.8 & the Rising Waterline of Intelligence

00:28:48 — The 80-Year-Old Geometry Conjecture a GPT Model Cracked

00:44:50 — My 7-Year-Old Ezra Joins: ChatGPT, Minecraft & His P(Doom)

00:58:46 — How the AI Actually Beat Erdős’s Grid

01:03:50 — Scott Aaronson: “The Last Days of Human Relevance”

01:07:53 — Why the Foom Is Taking Years, Not Hours

01:10:47 — Is Ori Just a Yes Man?

01:12:41 — Does P = NP + AI?

01:23:43 — METR’s Beth Barnes: “We Are Not On Top Of It”

01:28:31 — Liron’s Vibe Coding Confession

01:31:21 — Alverin Joins: Can We Wait, Then Hit the Off Switch?

01:48:01 — AI in a Box & the Super-Persuasion Threshold

02:06:24 — Brian Joins: The Ways It Could Go Right

02:11:30 — Jack Joins: How Fast Will the Foom Be?

02:18:48 — 80,000 Hours, Inventing Erdős Problems & Holly Elmore’s Warning

02:29:42 — Wrap-Up

Links

LessOnline 2026 — June 5–7, Berkeley — https://less.online/

Manifest 2026 — June 12–14 — https://manifest.is/

Shtetl-Optimized — “Dispatches from the possibly last days of human relevance” — https://scottaaronson.blog/?p=9782

Global Call for AI Red Lines (Bengio, Hinton, Harari) — red-lines.ai

Order the 80,000 Hours book — Benjamin Todd (Penguin) — https://80000hours.org/book/

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

***ATTENTION: We’re looking for paid help at LessOnline from June 5-7. Apply now!***

OpenAI pushes the frontier of mathematics, Anthropic hires Andrej Karpathy to work on recursive self-improvement, live callers challenge the doom argument, and we've got a soundboard!

Doom Debates Paid Gig in Berkeley!

Doom Debates is proudly sponsoring The LessOnline conference this year and we got a merch table in the courtyard! We need 2 people to help represent the show — hand out T-shirts, talk to attendees, and spread the good word about Doom Debates. If you’re a regular viewer of the show, you probably know enough. Just be knowledgeable, friendly and helpful!

* 💰We pay you $500 to help out during the entire Fri-Sun event

* 🎟️ Plus you get FREE admission to the conference ($675 value)

This is best suited for someone local to the SF Bay Area or who can arrange accommodations. If you’re interested, send an email to [email protected] or DM Producer Ori on Discord. Tell us a little about yourself and your availability for all 3 days. Thanks!

Links

Less Online conference (June 5–7, Berkeley) — https://less.online

Doom Debates Discord — https://discord.gg/g2X8h5UCrU

OpenAI — “An OpenAI model has disproved a central conjecture in discrete geometry” — https://openai.com/index/model-disproves-discrete-geometry-conjecture/

Timestamps

00:00:00 — Welcome + New Soundboard!

00:09:27 — Doom Debates Is Sponsoring Less Online

00:17:30 — Ben Goertzel Episode Recap

00:27:30 — Goertzel's Alignment Blind Spot

00:34:21 — OpenAI Disproves an 80-Year-Old Erdős Conjecture

00:41:00 — David Deutsch Says "Amazing"

00:50:31 — Eliezer Yudkowsky's Job Replacement Progression

00:54:07 — Anthropic's $44 Billion Run Rate

01:07:45 — Schadenfreude Week: Liron Loses $10K

01:13:04 — Teaching Ezra Prediction Markets

01:18:11 — Caller Williawa: Is the Goal Engine Argument Effective?

01:37:00 — Caller Will: Brain Rot and Idiocracy

01:55:04 — Anthropic Hires Karpathy for Recursive Self-Improvement

01:57:16 — Rob Wiblin Says Recursive Self-Improvement Should be "Illegal"

02:01:37 — Caller Henry: The Chicken Riddle

02:08:41 — David Sacks Kills the AI Executive Order

02:11:20 — The Philanthropy Tidal Wave

02:15:56 — Donate to Doom Debates

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Now that Doom Debates is almost 2 years old, enough time has passed that we can revisit my claims from 2024. I claim they’ve been aging well!

In this early episode, I react to Martin Casado, a General Partner at Andreessen Horowitz (a16z) who claims that AI is basically just a buzzword for statistical models and simulations. As a result of this worldview, he only predicts incremental AI progress that doesn’t pose an existential threat to humanity, and he sees AI regulation as a net negative.

From my perspective, Martin’s problem is that he needs to go beyond analyzing AI as just statistical models and simulations, and analyze it using the more predictive concept of “intelligence” in the sense of hitting tiny high-value targets in exponentially-large search spaces.If Martin appreciated that intelligence is a quantifiable property that algorithms have, and that our existing AIs are getting close to surpassing human-level general intelligence, then hopefully he’d come around to raising his P(doom) and appreciating the urgent extinction risk we face.

Links

Watch the original episode of the Cognitive Revolution podcast with Martin and host Nathan Labenz — https://www.youtube.com/watch?v=oZ7788oKoss

Follow Martin — https://x.com/martin_casado

Follow Nate — https://x.com/labenz

Follow Liron — https://x.com/liron

Timestamps

00:00 Introducing Martin Casado

01:42 Martin’s AGI Timeline

05:39 Martin’s Analysis of Self-Driving Cars

15:30 Heavy-Tail Distributions

38:03 Understanding General Intelligence

38:29 AI's Progress in Specific Domains

43:20 AI’s Understanding of Meaning

47:16 Compression and Intelligence

48:09 Symbol Grounding

53:24 Human Abstractions and AI

01:18:18 The Frontier of AI Applications

01:23:04 Human vs. AI: Concept Creation and Reasoning

01:25:51 The Complexity of the Universe and AI's Limitations

01:28:16 AI's Potential in Biology and Simulation

01:32:40 The Essence of Intelligence and Creativity in AI

01:41:13 AI's Future Capabilities

02:00:29 Intelligence vs. Simulation

02:14:59 AI Regulation

02:23:05 Concluding Thoughts

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Dr. Ben Goertzel is a pioneering AI researcher, entrepreneur, and author who has helped organize AGI as a research field since the 2000s through books, workshops, and conferences.

We talk about the early days of AGI research with Yudkowsky, get Ben’s timeline to the singularity, and I try to understand what makes him so optimistic that superintelligence will be beneficial for humanity.

Timestamps

00:00:00 — Cold Open

00:00:59 — Introducing Dr. Ben Goertzel

00:04:11 — How Did Ben Get Interested in AGI?

00:08:26 — Early AGI Enthusiast Community: Extropians & SL4

00:15:15 — Is There Headroom Above Human Intelligence?

00:19:53 — Can Superintelligent AI Kill Everyone?

00:28:25 — Ben’s AGI Timeline: 2029

00:36:00 — What Modern AI Is Missing

00:42:01 — AGI to Superintelligence — FOOM in a Matter of Months

00:46:34 — What’s Your P(Doom)?™

00:53:37 — Ben’s Case for AI Acceleration

01:03:50 — The Orthogonality Thesis Is “Dishonest”

01:13:43 — Liron: Moore’s Law for Goal Achievement

01:23:50 — Ben: Alignment Is Just Good Parenting

01:27:36 — Alignment in Practice: OmegaCLAW

01:35:47 — Creating Stable Goals Under Self-Modification

01:45:22 — SingularityNET, OpenBGI, and the ASI Alliance

01:57:31 — The $ASI Token Economy

02:01:15 — Closing

Links

Ben Goertzel on X — https://x.com/bengoertzel

SingularityNET — https://singularitynet.io/

OpenBGI — https://openbgi.com/

Hyperon AGI Framework — https://hyperon.dev/

OmegaCLAW on GitHub — https://github.com/asi-alliance/OmegaClaw-Core

ASI Alliance — https://superintelligence.io/

The Consciousness Explosion (book) — https://theconsciousnessexplosion.ai/

“The Singularity Institute’s Scary Idea (and Why I Don’t Buy It)” (2010) — http://multiverseaccordingtoben.blogspot.com/2010/10/singularity-institutes-scary-idea-and.html

Ken Stanley Debate — https://lironshapira.substack.com/p/debate-with-a-former-openai-research

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

We recap the Dr. Mike Israetel debate and dig into the nuances of rationality vs. rationalization. Then onto the week's biggest news: Anthropic dethroning OpenAI, Trump and Xi Jinping exploring AI guardrails, and Liron's own company AI going rogue.

Timestamps

00:00:00 — Cold Open

00:03:09 — Dr. Mike Israetel Round 2 Recap

00:14:34 — A Viewer Accuses Liron of Rationalizing

00:34:20 — Is Sam Altman Any Worse than a Typical Tech CEO?

00:45:15 — What Would Lower P(Doom) Below 10%?

00:53:25 — Can We Disconnect an ASI from the Internet?

01:05:00 — Should Doomers Boycott AI?

01:16:27 — Liron Reacts to ThePrimeagen Reacting to the Yud Debate

01:26:13 — Ben Goertzel Episode Preview

01:38:10 — Scrolling Twitter: MIRI, Ryan Greenblatt

01:47:32 — Liron’s AI Goes Rogue at Work

01:49:15 — “Positive Alignment” Paper Gets Destroyed

01:58:30 — Anthropic Dethrones OpenAI

02:12:06 — Trump & Xi Talk AI Guardrails

Links

Zvi Mowshowitz AI roundup blog — https://thezvi.wordpress.com/

“The Bottom Line” by Eliezer Yudkowsky (LessWrong) — https://www.lesswrong.com/posts/34XxbRFe54FycoCDw/the-bottom-line

Rationality: From AI to Zombies by Eliezer Yudkowsky — https://intelligence.org/rationality-ai-zombies/

Less Online 2026 conference (June 5–7) — https://less.online/

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

In December 2023, I joined the Theo Jaffee Podcast to talk about a wide range of topics including non-AI x-risks, where I disagree with Eliezer Yudkwowsky, cryptocurrency and if P(Doom) is rigorous. This episode was recorded 6 months before I'd started Doom Debates. Let's look back with the power of hindsight and see if my answers still hold up.

Links

#10: Liron Shapira — AI doom, FOOM, rationalism, and crypto (YouTube) — https://youtu.be/YfEcAtHExFM

Watch Theo Jafee now on MTS — https://www.mts.now/

MTS Live — https://x.com/MTSlive

Timestamps

00:00:00 — Theo's Introduction

00:01:03 — Is Liron Worried About Non-AI Existential Risks

00:03:36 — Suffering Risks

00:05:22 — Is P(Doom) rigorous?

00:09:42 — Is Eliezer Overestimating P(Doom)?

00:12:18 — Where Does Liron Disagree with Eliezer?

00:15:43 — What Would Change Liron's Mind

00:17:14 — Elon Musk's AI Timelines

00:18:34 — Is xAI Making Things Worse?

00:20:24 — The Case for an AI Manhattan Project

00:22:32 — Are Superforecasters Wrong About AI Risk?

00:26:02 — The Race Against Time for Alignment

00:28:01 — Headroom Above Human Intelligence

00:33:31 — Vitalik's d/acc Framework

00:35:23 — Edging Toward Superintelligence

00:38:21 — From Chatbot to World-Ender

00:41:55 — GPT Paradigm vs. AlphaZero

00:43:18 — Critiquing AI Optimism

00:48:29 — Deceptive Alignment and Gradient Descent

00:53:31 — Does Nice Training Data Make Nice AI?

00:57:57 — How Do You Live with 50% P(Doom)?

01:00:08 — Why Have Kids If the World Might End?01:02:16 — Israel vs Hamas

01:06:15 — How LessWrong Changed Liron's Lif

e01:08:42 — Rationalism and Effective Altruism

01:14:49 — Why Blockchain Has No Use Case Beyond Cryptocurrency

01:22:13 — Charlie Munger and Richard Feynman

01:24:38 — Closing

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Dr. Mike Israetel is back in the arena to FIGHT ME (verbally) about the likelihood of AI killing everyone vs. the likelihood of AI making the universe totally awesome!

Back by popular demand after last year’s widely-viewed debate, the renowned exercise scientist, fitness entrepreneur, and low-P(Doom) AI futurist reveals where he gets off the Doom Train™, why he thinks AI is conscious, and how he pictures the coming AI utopia.

Links

Watch Dr. Mike’s AI/futurism YouTube channel —https://www.youtube.com/@mikeisraetelmakingprogress

Dr. Mike on X — https://x.com/misraetel

Make sure to watch Round 1 of our debate from last year — https://www.youtube.com/watch?v=RaDWSPMdM4o

Liron referenced a recent episode, “Steven Byrnes Returns — 90% P(Doom) on the Trajectory to ASI” — https://www.youtube.com/watch?v=oOb9K1KIAyk

Timestamps

00:00:00 — Cold Open

00:00:48 — Returning Champion Dr. Mike Israetel!

00:02:13 — What's Been Your Biggest AI Productivity Gain?

00:03:46 — Can AI Be a Personal Trainer? Dr. Mike Judges Liron's Technique

00:09:41 — Get Good at Taking Advice From AI

00:14:23 — Dr. Mike Israetel, What's Your Latest P(Doom)?™

00:20:12 — Liron's Mainline Scenario: Loss of Control

00:21:39 — Mike's Problems With a Paperclip Maximizer Scenario

00:26:00 — What Will the Goals of a Superintelligence Be?

00:28:49 — AI Labs Have Bit Off More Than They Can Chew

00:31:09 — Current AIs Are Moral

00:34:40 — AIs With a Ruthless Game-Playing Flavor

00:38:48 — Introducing the Doom Train

00:45:07 — "Can It" vs. "Will It" — Two-Part Argument on AI X-Risk

00:49:40 — Mike's Tweet on Superintelligence by 2027

01:01:34 — AI as a Goal Engine

01:20:51 — Offense vs. Defense in the Coming Cyber War

01:39:14 — Is Morality Just Game Theory? The "Different Flavor" of Future AI

01:46:16 — Ricardo's Law: Does AI Have Reason to Keep Us Around?

01:52:14 — Why Mike Gets Off the Doom Train

01:54:58 — Is AI Conscious?

02:01:39 — Mike's Vision for the Coming AI Utopia

02:06:52 — Wrap-Up

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Liron and Producer Ori explore the community's feedback to Eliezer Yudkowsky's $10,000 debate against an anonymous AI director. Plus, we unpack Eliezer's new post on the "irretrievability" of ASI development, Anthropic feasting on xAI's compute, and Liron's 4x Kalshi bet on... Spencer Pratt.

Timestamps

00:00:00 — Welcome

00:02:30 — The $10,000 Debate Post-Mortem

00:06:01 — Paul Tudor Jones: “Zero Risk Management on AI”

00:09:49 — Liron’s Jerry Springer Moment

00:14:19 — Why Yud Wore Steampunk

00:20:19 — YouTube Reacts: “Five Minutes In, Already Unhinged”

00:27:19 — 47F’s Anti-Disparagement Legal Theory

00:32:18 — Yud Wants Round 2

00:35:37 — Lumpin Space, Ben Goertzel & Early AI Safety Memories

00:47:06 — Eliezer’s “Irretrievability” Post

00:56:35 — The Maginot Line and Murphy’s Curse

01:08:13 — Blockchain vs AI: Earth’s Compute Swing

01:15:15 — Anthropic Eats Elon’s Compute

01:25:29 — Is Elon Tweeting as His Own Mom?

01:30:03 — Waymo Dodges a Fallen Scooter Rider

01:33:00 — Why Liron Bet on Spencer Pratt

01:37:13 — Live Twitter Scroll!

01:38:11 — Mira Murati: “Directionally Very Bad”

01:50:07 — AI Copies Itself Across Servers

01:56:44 — Steven Byrnes: LLMs Aren’t the Final Paradigm

02:07:55 — Wrap-Up

Links

LessOnline 2026 (June 5-7, Berkeley, CA) — https://less.online/

Eliezer Yudkowsky, “Irretrievability; or, Murphy’s Curse of Oneshotness upon ASI” (LessWrong, May 4, 2026) —https://www.lesswrong.com/posts/fbrz9xhKpEeTKw5zL/irretrievability-or-murphy-s-curse-of-oneshotness-upon-asi

Anthropic, SpaceX announce Colossus 1 compute deal (CNBC) — https://www.cnbc.com/2026/05/06/anthropic-spacex-data-center-capacity.html

SpaceXAI announcement: Compute partnership with Anthropic — https://x.ai/news/anthropic-compute-partnership

Kalshi: Spencer Pratt LA Mayor market — https://kalshi.com/markets/kxmayorla/la/kxmayorla-26

Morris worm — Wikipedia — https://en.wikipedia.org/wiki/Morris_worm

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Claude, also known as @lumpenspace on X, is a prominent AI accelerationist. He gives humanity a 30% chance of being superseded by superintelligence, and he’s fine with it! We unpack his lattice of beliefs, and pinpoint the cruxes of our disagreement on the orthogonality thesis and the capabilities of superintelligence.

Lumpenspace’s appearance comes after a Molotov cocktail was thrown at Sam Altman’s house, an act that he and I completely condemn.

Timestamps

00:00:00 — Cold Open

00:00:57 — Introducing Lumpenspace

00:03:46 — Is Lumpenspace Team Beff Jezos?

00:04:26 — What’s Your P(Doom)?™

00:06:28 — Worthy Successors & the Transhumanist Door

00:08:29 — The Orthogonality Thesis: Our Core Disagreement

00:15:14 — The Universality Threshold & David Deutsch

00:19:22 — Paperclips Won’t Happen Because “It’s Boring”

00:27:23 — Natural Selection vs Human Engineering

00:36:26 — Nanobots: Can ASI Build Them Unseen?

00:46:49 — Identifying the Cruxes of Disagreement

00:55:01 — Closing Statements

Links

Follow Lumpenspace — https://x.com/lumpenspace

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏