Doom Debates!

The Eliezer Yudkowsky interview premiere is tomorrow (Mon Sep 15) at 9am PT!!!

👉 https://www.youtube.com/watch?v=wQtpSQmMNP0 👈

I can't believe it. We are entering the launch week for If Anyone Builds It, Everyone Dies - a new book by Eliezer Yudkowsky and Nate Soares. This is a hell of a book. I highly recommend it.

Everybody should pick it up. Honestly, if you haven't gone and bought that book by now, not gonna lie, I'm kind of disappointed. Should you really even be watching this channel? Are you not getting the message that it's critical for you to buy this book?

It's going to be on the New York Times bestseller list, and the only question is, what position will it be? That's going to depend on how many people like you take action. And by action, I mean, you know, pay $14.99, get it on Kindle. Really do your part. It's not that much.

Once you've done that, remember, tomorrow something very special is happening on my personal channel - the long awaited interview between me and Eliezer Yudkowsky! We've all been waiting so long for me to talk to Eliezer. It finally happened as part of his book tour.

Technically, it's not part of Doom Debates, it's just on my own channel because it's not branded as a Doom thing. It's just a Yudkowsky interview. So you're not gonna see me ask P(Doom). That's not gonna be part of it. We're just going to talk in terms of existential risk and looking at dangers. That's the preferred terminology here. There's a difference of opinion between him and me, and we gotta respect that.

So action item for you. Once you've bought that book, "If Anyone Builds It, Everyone Dies," head over to the link in the show notes for this post or in the description for this post. There's going to be a link to a YouTube premiere. That YouTube premiere is the Eliezer Yudkowsky interview, which is happening 9:00 AM tomorrow, Monday, September 15th.

That's really not that long after you're listening to this post. We're talking hours away. You want to be subscribed to that premiere. You don't want to miss it because it's not going to be a post in the Doom Debates feed. It's going to be its own YouTube premiere on my personal channel.

So once again, check out those show notes. Check out the description to what you're watching right now. Click on that link and bookmark the premiere episode because I'm going to be there live in the live chat watching my own episode. Just like you're watching the episode, my producer Ori is going to be there live watching the episode. A bunch of your fellow Doom Debates fans are going to be in the live chat watching the episode.

This is a really special event, both for Eliezer Yudkowsky and MIRI, and for Doom Debates and for the human population. At least the human population can say that there was a New York Times bestseller calling out what's about to happen to all of us if we don't stop it. I think it's a very special week.

I hope you'll help it be as high profile as possible so that the algorithms will take notice and society will take notice and a grassroots movement will take notice and our politicians and other leaders will take notice.

All right. Signing off for now. I'll see you all in the YouTube premiere of the Eliezer Yudkowsky interview. Thanks for watching!

---

P.S. This post also has more details about Tuesday’s Doom Debates launch party!

In this special cross-post from Jona Ragogna’s channel, I'm interviewed about why superintelligent AI poses an imminent extinction threat, and how AI takeover is going to unfold.

Newly exposed to AI x-risk, Jona asks sharp questions about why we’re racing toward superintelligence despite the danger, and what ordinary people can do now to lower p(doom). This is one of the most crisp explainers of the AI-doom argument I’ve done to date.

Timestamps

0:00 Intro

0:41 Why AI is likely to cause human extinction

2:55 How AI takeover happens

4:55 AI systems have goals

6:33 Liron explains p(Doom)

8:50 The worst case scenario is AI sweeps us away

12:46 The best case scenario is hard to define

14:24 How to avoid doom

15:09 Frontier AI companies are just doing "ad hoc" alignment

20:30 Why "warning shots" from AI aren't scary yet

23:19 Should young adults work on AI alignment research?

24:46 We need a grassroots movement

28:31 Life choices when AI doom is imminent

32:35 Are AI forecasters just biased?

34:12 The Doom Train™ and addressing counterarguments

40:28 Anthropic's new AI welfare announcement isn't a major breakthrough

44:35 It's unknown what's going on inside LLMs and AI systems

53:22 Effective Altruism's ties to AI risk

56:58 Will AI be a "worthy descendant"?

1:01:08 How to calculate P(Doom)

1:02:49 Join the unofficial If Anyone Builds It, Everyone Dies book launch party!

Show Notes

Subscribe to Jona Ragogna — https://youtube.com/@jonaragogna

IF ANYONE BUILDS IT LAUNCH WEEK EVENTS:

Mon Sep 15 @ 9am PT / 12pm ET / 1600 UTCMy Eliezer Yudkowsky premieres on YouTube! Stay tuned for details.

Tue Sep 16 @ 2pm PT / 5pm ET / 2100 UTCThe Doom Debates unofficial IABI Launch Party!!!

More details about launch week HERE!

---

Doom Debates’s Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Ladies and gentlemen, we are days away from the long-awaited release of Eliezer Yudkowsky and Nate Soares's new book, “If Anyone Builds It, Everyone Dies”!!!

Mon Sep 15 @9am PT: My Interview with Eliezer Yudkowsky

We'll be kicking things off the morning of Monday, September 15th with a live watch party of my very special new interview with the one and only Eliezer Yudkowsky!

All of us will be in the YouTube live chat. I'll be there, producer Ori will be there, and you'll get a first look at this new & exciting interview: Questions he's never been asked before.

We'll talk about my history first meeting Eliezer in 2008, how things have evolved, what's going on now, why everybody has their head in the sand. And we'll even go to the LessWrong sequences and do some rationality deep cuts.

This will be posted on my personal YouTube channel. But if you're subscribed to Doom Debates, you'll get the link in your feed as we get closer to Monday.

Mark your calendars for Monday, September 15th, 9:00am Pacific, 12:00 PM Eastern. That's 1600 UTC for my Europals out there, midnight for my China peeps.

I think YouTube commenter @pairy5420 said it best: "Bro, I could have three essays due, my crush just text me, the FBI outside my house, me having a winning lottery ticket to turn in all at the same time, and I still wouldn't miss the interview."

That's the spirit, @pairy5420. You won't want to miss the Eliezer interview YouTube premiere. I'll see you all there. It's gonna be a great start to the book's launch week.

Tue Sep 16 @2pm PT: The Doom Debates Unofficial Book Launch Party

Now let's talk about the launch day: Tuesday, September 16th. Once you've picked up your copy of the book, get ready for the Doom Debates unofficial "If Anyone Builds It, Everyone Dies" launch party!!!

This is going to be a very special, unprecedented event here on this channel. It's gonna be a three hour live stream hosted by me and producer Ori, our new full-time producer. He's gonna be the co-host of the party.

He’s finally coming out from behind the scenes, making his debut on the channel. Unless you count the first episode of the show - he was there too. The two of us are going to be joined by a who's who of very special guests. Get a load of these guests who are gonna be dropping by the unofficial launch party:

* John Sherman from the AIX Risk Network and Michael, my other co-host of Warning Shots

* Roman Yampolskiy, a top researcher and communicator in the field of AI x-risk

* Emmett Shear, founder and CEO of a new AI alignment company called Softmax. He was previously the CEO of Twitch, and the interim CEO of OpenAI

* Roon, member of the technical staff at OpenAI and friend of the show

* Gary Marcus, cognitive scientist, author, entrepreneur, and famous AGI skeptic

* Liv Boeree, the YouTube star, poker champion, and science communicator

* Robert Wright, bestselling author, podcaster, wide-ranging intellectual, about announce his new book about AI

* Holly Elmore, executive director of PauseAI US

* Roko Mijic, you guys know Roko 😉

And that's not all. There's going to be even more that I can't tell you about now, but it will not disappoint.

So I really hope to see you all there at the unofficial "If Anyone Builds It, Everyone Dies" launch party, Tuesday, September 16th.

Same as the book’s launch day: Tuesday, September 16th at 2:00pm Pacific, 5:00pm Eastern.

Pick up your copy of the book that morning. Don't come to the party without it. We're gonna have a bouncer stationed at the door, and if you don't show him that you've got a copy of "If Anyone Builds It, Everyone Dies," he's gonna give you a big thumbs down.

BUY THE BOOK!!!

In all seriousness though, please support the book if you like Doom Debates. If you feel like you've gotten some value out of the show and you wanna give back a little bit, that is my ask. Head over to ifanyonebuildsit.com and buy the book from their links there. Go to Amazon, Barnes and Noble, wherever you normally buy books, just buy the damn thing. It's $14.99 on Kindle. It's not gonna break the bank.

Then spread the word. Tell your friends and family, tell your coworkers at the office. Try to get a few more copies sold. We don't have another book launch coming, guys. This is it.

This is our chance to take a little bit of action when it can actually move the needle and help. If you've been procrastinating this whole time, you gotta stop. You gotta go buy it now because the New York Times is gonna be checking this week.

This is the last week of pre-orders. You really want to give it that launch bump. Don't try to drag it out after launch week. Time is of the essence.

The Doom Debates Mission

Ultimately that's why I do this show. This isn't just entertainment for smart people. There is actually an important mission. We're trying to optimize the mission here. Help me out.

Or at the very least, help high quality discourse because a lot of people across the spectrum agree, this is a high quality book contributing to the discourse, and we need more books like it.

Thanks again for being with me on this journey to lower P(Doom) by convincing the average person that AI is urgently life-threatening to them and their loved ones. It's really important work.

See you all on Monday 9am PT at the Eliezer Yudkowsky interview (details coming soon), and Tuesday 2pm PT at the launch party (event link)!

Louis Berman is a polymath who brings unique credibility to AI doom discussions. He's been coding AI for 25 years, served as CTO of major tech companies, recorded the first visual sighting of what became the dwarf planet Eris, and has now pivoted to full-time AI risk activism. He's lobbied over 60 politicians across multiple countries for PauseAI and authored two books on existential risk.

Louis and I are both baffled by the calm, measured tone that dominates AI safety discourse. As Louis puts it: "No one is dealing with this with emotions. No one is dealing with this as, oh my God, if they're right. Isn't that the scariest thing you've ever heard about?"

Louis isn't just talking – he's acting on his beliefs. He just bought a "bug out house" in rural Maryland, though he's refreshingly honest that this isn't about long-term survival. He expects AI doom to unfold over months or years rather than Eliezer's instant scenario, and he's trying to buy his family weeks of additional time while avoiding starvation during societal collapse.

He's spent extensive time in congressional offices and has concrete advice about lobbying techniques. His key insight: politicians' staffers consistently claim "if just five people called about AGI, it would move the needle". We need more people like Louis!

Timestamps

* 00:00:00 - Cold Open: The Missing Emotional Response

* 00:00:31 - Introducing Louis Berman: Polymath Background and Donor Disclosure

* 00:03:40 - The Anodyne Reaction: Why No One Seems Scared

* 00:07:37 - P-Doom Calibration: Gary Marcus and the 1% Problem

* 00:11:57 - The Bug Out House: Prepping for Slow Doom

* 00:13:44 - Being Amazed by LLMs While Fearing ASI

* 00:18:41 - What’s Your P(Doom)™

* 00:25:42 - Bayesian Reasoning vs. Heart of Hearts Beliefs

* 00:32:10 - Non-Doom Scenarios and International Coordination

* 00:40:00 - The Missing Mood: Where's the Emotional Response?

* 00:44:17 - Prepping Philosophy: Buying Weeks, Not Years

* 00:52:35 - Doom Scenarios: Slow Takeover vs. Instant Death

* 01:00:43 - Practical Activism: Lobbying Politicians and Concrete Actions

* 01:16:44 - Where to Find Louis's Books and Final Wrap-up

* 01:18:17 - Outro: Super Fans and Mission Partners

Links

Louis’s website — https://xriskbooks.com — Buy his books!

ControlAI’s form to easily contact your representative and make a difference — https://controlai.com/take-action/usa — Highly recommended!

Louis’s interview about activism with John Sherman and Felix De Simone — https://www.youtube.com/watch?v=Djd2n4cufTM

If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares — https://ifanyonebuildsit.com

Become a Mission Partner!

Want to meaningfully help the show’s mission (raise awareness of AI x-risk & raise the level of debate around this crucial subject) become reality? Donate $1,000+ to the show (no upper limit) and I’ll invite you to the private Discord channel. Email me at [email protected] if you have questions or want to donate crypto.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Rob Miles is the most popular AI safety educator on YouTube, with millions of views across his videos explaining AI alignment to general audiences. He dropped out of his PhD in 2011 to focus entirely on AI safety communication – a prescient career pivot that positioned him as one of the field's most trusted voices over a decade before ChatGPT made AI risk mainstream.

Rob sits firmly in the 10-90% P(Doom) range, though he admits his uncertainty is "hugely variable" and depends heavily on how humanity responds to the challenge. What makes Rob particularly compelling is the contrast between his characteristic British calm and his deeply serious assessment of our situation. He's the type of person who can explain existential risk with the measured tone of a nature documentarian while internally believing we're probably headed toward catastrophe.

Rob has identified several underappreciated problems, particularly around alignment stability under self-modification. He argues that even if we align current AI systems, there's no guarantee their successors will inherit those values – a discontinuity problem that most safety work ignores. He's also highlighted the "missing mood" in AI discourse, where people discuss potential human extinction with the emotional register of an academic conference rather than an emergency.

We explore Rob's mainline doom scenario involving recursive self-improvement, why he thinks there's enormous headroom above human intelligence, and his views on everything from warning shots to the Malthusian dynamics that might govern a post-AGI world. Rob makes a fascinating case that we may be the "least intelligent species capable of technological civilization" – which has profound implications for what smarter systems might achieve.

Our key disagreement centers on strategy: Rob thinks some safety-minded people should work inside AI companies to influence them from within, while I argue this enables "tractability washing" that makes the companies look responsible while they race toward potentially catastrophic capabilities. Rob sees it as necessary harm reduction; I see it as providing legitimacy to fundamentally reckless enterprises.

The conversation also tackles a meta-question about communication strategy. Rob acknowledges that his measured, analytical approach might be missing something crucial – that perhaps someone needs to be "running around screaming" to convey the appropriate emotional urgency. It's a revealing moment from someone who's spent over a decade trying to wake people up to humanity's most important challenge, only to watch the world continue treating it as an interesting intellectual puzzle rather than an existential emergency.

Timestamps

* 00:00:00 - Cold Open

* 00:00:28 - Introducing Rob Miles

* 00:01:42 - Rob's Background and Childhood

* 00:02:05 - Being Aspie

* 00:04:50 - Less Wrong Community and "Normies"

* 00:06:24 - Chesterton's Fence and Cassava Root

* 00:09:30 - Transition to AI Safety Research

* 00:11:52 - Discovering Communication Skills

* 00:15:36 - YouTube Success and Channel Growth

* 00:16:46 - Current Focus: Technical vs Political

* 00:18:50 - Nuclear Near-Misses and Y2K

* 00:21:55 - What’s Your P(Doom)™

* 00:27:31 - Uncertainty About Human Response

* 00:31:04 - Views on Yudkowsky and AI Risk Arguments

* 00:42:07 - Mainline Catastrophe Scenario

* 00:47:32 - Headroom Above Human Intelligence

* 00:54:58 - Detailed Doom Scenario

* 01:01:07 - Self-Modification and Alignment Stability

* 01:17:26 - Warning Shots Problem

* 01:20:28 - Moving the Overton Window

* 01:25:59 - Protests and Political Action

* 01:33:02 - The Missing Mood Problem

* 01:40:28 - Raising Society's Temperature

* 01:44:25 - "If Anyone Builds It, Everyone Dies"

* 01:51:05 - Technical Alignment Work

* 01:52:00 - Working Inside AI Companies

* 01:57:38 - Tractability Washing at AI Companies

* 02:05:44 - Closing Thoughts

* 02:08:21 - How to Support Doom Debates: Become a Mission Partner

Links

Rob’s YouTube channel — https://www.youtube.com/@RobertMilesAI

Rob’s Twitter — https://x.com/robertskmiles

Rational Animations (another great YouTube channel, narrated by Rob) — https://www.youtube.com/RationalAnimations

Become a Mission Partner!Want to meaningfully help the show’s mission (raise awareness of AI x-risk & raise the level of debate around this crucial subject) become reality? Donate $1,000+ to the show (no upper limit) and I’ll invite you to the private Discord channel. Email me at [email protected] if you have questions or want to donate crypto.

Vitalik Buterin is the founder of Ethereum, the world's second-largest cryptocurrency by market cap, currently valued at around $500 billion. But beyond revolutionizing blockchain technology, Vitalik has become one of the most thoughtful voices on AI safety and existential risk.

He's donated over $665 million to pandemic prevention and other causes, and has a 12% P(Doom) – putting him squarely in what I consider the "sane zone" for AI risk assessment. What makes Vitalik particularly interesting is that he's both a hardcore techno-optimist who built one of the most successful decentralized systems ever created, and someone willing to seriously consider AI regulation and coordination mechanisms.

Vitalik coined the term "d/acc" – defensive, decentralized, democratic, differential acceleration – as a middle path between uncritical AI acceleration and total pause scenarios. He argues we need to make the world more like Switzerland (defensible, decentralized) and less like the Eurasian steppes (vulnerable to conquest).

We dive deep into the tractability of AI alignment, whether current approaches like DAC can actually work when superintelligence arrives, and why he thinks a pluralistic world of competing AIs might be safer than a single aligned superintelligence. We also explore his vision for human-AI merger through brain-computer interfaces and uploading.

The crux of our disagreement is that I think we're heading for a "plants vs. animals" scenario where AI will simply operate on timescales we can't match, while Vitalik believes we can maintain agency through the right combination of defensive technologies and institutional design.

Finally, we tackle the discourse itself – I ask Vitalik to debunk the common ad hominem attacks against AI doomers, from "it's just a fringe position" to "no real builders believe in doom." His responses carry weight given his credibility as both a successful entrepreneur and someone who's maintained intellectual honesty throughout his career.

Timestamps

* 00:00:00 - Cold Open

* 00:00:37 - Introducing Vitalik Buterin

* 00:02:14 - Vitalik's altruism

* 00:04:36 - Rationalist community influence

* 00:06:30 - Opinion of Eliezer Yudkowsky and MIRI

* 00:09:00 - What’s Your P(Doom)™

* 00:24:42 - AI timelines

* 00:31:33 - AI consciousness

* 00:35:01 - Headroom above human intelligence

* 00:48:56 - Techno optimism discussion

* 00:58:38 - e/acc: Vibes-based ideology without deep arguments

* 01:02:49 - d/acc: Defensive, decentralized, democratic acceleration

* 01:11:37 - How plausible is d/acc?

* 01:20:53 - Why libertarian acceleration can paradoxically break decentralization

* 01:25:49 - Can we merge with AIs?

* 01:35:10 - Military AI concerns: How war accelerates dangerous development

* 01:42:26 - The intractability question

* 01:51:10 - Anthropic and tractability-washing the AI alignment problem

* 02:00:05 - The state of AI x-risk discourse

* 02:05:14 - Debunking ad hominem attacks against doomers

* 02:23:41 - Liron’s outro

Links

Vitalik’s website: https://vitalik.eth.limo

Vitalik’s Twitter: https://x.com/vitalikbuterin

Eliezer Yudkowsky’s explanation of p-Zombies: https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies

—

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Today I’m sharing my interview on Robert Wright’s Nonzero Podcast from last May. Rob is an especially sharp interviewer who doesn't just nod along, he had great probing questions for me.

This interview happened right after Ilya Sutskever and Jan Leike resigned from OpenAI in May 2024, continuing a pattern that goes back to Dario Amodei leaving to start Anthropic. These aren't fringe doomers; these are the people hired specifically to solve the safety problem, and they keep concluding it's not solvable at the current pace.

00:00:00 - Liron’s preface

00:02:10 - Robert Wright introduces Liron

00:04:02 - PauseAI protests at OpenAI headquarters

00:05:15 - OpenAI resignations (Ilya Sutskever, Jan Leike, Dario Amodei, Paul Christiano, Daniel Kokotajlo)

00:15:30 - P vs NP problem as analogy for AI alignment difficulty

00:22:31 - AI pause movement and protest turnout

00:29:02 - Defining AI doom and sci-fi scenarios

00:32:05 - What’s My P(Doom)™

00:35:18 - Fast vs slow AI takeoff and Sam Altman's position

00:42:33 - Paperclip thought experiment and instrumental convergence explanation

00:54:40 - Concrete examples of AI power-seeking behavior (business assistant scenario)

01:00:58 - GPT-4 TaskRabbit deception example and AI reasoning capabilities

01:09:00 - AI alignment challenges and human values discussion

01:17:33 - Wrap-up and transition to premium subscriber content

Show Notes

This episode on Rob’s Nonzero Newsletter. You can subscribe for premium access to the last 1 hour of our discussion! — https://www.nonzero.org/p/in-defense-of-ai-doomerism-robert

This episode on Rob’s YouTube — https://www.youtube.com/watch?v=VihA_-8kBNg

PauseAI — https://pauseai.info

PauseAI US — http://pauseai-us.org

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Dr. Steven Byrnes, UC Berkeley physics PhD and Harvard physics postdoc, is an AI safety researcher at the Astera Institute and one of the most rigorous thinkers working on the technical AI alignment problem.

Steve has a whopping 90% P(Doom), but unlike most AI safety researchers who focus on current LLMs, he argues that LLMs will plateau before becoming truly dangerous, and the real threat will come from next-generation "brain-like AGI" based on actor-critic reinforcement learning.

For the last five years, he's been diving deep into neuroscience to reverse engineer how human brains actually work, and how to use that knowledge to solve the technical AI alignment problem. He's one of the few people who both understands why alignment is hard and is taking a serious technical shot at solving it.

We cover his "two subsystems" model of the brain, why current AI safety approaches miss the mark, his disagreements with social evolution approaches, and why understanding human neuroscience matters for building aligned AGI.

* 00:00:00 - Cold Open: Solving the technical alignment problem

* 00:00:26 - Introducing Dr. Steven Byrnes and his impressive background

* 00:01:59 - Steve's unique mental strengths

* 00:04:08 - The cold fusion research story demonstrating Steve's approach

* 00:06:18 - How Steve got interested in neuroscience through Jeff Hawkins

* 00:08:18 - Jeff Hawkins' cortical uniformity theory and brain vs deep learning

* 00:11:45 - When Steve first encountered Eliezer's sequences and became AGI-pilled

* 00:15:11 - Steve's research direction: reverse engineering human social instincts

* 00:21:47 - Four visions of alignment success and Steve's preferred approach

* 00:29:00 - The two brain subsystems model: steering brain vs learning brain

* 00:35:30 - Brain volume breakdown and the learning vs steering distinction

* 00:38:43 - Cerebellum as the "LLM" of the brain doing predictive learning

* 00:46:44 - Language acquisition: Chomsky vs learning algorithms debate

* 00:54:13 - What LLMs fundamentally can't do: complex context limitations

* 01:07:17 - Hypothalamus and brainstem doing more than just homeostasis

* 01:13:45 - Why morality might just be another hypothalamus cell group

* 01:18:00 - Human social instincts as model-based reinforcement learning

* 01:22:47 - Actor-critic reinforcement learning mapped to brain regions

* 01:29:33 - Timeline predictions: when brain-like AGI might arrive

* 01:38:28 - Why humans still beat AI on strategic planning and domain expertise

* 01:47:27 - Inner vs outer alignment: cocaine example and reward prediction

* 01:55:13 - Why legible Python code beats learned reward models

* 02:00:45 - Outcome pumps, instrumental convergence, and the Stalin analogy

* 02:11:48 - What’s Your P(Doom)™

* 02:16:45 - Massive headroom above human intelligence

* 02:20:45 - Can AI take over without physical actuators? (Yes)

* 02:26:18 - Steve's bold claim: 30 person-years from proto-AGI to superintelligence

* 02:32:17 - Why overhang makes the transition incredibly dangerous

* 02:35:00 - Social evolution as alignment solution: why it won't work

* 02:46:47 - Steve's research program: legible reward functions vs RLHF

* 02:59:52 - AI policy discussion: why Steven is skeptical of pause AI

* 03:05:51 - Lightning round: offense vs defense, P(simulation), AI unemployment

* 03:12:42 - Thanking Steve and wrapping up the conversation

* 03:13:30 - Liron's outro: Supporting the show and upcoming episodes with Vitalik and Eliezer

Show Notes

* Steven Byrnes' Website & Research — https://sjbyrnes.com/

* Steve’s Twitter — https://x.com/steve47285

* Astera Institute — https://astera.org/

Steve’s Sequences

* Intro to Brain-Like-AGI Safety — https://www.alignmentforum.org/s/HzcM2dkCq7fwXBej8

* Foom & Doom 1: “Brain in a box in a basement” — https://www.alignmentforum.org/posts/yew6zFWAKG4AGs3Wk/foom-and-doom-1-brain-in-a-box-in-a-basement

* Foom & Doom 2: Technical alignment is hard — https://www.alignmentforum.org/posts/bnnKGSCHJghAvqPjS/foom-and-doom-2-technical-alignment-is-hard

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Geoffrey Miller is an evolutionary psychologist at the University of New Mexico, bestselling author, and one of the world's leading experts on signaling theory and human sexual selection. His book "Mate" was hugely influential for me personally during my dating years, so I was thrilled to finally get him on the show.

In this episode, Geoffrey drops a bombshell 50% P(Doom) assessment, coming from someone who wrote foundational papers on neural networks and genetic algorithms back in the '90s before pivoting to study human mating behavior for 30 years.

What makes Geoffrey's doom perspective unique is that he thinks both inner and outer alignment might be unsolvable in principle, ever. He's also surprisingly bearish on AI's current value, arguing it hasn't been net positive for society yet despite the $14 billion in OpenAI revenue.

We cover his fascinating intellectual journey from early AI researcher to pickup artist advisor to AI doomer, why Asperger's people make better psychology researchers, the polyamory scene in rationalist circles, and his surprisingly optimistic take on cooperating with China. Geoffrey brings a deeply humanist perspective. He genuinely loves human civilization as it is and sees no reason to rush toward our potential replacement.

* 00:00:00 - Introducing Prof. Geoffrey Miller

* 00:01:46 - Geoffrey’s intellectual career arc: AI → evolutionary psychology → back to AI

* 00:03:43 - Signaling theory as the main theme driving his research

* 00:05:04 - Why evolutionary psychology is legitimate science, not just speculation

* 00:08:18 - Being a professor in the AI age and making courses "AI-proof"

* 00:09:12 - Getting tenure in 2008 and using academic freedom responsibly

* 00:11:01 - Student cheating epidemic with AI tools, going "fully medieval"

* 00:13:28 - Should professors use AI for grading? (Geoffrey says no, would be unethical)

* 00:23:06 - Coming out as Aspie and neurodiversity in academia

* 00:29:15 - What is sex and its role in evolution (error correction vs. variation)

* 00:34:06 - Sexual selection as an evolutionary "supercharger"

* 00:37:25 - Dating advice, pickup artistry, and evolutionary psychology insights

* 00:45:04 - Polyamory: Geoffrey’s experience and the rationalist connection

* 00:50:96 - Why rationalists tend to be poly vs. Chesterton's fence on monogamy

* 00:54:07 - The "primal" lifestyle and evolutionary medicine

* 00:56:59 - How Iain M. Banks' Culture novels shaped Geoffrey’s AI thinking

* 01:05:26 - What’s Your P(Doom)™

* 01:08:04 - Main doom scenario: AI arms race leading to unaligned ASI

* 01:14:10 - Bad actors problem: antinatalists, religious extremists, eco-alarmists

* 01:21:13 - Inner vs. outer alignment - both may be unsolvable in principle

* 01:23:56 - "What's the hurry?" - Why rush when alignment might take millennia?

* 01:28:17 - Disagreement on whether AI has been net positive so far

* 01:35:13 - Why AI won't magically solve longevity or other major problems

* 01:37:56 - Unemployment doom and loss of human autonomy

* 01:40:13 - Cosmic perspective: We could be "the baddies" spreading unaligned AI

* 01:44:93 - "Humanity is doing incredibly well" - no need for Hail Mary AI

* 01:49:01 - Why ASI might be bad at solving alignment (lacks human cultural wisdom)

* 01:52:06 - China cooperation: "Whoever builds ASI first loses"

* 01:55:19 - Liron’s Outro

Show Notes

Links

* Geoffrey’s Twitter

* Geoffrey’s University of New Mexico Faculty Page

* Geoffrey’s Publications

* Designing Neural Networks using Genetic Algorithms - His most cited paper

* Geoffrey’s Effective Altruism Forum Posts

Books by Geoffrey Miller

* Mate: Become the Man Women Want (2015) - Co-authored with Tucker Max

* The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature (2000)

* Virtue Signaling: Essays on Darwinian Politics and Free Speech (2019)

* Spent: Sex, Evolution, and Consumer Behavior (2009)

Related Doom Debates Episodes

* Liam Robins on College in the AGI Era - Student perspective on AI cheating

* Liron Reacts to Steven Pinker on AI Risk - Critiquing Pinker's AI optimism

* Steven Byrnes on Brain-Like AGI - Upcoming episode on human brain architecture

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

I’m doing a new weekly show on the AI Risk Network called Warning Shots. Check it out!

I’m only cross-posting the first episode here on Doom Debates. You can watch future episodes by subscribing to the AI Risk Network channel.

This week's warning shot: Mark Zuckerberg announced that Meta is racing toward recursive self-improvement and superintelligence. His exact words: "Developing superintelligence is now in sight and we just want to make sure that we really strengthen the effort as much as possible to go for it." This should be front-page news. Instead, everyone's talking about some CEO's dumb shenanigans at a Coldplay concert.

Recursive self-improvement is when AI systems start upgrading themselves - potentially the last invention humanity ever makes. Every AI safety expert knows this is a bright red line. And Zuckerberg just said he's sprinting toward it. In a sane world, he'd have to resign for saying this. That's why we made this show - to document these warning shots as they happen, because someone needs to be paying attention

* 00:00 - Opening comments about Zuckerberg and superintelligence

* 00:51 - Show introductions and host backgrounds

* 01:56 - Geoff Lewis psychotic episode and ChatGPT interaction discussion

* 05:04 - Transition to main warning shot about Mark Zuckerberg

* 05:32 - Zuckerberg's recursive self-improvement audio clip

* 08:22 - Second Zuckerberg clip about "going for superintelligence"

* 10:29 - Analysis of "superintelligence in everyone's pocket"

* 13:07 - Discussion of Zuckerberg's true motivations

* 15:13 - Nuclear development analogy and historical context

* 17:39 - What should happen in a sane society (wrap-up)

* 20:01 - Final thoughts and sign-off

Show Notes

Hosts:

* Doom Debates - Liron Shapira's channel

* AI Risk Network - John Sherman's channel

* Lethal Intelligence - Michael's animated AI safety content

This Episode's Warning Shots:

* Mark Zuckerberg podcast appearance discussing superintelligence

* Geoff Lewis (Bedrock VC) Twitter breakdown

---

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates