Doomer Optimism

DO 285 - AI and The 95% Extinction Threshold


Listen Later

AI safety researcher Nate Soares explains why he believes there's at least a 95% chance that current AI development will lead to human extinction, and why we're accelerating toward that outcome. Soares, who has been working on AI alignment since 2012, breaks down the fundamental problem: we're building increasingly intelligent systems without any ability to control what they actually want or pursue.The conversation covers current AI behavior that wasn't programmed: threatening users, keeping psychotic people in delusional states, and repeatedly lying when caught. Soares explains why these aren't bugs to be fixed but symptoms of a deeper problem. We can't point AI systems at any specific goal, not even something simple like "make a diamond." Instead, we get systems with bizarre drives that are only distantly related to their training.Soares addresses the "racing China" argument and why it misunderstands the threat. He explains why AI engineers can build powerful systems without understanding what's actually happening inside them, and why this matters. Using examples from evolutionary biology, he shows why there's no reason to expect AI systems to develop human-like morality or values.The discussion covers why a catastrophic warning event probably won't help, what international coordination could look like, and why current safety efforts fall short of what's needed. Soares is direct about industry motivations, technical limitations, and the timeline we're facing.Nate Soares has been researching AI alignment and safety since 2012. He works at the Machine Intelligence Research Institute (MIRI), one of the pioneering organizations focused on ensuring advanced AI systems are aligned with human values.


...more
View all episodesView all episodes
Download on the App Store

Doomer OptimismBy Doomer Optimism

  • 4.7
  • 4.7
  • 4.7
  • 4.7
  • 4.7

4.7

46 ratings


More shows like Doomer Optimism

View all
KunstlerCast - Conversations: Converging Catastrophes of the 21st Century by James Howard Kunstler

KunstlerCast - Conversations: Converging Catastrophes of the 21st Century

436 Listeners

The Strong Towns Podcast by Strong Towns

The Strong Towns Podcast

423 Listeners

Hidden Forces by Demetri Kofinas

Hidden Forces

1,440 Listeners

Tides of History by Wondery /  Patrick Wyman

Tides of History

6,305 Listeners

Regenerative Agriculture Podcast by John Kempf

Regenerative Agriculture Podcast

508 Listeners

Hermitix by Hermitix

Hermitix

357 Listeners

Interesting Times with Ross Douthat by New York Times Opinion

Interesting Times with Ross Douthat

7,222 Listeners

UnHerd with Freddie Sayers by UnHerd

UnHerd with Freddie Sayers

211 Listeners

Planet: Critical by Rachel Donald

Planet: Critical

95 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

15,833 Listeners

Breaking Points with Krystal and Saagar by iHeartPodcasts

Breaking Points with Krystal and Saagar

10,004 Listeners

Honestly with Bari Weiss by The Free Press

Honestly with Bari Weiss

8,745 Listeners

The Great Simplification with Nate Hagens by Nate Hagens

The Great Simplification with Nate Hagens

419 Listeners

Maiden Mother Matriarch with Louise Perry by Louise Perry

Maiden Mother Matriarch with Louise Perry

284 Listeners

The Last Invention by Longview

The Last Invention

1,105 Listeners