LessWrong (30+ Karma)

“Why do misalignment risks increase as AIs get more capable?” by ryan_greenblatt

Listen Later

It's generally agreed that as AIs get more capable, risks from misalignment increase. But there are a few different mechanisms by which more capable models are riskier, and distinguishing between those mechanisms is important when estimating the misalignment risk posed at a particular level of capabilities or by a particular model.

There are broadly 3 reasons why misalignment risks increase with capabilities:

  • As AIs get more capable, the chance of egregious misalignment increases: AIs with more powerful opaque reasoning abilities are more likely to become egregiously misaligned—sufficiently misaligned that they strategically deceive and work against their developers—without this being detected or trained out. We should expect opaque reasoning abilities generally increase as models get more capable, though it isn't entirely obvious what the relationship between opaque and non-opaque cognition will be in future AI systems. Additionally, I expect that as AIs get more capable their capabilities [...]

    • The original text contained 1 footnote which was omitted from this narration.

    ---

    First published:

    April 11th, 2025

    Source:

    https://www.lesswrong.com/posts/NDotm7oLHfR56g4sD/why-do-misalignment-risks-increase-as-ais-get-more-capable

    ---

    Narrated by TYPE III AUDIO.

    ...more
    View all episodesView all episodes
    Download on the App Store
    LessWrong (30+ Karma)By LessWrong

    More shows like LessWrong (30+ Karma)

    View all
    Making Sense with Sam Harris by Sam Harris

    Making Sense with Sam Harris

    26,364 Listeners

    Conversations with Tyler by Mercatus Center at George Mason University

    Conversations with Tyler

    2,438 Listeners

    The Peter Attia Drive by Peter Attia, MD

    The Peter Attia Drive

    9,079 Listeners

    Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas by Sean Carroll | Wondery

    Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

    4,153 Listeners

    ManifoldOne by Steve Hsu

    ManifoldOne

    92 Listeners

    Your Undivided Attention by The Center for Humane Technology, Tristan Harris, Daniel Barcay and Aza Raskin

    Your Undivided Attention

    1,596 Listeners

    All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

    All-In with Chamath, Jason, Sacks & Friedberg

    9,909 Listeners

    Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

    Machine Learning Street Talk (MLST)

    90 Listeners

    Dwarkesh Podcast by Dwarkesh Patel

    Dwarkesh Podcast

    70 Listeners

    Hard Fork by The New York Times

    Hard Fork

    5,469 Listeners

    The Ezra Klein Show by New York Times Opinion

    The Ezra Klein Show

    16,081 Listeners

    Moonshots with Peter Diamandis by PHD Ventures

    Moonshots with Peter Diamandis

    537 Listeners

    No Priors: Artificial Intelligence | Technology | Startups by Conviction

    No Priors: Artificial Intelligence | Technology | Startups

    131 Listeners

    Latent Space: The AI Engineer Podcast by swyx + Alessio

    Latent Space: The AI Engineer Podcast

    95 Listeners

    BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

    BG2Pod with Brad Gerstner and Bill Gurley

    521 Listeners