LessWrong (30+ Karma)

“Testing which LLM architectures can do hidden serial reasoning” by Filip Sondej


Listen Later

Summary

  • Recurrence enables hidden serial reasoning.
  • Not every recurrence though - connections between channels are needed. Notably Mamba architecture isn't capable of hidden reasoning.
  • Non-linearity isn’t needed for hidden reasoning.
  • It's hard for transformers to learn to use all the layers for serial computation. In my toy setup, to +1 the serial computation length, we need to +3 the number of layers.
  • If we expect recurrent architectures may ever become SOTA, it would be wise to preemptively ban them. (Preferably before they become SOTA, while it's easier.)

Motivation

There are many examples of unfaithful LLM reasoning - where the answer doesn't follow from the reasoning, but rather the reasoning is just a rationalization for the answer. E.g. Turpin et al. 2023 show LLMs rationalizing for sycophantic and stereotypical answers. However, these examples are cases of rather simple hidden reasoning. What would be most worrying, is LLMs doing complex [...]

---

Outline:

(00:05) Summary

(00:49) Motivation

(02:15) Toy task for hidden serial reasoning

(03:37) Experiments

(06:15) Bonus experiment 1 - Is non-linearity required for hidden serial reasoning?

(06:53) Bonus experiment 2 - Do more layers enable longer hidden reasoning in transformers?

(07:39) Caveats

The original text contained 3 footnotes which were omitted from this narration.

The original text contained 6 images which were described by AI.

---

First published:

December 16th, 2024

Source:

https://www.lesswrong.com/posts/ZB6guMhHH3NEyxA2k/testing-which-llm-architectures-can-do-hidden-serial-3

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

...more
View all episodesView all episodes
Download on the App Store

LessWrong (30+ Karma)By LessWrong


More shows like LessWrong (30+ Karma)

View all
Making Sense with Sam Harris by Sam Harris

Making Sense with Sam Harris

26,350 Listeners

Conversations with Tyler by Mercatus Center at George Mason University

Conversations with Tyler

2,392 Listeners

The Peter Attia Drive by Peter Attia, MD

The Peter Attia Drive

7,955 Listeners

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas by Sean Carroll | Wondery

Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas

4,128 Listeners

ManifoldOne by Steve Hsu

ManifoldOne

87 Listeners

Your Undivided Attention by Tristan Harris and Aza Raskin, The Center for Humane Technology

Your Undivided Attention

1,445 Listeners

All-In with Chamath, Jason, Sacks & Friedberg by All-In Podcast, LLC

All-In with Chamath, Jason, Sacks & Friedberg

8,909 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

88 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

372 Listeners

Hard Fork by The New York Times

Hard Fork

5,426 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

15,326 Listeners

Moonshots with Peter Diamandis by PHD Ventures

Moonshots with Peter Diamandis

466 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

122 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

76 Listeners

BG2Pod with Brad Gerstner and Bill Gurley by BG2Pod

BG2Pod with Brad Gerstner and Bill Gurley

450 Listeners