Sign up to save your podcasts
Or
Share “Showing SAE Latents Are Not Atomic Using Meta-SAEs” by Bart Bussmann, Michael Pearce, Patrick Leask, Joseph Bloom, Lee Sharkey, Neel Nanda
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Showing SAE Latents Are Not Atomic Using Meta-SAEs” by Bart Bussmann, Michael Pearce, Patrick Leask, Joseph Bloom, Lee Sharkey, Neel Nanda
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
Bart, Michael and Patrick are joint first authors. Research conducted as part of MATS 6.0 in Lee Sharkey and Neel Nanda's streams. Thanks to Mckenna Fitzgerald and Robert Krzyzanowski for their feedback!
TL;DR:
- Sparse Autoencoder (SAE) latents have been shown to typically be monosemantic (i.e. correspond to an interpretable property of the input). It is sometimes implicitly assumed that they are therefore atomic, i.e. simple, irreducible units that make up the model's computation.
- We provide evidence against this assumption by finding sparse, interpretable decompositions of SAE decoder directions into seemingly more atomic latents, e.g. Einstein -> science + famous + German + astronomy + energy + starts with E-
- We do this by training meta-SAEs, an SAE trained to reconstruct the decoder directions of a normal SAE.
- We argue that, conceptually, there's no reason to expect [...]
---
Outline:
(02:36) Introduction
(07:41) Defining Meta-SAEs
(09:46) Meta-latents form interpretable decompositions of SAE latents
(13:04) Are Meta-Latents different from SAE Latents?
(19:56) Using Meta-SAEs to Interpret Split Features
(24:02) Causally Intervening and Making Targeted Edits with Meta-Latents
(29:03) Discussion
The original text contained 2 footnotes which were omitted from this narration.
The original text contained 2 images which were described by AI.
---
First published:
August 24th, 2024
Source:
https://www.lesswrong.com/posts/TMAmHh4DdMr4nCSr5/showing-sae-latents-are-not-atomic-using-meta-saes
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongCrossposted from the AI Alignment Forum. May contain more technical jargon than usual.
Bart, Michael and Patrick are joint first authors. Research conducted as part of MATS 6.0 in Lee Sharkey and Neel Nanda's streams. Thanks to Mckenna Fitzgerald and Robert Krzyzanowski for their feedback!
TL;DR:
- Sparse Autoencoder (SAE) latents have been shown to typically be monosemantic (i.e. correspond to an interpretable property of the input). It is sometimes implicitly assumed that they are therefore atomic, i.e. simple, irreducible units that make up the model's computation.
- We provide evidence against this assumption by finding sparse, interpretable decompositions of SAE decoder directions into seemingly more atomic latents, e.g. Einstein -> science + famous + German + astronomy + energy + starts with E-
- We do this by training meta-SAEs, an SAE trained to reconstruct the decoder directions of a normal SAE.
- We argue that, conceptually, there's no reason to expect [...]
---
Outline:
(02:36) Introduction
(07:41) Defining Meta-SAEs
(09:46) Meta-latents form interpretable decompositions of SAE latents
(13:04) Are Meta-Latents different from SAE Latents?
(19:56) Using Meta-SAEs to Interpret Split Features
(24:02) Causally Intervening and Making Targeted Edits with Meta-Latents
(29:03) Discussion
The original text contained 2 footnotes which were omitted from this narration.
The original text contained 2 images which were described by AI.
---
First published:
August 24th, 2024
Source:
https://www.lesswrong.com/posts/TMAmHh4DdMr4nCSr5/showing-sae-latents-are-not-atomic-using-meta-saes
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,409 Listeners
Conversations with Tyler
2,387 Listeners
The Peter Attia Drive
7,908 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,131 Listeners
ManifoldOne
87 Listeners
Your Undivided Attention
1,457 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,042 Listeners
Machine Learning Street Talk (MLST)
87 Listeners
Dwarkesh Podcast
388 Listeners
Hard Fork
5,432 Listeners
The Ezra Klein Show
15,201 Listeners
Moonshots with Peter Diamandis
474 Listeners
No Priors: Artificial Intelligence | Technology | Startups
122 Listeners
Latent Space: The AI Engineer Podcast
75 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
454 Listeners