Sign up to save your podcasts
Or
Share “Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning” by Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, Lee Sharkey
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning” by Dan Braun, Jordan Taylor, Nicholas Goldowsky-Dill, Lee Sharkey
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.
A short summary of the paper is presented below.
This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) .
TL;DR: We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model and the model with SAE activations inserted. Compared to standard SAEs, e2e SAEs offer a Pareto improvement: They explain more network performance, require fewer total features, and require fewer simultaneously active features per datapoint, all with no cost to interpretability. We explore geometric and qualitative differences between e2e SAE features and standard SAE features.
Introduction.Current SAEs focus on the wrong goal: They are trained to minimize mean squared reconstruction error [...]
---
Outline:
(04:40) Key Results
(08:13) Acknowledgements
(08:50) Extras
---
First published:
May 17th, 2024
Source:
https://www.lesswrong.com/posts/xzJK3nENopiLmo77H/identifying-functionally-important-features-with-end-to-end
---
Narrated by TYPE III AUDIO.
View all episodes
By LessWrongCrossposted from the AI Alignment Forum. May contain more technical jargon than usual.This is a link post.
A short summary of the paper is presented below.
This work was produced by Apollo Research in collaboration with Jordan Taylor (MATS + University of Queensland) .
TL;DR: We propose end-to-end (e2e) sparse dictionary learning, a method for training SAEs that ensures the features learned are functionally important by minimizing the KL divergence between the output distributions of the original model and the model with SAE activations inserted. Compared to standard SAEs, e2e SAEs offer a Pareto improvement: They explain more network performance, require fewer total features, and require fewer simultaneously active features per datapoint, all with no cost to interpretability. We explore geometric and qualitative differences between e2e SAE features and standard SAE features.
Introduction.Current SAEs focus on the wrong goal: They are trained to minimize mean squared reconstruction error [...]
---
Outline:
(04:40) Key Results
(08:13) Acknowledgements
(08:50) Extras
---
First published:
May 17th, 2024
Source:
https://www.lesswrong.com/posts/xzJK3nENopiLmo77H/identifying-functionally-important-features-with-end-to-end
---
Narrated by TYPE III AUDIO.
More shows like LessWrong (30+ Karma)
View all
The Daily
112,101 Listeners
Astral Codex Ten Podcast
130 Listeners
Interesting Times with Ross Douthat
7,232 Listeners
Dwarkesh Podcast
576 Listeners
The Ezra Klein Show
16,129 Listeners
AI Article Readings
4 Listeners
Doom Debates!
14 Listeners
LessWrong posts by zvi
2 Listeners