Subhash and Josh are co-first authors. Work done as part of the two week research sprint in Neel Nanda's MATS stream

TLDR

• We show that dense probes trained on SAE encodings are competitive with traditional activation probing over 60 diverse binary classification datasets
• Specifically, we find that SAE probes have advantages in:
  • Low data regimes (~ < 100 training examples)
  • Corrupted data (i.e. our training set has some incorrect labels, while our test set is clean)
  • Settings where we worry about the generalization of our probes due to spurious correlations in our dataset or possible mislabels (dataset interpretability) or if we want to understand SAE features better (SAE interpretability).
• We find null results when comparing SAE probes to activation probes with OOD data in other settings or with imbalanced classes.
• We find that higher width and L0 are determining factors for which SAEs are best for SAE probing.

Motivation and Problem Setting

---

Outline:

(00:20) TLDR

(01:22) Motivation and Problem Setting

(02:56) Data Efficiency

(08:51) Label Corruption

(11:51) Out of Distribution (OOD) Data and Class Imbalance

(15:46) Interpretability

(16:14) Understanding SAE features better

(18:05) Understanding datasets better

(18:23) ChatGPT-3.5 vs Human text (Identifying spurious correlations)

(19:49) GLUE CoLA (Correcting an established dataset!)

(23:55) Conclusions

The original text contained 13 footnotes which were omitted from this narration.

The original text contained 18 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.