These are some of my notes from reading Anthropic's latest research report, Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.

TL;DR

In roughly descending order of importance:

1. Its great that Anthropic trained an SAE on a production-scale language model, and that the approach works to find interpretable features. Its great those features allow interventions like the recently-departed Golden Gate Claude. I especially like the code bug feature.
2. I worry that naming features after high-activating examples (e.g. "the Golden Gate Bridge feature") gives a false sense of security. Most of the time that feature activates, it is irrelevant to the golden gate bridge. That feature is only well-described as "related to the golden gate bridge" if you condition on a very high activation, and that's <10% of its activations (from an eyeballing of the graph).
3. This work does not address my major concern about dictionary learning: it is [...]

---

Outline:

(00:16) TL;DR

(02:23) A Feature Isnt Its Highest Activating Examples

(04:38) Finding Specific Features

(06:02) Architecture - The Classics, but Wider

(07:26) Correlations - Strangely Large?

(09:48) Future Tests

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.