We trained a SAE to find sparse features in image embeddings. We found many meaningful, interpretable, and steerable features. We find that steering image diffusion works surprisingly well and yields predictable and high-quality generations.

You can see the feature library here. We also have an intervention playground you can try.

Key Results

• We can extract interpretable features from CLIP image embeddings.
• We observe a diverse set of features, e.g. golden retrievers, there being two of something, image borders, nudity, and stylistic effects.
• Editing features allows for conceptual and semantic changes while maintaining generation quality and coherency.
• We devise a way to preview the causal impact of a feature, and show that many features have an explanation that is consistent with what they activate for and what they cause.
• Many feature edits can be stacked to perform task-relevant operations, like transferring a subject, mixing in a specific property of [...]

---

Outline:

(00:30) Key Results

(01:20) Interactive demo

(01:33) Introduction

(02:22) Steering Features

(04:22) Discovering and Interpreting Features

(06:22) Autointerpretation Labels

(07:13) Training Details

(08:34) Future Work

---

First published:

Source:

Narrated by TYPE III AUDIO.