LessWrong (30+ Karma)

“New ARENA material: 8 exercise sets on alignment science & interpretability” by CallumMcDougall


Listen Later

TLDR

This is a post announcing a lot of new ARENA material I've been working on for a while, which is now available for study here (currently on the alignment-science branch, but planned to be merged into main this Sunday).

There's a set of exercises (each one contains about 1-2 days of material) on the following topics:

  • Linear Probes (replication of the "Geometry of Truth" paper, plus Apollo's "Probing for Deception" work)
  • Activation Oracles (based around this demo notebook, with additional exercises on model diffing)
  • Attribution graphs (you can build them from scratch here including all the graph pruning implementations, and also use the circuit-tracer library)
  • Emergent Misalignment (mostly based on Soligo & Turner's work; this also covers a lot of "basics of how to work with model organisms" like writing autoraters, using LoRA finetunes, etc)
  • Science of Misalignment (walkthrough of 2 case studies: Palisade's "Shutdown Resistance" & GDM's follow-up, and Alignment Faking)
  • Reasoning Model Interpretability (guided replication of Thought Anchors plus the blackmail extension)
  • LLM Psychology & Persona Vectors (replicates the "assistant axis" paper including activation capping technique, and also has you create a persona vector extraction pipeline)
  • Investigator Agents (basically takes you through building mini-Petri from [...]

---

Outline:

(00:13) TLDR

(01:49) New material

(01:52) en-US-AvaMultilingualNeural__ Diagram showing eight AI safety and interpretability concepts including linear probes and activation oracles.

(03:19) (1.3.1) Linear Probes

(04:06) (1.3.4) Activation Oracles

(04:58) (1.4.2) Attribution Graphs

(06:15) (4.1) Emergent Misalignment

(07:05) (4.2) Science of Misalignment

(08:00) (4.3) Reasoning Model Interpretability

(08:52) (4.4) LLM Psychology & Persona Vectors

(09:51) (4.5) Investigator Agents

(10:45) New Site Features

(12:07) Logistics

(12:57) Why use, in vibe-code world?

(15:12) Feedback

The original text contained 1 footnote which was omitted from this narration.

---

First published:

February 27th, 2026

Source:

https://www.lesswrong.com/posts/nQAN2vxv2ASjowMda/new-arena-material-8-exercise-sets-on-alignment-science-and

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

...more
View all episodesView all episodes
Download on the App Store

LessWrong (30+ Karma)By LessWrong


More shows like LessWrong (30+ Karma)

View all
The Daily by The New York Times

The Daily

112,326 Listeners

Astral Codex Ten Podcast by Jeremiah

Astral Codex Ten Podcast

130 Listeners

Interesting Times with Ross Douthat by New York Times Opinion

Interesting Times with Ross Douthat

7,242 Listeners

Dwarkesh Podcast by Dwarkesh Patel

Dwarkesh Podcast

559 Listeners

The Ezra Klein Show by New York Times Opinion

The Ezra Klein Show

16,321 Listeners

AI Article Readings by Readings of great articles in AI voices

AI Article Readings

4 Listeners

Doom Debates! by Liron Shapira

Doom Debates!

14 Listeners

LessWrong posts by zvi by zvi

LessWrong posts by zvi

2 Listeners