Epistemic status: The important content here is the claims. To illustrate the claims, I sometimes use examples that I didn't research very deeply, where I might get some facts wrong; feel free to treat these examples as fictional allegories.

In a recent exchange on X, I promised to write a post with my thoughts on what sorts of downstream problems interpretability researchers should try to apply their work to. But first, I want to explain why I think this question is important.

In this post, I will argue that interpretability researchers should demo downstream applications of their research as a means of validating their research. To be clear about what this claim means, here are different claims that I will not defend here:

Not my claim: Interpretability researchers should demo downstream applications of their research because we terminally care about these applications; researchers should just directly work on the [...]

---

Outline:

(02:30) Two interpretability fears

(07:21) Proposed solution: downstream applications

(11:04) Aside: fair fight vs. no-holds barred vs. in the wild

(12:54) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Now and then, at work, we’ll have a CEO on from the megacorp that owns the company I work at. It's a Zoom meeting with like 300 people, the guy is usually giving a speech that is harmless and nice (if a bit banal), and I’ll turn on my camera and ask a question about something that I feel is not going well.

Invariably, within 10 minutes, I receive messages from coworkers congratulating me on my boldness. You asked him? That question? So brave!

This is always a little strange to me. Of course I feel free to question the CEO!

What really makes me nervous is asking cutting questions to people newer to the company than me, who have less experience, and less position. If I screw up how I ask them a question, and am a little too thoughtless or rude, or imagine I was, I agonize [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

PDF version. berkeleygenomics.org.

William Thurston was a world-renowned mathematician. His ideas revolutionized many areas of geometry and topology[1]; the proof of his geometrization conjecture was eventually completed by Grigori Perelman, thus settling the Poincaré conjecture (making it the only solved Millennium Prize problem). After his death, his students wrote reminiscences, describing among other things his exceptional vision.[2] Here's Jeff Weeks:

Bill's gift, of course, was his vision, both in the direct sense of seeing geometrical structures that nobody had seen before and in the extended sense of seeing new ways to understand things. While many excellent mathematicians might understand a complicated situation, Bill could look at the same complicated situation and find simplicity.

Thurston emphasized clear vision over algebra, even to a fault. Yair Minksy:

Most inspiring was his insistence on understanding everything in as intuitive and immediate a way as possible. A clear mental [...]

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Includes a link to the full Gemini conversation, but setting the stage first:

There is a puzzle. If you are still on Tumblr, or live in Berkeley where I can and have inflicted it on you in person[1], you may have seen it. There's a series of arcane runes, composed entirely of []- in some combination, and the runes - the strings of characters - are all assigned to numbers; mostly integers, but some fractions and integer roots. You are promised that this is a notation, and it's a genuine promise.

If you enjoy math puzzles, try it out. This post will [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

A new AI alignment player has entered the arena.

Emmett Shear, Adam Goldstein and David Bloomin have set up shop in San Francisco with a 10-person start-up called Softmax. The company is part research lab and part aspiring money maker and aimed at figuring out how to fuse the goals of humans and AIs in a novel way through what the founders describe as “organic alignment.” It's a heady, philosophical approach to alignment that seeks to take cues from nature and the fundamental traits of intelligent creatures and systems, and we’ll do our best to capture it here.

“We think there are these general principles that govern the alignment of any group of intelligent learning agents or beings, whether it's an ant colony or humans on a team or cells in a body,” Shear said, during his first interview to discuss the new company. “And organic alignment is the [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Epistemic status: This post aims at an ambitious target: improving intuitive understanding directly. The model for why this is worth trying is that I believe we are more bottlenecked by people having good intuitions guiding their research than, for example, by the ability of people to code and run evals.

Quite a few ideas in AI safety implicitly use assumptions about individuality that ultimately derive from human experience.

When we talk about AIs scheming, alignment faking or goal preservation, we imply there is something scheming or alignment faking or wanting to preserve its goals or escape the datacentre.

If the system in question were human, it would be quite clear what that individual system is. When you read about Reinhold Messner reaching the summit of Everest, you would be curious about the climb, but you would not ask if it was his body there, or his [...]

---

Outline:

(01:38) Individuality in Biology

(03:53) Individuality in AI Systems

(10:19) Risks and Limitations of Anthropomorphic Individuality Assumptions

(11:25) Coordinating Selves

(16:19) Whats at Stake: Stories

(17:25) Exporting Myself

(21:43) The Alignment Whisperers

(23:27) Echoes in the Dataset

(25:18) Implications for Alignment Research and Policy

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Gemini 2.5 Pro Experimental is America's next top large language model.

That doesn’t mean it is the best model for everything. In particular, it's still Gemini, so it still is a proud member of the Fun Police, in terms of censorship and also just not being friendly or engaging, or willing to take a stand.

If you want a friend, or some flexibility and fun, or you want coding that isn’t especially tricky, then call Claude, now with web access.

If you want an image, call GPT-4o.

But if you mainly want reasoning, or raw intelligence? For now, you call Gemini.

The feedback is overwhelmingly positive. Many report Gemini 2.5 is the first LLM to solve some of their practical problems, including favorable comparisons to o1-pro. It's fast. It's not $200 a month. The benchmarks are exceptional.

(On other LLMs I’ve used in [...]

---

Outline:

(01:22) Introducing Gemini 2.5 Pro

(03:45) Their Lips are Sealed

(07:25) On Your Marks

(12:06) The People Have Spoken

(26:03) Adjust Your Projections

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What if they released the new best LLM, and almost no one noticed?

Google seems to have pulled that off this week with Gemini 2.5 Pro.

It's a great model, sir. I have a ton of reactions, and it's 90%+ positive, with a majority of it extremely positive. They cooked.

But what good is cooking if no one tastes the results?

Instead, everyone got hold of the GPT-4o image generator and went Ghibli crazy.

I love that for us, but we did kind of bury the lede. We also buried everything else. Certainly no one was feeling the AGI.

Also seriously, did you know Claude now has web search? It's kind of a big deal. This was a remarkably large quality of life improvement.

Table of Contents

---

Outline:

(01:01) Table of Contents

(01:08) Google Fails Marketing Forever

(04:01) Language Models Offer Mundane Utility

(10:11) Language Models Don't Offer Mundane Utility

(11:01) Huh, Upgrades

(13:48) On Your Marks

(15:48) Copyright Confrontation

(16:44) Choose Your Fighter

(17:22) Deepfaketown and Botpocalypse Soon

(19:35) They Took Our Jobs

(22:19) The Art of the Jailbreak

(24:40) Get Involved

(25:16) Introducing

(25:52) In Other AI News

(28:23) Oh No What Are We Going to Do

(31:50) Quiet Speculations

(34:24) Fully Automated AI RandD Is All You Need

(37:39) IAPS Has Some Suggestions

(42:56) The Quest for Sane Regulations

(48:32) We The People

(51:23) The Week in Audio

(52:29) Rhetorical Innovation

(59:42) Aligning a Smarter Than Human Intelligence is Difficult

(01:05:18) People Are Worried About AI Killing Everyone

(01:05:33) Fun With Image Generation

(01:12:17) Hey We Do Image Generation Too

(01:14:25) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The other day I discussed how high monitoring costs can explain the emergence of “aristocratic” systems of governance:

Aristocracy and Hostage Capital

Arjun Panickssery · Jan 8
There's a conventional narrative by which the pre-20th century aristocracy was the "old corruption" where civil and military positions were distributed inefficiently due to nepotism until the system was replaced by a professional civil service after more enlightened thinkers prevailed ...

An element of Douglas Allen's argument that I didn’t expand on was the British Navy. He has a separate paper called “The British Navy Rules” that goes into more detail on why he thinks institutional incentives made them successful from 1670 and 1827 (i.e. for most of the age of fighting sail).

In the Seven Years’ War (1756–1763) the British had a 7-to-1 casualty difference in single-ship actions. During the French Revolutionary and Napoleonic Wars (1793–1815) the British had a 5-to-1 [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[This is our blog post on the papers, which can be found at https://transformer-circuits.pub/2025/attribution-graphs/biology.html and https://transformer-circuits.pub/2025/attribution-graphs/methods.html.]

Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data. During that training process, they learn their own strategies to solve problems. These strategies are encoded in the billions of computations a model performs for every word it writes. They arrive inscrutable to us, the model's developers. This means that we don’t understand how models do most of the things they do.

Knowing how models like Claude think would allow us to have a better understanding of their abilities, as well as help us ensure that they’re doing what we intend them to. For example:

• Claude can speak dozens of languages. What language, if any, is it using "in its head"?
• Claude writes text one word at a time. Is it only focusing on predicting the [...]

---

Outline:

(06:02) How is Claude multilingual?

(07:43) Does Claude plan its rhymes?

(09:58) Mental Math

(12:04) Are Claude's explanations always faithful?

(15:27) Multi-step Reasoning

(17:09) Hallucinations

(19:36) Jailbreaks

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.