This is a crosspost of my ICLR 2026 blogpost track post. All code and experiments are available at github.com/andyrdt/iclr_induction.

Summary

Park et al., 2025 show that when large language models (LLMs) process random walks on a graph, their internal representations come to mirror the underlying graph's structure. The authors interpret this broadly, suggesting that LLMs can "manipulate their representations in order to reflect concept semantics specified entirely in-context". In this post, we take a closer look at the underlying mechanism, and suggest a simpler explanation. We argue that induction circuits (Elhage et al., 2021; Olsson et al., 2022), a well-known mechanism for in-context bigram recall, suffice to explain both the task performance and the representation geometry observed by Park et al.

Recapitulation and reproduction of Park et al., 2025

We begin by describing the experimental setup of Park et al., 2025 and reproducing their main results on Llama-3.1-8B.

---

Outline:

(00:20) Summary

(01:06) Recapitulation and reproduction of Park et al., 2025

(02:03) The grid tracing task

(04:36) Reproduction and Park et al.s interpretation

(06:19) A simpler explanation: induction circuits

(07:32) Testing the induction hypothesis

(08:45) Results

(11:25) Previous-token mixing can account for representation geometry

(11:52) The neighbor-mixing hypothesis

(12:50) A toy model of previous-token mixing

(13:56) Evidence of neighbor mixing in individual model activations

(15:34) Limitations

(17:57) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Have you guys heard about this Epstein stuff? Shit's pretty crazy.

Note: I'm not going to provide a summary of the situation or talk about evidence; this piece is for people that already know these things. I'm going to avoid specifics about what Epstein and co did, and instead will use vague terms like "Epstein stuff". This is a short post about how I've updated my world model.

Particular things that I find very surprising: that so many people basically knew what was going on and didn't say anything; that so many people were involved themselves in incriminating heinous acts; that the Epstein stuff and associated conspiracy to hide/protect it spanned not only lots of people but lots of time (~20 years!); that they got away with it for so long.

Conspiracy

Scott Alexander writes:

The Basic Argument Against Conspiracy Theories goes: “You can’t run a big organization in secret without any outsiders noticing or any insiders blowing the whistle."

He offers a number of heuristics regarding the plausibility of conspiracy theories, including:

A. You generally can’t keep the existence of a large organization that engages in clandestine activities secret.

Before I learned about this Epstein stuff, I thought [...]

---

Outline:

(00:58) Conspiracy

(01:39) Things I think are much more prevalent/likely than I did before

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is the long version of what happened so far. I will strive for shorter ones later, when I have the time to write them.

Most of you should read the first two sections, then choose the remaining sections that are relevant to your interests.

But first, seriously, read Dean Ball's post Clawed. Do that first. I will not quote too extensively from it, because I am telling all of you to read it. Now. You’re not allowed to keep reading this or anything else until after you do. I’m not kidding.

That's out of the way? Good. Let's get started.

What Happened

President Trump enacted a perfectly reasonable solution to the situation with Anthropic and the Department of War. He cancelled the Anthropic contract with a six month wind down period, after which the Federal Government would be told not to use Anthropic software.

Everyone thought the worst was now over. The situation was unfortunate for Anthropic and also for national security, but this gave us six months to transition, it gave us six months to negotiate another solution, and it avoided any of the extreme highly damaging options that Secretary of [...]

---

Outline:

(00:49) What Happened

(10:02) The Timeline Of Events

(21:39) I Did Not Have Time To Write You A Short One

(22:40) The Unhinged Declaration of the Secretary of War

(25:08) Altman Has Been Excellent On The Question of Supply Chain Risk, But May Need To Do More

(27:35) Arrogance Here Means Insisting On Meaningful Red Lines On Mass Domestic Surveillance and Lethal Autonomous Weapons

(29:35) Not Doing Business Is Totally Fine

(30:22) The Demand For Unrestricted Access Is New And Is Selective And Fake

(32:47) Claims Of Strongarming Are Ad Hominem Bad Faith Obvious Nonsense

(34:55) Hegseth Equates Not Being a Dictator With Companies Having Veto Power Over Operational Military Decisions

(40:54) The Part That If Enacted Would Be A Historically Epic Clusterfuck

(48:54) The Other Part Of The Clusterfuck

(52:49) The Department of War Had Many Excellent Options

(55:53) And Then Theres Emil Michael

(01:02:42) Anthropic Will Probably Survive

(01:05:14) The Goal of DoW Was Largely Mass Domestic Surveillance

(01:15:32) What Are The Key Differences Between The Two Contracts?

(01:22:54) OpenAIs Contract Terms

(01:27:04) What OpenAIs Contract Terms Actually Do

(01:29:16) OpenAI Is Trusting DoW And Sam Altman Misrepresented This

(01:33:13) OpenAI Accepted Terms Anthropic Explicitly Declined And That Would Not Have Protected Anthropics Red Lines

(01:35:26) How Altman Initially Described His Deal

(01:42:26) OpenAI Allowed All Lawful Use And Trusts DoW On This

(01:47:21) The DoW Could Alter This Deal

(01:49:29) Why OpenAIs Shared Legal Language Offers Almost No Protections

(01:59:11) So How Does OpenAI Hope For This To Work Out?

(02:01:56) This Was Never About Money

(02:05:03) OpenAI Tells Us How They Really Feel

(02:06:45) First The Good News

(02:11:40) The OpenAI Redlines Only Forbid Currently Illegal Activity

(02:16:28) Altman Does Not Present As Understanding The Difference In Redlines

(02:18:46) Meeting Of The Minds

(02:21:21) Anthropics Position Was The Opposite Of How This Is Portrayed

(02:21:56) The Room Where It Happened

(02:25:45) You Dont Have The Right

(02:32:25) I Ask Questions And Get Answers

(02:37:05) Does This Contract Apply To NSA?

(02:38:07) Can OpenAI Models Be Used To Analyze Commercially Available Data At Scale?

(02:48:21) Employee Activism

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What a weekend. Two new wars in Asia don't qualify as top news.

My first reaction to Hegseth's conflict with Anthropic was along the

On closer inspection, it doesn't look very much like nationalization. A

On the other hand, his attempts to look like he's punishing Anthropic

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Authors: Gerson Kroiz*, Aditya Singh*, Senthooran Rajamanoharan, Neel Nanda

Gerson and Aditya are co-first authors. This work was conducted during MATS 9.0 and was advised by Senthooran Rajamanoharan and Neel Nanda.

TL;DR

Understanding why a model took an action is a key question in AI Safety. It is a difficult question, as often many motivations could plausibly lead to the same action. We are particularly concerned about the case of model incrimination, i.e. observing a model taking a bad action and distinguishing between benign and malign motivations, e.g. scheming vs confusion.

To practice the skill of understanding motivations, we need to build high-quality environments that serve as realistic proxy tasks for investigating future models. But we’ve found that there are many ways in which environments can be contrived. It is valuable to invest effort in ruling out confounds like ambiguity, instruction conflict, or confusion.

Over the course of building and iterating on more than 20 environments, we have refined 5 design principles for creating such settings. Generally, the more principles that an environment adheres to, the more fruitful and interesting the investigation will be.

1. Have uncertain causes: The environment should elicit behavior that admits either multiple plausible explanations [...]

---

Outline:

(00:32) TL;DR

(02:47) Introduction

(05:10) Design principles

(05:35) 1. Have uncertain causes

(07:59) 2. Maximize investigative surprise

(09:48) 3. Clear user intent

(11:38) 4. Avoid implicit nudges

(13:26) 5. Make the environment realistic

(15:03) Example of cleaning up an environment

(20:03) Appendix

(20:06) A1. Math Tutor Environment Details

(20:49) A2. Directory Summary Environment Details

(21:48) A3. Funding Email Environment Details

(22:38) A4. Evaluation tampering environment details

(23:20) A5. Secret number environment details

(24:06) A6. Sandbagging environment details

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TLDR

• We train and many-shot prompt base models to generate user prompts that are harder to distinguish from deployment (WildChat) prompts. 
• Then we give Petri, an automated auditing agent, a tool to use a prompt generator model for sycophancy audits. It doesn’t help with making the full audit transcripts harder to distinguish from deployment transcripts, despite more realistic single-turn prompts. 
• We find evidence that full audit realism is bottlenecked not by prompt quality, but by higher-level features: the harmful nature of the scenarios and the unnatural structure of the multi-turn conversations.

💻Code

Research done as part of the Anthropic Fellows Program.

Introduction

Automated auditing agents like Petri are increasingly used to evaluate the propensities of LLMs. But one growing challenge is evaluation awareness: frontier models can often detect when they’re in an eval vs deployment, making audits harder to trust. As models improve at detecting LLM generated text, they may identify some audit scenarios by noticing that their input prompts are synthetic rather than human written. This motivates the need for better methods to generate realistic synthetic prompts.

In this post we:

1. Train and many-shot prompt base models to generate realistic user prompts, and show that these [...]

---

Outline:

(00:14) TLDR

(01:01) Introduction

(03:25) Methods

(03:28) Training realistic user prompt generators

(04:55) Evaluating prompt generators

(05:44) Results

(05:47) Prompt generator models produce realistic user prompts

(09:06) Prompt generators dont improve Petri audit realism

(12:07) User prompts arent the bottleneck: realism judge CoT analysis

(14:28) Appendix

(14:31) SFT prompt format

(14:49) One failed attempt at training conditional text GAN

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Americans don't like OpenAI very much anymore, and you know why. Of course, AI systems it helped make have caused various problems already, like:

• bots pushing politics on social media
• maintainers of open source projects having their time wasted, and sometimes quitting
• AI-generated scientific papers crowding out good research
• scammers who clone the voice of a family member
• AI-generated fake pictures fooling people on Facebook

And gamers haven't been huge fans of OpenAI since it bought up half the RAM wafer production - mostly just to keep it off the market so other people can't use it, since they only bought the raw wafers, not finished RAM.

But the popularity of OpenAI has fallen abruptly this week, because it decided to help the US military make automated weapons and mass surveillance systems that Anthropic refused to make on ethical grounds.

The leadership of OpenAI is trying to pretend that that they got the same deal Anthropic wanted but they were just nicer about it. They are lying. Anthropic wanted a human in the loop of autonomous weapons. OpenAI's contract just says that there must be human oversight to whatever extent is required by law. But of course, the [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's an open letter opposing the DoW's assignment of Anthropic as a supply chain risk. It needs more signatures before it gets sent. If you're a tech founder, engineer, or investor and you agree with the letter, then your signature would help bolster its message. Your signature would be especially impactful if you work at a non-Anthropic SOTA AI lab.

You can sign the letter here: https://app.dowletter.org/

Full text follows:

We write as founders, engineers, investors, and executives in the American technology industry. We strongly believe the federal government should not retaliate against a private company for declining to accept changes to a contract.

When two parties cannot agree on terms, the normal course is to part ways and work with a competitor. Instead, the Department of War has designated Anthropic a “supply chain risk” (a label normally reserved for foreign adversaries), stating that “no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic.”

This situation sets a dangerous precedent. Punishing an American company for declining to accept changes to a contract sends a clear message to every technology company in America: accept whatever terms the government demands, or [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR

Your base LLM has no examples of superintelligent AI in its training data. When you RL it into superintelligence, it will have to extrapolate to how a superintelligent Claude would behave. The LLM's extrapolation may not converge optimizing for what humanity would, on reflection, like to optimize, because these are different processes with different inductive biases.

Intro

I'm going to take the Persona Selection Model as being roughly true, for now. Even on its own terms, it will fail. If the Persona Selection Model is false, we die in a different way.

I'm going to present some specific arguments and secnarios, but the core of it is a somewhat abstract point: the Claude persona, although it currently behaves in a human-ish way, will not grow into a superintelligence in the same way that humans would. This means it will not grow into the same kind of superintelligence with the same values that human values would converge on. Since value is fragile, this is fatal for the future.

I don't think this depends on the specifics of Claude's training, nor how human values are instantiated, unless Claude's future training methods are specifically designed to work in the exact [...]

---

Outline:

(00:10) TL;DR

(00:36) Intro

(01:35) LLMs

(01:39) Persona Selection and Other Models

(03:06) Persona Theory As Alignment Plan

(04:21) Gears of Personas

(07:21) Complications

(08:20) Reasoning and Chain-of-thought

(10:00) Reinforcement Learning

(11:40) Humans

(11:43) Human Values

(12:00) TL;DR

(12:30) Goal-Models and Inductors

(13:49) These Are Not The Same

(16:34) Final Thoughts

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.