This is my personal take, not an organizational one. Originally written May 2025, revived for the EA Forum's Draft Amnesty Week. Cross-posted from the EA Forum but cross-posting didn't work the first time.

When I discovered the beginnings of EA, it felt like coming home. I had finally found a group of people who cared as much as I did about saving lives.

When I discovered rationality not long after, it felt new. The intensity of LessWrong was both offputting and fascinating to me, and I read a lot of the site in the summer of 2011. I had studied sociology and was used to thinking in terms of ethics and emotions, but not in terms of statistics or economics. LessWrongers were picky and disagreeable, but I learned reasoning skills from them.

Before EA even had a name, LessWrong was one of the best places to talk [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Audio note: this article contains 63 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Summary

Kernel regression with the empirical neural tangent kernel (eNTK) gives a closed-form approximation to the function learned by a neural network in parts of the model space. We provide evidence that the eNTK can be used to find features in toy models for interpretability. We show that in Toy Models of Superposition and a MLP trained on modular arithmetic, the eNTK eigenspectrum exhibits sharp cliffs whose top eigenspaces align with the ground-truth features. Moreover, in the modular arithmetic experiment, the evolution of the eNTK spectrum can be used to track the grokking phase transition. These results suggest that eNTK analysis may provide a new practical handle for feature discovery and for detecting phase changes in small models.

[...]

---

Outline:

(00:23) Summary

(01:25) Background

(04:57) Results

(05:10) Toy models of Superposition

(06:34) Modular arithmetic

(10:00) Next steps

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In a previous post, I discussed mitigating risks from scheming by studying examples of actual scheming AIs.[1] In this post, I'll discuss an alternative approach: directly training (or instructing) an AI to behave how we think a naturally scheming AI might behave (at least in some ways). Then, we can study the resulting models. For instance, we could verify that a detection method discovers that the model is scheming or that a removal method removes the inserted behavior. The Sleeper Agents paper is an example of directly training in behavior (roughly) similar to a schemer.

This approach eliminates one of the main difficulties with studying actual scheming AIs: the fact that it is difficult to catch (and re-catch) them. However, trained-in scheming behavior can't be used to build an end-to-end understanding of how scheming arises, or to test techniques focused on preventing scheming (rather than removing it even [...]

---

Outline:

(02:28) Studying detection and removal techniques using trained-in scheming behavior

(05:31) Issues

(18:21) Conclusion

(19:31) Canary string

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Once AI companies build powerful AIs, they may:

• Give internal AIs access to sensitive internal privileges (e.g. access to the internal infra that touches model weights, help with training of the next generation of AIs, …);
• Have mitigations specific to those internal deployments enforced via monitoring of the internal LLM API;
• Have an external API deployment without those mitigations.

In this situation, an internal AI may build an agent scaffold that makes calls to the external API and uses the internal sensitive privileges. Such agent scaffold would be in a better position to cause a catastrophe because it would not be subject to the same monitoring as the internal API. I call this a rogue internal deployment via external APIs. (It is “internal” because the model weights and the agent scaffold never leave the cluster.)

I think preventing those is similarly important from a misalignment perspective as preventing [...]

---

Outline:

(01:43) Rogue internal deployments via external APIs

(03:29) A variation: rogue internal deployments via cross-company APIs

(04:28) A possible mitigation: preventing the creation of scaffolds that use external LLM APIs via monitoring

(06:29) Why I am more pessimistic about other solutions

(06:34) Monitoring the external API

(08:47) Preventing access to external APIs

(10:22) Monitoring access to sensitive permissions

(10:57) Final thoughts

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I recently visited my girlfriend's parents in India. Here is what that experience taught me:

Yudkowsky has this facebook post where he makes some inferences about the economy after noticing two taxis stayed in the same place while he got his groceries. I had a few similar experiences while I was in India, though sadly I don't remember them in enough detail to illustrate them in as much detail as that post. Most of the thoughts relating to economics revolved around how labour in India is extremely cheap.

I knew in the abstract that India is not as rich as countries I had been in before, but it was very different seeing that in person. From the perspective of getting an intuitive feel for economics, it was very interesting to be thrown into a very different economy and seeing a lot of surprising facts and noticing how [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

There are several scientific papers that claim that the variability in luminosity of young stars can be explained by the collisions of these stars with comet-like bodies, and that during such collisions, energy release would be rapid and explosive in the upper layers of the solar atmosphere (similar to the explosion of the Chelyabinsk meteorite).

Furthermore, it is known that Kreutz comet progenitor with a diameter of 100-200 kilometers passed at a distance of 1.3 solar radii from the Sun around 2,300 years ago. Due to the very high escape velocity at the Sun, a body falling from infinity would have a velocity of 618 kilometers per second at its surface. This gives us energy estimates: when a body with a diameter of 100 km falls, the flare energy would be 100 times greater than the energy radiated by the Sun per second, and if a 1000-kilometer body falls [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Claude Code has radically changed what it means for me to be a programmer. It's made me much more productive. I’m able to get work done in hours that would have previously taken me days. On the surface it probably looks like I’m only about 1.5x more productive in terms of time to deliver results, but when you factor in that the quality of the code I can write in the same amount of time is now much higher because coding agents make it cheaper to put in the effort to write high quality code, it's easily a 3x-5x increase in total productivity, assuming you agree with me that quality code is valuable.

The natural questions to ask are “how do I do it?” and “can you do it, too?”

Let's tackle these questions in order.

The Mechanics of 5x Engineering

Arguably, the way I achieve 5x productivity at [...]

---

Outline:

(00:53) The Mechanics of 5x Engineering

(10:02) Can You Become a 5x Engineer?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

"I heard Chen started distilling the day after he was born. He's only four years old, if you can believe it. He's written 18 novels. His first words were, "I'm so here for it!" Adrian said.

He's my little brother. Mom was busy in her world model. She says her character is like a "villainess" or something - I kinda worry it's a sex thing. It's for sure a sex thing. Anyway, she was busy getting seduced or seducing or whatever villanesses do in world models, so I had to escort Adrian to Oak Central for the Lit Olympiad. Mom doesn't like supervision drones for some reason. Thinks they're creepy. But a gangly older sister looming over him and witnessing those precious adolescent memories for her - that's just family, I guess.

"That sounds more like a liability to me," I said. "Bad data, old models."

Chen waddled [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Audio note: this article contains 41 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Abstract

We present a formal model demonstrating how utilitarian reasoning creates a structural vulnerability that allows AI corporations to acquire a public veneer of safety at arbitrary low cost.

Drawing from the work from Houy [2014], we prove that an organisation can acquire _k_ safety minded employees for a vanishingly small premium _epsilon_.

This results formalises a well known phenomenon in AI safety, wherein researchers concerned about existential risks from AI joins an accelerationist corporation under the rationale of "changing things from the inside", without ever producing measurable safety improvements.

We discuss implications for AI governance, organisational credibility, and the limitations of utilitarian decision-making in competitive labour markets.

1) Introduction

The title is a play on It will [...]

---

Outline:

(00:22) Abstract

(01:13) 1) Introduction

(02:06) 2) Formal Framework

(04:42) 3) Implications

(06:22) 4) Future Work

(08:10) Conclusion

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

On some level, calories in calories out has to be true. But these variables are not independent. Bodies respond to exercise by getting hungry and to calorie deficit by getting tired. Even absent that, bodies know how much food they want, and if you don’t give it to them they will tell you at increasing volume until you give in (not all bodies, of course, but quiet stomachs aren’t the target market for GLP-1s). A new breed of drugs, GLP-1 agonists, offer a way out of the latter trap by telling your body you’ve eaten, even when you haven’t, but leave many people fatigued. The newest GLP-1, retatrutide, may escape that trap too, with a mechanism so beautiful I almost don’t believe it.

How Jelly Beans Becomes Fat

Unfortunately in order to understand the beauty of retatrutide, you’re going to have to learn the basics of [...]

---

Outline:

(00:56) How Jelly Beans Becomes Fat

(04:19) The Power Plant Managers

(08:21) How do GLP-1 Medications Work?

(11:04) The Side Effects

(13:32) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.