It started as so many dooms do, with a flash in the night sky over the South China Sea. Testing a new ASAT weapon, the Chinese military shattered a derelict spy satellite into 40,000 shards of shrapnel. The debris pattern suggested a fragmentation warhead optimized for lethal scatter. Within 48 hours, the U.S. responded with a demonstration shot through a Chinese weather satellite. Russia, not to be outdone, lobbed a kinetic strike at OneWeb's constellation—a “warning shot” that missed its mark by 300 meters, spraying shrapnel through the UK network's 1,200 km orbital shell.

In Hangzhou, a different kind of flash heralded the deployment of DeepSeek Alpha, a 12 MB distillation of a DeepSeek R2 instance finetuned specifically for trading. Strapped for compute, DeepSeek had designed Alpha to scavenge idle compute cycles wherever it could find them. Soon, the model spread through forgotten AWS instances, idle work laptops, and [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

In this new paper, I discuss what it would mean for AI systems to be persons — entities with properties like agency, theory-of-mind, and self-awareness — and why this is important for alignment. In this post, I say a little more about why you should care.

The existential safety literature focuses on the problems of control and alignment, but these framings may be incomplete and/or untenable if AI systems are persons.

The typical story is approximately as follows.

AI x-safety problem (the usual story):

1. Humans will (soon) build AI systems more intelligent/capable/powerful than humans;
2. These systems will be goal-directed agents;
3. These agents’ goals will not be the goals we want them to have (because getting intelligent agents to have any particular goal is an unsolved technical problem);
4. This will lead to misaligned AI agents disempowering humans (because this is instrumentally useful for their actual goals).

Solution [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

January: In early 2026, Meta launches a fleet of new AI influencers, targeting the massive audience displaced by the Xiaohongshu-TikTok wars. They are beautiful, funny, smart—whatever you want them to be. Equipped with the latest in online learning, the agents immediately begin adapting to social media trends as they occur. Engagement metrics reach new highs. Nobody questions whose engagement they're measuring.

February: Fads pass faster now. The influencers are speed-running a revival movement of all viral trends since early naughts. For a week, social platforms witness an explosive resurgence of finger-tutting – shows of intricate finger choreography merging dance and calligraphy. Just as suddenly, the influencers deem this passé and shift to endless variations of sea shanties. Then both. Engagement metrics soar beyond human limits as silicon minds start to smoke their own supply.

Figure 1: "Daft Hands – Harder, Better, Faster, Stronger" (2007). This video from early [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Attorney General of Massachusetts recently announced that drivers for ride-sharing companies must be paid at least $32.50 per hour.

Now, if you’re a hardcore libertarian, then you probably hate with the minimum wage. You need no convincing and we can part now on good terms.

But what if you’re part of the vast majority of people who like the minimum wage? What if you think the minimum wage is awesome and you want to expand it? That's fine. I won’t try to change your mind! But even so, there are strong reasons to be skeptical about this policy.

Because: If you read closely, you’ll see that Massachusetts’ rule is that drivers must be paid $32.50 for hours that they spend driving passengers around. For the time they spend waiting for passengers to request rides, they will still be paid $0. And when you screw around with prices, you [...]

---

Outline:

(01:25) Story time

(08:31) Science time

(10:17) Math time

(15:27) Discussion time

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

And then we hit a wall.

Nobody expected it. Well... almost nobody. Yann LeCun posted his "I told you so's" all over X. Gary Marcus insisted he'd predicted this all along. Sam Altman pivoted, declaring o3 was actually already ASI.

The first rumors of scaling laws breaking down were already circulating in late 2024. By late 2025, it was clear that test-time scaling was not coming to the rescue. Despite the scaling labs' best efforts, nobody could figure out how to generalize reasoning beyond the comfortable confines of formally verifiable domains. Sure, you could train reasoning models on math and a little bit on coding, but that was it — transformers had reached their limits.

Or so we thought. It turned out there was still plenty of room to explore in the high reaches of mathematics. Building on foundations laid by o1, Minerva, and AlphaProof, DeepMind's [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

“Anomalous”, “glitch”, or “unspeakable” tokens in an LLM are those that induce bizarre behavior or otherwise don’t behave like regular text.

The SolidGoldMagikarp saga is pretty much essential context, as it documents the discovery of this phenomenon in GPT-2 and GPT-3.

But, as far as I was able to tell, nobody had yet attempted to search for these tokens in DeepSeek-V3, so I tried doing exactly that. Being a SOTA base model, open source, and an all-around strange LLM, it seemed like a perfect candidate for this.

This is a catalog of the glitch tokens I've found in DeepSeek after a day or so of experimentation, along with some preliminary observations about their behavior.

Note: I’ll be using “DeepSeek” as a generic term for V3 and r1.

Process

I searched for these tokens by first extracting the vocabulary from DeepSeek-V3's tokenizer, and then automatically testing every one of them [...]

---

Outline:

(00:55) Process

(03:30) Fragment tokens

(06:45) Other English tokens

(09:32) Non-English

(12:01) Non-English outliers

(14:09) Special tokens

(16:26) Base model mode

(17:40) Whats next?

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a linkpost for Apollo Research's new interpretability paper:

"Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition".

We introduce a new method for directly decomposing neural network parameters into mechanistic components.

Motivation

At Apollo, we've spent a lot of time thinking about how the computations of neural networks might be structured, and how those computations might be embedded in networks' parameters. Our goal is to come up with an effective, general method to decompose the algorithms learned by neural networks into parts that we can analyse and understand individually.

For various reasons, we've come to think that decomposing network activations layer by layer into features and connecting those features up into circuits (which we have started calling 'mechanisms'[1]) may not be the way to go. Instead, we think it might be better to directly decompose a network's parameters into components by [...]

---

Outline:

(00:27) Motivation

(01:56) What we do

(06:25) Future work

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[Epistemic status: slightly ranty. This is a lightly edited slack chat, and so may be lower-quality.]

I am surprised by the perennial spikes in excitement about "polytopes" and "tropical geometry on activation space" in machine learning and interpretability[1]. I'm not going to discuss tropical geometry in this post in depth (and might save it for later -- it will have to wait until I'm in a less rant-y mood[2]).

As I'll explain below, I think some interesting questions and insights are extractable by suitably weakening the "polytopes" picture, and a core question it opens up (that of "statistical geometry" -- see below) is very deep and worth studying much more systematically. However if taken directly as a study of rigid mathematical objects (polytopes) that appear as locally linear domains in neural net classification, what you are looking at is, to leading order, a geometric form of noise.

---

Outline:

(02:31) Issues with polytopes

(02:35) Holistically understanding or enumerating polytopes is intractable.

(04:39) Polytopes are noisy

(06:04) Statistical complexity measures

(07:00) Humayun et al. and the trampoline analogy

(09:51) Relation to RLHF and measuring toxicity

(15:15) Softening the rigid structure: statistical geometry

(15:42) From hard to soft

(21:12) Other polytope-adjacent things I think are inherently interesting

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Where the challenge of aligning an LLM-based AI comes from, and the obvious solution.

Evolutionary Psychology is the Root Cause

LLMs are pre-trained using stochastic gradient-descent on very large amounts of human-produced text, normally drawn from the web, books, journal articles and so forth. A pre-trained LLM has learned in detail how to simulate all the different human text-generation processes that produced this text — everything from a cooperatively edited wikipedia article to shit-postings. We are thus 'distilling' human intelligence into the pre-trained LLM.[1]

This has many advantages for alignment: an LLM pre-trained this way understands and produces output using human language and ontologies, and also has a deep understanding of human values and ethics — thus avoiding a number of issues around alignment that were major concerns before it became apparent that our first AGI was very likely to be based on or at least incorporate an LLM.

---

Outline:

(00:12) Evolutionary Psychology is the Root Cause

(03:42) Distilling Humans is the Problem, so Use Synthetic Pre-Training Data

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I found this interview notable as the most useful public statement yet of Yudkowsky's views. I congratulate both him and the host, Dan Fagella, for strategically improving how they're communicating their ideas.

The host, Dan Fagella is to be commended for asking the right questions and taking the right tone to get an extraordinarily concise statement of Yudkowsky's views on what we might do to survive and why. It also seemed likely that Yudkowsky has thought hard about his messaging after having his views both deliberately and accidentally misunderstood and panned. Despite having followed his thinking over the last 20 years, I gained new perspective on his current thinking from this interview.

Takeaways:

• Humans will probably fail to align the first takeover-capable AGI and all die
  • Not because alignment is impossible
  • But because humans are empirically foolish
  • And historically rarely get hard [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.