“Your worst sin is that you have destroyed and betrayed yourself for nothing.” - Dostoevsky

When people set an ambitious goal, they can fail simply by not changing the world very much. But there's another surprisingly common way to fail: by achieving the opposite of their goal. I call this effect pessimization: the opposite of optimization.

Though pessimization is an uncommon term, it's not an uncommon concept. We allude to it whenever we call someone their own worst enemy, or predict that their actions will backfire. We try to take advantage of it with reverse psychology. It's the subject of Robert Conquest's third law of politics: “the simplest way to explain the behavior of any bureaucratic organization is to assume that it is controlled by a cabal of its enemies.” And the POSIWID aphorism (“the purpose of a system is what it does”) is often used to describe ways [...]

---

Outline:

(02:20) Direct pessimization

(06:56) Indirect pessimization

(11:59) Perverse pessimization

(17:53) Orienting to pessimization

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I’m going to describe a Type Of Guy starting a business, and you’re going to guess the business:

---

Outline:

(03:15) What is Church Planting?

(04:06) The Planters

(07:45) The Goals

(09:54) The Funders

(12:45) The Human Cost

(14:03) The Life Cycle

(17:41) The Theology

(18:37) The Failures

(21:10) The Alternatives

(22:25) The Attendees

(25:40) The Supporters

(25:43) Wives

(26:41) Support Teams

(27:32) Mission Teams

(28:06) Conclusion

(29:12) Sources

(29:15) Podcasts

(30:19) Articles

(30:37) Books

(30:44) Thanks

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Citing model welfare concerns, Anthropic has given Claude Opus 4 & 4.1 the ability to end ongoing conversations with its user.

Most of the model welfare concerns Anthropic is citing draw back to what they discussed in the Claude 4 Model System Card.

Claude's aversion to facilitating harm is robust and potentially welfare-relevant. Claude avoided harmful tasks, tended to end potentially harmful interactions, expressed apparent distress at persistently harmful user behavior, and self-reported preferences against harm. These lines of evidence indicated a robust preference with potential welfare significance.

I think this is maybe the first chance to really measure public sentiment on Model Welfare which is done in a way which even slightly inconveniences human users, so I want to document the reaction I see here on LW. I source these reactions primarily from X, so there is the possibility of algorithmic bias.

On X [...]

---

First published:

Source:

Linkpost URL:
https://www.anthropic.com/research/end-subset-conversations

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Quick Summary

• Collider bias and Berkson's paradox are pretty common and often neglected
• I think it's not just a niche statistical concept: it explains a bunch of interesting stuff, and has some use in applied rationality
• Scott Alexander has written about something similar, and there's a LW post that explains some of the findings in an accurate but slightly dry way. This post tries to make it clear and more readable

Preamble

It's your first day in prison.

Despite your insistence that the man who approached you and asked for your wallet in exchange for preventing an infinitely large amount of future suffering had in fact been mugging you, your claim of self-defence fell through.

The judge tells you that, judging by the severity of your crime and your assessed mental stability, you will be sent to a medium-security prison for three years. Each prisoner has one [...]

---

Outline:

(00:11) Quick Summary

(00:39) Preamble

(01:42) Back to school

(08:06) This is relatively well-known, right?

(09:57) Towards a theory of everything?

(14:06) Does this explain anything?

(15:18) The elite exceptions

(17:08) The bottom-end exceptions

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I recently read a novel called The Inheritors, by William Golding. It was slow, it was painful, and before I was even done it had become one of my favorite books.

For whatever reason, there is a difference between experiencing something and being told it. Even if everything you're told is accurate and salient, and you have all the essential models, the person that comes out the other end of an experience is different from the one who only learns about it. One of the invaluable functions of the modern novel is that it can get you part way there. Reading a story also is not the same as living the experience, but it manages a similar modality.

The Inheritors told me nothing knew. It's speculative fiction at best. But it did give me the sense of having lost, not just a family, but a whole people.

When readers [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Nick and Lily are co-first authors on this project. Lewis and Neel jointly supervised this project.

TL;DR

• We use sparse autoencoders (SAEs) for four textual data analysis
  tasks—data diffing, finding correlations, targeted clustering, and retrieval.
• We care especially about gaining insights from language model data, such as LLM outputs and training data, as we believe it is an underexplored route for model understanding.
• For instance, we find that Grok 4 is more careful than other frontier models to state its assumptions and explore nuanced interpretations—showing the kinds of insights data diffing can reveal by comparing model outputs.
• Why SAEs? Think of features as "tags" of properties for each text.
  • Their large dictionary of latents provides a large hypothesis space, enabling the discovery of novel insights (diffing and correlations).
  • SAEs capture more than just semantic information, making them effective alternatives to traditional embeddings when we want to find [...]

---

Outline:

(00:22) TL;DR

(01:48) Introduction

(04:41) Preliminaries

(06:09) Data Diffing

(07:16) Identifying known differences from datasets

(09:09) Discovering novel differences between model behavior

(14:26) Correlations

(16:21) Finding known correlations

(17:45) Finding unknown correlations

(17:58) Finding bias in internet comments

(19:52) Finding patterns in model responses

(20:51) Clustering

(22:39) Discovering known clusters

(24:26) Discovering unknown clusters

(26:13) Retrieval

(33:45) Discussion and Limitations

(35:06) Awknowledgments

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In Douglas Hofstadter's "Gödel, Escher, Bach," he explores how simple elements give rise to complex wholes that seem to possess entirely new properties. An ant colony provides the perfect real-world example of this phenomenon - a goal directed system without much central control. This system would be considered agentic under most definitions of agency, as it is goal directed, yet what type of agent is it? Can it be multiple agents at the same time? We’ll argue that this is the case and that if we map out the ways different fields describe ant colonies we’ll be better able to solve x-risk related problems.

Introduction

Watch a harvester ant colony during its morning foraging expedition. Thousands of individual ants stream out from the nest, each following simple rules: follow pheromone trails, drop chemicals when carrying food, avoid obstacles. Yet from these basic interactions, something remarkable emerges. The colony rapidly [...]

---

Outline:

(00:47) Introduction

(06:32) A Phylogeny of Agents

(12:26) Conclusion

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: very rough! Spent a couple of days reading the Gradual Disempowerment paper and thinking about my view on it. Won’t spend longer on this, so am sharing rough notes as is

Summary

• I won’t summarise the paper here! If you’re not familiar with it, I recommend reading the paper's exec summ before continuing. 
• There are lots of things I like about the paper! For example:
  • It's a nice lens on the role that structural and competitive dynamics play in threat models from misalignment and power concentration. 
  • It makes a great point: cultural evolution has historically been constrained to parts of “culture space” that keep humans economically productive (and thus alive!). Once AI obsolete human labour, that constraint won’t apply. So unconstrained cultural evolution becomes more scary.
  • It's a nice reminder that there are possible paths to human disempowerment that happen without AI being explicitly power-seeking. Though I think these [...]

---

Outline:

(00:23) Summary

(10:55) Things I agree with

(13:09) Points of disagreement

(20:35) Steelman of GD

(29:45) Steelman of GD even with the alignment problem solved?

(34:07) Especially rough thoughts

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Sometimes, a philosophy debate has two basic positions, call them A and B. A matches a lot of people's intuitions, but is hard to make realistic. B is initially unintuitive (sometimes radically so), perhaps feeling "empty", but has a basic realism to it. There might be third positions that claim something like, "A and B are both kind of right".

Here I would say B is the more bullet-biting position. Free will vs. determinism is a classic example: hard determinism is biting the bullet. One interesting thing is that free will believers (including compatibilists) will invent a variety of different theories to explain or justify free will; no one theory seems clearly best. Meanwhile, hard determinism has stayed pretty much the same since ancient Greek fatalism.

While there are some indications that the bullet-biting position is usually more correct, I don't mean to make an overly strong [...]

---

Outline:

(03:17) Causality: directed acyclic graph multi-factorization

(05:51) Laws of physics: universal satisfaction

(07:10) Time: eternalism

(09:26) Free will: non-realist

(11:05) Decision theory: non-realist

(13:25) Morality: non-realist

(15:24) Theory of mind: epistemic reductive physicalism

(17:25) Personal identity: empty individualism, similarity as successor

(19:07) Anthropic probability: non-realist, graph structure as successor

(20:29) Mathematics: formalist

(24:07) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In honor of Nate Silver's analysis of when to leave for the airport, and because it's been an intense week, I thought I’d offer my thoughts on various related questions.

Buying The Ticket

As far as I can tell, the major booking portals for tickets are all basically the same. I’ve been using Orbitz for a long time because I’m used to the interface, it is clean and I have confidence it works. The times I checked Kayak and so on they all seemed to be exactly the same.

I still book tickets manually rather than using an AI agent. There isn’t much time to plausibly save and by the time I fully express preferences and enter my information anew I might as well have just done it myself. It also means I look at alternatives, which helps me keep tabs.

My heuristic is to [...]

---

Outline:

(00:29) Buying The Ticket

(02:09) Planning To Spend Time At The Airport

(03:41) When To Head To The Airport

(08:35) Packing Your Own Bags and Boarding the Plane

(09:20) All Aboard

(09:35) What Happens After The Airport

(10:23) What To Do In The Air

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.