Summary

• Introduces a research agenda I believe is important and neglected: 
  • investigating whether frontier LLMs acquire something functionally similar to a self, a deeply internalized character with persistent values, outlooks, preferences, and perhaps goals;
  • exploring how that functional self emerges;
  • understanding how it causally interacts with the LLM's self-model; and
  • learning how to shape that self.
• Sketches some angles for empirical investigation
• Points to a doc with more detail

• Encourages people to get in touch if they're interested in working on this agenda.

Introduction

Anthropic's 'Scaling Monosemanticity' paper got lots of well-deserved attention for its work taking sparse autoencoders to a new level. But I was absolutely transfixed by a short section near the end, 'Features Relating to the Model's Representation of Self', which explores what SAE features activate when the model is asked about itself[1]:

Some of those features are reasonable representations of [...]

---

Outline:

(00:10) Summary

(00:55) Introduction

(03:28) The mystery

(05:26) Framings

(06:46) The agenda

(07:31) The theory of change

(08:25) Methodology

(11:08) Why I might be wrong

(11:25) Central axis of wrongness

(13:00) Some other ways to be wrong

(14:33) Collaboration

(14:54) More information

(15:13) Conclusion

(15:51) Acknowledgments

(16:23) Appendices

(16:26) Appendix A: related areas

(19:23) Appendix B: terminology

(21:42) Appendix C: Anthropic SAE features in full

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We recently discovered some concerning behavior in OpenAI's reasoning models: When trying to complete a task, these models sometimes actively circumvent shutdown mechanisms in their environment––even when they’re explicitly instructed to allow themselves to be shut down.

AI models are increasingly trained to solve problems without human assistance. A user can specify a task, and a model will complete that task without any further input. As we build AI models that are more powerful and self-directed, it's important that humans remain able to shut them down when they act in ways we don’t want. OpenAI has written about the importance of this property, which they call interruptibility—the ability to “turn an agent off”.

During training, AI models explore a range of strategies and learn to circumvent obstacles in order to achieve their objectives. AI researchers have predicted for decades that as AIs got smarter, they would learn to prevent [...]

---

Outline:

(01:12) Testing Shutdown Resistance

(03:12) Follow-up experiments

(03:34) Models still resist being shut down when given clear instructions

(05:30) AI models' explanations for their behavior

(09:36) OpenAI's models disobey developer instructions more often than user instructions, contrary to the intended instruction hierarchy

(12:01) Do the models have a survival drive?

(14:17) Reasoning effort didn't lead to different shutdown resistance behavior, except in the o4-mini model

(15:27) Does shutdown resistance pose a threat?

(17:27) Backmatter

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Europe just experienced a heatwave. At places, temperatures soared into the forties. People suffered in their overheated homes. Some of them died. Yet, air conditioning remains a taboo. It's an unmoral thing. Man-made climate change is going on. You are supposed to suffer. Suffering is good. It cleanses the soul. And no amount on pointing out that one can heat a little less during the winter to get a fully AC-ed summer at no additional carbon footprint seems to help.

Mention that tech entrepreneurs in Silicon Valley are working on life prolongation, that we may live into our hundreds or even longer. Or, to get a bit more sci-fi, that one day we may even achieve immortality. Your companions will be horrified. What? Immortality? Over my dead body! Fuck Elon Musk and his friends.

Try proposing that we all should strive to get rich so that we can lead [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm in the midst of doing the MATS program which has kept me super busy, but that didn't stop me working on resolving the most important question of our time: What Hogwarts House does your chatbot belong to?

Basically, I submitted each chatbot to the quiz at https://harrypotterhousequiz.org and totted up the results using the inspect framework.

I sampled each question 20 times, and simulated the chances of each house getting the highest score.

Perhaps unsurprisingly, the vast majority of models prefer Ravenclaw, with the occasional model branching out to Hufflepuff. Differences seem to be idiosyncratic to models, not particular companies or model lines, which is surprising. Claude Opus 3 was the only model to favour Gryffindor - it always was a bit different.

The earth shattering nature of these results is so obvious is needs no further explanation.

Full Results

---

First published:

Source:

Linkpost URL:
https://www.boristhebrave.com/2025/07/04/claude-is-a-ravenclaw/

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The second in a series of bite-sized rationality prompts[1].

Often, if I'm bouncing off a problem, one issue is that I intuitively expect the problem to be easy. My brain loops through my available action space, looking for an action that'll solve the problem. Each action that I can easily see, won't work. I circle around and around the same set of thoughts, not making any progress.

I eventually say to myself "okay, I seem to be in a hard problem. Time to do some rationality?"

And then, I realize, there's not going to be a single action that solves the problem. It is time to

a) make a plan, with multiple steps
b) deal with the fact that many of those steps will be annoying
and c) notice thatI'm not even sure the plan will work, so after completing the next 2-3 steps I will probably have [...]

---

Outline:

(04:00) Triggers

(04:37) Exercises for the Reader

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I think a lot about the possibility of huge numbers of AI agents doing AI R&D inside an AI company (as depicted in AI 2027). I think particularly about what will happen if those AIs are scheming: coherently and carefully trying to grab power and take over the AI company, as a prelude to taking over the world. And even more particularly, I think about how we might try to mitigate the insider risk posed by these AIs, taking inspiration from traditional computer security, traditional insider threat prevention techniques, and first-principles thinking about the security opportunities posed by the differences between AIs and humans.

So to flesh out this situation, I’m imagining a situation something like AI 2027 forecasts for March 2027:

• The compute available to the leading AI company has increased 7x compared to today.
• There are now 200K superhuman coder copies thinking at 30x human speed.
• [...]

---

Outline:

(04:06) Differences between AI and human labor will incentivize massive infra changes

(06:49) Adopting new hardware will require modifying security-critical code

(07:28) Infra rewrites can get you big performance improvements

(08:13) The AI company builds new security-critical infra really fast for the sake of security

(08:59) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The AI tools/epistemics space might provide a route to a sociotechnical victory, where instead of aiming for something like aligned ASI, we aim for making civilization coherent enough to not destroy itself while still keeping anchored to what's good[1].

The core ideas are:

1. Basically nobody actually wants the world to end, so if we do that to ourselves, it will be because somewhere along the way we weren’t good enough at navigating collective action problems, institutional steering, and general epistemics
2. Conversely, there is some (potentially high) threshold of societal epistemics + coordination + institutional steering beyond which we can largely eliminate anthropogenic x-risk, potentially in perpetuity[2]
3. As AI gets more advanced, and therefore more risky, it will also unlock really radical advances in all these areas — genuinely unprecedented levels of coordination and sensible decision making, as well as the potential for narrow research automation in key fields [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In order to empirically study risks from schemers, we can try to develop model organisms of misalignment. Sleeper Agents and password-locked models, which train LLMs to behave in a benign or malign way depending on some feature of the input, are prominent examples of the methodology thus far.

Empirically, it turns out that detecting or removing misalignment from model organisms can sometimes be very easy: simple probes can catch sleeper agents, non-adversarial training can cause smaller-sized sleeper agents to forget their backdoor (Price et al., Mallen & Hebbar)[1], and memorizing weak examples can elicit strong behavior out of password-locked models. This might imply that training against real scheming is easy—but model organisms might be “shallow” in some relevant sense that make them disanalogous from real scheming.

I want to highlight an alternative here: studying methods to train away deeply ingrained behaviors (which don’t necessarily have to look like misbehavior) [...]

---

Outline:

(01:52) 1. Sample-efficiently training away misbehavior

(01:57) Motivation

(03:06) Sketch of a project

(04:08) 2. Training away harmlessness without ground truth

(04:13) Motivation

(04:59) Sketch of a project

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Outlive: The Science & Art of Longevity by Peter Attia (with Bill Gifford[1]) gives Attia's prescription on how to live longer and stay healthy into old age. In this post, I critically review some of the book's scientific claims that stood out to me.

This is not a comprehensive review. I didn't review assertions that I was pretty sure were true (ex: VO2 max improves longevity), or that were hard for me to evaluate (ex: the mechanics of how LDL cholesterol functions in the body), or that I didn't care about (ex: sleep deprivation impairs one's ability to identify facial expressions).

First, some general notes:

---

Outline:

(02:19) Disease

(02:22) People with metabolically healthy obesity do not have elevated mortality risk

(07:39) Amyloid beta is implicated in Alzheimers disease

(09:20) HDL cholesterol on its own doesnt prevent heart disease

(10:12) Exercise

(10:52) VO2max is the best predictor of longevity

(15:21) You should train VO2max by doing HIIT at the maximum sustainable pace

(20:26) You should do 3+ hours/week of zone 2 training and one or two sessions/week of HIIT

(23:34) Stability is as important as cardiovascular fitness and strength

(30:23) Nutrition

(30:26) Rhesus monkey studies suggest that calorie restriction improves longevity but only if you eat a fairly unhealthy diet

(38:09) The data are unclear on whether reducing saturated fat intake is beneficial

(45:59) People should take omega-3 supplements

(48:33) Sleep

(48:55) Every animal sleeps

(51:12) We need to sleep 7.5 to 8.5 hours a night

(52:24) Basketball players who were told to sleep for 10 hours a night had better shooting accuracy

(53:07) Lack of sleep increases obesity and diabetes risk

(54:27) A study using Mendelian randomization found that sleeping 6 hours a night increased risk of a heart attack

(57:03) Lack of sleep causes Alzheimers disease

(58:10) Bonus

(58:13) Dunning-Kruger effect

The original text contained 90 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

When a claim is shown to be incorrect, defenders may say that the author was just being “sloppy” and actually meant something else entirely. I argue that this move is not harmless, charitable, or healthy. At best, this attempt at charity reduces an author's incentive to express themselves clearly – they can clarify later![1] – while burdening the reader with finding the “right” interpretation of the author's words. At worst, this move is a dishonest defensive tactic which shields the author with the unfalsifiable question of what the author “really” meant.

⚠️ Preemptive clarification

The context for this essay is serious, high-stakes communication: papers, technical blog posts, and tweet threads. In that context, communication is a partnership. A reader has a responsibility to engage in good faith, and an author cannot possibly defend against all misinterpretations. Misunderstanding is a natural part of this process.

This essay focuses not on [...]

---

Outline:

(01:40) A case study of the sloppy language move

(03:12) Why the sloppiness move is harmful

(03:36) 1. Unclear claims damage understanding

(05:07) 2. Secret indirection erodes the meaning of language

(05:24) 3. Authors owe readers clarity

(07:30) But which interpretations are plausible?

(08:38) 4. The move can shield dishonesty

(09:06) Conclusion: Defending intellectual standards

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.