Executive Summary

---

Outline:

(00:11) Executive Summary

(02:57) Theories Of Change

(04:25) Science Of Misalignment

(06:59) Empowering Other Areas Of AGI Safety

(07:17) Evaluation Awareness

(07:53) Better Feedback On Safety Research

(08:11) Conceptual Progress On Model Psychology

(08:44) Maintaining Monitorability Of Chain Of Thought

(09:20) Preventing Egregiously Misaligned Actions

(11:20) Directly Helping Align Models

(13:30) Research Areas Directed Towards Theories Of Change

(13:53) Model Biology

(15:48) Helping Direct Model Training

(15:55) Monitoring

(17:10) Research Areas About Robustly Useful Settings

(18:00) Reasoning Model Interpretability

(22:42) Automating Interpretability

(23:51) Basic Science Of AI Psychology

(24:08) Finding Good Proxy Tasks

(25:05) Discovering Unusual Behaviours

(26:12) Data-Centric Interpretability

(27:16) Model Diffing

(28:05) Applied Interpretability

(29:01) Appendix: Motivating Example For Why Reasoning Model Interp Breaks Standard Techniques {#appendix:-motivating-example-for-why-reasoning-model-interp-breaks-standard-techniques}

The original text contained 16 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Executive Summary

---

Outline:

(00:10) Executive Summary

(03:00) Introduction

(03:44) Motivating Example: Steering Against Evaluation Awareness

(06:21) Our Core Process

(08:20) Which Beliefs Are Load-Bearing?

(10:25) Is This Really Mech Interp?

(11:27) Our Comparative Advantage

(14:57) Why Pivot?

(15:20) Whats Changed In AI?

(16:08) Reflections On The Fields Progress

(18:18) Task Focused: The Importance Of Proxy Tasks

(18:52) Case Study: Sparse Autoencoders

(21:35) Ensure They Are Good Proxies

(23:11) Proxy Tasks Can Be About Understanding

(24:49) Types Of Projects: What Drives Research Decisions

(25:18) Focused Projects

(28:31) Exploratory Projects

(28:35) Curiosity Is A Double-Edged Sword

(30:56) Starting In A Robustly Useful Setting

(34:45) Time-Boxing

(36:27) Worked Examples

(39:15) Blending The Two: Tentative Proxy Tasks

(41:23) What's Your Contribution?

(43:08) Jack Lindsey's Approach

(45:44) Method Minimalism

(46:12) Case Study: Shutdown Resistance

(48:28) Try The Easy Methods First

(50:02) When Should We Develop New Methods?

(51:36) Call To Action

(53:04) Acknowledgments

(54:02) Appendix: Common Objections

(54:08) Aren't You Optimizing For Quick Wins Over Breakthroughs?

(56:34) What If AGI Is Fundamentally Different?

(57:30) I Care About Scientific Beauty and Making AGI Go Well

(58:09) Is This Just Applied Interpretability?

(58:44) Are You Saying This Because You Need To Prove Yourself Useful To Google?

(59:10) Does This Really Apply To People Outside AGI Companies?

(59:40) Aren't You Just Giving Up?

(01:00:04) Is Ambitious Reverse-engineering Actually Overcrowded?

(01:00:48) Appendix: Defining Mechanistic Interpretability

(01:01:44) Moving Toward Mechanistic OR Interpretability

The original text contained 47 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In this paper, we make recommendations for how middle powers may band together through a binding international agreement and achieve the goal of preventing the development of ASI, without assuming initial cooperation by superpowers.

You can read the paper here: asi-prevention.com

In our previous work Modelling the Geopolitics of AI, we pointed out that middle powers face a precarious predicament in a race to ASI. Lacking the means to seriously compete in the race or unilaterally influence superpowers to halt development, they may need to resort to a strategy we dub “Vassal's Wager”: allying themselves with a superpower and hoping that their sovereignty is respected after the superpower attains a DSA.

Of course, this requires superpowers to avert the extinction risks posed by powerful AI systems, something over which middle powers have little or no control over. Thus, we argue that it is in the interest of most middle powers to collectively deter and prevent the development of ASI by any actor, including superpowers.

In this paper, we design an international agreement that could enable middle powers to form a coalition capable of achieving this goal. The agreement we propose is complementary to a “verification [...]

---

Outline:

(01:48) Key Mechanisms

(03:36) Path to Adoption

(04:48) Urgency

---

First published:

Source:

Linkpost URL:
https://asi-prevention.com

---

Narrated by TYPE III AUDIO.

Claude Opus 4.5 is the best model currently available.

No model since GPT-4 has come close to the level of universal praise that I have seen for Claude Opus 4.5.

It is the most intelligent and capable, most aligned and thoughtful model. It is a joy.

There are some auxiliary deficits, and areas where other models have specialized, and even with the price cut Opus remains expensive, so it should not be your exclusive model. I do think it should absolutely be your daily driver.

Table of Contents

It's The Best Model, Sir

Here is the full picture of where we are now (as mostly seen in Friday's post):

You want to be using Claude Opus 4.5.

That is especially true for coding, or if [...]

---

Outline:

(00:59) It's The Best Model, Sir

(03:18) Huh, Upgrades

(04:50) On Your Marks

(09:12) Anthropic Gives Us Very Particular Hype

(13:35) Employee Hype

(15:40) Every Vibe Check

(18:16) Spontaneous Positive Reactions

(21:44) Reaction Thread Positive Reactions

(28:39) Negative Reactions

(30:34) The Lighter Side

(31:27) Popularity

(33:26) You've Got Soul

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In my previous post Traditional Food*, I explained how what we think of as a "traditional" diet is a nationalist propaganda campaign that's making us sick. In this post I'll go into the biological mechanisms.

There are four substances that the body can metabolize: carbohydrates, fats, protein and alcohol.

In this post I'll focus on how modern carbohydrate-heavy foods (like pasta, bread and rice) are related to insulin resistance. This doesn't mean that seed oils are good for you, or that the industrial revolution hasn't changed how people consume meat. Seed oils are bad for you and people today don't eat meats the way peasants did—they ate organs and other stuff that gets fed to animals today. Alcohol is just a poison you can metabolize.

With that out of the way, it's time to learn about insulin resistance.

Insulin & Resistance

Your blood has glucose dissolved in it. Blood glucose is important because your brain relies on glucose to function. If there wasn't glucose in your blood, then you'd die. However, too much blood glucose causes problems too. Precisely what problems hyperglycemia causes isn't important right now. Instead, what matters to this post is [...]

---

Outline:

(01:04) Insulin & Resistance

(04:37) How diet changed

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Throughout November, I’ve been keeping up with the Inkhaven mandate to write and post a blogpost, of at least 500 words, every day. It's the last day of November, so how’d that go?

First and foremost: most of my blogposts from this month are pretty mediocre, by my own standards. Not necessarily bad, plausibly worthwhile, but I am not particularly impressed by them.

Largely, that's because (unlike the Inkhaven program proper) I did not set aside the entire month for post-writing. I worked most of the month, took a vacation in the last week and a half, and none of that time was primarily focused on writing. The large majority of posts were low-effort in one way or another.

About a third were long standing denizens of my drafts area. For those, I mostly got them done by abandoning whatever vision I originally had for the post, instead filling in only what was already in my head, just enough to make the post shippable at all. Thus posts like e.g.:

• Moonrise, which is one half of a really good post on AI alignment
• Where's The Capital, which is a minimal viable writeup of a project from years ago
• [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

This will be the 30th post of at least 500 words I have written this month. (I did somewhat cheat two days ago, by making a 500+ word edit to Legitimate Deliberation, which I also posted independently as a shortform.)

Inkhaven has been very much what I was hoping for. I have been wanting to write more, and this certainly did the trick. I think it will be easy to hit a once-a-week target now, something I was struggling to do before.

I came with lots of drafts I wanted to finish, and outlines, and lists of ideas. Now I've got 29 posts, many of which probably would have come into existence without Inkhaven, and certainly not so quickly.

The Inkhaven posts are certainly of lower quality, for the most part, than what I usually do. I had to average at most a day on each, by design. Many of my posts are even lower-effort sacrifice posts written quickly so that I can spend more time on a few high-effort posts such as Condensation.

During the second week, I was feeling too exhausted to write by the afternoon of each day (but recovered somewhat by the evening, to finish [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

If you’re going to learn a new skill or change in some way, going hard at it for a short intensive period beats spreading a gentler effort across months or years.

I’m on day 29 of Inkhaven, where we committed to writing a blog post a day for a month. It has been great; one of the best periods of “self-development” I’ve been in. I’ve progressed far more at the skill of putting my thoughts on the internet than some counterfactual where I wrote twice a month for a year.

The quintessential example of explosive skill acquisition is foreign language learning. It's standard advice that if you really want to speak Spanish, you should do a language immersion—travel to Mexico and only speak Spanish while there—rather than practicing with apps and textbooks for an hour a week. I’d bet that the person who spent two months hanging around Tijuana - or who immersed themselves in spanish media and telenovellas for a few months, is going to be better at Spanish than the person who has a million Duolingo points.

Why explosive acquisition works

Several reasons compound together:

Overlapping forgetting curves. If you practice a skill, the clock starts [...]

---

Outline:

(01:06) Why explosive acquisition works

(04:36) Why we don't do this more

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Here's a fun format: get a big white board, and write the years of the 21st century. Write a category; something that has many variations come out every year. Next, write your picks or favourites. Now invite everyone attending to replace a year's pick if they want and replace it with something they like.

This was a rolling game played last month at Lighthaven. I put up “Best blog post of every year” when the opportunity arose.

2000

My pick: Painless Software Schedules

This Joel SPolsky piece is applicable to far more than software. It's been the foundation or ancestor of all my good project management.

Crowd pick: Why the Future Doesn’t Need Us

Billy Joy writes with foresight about new technologies and their ethical dimensions.

2001

My pick: Suck in the Middle with Bruce

A stream of conciousness, half manic fever dream, half calculatingly ruthless ode to victory. Don’t worry about the Magic: The Gathering details, they don’t matter.

Crowd pick:

2002

My pick: Taste for Makers

Paul Graham had not yet learned brevity, but he had learned design. This is a beachhead of aesthetic - perhaps just the legible outcrop into software engineering.

Crowd pick:

2003

My [...]

---

Outline:

(00:39) 2000

(00:42) My pick: Painless Software Schedules

(00:56) Crowd pick: Why the Future Doesn't Need Us

(01:07) 2001

(01:10) My pick: Suck in the Middle with Bruce

(01:27) Crowd pick:

(01:30) 2002

(01:34) My pick: Taste for Makers

(01:49) Crowd pick:

(01:52) 2003

(01:56) My pick: Welcome To The Site

(02:13) Crowd pick:

(02:16) 2004

(02:20) My pick: What You Can't Say

(02:33) Crowd pick: Mind the Gap

(02:53) 2005

(02:57) My pick: Fable of the Dragon Tyrant

(03:11) Crowd pick:

(03:14) 2006

(03:18) My pick: The Martial Art of Rationality

(03:33) Crowd pick:

(03:36) 2007

(03:40) My pick: Policy Tug of War

(03:54) Crowd pick: The Alameda-Weehawken Burrito Tunnel

(04:10) 2008

(04:13) My pick: Security Mindset

(04:29) Crowd pick: 37 Ways Words Can Be Wrong

(04:40) 2009

(04:43) My pick: Descent: The Game That Ruined Me

(05:01) Crowd pick:

(05:04) 2010

(05:08) My pick: Ureshiku Naritai

(05:26) Crowd pick:

(05:30) 2011

(05:33) My pick: Hack Away At The Edges

(05:47) Crowd pick: The Alouside Boytmend

(05:56) 2012

(06:00) My pick: Participation in the LW Community Associated With Less Bias

(06:22) Crowd pick: Salary Negotiation: Get paid more, be more valued

(06:47) 2013

(06:50) My pick: The Lottery of Fascinations

(07:05) Crowd pick:

(07:08) 2014

(07:11) My pick: The Control Group Is Out Of Control

(07:29) Crowd pick: Meditations on MOloch

(07:44) 2015

(07:47) My pick: Not Because You Should

(08:01) Crowd pick: The Value Of A Life

(08:23) 2016

(08:26) My pick: The Pyramid and the Garden

(08:40) Crowd pick: A Possible Way To Achieve All Your Goals

(08:50) 2017

(08:54) My pick: Melting Gold, and Organizational Capacity

(09:08) Crowd pick: The Face Of The Ice

(09:28) 2018

(09:31) My pick: Write a Thousand Roads To Rome

(09:46) Crowd pick:

(09:49) 2019

(09:53) My pick: Asymetric Justice

(10:08) Crowd pick:

(10:11) 2020

(10:15) My pick:

(10:18) Crowd pick: Pain is Not the Unit of Effort

(10:32) 2021

(10:36) My pick: Self Integrity and the Drowning Child

(10:49) Crowd pick: Willingness to Look Stupid

(11:06) 2022

(11:10) My pick: Sazen

(11:25) Crowd pick:

(11:28) 2023

(11:32) My pick: Things I Learned By Spending Five Thousand Hours In Non-EA Charities

(11:51) Crowd pick:

(11:54) 2024

(11:58) My pick: Situational Awareness

(12:17) Crowd pick: No Good Along

(12:47) 2025

(12:50) My pick: On Priesthoods

(13:02) Crowd pick: An upcoming post by one of Inkhaven Residents

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I have made many many feedback forms for events I have run or been a part of. Here are some simple heuristics of mine, that I write for others' to learn from and for my collaborators in the future. Most of my events have had between 50 and 500 people in them, that's the rough range I have in mind.

1. The default format for any question is a mandatory multiple-choice, then an optional text box

Most of your form should be 1-10 questions! (e.g. "How was the food, from 1-10?") Then next to it give people an optional space to provide additional text.

All forms I make primarily look like a stack of these questions.

This is because you can get a lot of signal cheaply through getting ~100 people giving a 1-10 on how the food was, or how the talks were, or some other thing. An average of 7/10 is very different from an average of 4/10, and the latter suggests you screwed up and [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.