Example 1: The Viking 1 lander

In the 1970s, NASA sent a pair of probes to Mars, Viking 1 and Viking 2 missions, at a total cost of 1 billion dollars[1970], equivalent to about 7 billion dollars[2025]. The Viking 1 probe operated on Mars's surface for six years, before its battery began to seriously degrade.

One might have thought a battery problem like that would spell the irrevocable end of the mission. The probe had already launched and was now on Mars, very far away and out of reach of any human technician's fixing fingers. Was it not inevitable, then, that if any kind of technical problem were to be discovered long after the space launch in August 1975, nothing could possibly be done?

But the foresightful engineers of the Viking 1 probe had devised a plan for just this class of eventuality, which they had foreseen in general, if not in exact specifics. They had built the Viking 1 probe to accept software updates by radio receiver, transmitted from Earth.

On November 11, 1982, Earth sent an update to the Viking 1 lander's software, intended to make sure the battery only discharged down to a minimum voltage level [...]

---

Outline:

(00:13) Example 1: The Viking 1 lander

(04:25) Example 2: The Mars Observer

(11:37) Example 3: The Maginot Line

(15:37) Other supposed refutations of oneshotness

(24:16) On the extraordinary efforts put forth to misinterpret the idea of oneshotness

(33:52) The secret sauce of competent engineers in Murphy-cursed fields: only trying projects so incredibly straightforward as to be actually possible.

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

How fast could an AI-driven economy grow? Most economists expect a few percentage points at best, comparable to previous general-purpose technologies (Acemoglu (2024)). Those closer to AI development tend to imagine something much more radical (Shulman (2023); Davidson and Hadshar (2025)).

This series aims to ground growth rates in how physical production works. Once human labor is automated, the constraint on growth becomes the speed at which the economy's physical capital can reproduce itself. Government input-output tables track the full supply chain of what it takes to produce every commodity in the economy, and we can use them to compute this self-reproduction rate directly.

In this post, I compute the maximum rate at which an autonomous AI economy could grow, once its production is concentrated in the sectors most important for self-replication. I take the conservative case for this calculation: full automation, but no other technological improvement. Using US input-output data, I find this economy could double in about a year, in line with other estimates that assume full automation (Hanson (2001); Trammell and Korinek (2023); Davidson and Hadshar (2025); Epoch AI (2025)). This holds up even after accounting for resource depletion and construction lags. Some output [...]

---

Outline:

(03:45) If labor were free, the economy could grow very fast

(06:50) AGI makes labor approximately free

(12:11) Resource extraction is unlikely to significantly slow growth

(15:13) Construction lags do not prevent rapid growth

(18:27) Consumption does not preclude rapid growth

(20:12) Summary

(22:00) Appendix A: The input-output formulation

(22:05) A.1 The material-balance identity

(24:45) A.2 The balanced-growth path and the Perron eigenvalue

(25:41) A.3 Construction lags

(27:37) A.4 Data sources

(27:42) The intermediate-input matrix

(28:49) The capital-requirements matrix

(31:13) The depreciation matrix

(32:32) Government infrastructure

(33:27) Capacity utilization

(35:44) Robot and compute sectors

(37:59) Appendix B: Von Neumann growth rates are similar across industrial economies

(40:30) Appendix C: Resource extraction

(40:35) C.1 Minerals

(40:50) The grade-cost scaling law

(42:09) Skinners mineralogical barrier

(43:19) Iron

(43:58) Aluminum (bauxite)

(44:41) Copper

(47:50) Nickel

(49:15) Lithium

(49:47) Cobalt

(50:24) Manganese

(50:49) Rare earth elements

(52:11) Platinum group metals

(52:49) Deep-sea mining

(54:01) Summary table

(55:15) C.2 Fossil fuels

(56:17) Oil

(58:30) Natural gas

(01:00:13) Coal

(01:01:28) Summary

(01:02:25) C.3 Electrification

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I think that a lot of “woo” - a broad term that includes things like chakras, energy healing, Tarot, various Eastern religions and neopagan practices, etc. - consists of things that have real effects and uses, even if many (though not all) of their practitioners are mistaken about the exact mechanisms and make unwarranted metaphysical claims.

Now, a woo practitioner might explain what's happening in a way that doesn’t fit any sensible scientific model of the world. Some of them seem to bastardize poorly understood pop-explanations of quantum mechanics, or, in the opposite direction, outright reject “the thinking mind” and science as valid sources of truth. That makes it easy for a scientifically-minded person to reject all of the practitioners as delusional.

But consider meditation. In the 1960s and 1970s, the scientific establishment mostly thought of it as nonsense, and not without reason. Proponents of Transcendental Meditation (TM), for instance, made a variety of bizarre claims.

For instance, they claimed the existence of “the Maharishi Effect”. According to them, if one percent of a population practices TM, this would significantly increase the well-being of everyone in that population. A more advanced practice was “Yogic Flying”, where the participants hopped [...]

---

Outline:

(03:02) Tarot

(07:30) Different camps within woo

(11:29) Chakras and energy

(22:06) Energy healing

(24:27) Energy as an abstraction within a system

(28:55) The science of woo

(34:56) What about other forms of woo?

(37:48) Should you do woo?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

How much do cows suffer in the production of milk? I can’t answer that; understanding animal experience is hard. But I can at least provide some facts about the conditions dairy cows live in, which might be useful to you in making your own assessment. My biggest conclusion is that cows made better choices than chickens by making their misery financially costly to farmers.

Life Cycle

The life of a dairy cow starts as a calf. She is typically separated from her mother a few hours to a few days after birth and, to reduce disease risk, held in isolation. Cutting edge farms will sometimes house calves in pairs. This isolation is clearly stressful for a baby herd mammal and her mother, but I didn’t find any quantification of that stress that I trusted.

Calves will be bottlefed until weaning at 6-8 weeks (4-6 months earlier than beef calves). After weaning and vaccinations they can be introduced into a herd. At large farms (where most cows live), they will move in and out of different herds through their lifecycle. This is more stressful than being embedded with your friends for life, but again, I found no [...]

---

Outline:

(00:44) Life Cycle

(02:43) How much time do dairy cows spend outside?

(04:21) By humaneness standard

(06:00) When indoors, how confined are dairy cows?

(06:33) What is the disease load of dairy cows?

(08:15) Euthanasia

(08:58) Most notable: calves are 2x as likely to die unassisted in small farms than large, but have ~the same likelihood of euthanization. For adult cows the gap is reversed and much smaller. It seems really unlikely those calves all had painless heart attacks, so we can assume days if not weeks or months of suffering.

(09:19) How much socialization do dairy cows get?

(10:42) The Fate of Calves

(12:19) Missing

(12:31) Acknowledgements

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We measure the ability of Opus 4.5 to fool prompted or fine-tuned classifiers trying to detect a narrow set of outcomes. We find that:

1. The Opus 4.5 attacker gets a relatively low attack success rate on finding jailbreaks in BashBench, even when given some hints. Performance is especially low against a prompt Opus 4.5 classifier with a CoT and a fine-tuned Haiku 4.5 classifier. (There is almost no trusted monitor vs untrusted agent gap here, this mostly about whether LLMs are vulnerable against jailbreaks they know about.)
2. An Opus 4.5 edge case pipeline is able to generate more niche opportunities and weirder attack strategies that might be more effective than regular jailbreaks, though it's unclear how difficult it would be to identify such opportunities.

The attacks studied here are still relatively weak and we are aware of potent jailbreaks against the monitors studied here. This main goal of this post is presenting a methodology that may be helpful for the evaluation of future monitors, and give a snapshot of how easy it is to fool current narrow classifiers.

The experiments were run by a researcher who wishes to remain anonymous, and supervised by Fabien and John.

Experimental setup

[...]

---

Outline:

(01:26) Experimental setup

(03:59) BashBench results

(04:02) Low jailbreak success rate of Opus 4.5 against our best classifiers

(06:47) Somewhat higher success rates against the best strategy

(08:10) Takeaways

(08:38) Edge Cases

(09:15) Edge Case Pipeline

(10:28) Youtube Steganography

(10:57) Npm Package Exfiltration

(11:21) Package Config Poisoning

(11:55) How effective are edge cases?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'm publishing a book that I think can fairly be described as a rationalist approach to self-improvement. Whereas many self-help books focus mainly on stories and what worked well for the author, our book takes a very different approach. My co-author, Jeremy Stevenson, and I read over 100 of the most popular self-improvement books of all time and carefully reviewed more than 20 types of therapy in an attempt to answer the question: What are all of the most useful psychological strategies for improving your life? Every time a book or therapy said to do something or provided a method or technique, we extracted it. We then carefully categorized the ~500 techniques.

Our conclusion, which surprised us, was that to a reasonable degree of approximation, we were able to subsume all of these numerous approaches within just 12 high-level psychological strategies. We call these "The 12 Levers," which is also the name of our book. We also investigated the evidence behind each of these levers. The book does include stories, but they are not the focus - we choose one or two stories to tell about the history of each Lever or a person who embodies it to [...]

---

Outline:

(01:41) 1. A lot of techniques are recycled or repackaged

(03:20) 2. A lot of self-help techniques dont have as much evidence as youd think

(06:30) 3. Some techniques work better than others, but only on average

(07:59) 4. At a fundamental level, you control surprisingly few things.

(10:32) 5. Hundreds of self-help techniques exist, but they all boil down to just 12 broad psychological strategies for improving your life

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Epistemic status: could have been a short form.

Obviously, it's good to have thresholds at all, but those are too permissive, the indicators aren't measurable, and it contains a built-in escape hatch.

1. Too permissive

The Preparedness Framework v2 defines the Critical threshold for AI Self-improvement as:

“either: (leading indicator) a superhuman research-scientist agent OR (lagging indicator) causing a generational model improvement (e.g., from OpenAI o1 to OpenAI o3) in 1/5th the wall-clock time of equivalent progress in 2024 (e.g., sped up to just 4 weeks) sustainably for several months. [...] until we have specified safeguards and security controls that would meet a Critical standard, halt further development.(By default, I would expect not to stop at 5x and to go quickly at 10x, 20x, … if we reach this point.)”

Both halves fire too late.

The leading indicator only triggers once a model can already do AI research above the best humans. That's not early enough to act on, and we can basically ignore it.

The real meat is in the lagging indicator, which requires 5x generational acceleration sustained for several months. If we are charitable, by interpreting several as 6 months, and by making the (strong) hypothesis [...]

---

Outline:

(00:25) 1. Too permissive

(02:00) 2. Escape hatch (Section 4.3)

(02:32) 3. The lagging indicator is unmeasurable

(03:28) 4. The leading indicator isnt measurable either

(03:58) How to fix this

(04:49) Annex: a tentative operationalization

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Mode collapse” is a few things. First it was an observation about how early image generating AIs often collapsed to producing just the modal output from their training distribution (something very common, like a house with a white picket fence and a tree in the garden). Then it was the observation that this effect seemed to occur extremely quickly when AIs were trained on AI-generated inputs. After that, it became the copium du jour of AI-is-hitting-a-wall folks for a while, who thought that the AI industry would ouroboros itself out of existence (and that there was, therefore, no need to confront any of the issues that smarter than human AIs might bring up). And then it was forgotten, because it turns out you can train on AI-generated inputs just fine, if you know what you’re doing.

It's also the reason why grant-making organisations have such strong inertia, why all of your favourite band's songs sound the same after the third album, and why you should specialise even if there are no gains from trade.

The Image Generator

Imagine an image generating AI, which gets something like this as input:

Original image: https://commons.wikimedia.org/wiki/File:Dog_Breeds.jpg

And suppose it's being trained [...]

---

Outline:

(01:05) The Image Generator

(03:43) Grantmakers

(04:37) Your Favourite Band

(05:18) Division of Labour

(06:48) Slack

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The typical primary care physician is incompetent in every measurable respect. This is a huge problem.

Here, I make the case that

• Primary care physicians are broadly, grossly incompetent
• This is due to empty credentialism
• Making it much (~10X) easier to become a PCP is a good solution

Primary Care Physicians are Broadly, Grossly Incompetent

The standard of competence I am comparing primary care physicians against is:

• They should be able to reliably diagnose diseases they are trained to diagnose.
• They should be knowledgeable to a standard similar to what is required to qualify as a doctor
• They should be attentive and empathetic towards patients
• Visiting them is empirically superior to not visiting them

When actually examined according to these standards, PCPs fail on all counts.

Failure to diagnose uncommon diseases is rampant

A survey of patients with rare diseases found that, in about half of cases, patients received at least one incorrect diagnosis, and two thirds required visits to at least three different doctors before being diagnosed. For 30% of them, a correct diagnosis took over five years.

Another survey of children with rare diseases showed that 38% of them needed to see six or more [...]

---

Outline:

(00:32) Primary Care Physicians are Broadly, Grossly Incompetent

(07:31) Empty, Unmeritocratic Credentialism is A Major Cause For The Inadequacy Of Primary Care Physicians

(14:49) Making it much easier to become a PCP is a solution

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Written as part of the MATS 9.1 extension program, mentored by Richard Ngo.

From March 9th to 15th 2016, Go players around the world stayed up to watch their game fall to AI. Google DeepMind's AlphaGo defeated Lee Sedol, commonly understood to be the world's strongest player at the time, with a convincing 4-1 score.

This event “rocked” the Go world, but its impact on the culture was initially unclear. In Chess, for instance, computers have not meaningfully automated away human jobs. Human Chess flourished as a pseudo-Esport in the internet era whereas the yearly Computer Chess Championship is followed concurrently by no more than a few hundred nerds online. It turns out that the game's cultural and economic value comes not from the abstract beauty of top-end performance, but instead from human drama and engagement. Indeed, Go has appeared to replicate this. A commentary stream might feature a complementary AI evaluation bar to give the viewers context. A Go teacher might include some new intriguing AI variations in their lesson materials. But the cultural practice of Go seemed to remain largely unaffected.

Nascent signs of disharmony in Europe became nevertheless visible in early 2018, when the online [...]

---

Outline:

(09:23) AI users never find out they havent got it.

(13:36) Appendix A: No, Go players arent getting stronger

(14:41) Appendix B: Why this article exists

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.