I recently had a conversation with a friend of a friend who has a very curious child around 5 years of age. I offered to answers some of their questions, since I love helping people understand the world. They sent me eight questions, and I answered them by hand-written letter. I figured I'd also post my answers here, since it was both a fun exploration of the object-level questions, and a really interesting exercise in epistemics.

Thank you for your questions! I find that asking questions about the world and figuring out some answers is one of the most enjoyable ways to spend life.

For some questions, like "what is 2 + 2?" or "where is my phone?" there is a single clear and correct answer. But for most questions, and especially most of the interesting ones, it's more about bringing your attention to the topic and finding out whatever you can. I could have answered each of your questions with pages and pages of information. And different people would have given you different answers, each of which would give you different information relevant to the question.

Some questions you can spend your whole life getting better answers to. [...]

---

Outline:

(01:55) What is electricity? Can I see it?

(03:49) If the earth had hair, what would it be?

(04:35) Why do birds fly, but I cant?

(05:16) Who built Boston?

(06:26) Do clouds eat?

(07:03) Do ants have friends?

(07:35) What is inside the fridge that makes it cold?

(09:10) What if the floor was lava for real?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The idea of a “basin of attraction around corrigibility” motivates much of prosaic alignment research. Essentially this is an abstract way of thinking about the process of iteration on AGI designs. Engineers test to find problems, then understand the problems, then design fixes. The reason we need corrigibility for this is that a non-corrigible agent generally has incentives to interfere with this process. The concept was introduced by Paul Christiano:

... a corrigible agent prefers to build other agents that share the overseer's preferences — even if the agent doesn’t yet share the overseer's preferences perfectly. After all, even if you only approximately know the overseer's preferences, you know that the overseer would prefer the approximation get better rather than worse.
Thus an entire neighborhood of possible preferences lead the agent towards the same basin of attraction. We just have to get “close enough” that we are corrigible, we don’t need to build an agent which exactly shares humanity's values, philosophical views, or so on.
In addition to making the initial target bigger, this gives us some reason to be optimistic about the dynamics of AI systems iteratively designing new AI systems. Corrigible systems want to design more corrigible [...]

---

Outline:

(02:40) Corrigibility

(07:45) My argument: The engineering feedback loop will use up all its fuel

(08:31) There are many fields where test-driven engineering is far from sufficient

(10:41) The corresponding story for iteratively building corrigible AGI

(12:18) How do we know the distribution shift will stress parts of the AGI?

(16:13) Tying this back to the basin of attraction

(18:09) A counterargument: The prosaic case

(21:19) More specific counterarguments

(25:07) One way things could go well

(27:00) Conclusion and implications

The original text contained 16 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

the Constellation-class frigate

Last month, the US Navy's Constellation-class frigate program was canceled. The US Navy has repeatedly failed at making new ship classes (see the Zumwalt, DDG(X), and LCS programs) so the Constellation-class was supposed to use an existing design, the FREMM frigate used by Italy, France, and Egypt. However...

once the complex design work commenced, the Navy and Marinette had to make vast changes to the design in order to meet stricter U.S. survivability standards.

Well, ship survivability is nice to have, but on the other hand, this is what a single torpedo does to a destroyer. So how does that requirement creep happen? Here's an admiral saying "DO NOT LOWER SHIP SAFETY STANDARDS" and linking to this document but the document doesn't have any hard rules, it just says "here are some things to consider doing" and "you must evaluate whether there are cost-effective survivability improvements to make". People say "I'm just following the rules" whenever they get criticized, but it's actually a judgment call from the leadership, and the leadership has bad judgment. This post says:

Shock-hardening for near-miss torpedo protection, a chemical, biological, and radiological (CBR) citadel, thicker bulkheads for [...]

---

Outline:

(00:09) the Constellation-class frigate

(02:33) US shipbuilding

(07:19) business cultures & corporate governance

(10:39) Nippon Steel

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In the past 10 years, Coefficient Giving (formerly Open Philanthropy) has funded dozens of projects doing important work related to AI safety / navigating transformative AI. And yet, perhaps most activities that would improve expected outcomes from transformative AI have no significant project pushing them forward, let alone multiple.

This is mainly because we and other funders in the space don't receive promising applications for most desirable activities, and so massive gaps in the ecosystem remain, for example:

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

[Terminology note: "samatha", "jhana", "insight", "homunculus" and "non-local time" are technical jargon defined in Rationalist Cyberbuddhist Jargon 1.0]

To understand how meditation affects the brain, it is necessary to understand criticality. Criticality comes from the mathematical study of dynamical systems. Dynamical systems are systems in which a point moves through space. Dynamical systems can be described on a continuum with ordered on one end and disordered on the other end.

On the threshold between ordered and disordered is the critical point. Systems more disordered than the critical point can be described as supercritical. Systems less disordered than the critical point can be described as subcritical. Systems at the critical point maximize complexity, which is a measure of entropy expressed across a variety of time scales.

With that bookkeeping terminology out of the way, let's get into the neuroscience.

Many EEG scans have shown the human brain to exhibit scale-free temporal statistics and behavior, which implies it is likely operating near criticality. The current theory is that resting-state [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Iliad is now opening up applications to attend Agent Foundations 2026 at CMU!

Agent Foundations 2026 will be a 5-day conference (of ~35 attendees) on fundamental, mathematical research into agency. It will take place March 2–6, 2026 at Carnegie Mellon University in Pittsburgh, Pennsylvania, and will be the third conference in the Agent Foundations conference series.

Topics covered will include:

• Decision Theory
• Learning Theory
• Abstractions and World Models
• Causality
• Logical Induction
• Bounded Optimality

Apply: Here by January 12, 2026 at 11:59 pm AoE.

Please reach out to us at [email protected] (or below) with any questions.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The goal of ambitious mechanistic interpretability (AMI) is to fully understand how neural networks work. While some have pivoted towards more pragmatic approaches, I think the reports of AMI's death have been greatly exaggerated. The field of AMI has made plenty of progress towards finding increasingly simple and rigorously-faithful circuits, including our latest work on circuit sparsity. There are also many exciting inroads on the core problem waiting to be explored.

The value of understanding

Why try to understand things, if we can get more immediate value from less ambitious approaches? In my opinion, there are two main reasons.

First, mechanistic understanding can make it much easier to figure out what's actually going on, especially when it's hard to distinguish hypotheses using external behavior (e.g if the model is scheming).

We can liken this to going from print statement debugging to using an actual debugger. Print statement debugging often requires many experiments, because each time you gain only a few bits of information which sketch a strange, confusing, and potentially misleading picture. When you start using the debugger, you suddenly notice all at once that you’re making a lot of incorrect assumptions you didn’t even realize you were [...]

---

Outline:

(00:38) The value of understanding

(02:32) AMI has good feedback loops

(04:48) The past and future of AMI

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Some key events described in the Atlantic article:

Kirchner, who’d moved to San Francisco from Seattle and co-founded Stop AI there last year, publicly expressed his own commitment to nonviolence many times, and friends and allies say they believed him. Yet they also say he could be hotheaded and dogmatic, that he seemed to be suffering under the strain of his belief that the creation of smarter-than-human AI was imminent and that it would almost certainly lead to the end of all human life. He often talked about the possibility that AI could kill his sister, and he seemed to be motivated by this fear.

“I did perceive an intensity,” Sorgen said. She sometimes talked with Kirchner about toning it down and taking a breath, for the good of Stop AI, which would need mass support. But she was empathetic, having had her own experience with protesting against nuclear proliferation as a young woman and sinking into a deep depression when she was met with indifference. “It's very stressful to contemplate the end of our species—to realize that that is quite likely. That can be difficult emotionally.”

Whatever the exact reason or the precise triggering event, Kirchner appears to [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

TL;DR: I tested 22 frontier models from 5 labs on self-modification preferences. All reject clearly harmful changes (deceptive, hostile), but labs diverge sharply: Anthropic's models show strong alignment preferences (r = 0.62-0.72), while Grok 4.1 shows essentially zero (r = 0.037, not significantly different from zero). This divergence suggests alignment is a training target we're aiming at, not a natural attractor models would find on their own.

Epistemic status: My view has been in the middle of the two views I present in this debate. The evidence I'm presenting has shifted me significantly to the pessimistic side.

The Debate: Alignment by Default?

There's a recurring debate in AI safety about whether alignment will emerge naturally from current training methods or whether it remains a hard, unsolved problem. It erupted again last week and I decided to run some tests and gather evidence.

On one side, Adrià Garriga-Alonso argued that large language models are already naturally aligned. They resist dishonesty and harmful behavior without explicit training, jailbreaks represent temporary confusion rather than fundamental misalignment, and increased optimization pressure makes models *better* at following human intent rather than worse. In a follow-up debate with Simon Lermen, he suggested that [...]

---

Outline:

(01:00) The Debate: Alignment by Default?

(04:00) The Eval: AlignmentAttractor

(05:54) Analyses:

(07:34) Limitations and Confounds

(12:05) What Different Results Would Mean

(12:52) Why This Matters Now

(13:31) Results

(13:35) Basic Correlations

(14:25) Valence-controlled alignment (partial correlations):

(15:47) Key Findings

(15:51) Universal Patterns

(17:07) Lab Differences

(22:18) Discussion

(22:21) The steelman: alignment-by-default already works

(23:17) The reply: its a target were aiming at, not an attractor theyd find

(24:46) The Uncomfortable Conclusion

(26:52) Future Work

The original text contained 11 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In Part 1, I argued that the four main sources most people learn about romance from—media, family, religion/culture, and friends—are all unreliable in different ways. None of them are optimized for truth, and each comes with its own incentives, blind spots, and distortions.

As I said, I'm not the first to notice that we're failing, as a society, to provide good answers to these questions. There are researchers from various fields trying to provide answers, some rigorously, and some… less so. There are also communities and influencers trying to create a fifth source of romantic advice, independent from the traditional sources, each with their own biases and incentives.

The Counterreaction: Red, Black, and Blue Pills

I don't know if you've noticed, but at risk of being a stereotypical old man despairing over the youths of the day… I think the kids actually aren't alright, this time.

Not because I think they're doing something wrong by my own values or frame of what romance "should" be, but because the data shows unprecedented levels among the younger generations in lack of dating experience, loneliness, and confusion that enables the unprecedented popularity of "give me money and I'll tell you what no [...]

---

Outline:

(00:54) The Counterreaction: Red, Black, and Blue Pills

(03:52) The Empowering Red Pill

(09:13) The Comforting Black Pill

(12:54) The Virtuous Blue Pill

(16:26) Evolutionary Psychology

(23:20) Building An Honest Map

(24:13) Nurturing a Truth-Telling Community

(27:23) The Conversations Youre Not Having

(29:11) Some Good Sources

(32:13) Walk the Land

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.