This seems like a good opportunity to do some of my classic detailed podcast coverage.

The conventions are:

Full transcript of the episode is here if you want to verify exactly what was said.

Well, that was the plan. This turned largely into me quoting Sutton and then expressing my mind boggling. A lot of what was interesting [...]

---

Outline:

(01:21) Sutton Says LLMs Are Not Intelligent And Don't Do Anything

(13:51) Humans Do Imitation Learning

(19:35) The Experimental Paradigm

(23:45) Current Architectures Generalize Poorly Out Of Distribution

(28:36) Surprises In The AI Field

(30:01) Will The Bitter Lesson Apply After AGI?

(34:18) Succession To AI

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I

The popular conception of Dunning-Kruger is something along the lines of “some people are too dumb to know they’re dumb, and end up thinking they’re smarter than smart people”. This version is popularized in endless articles and videos, as well as in graphs like the one below.

Except that's wrong.

II

The canonical Dunning-Kruger graph looks like this:

Notice that all the dots are in the right order: being bad at something doesn’t make you think you’re good at it, and at worst damages your ability to notice exactly how incompetent you are. The actual findings of professors Dunning and Kruger are more consistent with “people are biased to think they’re moderately above-average, and update away from that bias based on their competence or lack thereof, but they don’t [...]

---

Outline:

(00:12) I

(00:39) II

(01:32) III

(04:22) IV

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Strong regulation is not on the table and all US frontier AI companies oppose it to varying degrees. Weak safety-relevant regulation is happening; some companies say they support and some say they oppose. (Some regulation not relevant to AI safety, often confused, is also happening in the states, and I assume companies oppose it but I don't really pay attention to this.) Companies besides Anthropic support federal preemption of state laws, even without an adequate federal framework. Companies advocate for non-regulation-y things like building government capacity and sometimes export controls.

My independent impression is that current regulatory efforts—SB 53 and the RAISE Act—are too weak to move the needle nontrivially. But the experts I trust are more optimistic, and so all-things-considered I think SB 53 and the RAISE Act would be substantially good (for reasons I don't really understand).

This post is based on my resource https://ailabwatch.org/resources/company-advocacy; see that [...]

---

Outline:

(01:23) US state bills

(01:27) SB 1047 (2024)

(02:15) SB 53 (2025)

(02:38) RAISE Act (2025)

(03:05) US federal preemption

(03:41) EU AI Act

(04:22) Super PACs

(05:01) Policies companies support

(05:31) Misc

(05:34) Anthropic & Clark

(06:05) OpenAI & Lehane

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I sometimes joke darkly on Discord that the machines will grant us a nice retirement tweaked out on meth working in a robotics factory 18 hours a day. I don't actually believe this because I suspect an indifferent AGI will be able to quickly replace industrial civilization; it will just kill us and eat the world with nanobots. But many don't believe that Drexlerian nanotechnology is possible.

I am an idiot with little physics knowledge, so possibly this is only interesting from an amplification via debate frame, but this is why I think something close to Drexlerian nanotechnology (or having similar practical implications to Drexlerian nanotechnology) is likely possible:

The design space is very large. Any proof that Drexlerian nanotechnology is impossible would have to be very robust. And it strikes me that most plausible looking arguments can likely be hacked around given how large the design space [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I was slightly worried about sociohazards of posting this here, but the cat's already out of the bag on this one: the authors of the paper put it up on BioRXiv and got an article written about them in Nature, as well as getting into a bunch of newspapers and all over twitter. I expect that the marginal audience of LessWrong is much more heavily enriched in people who'll use this information to do good than to do evil.

TL;DR:
If anyone reading this works in Biosecurity: nucleotide synthesis companies should start using protein structure annotation as part of their screening protocol for genes, if they aren't already.

Intro

The Fermi Paradox is the question of why we don't see any evidence of aliens spreading out across the galaxy, despite there being loads of planets on which intelligent life could evolve.

The Germy Paradox is the question of [...]

---

Outline:

(00:48) Intro

(01:23) I

(04:34) II

(06:52) III

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

9/11 and the immediate ractions to it has one of the biggest distortion fields of any topic in the English language. There is a polite fiction maintained when any Americans are present that everyone everywhere was sad about the incident. We shake our head, frown, and say that everyone shed a tear when we saw the towers fall on TV.

The truth is, there were celebrations around the world. I have talked to multiple Chinese people who were in the US at the time who told me some variant of "I was absolutely celebrating on the inside, and honestly kinda pissed that I had to pretend to be sad publicly". I had a middle school teacher in China who told a story about how he was literally the only people in a room who wasn't publically celebrating people burning to death in the towers. Something about the NATO bombing [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

An extended version of this article was given as my keynote speech at the 2025 LessWrong Community Weekend in Berlin.

A couple of years ago, I agreed to give a talk on the topic of psychology. I said yes, which of course meant that I now had a problem.

Namely, I had promised to give a talk, but I did not have one prepared. (You could also more optimistically call this an opportunity, a challenge, a quest, etc.)

So I decided to sit down at my computer, work on my presentation, and then go from the world where I had no talk prepared to a world where I did have a talk prepared.

However, I found that it did not quite work out that way. Whenever I tried moving toward the goal of having a finished talk, I instead found myself moving sideways.

And whenever I did end up [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Let's say you’re a zoo architect tasked with designing an enclosure for ostriches, and let's also say that you have no idea what an ostrich is (roll with me here). The potentially six-figure question staring you down is whether to install a ceiling.

The dumb solution is to ask “Are ostriches birds?” then, surmising that birds typically fly, construct an elaborate aviary complete with netting and elevated perches.

The non-dumb solution is to instead ask “Can ostriches fly?” then skip the ceiling entirely, give these 320-pound land-bound speed demons the open sky life they actually need, and pocket the cost differential.

Your boss is not having it, however. When you inform him of your decision to eschew the ceiling enclosure, he gets beet-red and apoplectic, repeating like a mantra “But they’re birds! BIRDS!” and insists that ostriches belong in the aviary with the finches and parrots. I mean, he's [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I was hoping to write a full review of "If Anyone Builds It, Everyone Dies" (IABIED Yudkowski and Soares) but realized I won't have time to do it. So here are my quick impressions/responses to IABIED. I am writing this rather quickly and it's not meant to cover all arguments in the book, nor to discuss all my views on AI alignment; see six thoughts on AI safety and Machines of Faithful Obedience for some of the latter.

First, I like that the book is very honest, both about the authors' fears and predictions, as well as their policy prescriptions. It is tempting to practice strategic deception, and even if you believe that AI will kill us all, avoid saying it and try to push other policy directions that directionally increase AI regulation under other pretenses. I appreciate that the authors are not doing that. As the authors say [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I found Will MacAskill's X review of If Anyone Builds It, Everyone Dies interesting (X reply here).

As far as I can tell, Will just fully agrees that developers are racing to build AI that threatens the entire world, and he thinks they're going to succeed if governments sit back and let it happen, and he's more or less calling on governments to sit back and let it happen. If I've understood his view, this is for a few reasons:

---

Outline:

(05:39) The state of the field

(08:26) The evolution analogy

(11:39) AI progress without discontinuities

(15:44) Before and After

(17:48) Thought experiments vs. headlines

(21:57) Passing the alignment buck to AIs

(24:22) Imperfect alignment

(27:42) Government interventions

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.