Epistemic status: thing people have told me that seems right. Also primarily relevant to US audiences. Also I am speaking in my personal capacity and not representing any employer, present or past.

Sometimes, I talk to people who work in the AI governance space. One thing that multiple people have told me, which I found surprising, is that there is apparently a real problem where people accidentally rule themselves out of AI policy positions by making political donations of small amounts—in particular, under $10.

My understanding is that in the United States, donations to political candidates are a matter of public record, and that if you donate to candidates of one party, this might look bad if you want to gain a government position when another party is in charge. Therefore, donating approximately $3 can significantly damage your career, while not helping your preferred candidate all that [...]

---

First published:
May 11th, 2025

Source:
https://www.lesswrong.com/posts/tz43dmLAchxcqnDRA/consider-not-donating-under-usd100-to-political-candidates

---

Narrated by TYPE III AUDIO.

"If you kiss your child, or your wife, say that you only kiss things which are human, and thus you will not be disturbed if either of them dies." - Epictetus

"Whatever suffering arises, all arises due to attachment; with the cessation of attachment, there is the cessation of suffering." - Pali canon

"He is not disturbed by loss, he does not delight in gain; he is not disturbed by blame, he does not delight in praise; he is not disturbed by pain, he does not delight in pleasure; he is not disturbed by dishonor, he does not delight in honor." - Pali Canon (Majjhima Nikaya)

"An arahant would feel physical pain if struck, but no mental pain. If his mother died, he would organize the funeral, but would feel no grief, no sense of loss." - the Dhammapada

"Receive without pride, let go without attachment." - Marcus Aurelius

[...]

---

First published:
May 10th, 2025

Source:
https://www.lesswrong.com/posts/aGnRcBk4rYuZqENug/it-s-okay-to-feel-bad-for-a-bit

---

Narrated by TYPE III AUDIO.

The other day I discussed how high monitoring costs can explain the emergence of “aristocratic” systems of governance:

Aristocracy and Hostage Capital

Arjun Panickssery · Jan 8
There's a conventional narrative by which the pre-20th century aristocracy was the "old corruption" where civil and military positions were distributed inefficiently due to nepotism until the system was replaced by a professional civil service after more enlightened thinkers prevailed ...

An element of Douglas Allen's argument that I didn’t expand on was the British Navy. He has a separate paper called “The British Navy Rules” that goes into more detail on why he thinks institutional incentives made them successful from 1670 and 1827 (i.e. for most of the age of fighting sail).

In the Seven Years’ War (1756–1763) the British had a 7-to-1 casualty difference in single-ship actions. During the French Revolutionary and Napoleonic Wars (1793–1815) the British had a 5-to-1 [...]



---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/YE4XsvSFJiZkWFtFE/explaining-british-naval-dominance-during-the-age-of-sail

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Eliezer and I wrote a book. It's titled If Anyone Builds It, Everyone Dies. Unlike a lot of other writing either of us have done, it's being professionally published. It's hitting shelves on September 16th.

It's a concise (~60k word) book aimed at a broad audience. It's been well-received by people who received advance copies, with some endorsements including:

The most important book I've read for years: I want to bring it to every political and corporate leader in the world and stand over them until they've read it. Yudkowsky and Soares, who have studied AI and its possible trajectories for decades, sound a loud trumpet call to humanity to awaken us as we sleepwalk into disaster.

- Stephen Fry, actor, broadcaster, and writer

If Anyone Builds It, Everyone Dies may prove to be the most important book of our time. Yudkowsky and Soares believe [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:
May 14th, 2025

Source:
https://www.lesswrong.com/posts/iNsy7MsbodCyNTwKs/eliezer-and-i-wrote-a-book-if-anyone-builds-it-everyone-dies

---

Narrated by TYPE III AUDIO.

It was a cold and cloudy San Francisco Sunday. My wife and I were having lunch with friends at a Korean cafe.

My phone buzzed with a text. It said my mom was in the hospital.

I called to find out more. She had a fever, some pain, and had fainted. The situation was serious, but stable.

Monday was a normal day. No news was good news, right?

Tuesday she had seizures.

Wednesday she was in the ICU. I caught the first flight to Tampa.

Thursday she rested comfortably.

Friday she was diagnosed with bacterial meningitis, a rare condition that affects about 3,000 people in the US annually. The doctors had known it was a possibility, so she was already receiving treatment.

We stayed by her side through the weekend. My dad spent every night with her. We made plans for all the fun things we would when she [...]

---

First published:
May 13th, 2025

Source:
https://www.lesswrong.com/posts/reo79XwMKSZuBhKLv/too-soon

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

At the bottom of the LessWrong post editor, if you have at least 100 global karma, you may have noticed this button.

The button Many people click the button, and are jumpscared when it starts an Intercom chat with a professional editor (me), asking what sort of feedback they'd like.

So, that's what it does. It's a summon Justis button.

Why summon Justis?

To get feedback on your post, of just about any sort. Typo fixes, grammar checks, sanity checks, clarity checks, fit for LessWrong, the works. If you use the LessWrong editor (as opposed to the Markdown editor) I can leave comments and suggestions directly inline. I also provide detailed narrative feedback (unless you explicitly don't want this) in the Intercom chat itself.

The feedback is totally without pressure. You can throw it all away, or just keep the bits you like. Or use it all!

In any case [...]

---

Outline:

(00:35) Why summon Justis?

(01:19) Why Justis in particular?

(01:48) Am I doing it right?

(01:59) How often can I request feedback?

(02:22) Can I use the feature for linkposts/crossposts?

(02:49) What if I click the button by mistake?

(02:59) Should I credit you?

(03:16) Couldnt I just use an LLM?

(03:48) Why does Justis do this?

---

First published:
May 12th, 2025

Source:
https://www.lesswrong.com/posts/bkDrfofLMKFoMGZkE/psa-the-lesswrong-feedback-service

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For months, I had the feeling: something is wrong. Some core part of myself had gone missing.

I had words and ideas cached, which pointed back to the missing part.

There was the story of Benjamin Jesty, a dairy farmer who vaccinated his family against smallpox in 1774 - 20 years before the vaccination technique was popularized, and the same year King Louis XV of France died of the disease.

There was another old post which declared “I don’t care that much about giant yachts. I want a cure for aging. I want weekend trips to the moon. I want flying cars and an indestructible body and tiny genetically-engineered dragons.”.

There was a cached instinct to look at certain kinds of social incentive gradient, toward managing more people or growing an organization or playing social-political games, and say “no, it's a trap”. To go… in a different direction, orthogonal [...]

---

Outline:

(01:19) In Search of a Name

(04:23) Near Mode

---

First published:
May 8th, 2025

Source:
https://www.lesswrong.com/posts/Wg6ptgi2DupFuAnXG/orienting-toward-wizard-power

---

Narrated by TYPE III AUDIO.

(Disclaimer: Post written in a personal capacity. These are personal hot takes and do not in any way represent my employer's views.)

TL;DR: I do not think we will produce high reliability methods to evaluate or monitor the safety of superintelligent systems via current research paradigms, with interpretability or otherwise. Interpretability seems a valuable tool here and remains worth investing in, as it will hopefully increase the reliability we can achieve. However, interpretability should be viewed as part of an overall portfolio of defences: a layer in a defence-in-depth strategy. It is not the one thing that will save us, and it still won’t be enough for high reliability.

Introduction

There's a common, often implicit, argument made in AI safety discussions: interpretability is presented as the only reliable path forward for detecting deception in advanced AI - among many other sources it was argued for in [...]

---

Outline:

(00:55) Introduction

(02:57) High Reliability Seems Unattainable

(05:12) Why Won't Interpretability be Reliable?

(07:47) The Potential of Black-Box Methods

(08:48) The Role of Interpretability

(12:02) Conclusion

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
May 4th, 2025

Source:
https://www.lesswrong.com/posts/PwnadG4BFjaER3MGf/interpretability-will-not-reliably-find-deceptive-ai

---

Narrated by TYPE III AUDIO.

It'll take until ~2050 to repeat the level of scaling that pretraining compute is experiencing this decade, as increasing funding can't sustain the current pace beyond ~2029 if AI doesn't deliver a transformative commercial success by then. Natural text data will also run out around that time, and there are signs that current methods of reasoning training might be mostly eliciting capabilities from the base model.

If scaling of reasoning training doesn't bear out actual creation of new capabilities that are sufficiently general, and pretraining at ~2030 levels of compute together with the low hanging fruit of scaffolding doesn't bring AI to crucial capability thresholds, then it might take a while. Possibly decades, since training compute will be growing 3x-4x slower after 2027-2029 than it does now, and the ~6 years of scaling since the ChatGPT moment stretch to 20-25 subsequent years, not even having access to any [...]

---

Outline:

(01:14) Training Compute Slowdown

(04:43) Bounded Potential of Thinking Training

(07:43) Data Inefficiency of MoE

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
May 1st, 2025

Source:
https://www.lesswrong.com/posts/XiMRyQcEyKCryST8T/slowdown-after-2028-compute-rlvr-uncertainty-moe-data-wall

---

Narrated by TYPE III AUDIO.

In this blog post, we analyse how the recent AI 2027 forecast by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean has been discussed across Chinese language platforms. We present:

1. Our research methodology and synthesis of key findings across media artefacts
2. A proposal for how censorship patterns may provide signal for the Chinese government's thinking about AGI and the race to superintelligence
3. A more detailed analysis of each of the nine artefacts, organised by type: Mainstream Media, Forum Discussion, Bilibili (Chinese Youtube) Videos, Personal Blogs.

Methodology We conducted a comprehensive search across major Chinese-language platforms–including news outlets, video platforms, forums, microblogging sites, and personal blogs–to collect the media featured in this report. We supplemented this with Deep Research to identify additional sites mentioning AI 2027. Our analysis focuses primarily on content published in the first few days (4-7 April) following the report's release. More media [...]

---

Outline:

(00:58) Methodology

(01:36) Summary

(02:48) Censorship as Signal

(07:29) Analysis

(07:53) Mainstream Media

(07:57) English Title: Doomsday Timeline is Here! Former OpenAI Researcher's 76-page Hardcore Simulation: ASI Takes Over the World in 2027, Humans Become NPCs

(10:27) Forum Discussion

(10:31) English Title: What do you think of former OpenAI researcher's AI 2027 predictions?

(13:34) Bilibili Videos

(13:38) English Title: \[AI 2027\] A mind-expanding wargame simulation of artificial intelligence competition by a former OpenAI researcher

(15:24) English Title: Predicting AI Development in 2027

(17:13) Personal Blogs

(17:16) English Title: Doomsday Timeline: AI 2027 Depicts the Arrival of Superintelligence and the Fate of Humanity Within the Decade

(18:30) English Title: AI 2027: Expert Predictions on the Artificial Intelligence Explosion

(21:57) English Title: AI 2027: A Science Fiction Article

(23:16) English Title: Will AGI Take Over the World in 2027?

(25:46) English Title: AI 2027 Prediction Report: AI May Fully Surpass Humans by 2027

(27:05) Acknowledgements

---

First published:
April 30th, 2025

Source:
https://www.lesswrong.com/posts/JW7nttjTYmgWMqBaF/early-chinese-language-media-coverage-of-the-ai-2027-report

---

Narrated by TYPE III AUDIO.