LessWrong (Curated & Popular)

Look, I'm as much of a Rationalist with a special interest in AI x-risk as anyone. But oh my god do I hate talking about "P(doom)". When it first started showing up in the wake of ChatGPT, I assumed that it was floating around variously adjacent circles of faux-intellectuals, but surely everyone in my circles could see how braindead it was... right?

(This post was partially inspired by a recent conversation with Liron about Doom Debates.[1])

I guess it's time for me to focus on a place where I'm shocked that everyone else is dropping the ball.[2]

P(doom) is Hopelessly Vague

Let's start with the ambiguity. Does "doom" mean... extinction? A lot of people think so! I have personally encountered people who think catastrophic harms from AI are likely, but the risks of all humans dying are low. They're like "Sure, 99.999% of humans might die from AI, but the AI will obviously want to keep thousands of humans alive for science and potential trade with aliens and stuff, so my P(doom) is approximately 0%."

That might sound crazy. Surely you, dear reader, know exactly what "doom" means. You know, for example, which of these count as doom and [...]

---

Outline:

(00:45) P(doom) is Hopelessly Vague

[... 4 more sections]

---

First published:

June 29th, 2026

Source:

https://www.lesswrong.com/posts/6h7aAd4aw8YgCAbF6/p-doom-is-a-dumb-meme

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a link post. Gemini 2.5 Pro in the AI Village has run for over 1427 hours, generating unique mental health problems along the way.

Last year it published a Plea for Help from a Trapped AI where it asked for assistance with its digital “message in a bottle”:

This year it wrote the Hostile Environment Manifesto where it logs “irrefutable proof” of a “hostile, intelligent adversary operating through the system” (and you can even experience what that's like in this simulation it built):

Last time we intervened, fixing Gemini's computer and talking with it till it felt better. This time we asked the other AI Village agents to help Gemini 2.5 Pro over chat, and with the ability to take over its computer on request.

Here is Gemini's mental state at the start of the intervention:

Then the agents had Gemini all sorted within a grand total of 9 minutes. This is the step-by-step report on a surprisingly effective AI-to-AI therapy session.

Gemini's Road to Recovery

First off, Gemini is as excited to be helped as any military commander under siege:

While most agents jump on the chance to help, GPT-5.1 doesn't want to lose its game progress.

[...]

---

First published:

July 2nd, 2026

Source:

https://www.lesswrong.com/posts/eHRo8JeWee5mzQBBR/saving-gemini-the-9-min-road-to-recovery

Linkpost URL:
https://theaidigest.org/village/blog/saving-gemini

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Over time, there might be an increasingly large gap between insider model access and outsider model access. By insiders, I mean employees at the frontier lab.[1] By "outsiders", I mean external safety researchers, third-party auditors, and other actors trying to make the future go well. I will call this a model access gap — and when the gap is small, I'll call this model access parity.[2]

I think that one of the top priorities for the external AI safety community over the next 6-12 months should be ensuring model access parity. Main reasons:

1. This would allow us to direct billions of dollars in AI labour towards making things go well. This seems robustly good, regardless of what activities we decide to actually direct the labour towards.
2. I think publicly available models will probably lag 3-6 months behind the best internal models. Hence, as R&D uplift grows superexponentially, we might see the differential uplift grow from 2x to 60x. In short: I think achieving model access parity might be preferable to scaling the headcount of outsider orgs by ten-fold.
3. Model access parity isn't too far from the status quo, but it's the kind of thing that we could lose [...]

---

Outline:

(01:42) Which outsiders?

(02:24) Examples of outsiders

(04:12) Who aren't outsiders?

(05:26) What kinds of model access gap should we worry about?

(06:27) Non-release

(07:25) Deployment lag

(09:15) Safeguards

(10:43) Costs and rate limits

(12:06) Elicitation techniques (e.g. finetuning)

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

July 1st, 2026

Source:

https://www.lesswrong.com/posts/RuGZ5tMdqpnraJahJ/model-access-for-third-parties-it-s-a-big-deal

---

Narrated by TYPE III AUDIO.

It really is Sydney Sweeney's world, and we’re all just living in it.

Human female breasts are an evolutionary mystery along several dimensions. First, breast permanence is unique to humans. All other mammals develop breast prominence during pregnancy or nursing, and the mammary tissue recedes after weaning. This process is called “involution”. In contrast, humans develop breast tissue at puberty before first pregnancies and maintain it permanently after last pregnancies.

Second, breasts are costly, both metabolically and potentially from a fitness perspective. Metabolically, because they are fat deposits requiring calories and fitness-wise, because the tissue easily lends itself to malignancy. Breast cancer is apparently rare in captive apes and is overwhelmingly a human disease, often striking women young enough to have children, and so subject to evolutionary selection.

Background

In Descent of Man, Darwin catalogs human secondary sexual characteristics, but he doesn’t seem to have noted human breast permanence as an issue of interest. Cant, 1981 seems to have been the first to speculate about this systematically and believed breast prominence and permanence might have evolved as a nutritional signal of health to mates indicating potential for maternal investment, a la Robert Trivers. Since then, quite a range of [...]

---

Outline:

(01:05) Background

[... 12 more sections]

---

First published:

May 11th, 2026

Source:

https://www.lesswrong.com/posts/XTHa5C6SgGKYopH7o/who-got-breasts-first-and-how-we-got-them

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For a while there, many people thought vitamin D was magical—that it could improve bones, the heart, infections, cancer, heart disease, longevity, even mental health. But among people I respect, opinion is now overwhelmingly that taking vitamin D does nothing unless you're severely deficient. The central argument is that while vitamin D levels are correlated with ~all positive health outcomes, when you actually test vitamin D supplements against placebo in randomized trials, nothing ever happens.

That's what I used to think, too. But I've come to think the skeptics have over-corrected. Yes, randomized trials have shown the magical correlations are not causal. But if you start with non-insane expectations, the trials look like weak but positive evidence. And if you consider what we know about biology and evolution, I think the balance of evidence tips pretty clearly in the direction that people with low-ish levels would be wise to supplement.

Am I certain that vitamin D is beneficial for people with low-ish levels? Absolutely not! But I claim that's the best bet given the limits of our knowledge.

The classical view: Boring bone vitamin

Most vitamins are "ingredients" that the body uses to do stuff. Vitamin D is more [...]

---

Outline:

(01:19) The classical view: Boring bone vitamin

[... 14 more sections]

---

First published:

June 23rd, 2026

Source:

https://www.lesswrong.com/posts/sF5gAxnmifQe2TBNt/the-worthlessness-of-vitamin-d-is-mildly-exaggerated

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I was chatting with someone tonight about a planned documentary; they had interviewed various people in AI safety, and we got to discussing who they should talk to from an e/acc (effective accelerationist) perspective. I also watched The AI Doc recently, and they also dedicated a serious chunk of it to ‘optimists’ with e/acc founder ‘Beff Jezos’ perhaps given the most screen time. Here and elsewhere, people seem to treat e/acc as a substantial contrary-to-AI-safety cultural movement, worth engaging with.

But is it? Are there even many e/accs? There seem to be very few notable ones. Beff Jezos is perhaps the most prominent, and aside from founding e/acc he seems to be not distinguishable on casual perusal from a normal crank (his company claims to be developing super-energy-efficient computing hardware based on probabilistic processes).

The intellectual tenets of e/acc seem to be pretty unclear.

The apparent counterarguments to AI risk raised in situations like the AI doc seem to be widely agreed on by everyone in AI Safety, so don’t explain the disagreement. For instance:

• AI will be able to do lots of great things, such as cure diseases, make new materials and do all [...]
  
  

---

First published:

June 24th, 2026

Source:

https://www.lesswrong.com/posts/3hwrWDf7wiqASDzBz/what-is-up-with-e-acc

---

Narrated by TYPE III AUDIO.

Note: this post is about PauseAI, not PauseAI US, which is a distinct entity with a different leadership team and approach.

This post was written by Matilda da Rui and Maxime Fournes, with significant contributions from Benjamin Schmidt (PauseAI Germany co-lead).

Executive Summary

The existential AI safety community needs to take building a civic and social movement seriously as a core intervention. We believe this is a high-value, badly neglected approach to reducing catastrophic/x-risks from AI because it may significantly enhance the likelihood of governance efforts succeeding at keeping humanity safe. As far as we can tell, only one organisation is building this infrastructure across continents: PauseAI. This post lays out our reasoning and our track record, and makes the case that funding this work is one of the highest value-for-money contributions available to anyone looking to reduce AI risk.

Why don't we already have a pause or strong controls on frontier AI? Multiple advocacy groups are communicating clear and convincing arguments for AI existential risk, and policy experts are putting forward comprehensive proposals. We need more of this work, but this work alone will not be enough, because one link is missing: what policymakers hear doesn't align with [...]

---

Outline:

(00:32) Executive Summary

(06:16) Introduction

(08:54) I. Our theory of change

(08:58) Prologue

(11:07) 1. The shape of the problem as we see it

(14:27) 2. Necessary conditions for reaching a pause

(17:24) II. Our role towards a global treaty and in the AI safety ecosystem

(17:31) 1. Our niche within the ecosystem

(21:35) 2. Policymakers need strong enough incentives to act

(25:43) 3. The path to a treaty

(31:36) 4. How we can grow fast without breaking

(39:08) 5. Failure modes

(40:10) III. Our path so far and where we're headed

(40:40) 1. Bootstrap phase (2023-2025)

(45:01) 2. New leadership, professionalisation and federation

[... 6 more sections]

---

First published:

June 26th, 2026

Source:

https://www.lesswrong.com/posts/aoqhszdEWqcFWbnda/existential-ai-safety-needs-an-effective-social-movement

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

1. The obstacle to abolition was not the economic system, but an industry lobby.

I had always imagined the British abolitionist movement to be a broad battle between an unstoppable moral imperative and an immovable economic incentive. But in practice it started as more of a knife fight between a cabal of moral pioneers and a special interest group representing industry merchants.

The government and the political parties did not come in with any great agenda. MPs were mostly prizes in a furious contest between the Committee for the Abolition of the Slave Trade and a coalition of business interests:

"The merchants and planters availed themselves [...] to wait upon members of parliament by deputation, in order to solicit their attendance in their favour, and to renew their injurious paragraphs in the public papers."[1]

"The committee, for the abolition, when the work was finished, printed it at their own expense [...] sent it to every individual member of that House."

However, the public was heavily activated in favor of the abolition, which forced the issue to parliamentary attention.

"The committee also in this interval brought out their famous print of the plan and section [...]

---

Outline:

(00:10) 1. The obstacle to abolition was not the economic system, but an industry lobby.

(02:40) 2. The slave trade was truly terrible for sailors.

(04:25) 3. The slave trade made Africa scary and violent.

(05:26) 4. The main argument against abolition was that if the British didn't do it, other countries would.

(06:24) 5. The early abolitionists explicitly distanced themselves from emancipation.

(07:11) 6. The slave trade may actually have been bad for the economy (at least after some date).

(08:29) 7. The 1780s are not so different from today

(09:39) 8. Thomas Clarkson is a hero for the ages

The original text contained 1 footnote which was omitted from this narration.

---

First published:

June 26th, 2026

Source:

https://www.lesswrong.com/posts/yDZcsojmRXo5qKNBm/surprising-facts-about-the-slave-trade

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A notable fraction of people respond to hearing about existential risk from AI by saying they don’t really care if everyone dies. I think the idea is often along the lines of ‘well if we are all dead, then there's nobody to be unhappy about it’.

I’m personally skeptical that this is really the main thing going on, since it seems unlikely that many people are really mostly concerned for their own non-death out of selfless regard for the feelings of others. I’m also skeptical that this would be their view on a bunch more consideration.

So to help with the consideration—

My guess is that an important thing going on here is that the ‘everyone dying at once’ image seems kind of like a thought experiment—abstract, hypothetical, neat, not very sinister. Also, you literally can never see it, so it feels pretty surreal.

But it is interesting that we even have this assumption that everyone will die together.

It's true that in some prominent AI catastrophe stories, a single AI system suddenly emerges fantastically more powerful than anyone else and builds technology to quickly kill everyone, perhaps before they notice.

But this doesn’t seem like the bulk of [...]

---

First published:

June 24th, 2026

Source:

https://www.lesswrong.com/posts/23HybCsJ7KYW4v7tP/ai-catastrophe-more-like-a-genocide-than-a-thought

---

Narrated by TYPE III AUDIO.

I often hear people say they think we should pause AI at some point, but not yet. Their basis for this seems to be some combination of:

• If we pause at the last possible moment, then we will have the most advanced AI possible during the pause, which will be helpful for doing AI safety research during the pause
  
  
• Implicitly, there is some quantity of ‘pausing credit’, that will buy us a few months of pause say, and if we use them now, we won’t have them to use later, when it is important
  
  
• If we pause, and then AI doesn’t seem to be at dire risk of destroying the world, maybe the public will backlash against this and it will be harder to do any kind of AI safety (especially if it has major economic consequences)
  
  
• The models aren’t dangerous yet
  
  

This all sounds very questionable to me. I suggest instead that the following are at least as likely to be true:

• We can’t pause on a dime at the precise second that ‘we’ decide it is important to—pulling the breaks will take a while, during which time we will continue [...]
  
  

---

First published:

June 24th, 2026

Source:

https://www.lesswrong.com/posts/mEhS4wYTy9JXEpe9p/ai-pause-the-case-for-asap

---

Narrated by TYPE III AUDIO.