LessWrong (Curated & Popular)

Crosspost from my blog.

Synopsis

1. When we share words with each other, we don't only care about the words themselves. We care also—even primarily—about the mental elements of the human mind/agency that produced the words. What we want to engage with is those mental elements.
2. As of 2025, LLM text does not have those elements behind it.
3. Therefore LLM text categorically does not serve the role for communication that is served by real text.
4. Therefore the norm should be that you don't share LLM text as if someone wrote it. And, it is inadvisable to read LLM text that someone else shares as though someone wrote it.

Introduction

One might think that text screens off thought. Suppose two people follow different thought processes, but then they produce and publish identical texts. Then you read those texts. How could it possibly matter what the thought processes were? All you interact with is the text, so logically, if the two texts are the same then their effects on you are the same.

But, a bit similarly to how high-level actions don’t screen off intent, text does not screen off thought. How [...]

---

Outline:

(00:13) Synopsis

(00:57) Introduction

(02:51) Elaborations

(02:54) Communication is for hearing from minds

(05:21) Communication is for hearing assertions

(12:36) Assertions live in dialogue

---

First published:
November 1st, 2025

Source:
https://www.lesswrong.com/posts/DDG2Tf2sqc8rTWRk3/llm-generated-text-is-not-testimony

---



Narrated by TYPE III AUDIO.

An Overture

Famously, trans people tend not to have great introspective clarity into their own motivations for transition. Intuitively, they tend to be quite aware of what they do and don't like about inhabiting their chosen bodies and gender roles. But when it comes to explaining the origins and intensity of those preferences, they almost universally to come up short. I've even seen several smart, thoughtful trans people, such as Natalie Wynn, making statements to the effect that it's impossible to develop a satisfying theory of aberrant gender identities. (She may have been exaggerating for effect, but it was clear she'd given up on solving the puzzle herself.)

I'm trans myself, but even I can admit that this lack of introspective clarity is a reason to be wary of transgenderism as a phenomenon. After all, there are two main explanations for trans people's failure to thoroughly explain their own existence. One is that transgenderism is the result of an obscenely complex and arcane neuro-psychological phenomenon, which we have no hope of unraveling through normal introspective methods. The other is that trans people are lying about something, including to themselves.

Now, a priori, both of these do seem like real [...]

---

Outline:

(00:12) An Overture

(04:55) In the Case of Fiora Starlight

(16:51) Was it worth it?

The original text contained 3 footnotes which were omitted from this narration.

---

First published:
November 1st, 2025

Source:
https://www.lesswrong.com/posts/gEETjfjm3eCkJKesz/post-title-why-i-transitioned-a-case-study

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR: AI progress and the recognition of associated risks are painful to think about. This cognitive dissonance acts as fertile ground in the memetic landscape, a high-energy state that will be exploited by novel ideologies. We can anticipate cultural evolution will find viable successionist ideologies: memeplexes that resolve this tension by framing the replacement of humanity by AI not as a catastrophe, but as some combination of desirable, heroic, or inevitable outcome. This post mostly examines the mechanics of the process.

Most analyses of ideologies fixate on their specific claims - what acts are good, whether AIs are conscious, whether Christ is divine, or whether Virgin Mary was free of original sin from the moment of her conception. Other analyses focus on exegeting individual thinkers: 'What did Marx really mean?' In this text, I'm trying to do something different - mostly, look at ideologies from an evolutionary perspective. I [...]

---

Outline:

(01:27) What Makes Memes Fit?

(03:30) The Cultural Evolution Search Process

(04:31) The Fertile Ground: Sources of Dissonance

(04:53) 1. The Builders Dilemma and the Hero Narrative

(05:35) 2. The Sadness of Obsolescence

(06:06) 3. X-Risk

(06:24) 4. The Wrong Side of History

(06:36) 5. The Progress Heuristic

(06:57) The Resulting Pressure

(07:52) The Meme Pool: Raw Materials for Successionism

(08:14) 1. Devaluing Humanity

(09:10) 2. Legitimizing the Successor AI

(12:08) 3. Narratives of Inevitability

(12:13) Memes that make our obsolescence seem like destiny rather than defeat.

(14:14) Novel Factor: the AIs

(16:05) Defense Against Becoming a Host

(18:13) Appendix: Some memes

---

First published:
October 28th, 2025

Source:
https://www.lesswrong.com/posts/XFDjzKXZqKdvZ2QKL/the-memetics-of-ai-successionism

---



Narrated by TYPE III AUDIO.

This is the latest in a series of essays on AI Scaling.
You can find the others on my site.

Summary: RL-training for LLMs scales surprisingly poorly. Most of its gains are from allowing LLMs to productively use longer chains of thought, allowing them to think longer about a problem. There is some improvement for a fixed length of answer, but not enough to drive AI progress. Given the scaling up of pre-training compute also stalled, we'll see less AI progress via compute scaling than you might have thought, and more of it will come from inference scaling (which has different effects on the world). That lengthens timelines and affects strategies for AI governance and safety.



The current era of improving AI capabilities using reinforcement learning (from verifiable rewards) involves two key types of scaling:

1. Scaling the amount of compute used for RL during training
2. Scaling [...]

---

Outline:

(09:46) How do these compare to pre-training scaling?

(14:16) Conclusion

---

First published:
October 22nd, 2025

Source:
https://www.lesswrong.com/posts/xpj6KhDM9bJybdnEe/how-well-does-rl-scale

---



Narrated by TYPE III AUDIO.

---

Images from the article:

I've created a highly specific and actionable privacy guide, sorted by importance and venturing several layers deep into the privacy iceberg. I start with the basics (password manager) but also cover the obscure (dodging the millions of Bluetooth tracking beacons which extend from stores to traffic lights; anti-stingray settings; flashing GrapheneOS on a Pixel). I feel strongly motivated by current events, but the guide also contains a large amount of timeless technical content. Here's a preview.

Digital Threat Modeling Under Authoritarianism by Bruce Schneier

Being innocent won't protect you.

This is vital to understand. Surveillance systems and sorting algorithms make mistakes. This is apparent in the fact that we are routinely served advertisements for products that don’t interest us at all. Those mistakes are relatively harmless—who cares about a poorly targeted ad?—but a similar mistake at an immigration hearing can get someone deported.

An authoritarian government doesn't care. Mistakes are a feature and not a bug of authoritarian surveillance. If ICE targets only people it can go after legally, then everyone knows whether or not they need to fear ICE. If ICE occasionally makes mistakes by arresting Americans and deporting innocents, then everyone has to [...]

---

Outline:

(01:55) What should I read?

(02:53) Whats your risk level?

(03:46) What information this guide will and wont help you protect

(05:00) Overview of the technical recommendations in each post

(05:05) Privacy Despite Authoritarianism

(06:08) Advanced Privacy Despite Authoritarianism

---

First published:
October 29th, 2025

Source:
https://www.lesswrong.com/posts/BPyieRshykmrdY36A/an-opinionated-guide-to-privacy-despite-authoritarianism

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There is a very famous essay titled ‘Reality has a surprising amount of detail’. The thesis of the article is that reality is filled, just filled, with an incomprehensible amount of materially important information, far more than most people would naively expect. Some of this detail is inherent in the physical structure of the universe, and the rest of it has been generated by centuries of passionate humans imbibing the subject with idiosyncratic convention. In either case, the detail is very, very important. A wooden table is “just” a flat slab of wood on legs until you try building one at industrial scales, and then you realize that a flat slab of wood on legs is but one consideration amongst grain, joint stability, humidity effects, varnishes, fastener types, ergonomics, and design aesthetics. And this is the case for literally everything in the universe.

Including cancer.

But up until just [...]

---

First published:
October 26th, 2025

Source:
https://www.lesswrong.com/posts/w7eojyXfXiZaBSGej/cancer-has-a-surprising-amount-of-detail

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's a strong argument that humans should stop trying to build more capable AI systems, or at least slow down progress. The risks are plausibly large but unclear, and we’d prefer not to die. But the roadmaps of the companies pursuing these systems envision increasingly agentic AI systems taking over the key tasks of researching and building superhuman AI systems, and humans will therefore have a decreasing ability to make many key decisions. In the near term, humanity could stop, but seem likely to fail. That said, even though humans have relatively little ability to coordinate around such unilateralist dilemmas, AI systems may have different abilities or incentives. Not only that, but it seems at least plausible that the systems used to continue work on ASI development will need a higher degree of self-directedness and internal goal-directed behavior. The below, arguing that they should refrain from developing more advanced [...]

---

Outline:

(01:10) Cui bono?

(02:39) Should the AI Systems Care?

(04:29) Who might be convinced?

---

First published:
October 27th, 2025

Source:
https://www.lesswrong.com/posts/CFA8W6WCodEZdjqYE/ais-should-also-refuse-to-work-on-capabilities-research

---



Narrated by TYPE III AUDIO.

(23K words; best considered as nonfiction with a fictional-dialogue frame, not a proper short story.)

Prologue:

Klurl and Trapaucius were members of the machine race. And no ordinary citizens they, but Constructors: licensed, bonded, and insured; proven, experienced, and reputed. Together Klurl and Trapaucius had collaborated on such famed artifices as the Eternal Clock, Silicon Sphere, Wandering Flame, and Diamond Book; and as individuals, both had constructed wonders too numerous to number.

At one point in time Trapaucius was meeting with Klurl to drink a cup together. Klurl had set before himself a simple mug of mercury, considered by his kind a standard social lubricant. Trapaucius had brought forth in turn a far more exotic and experimental brew he had been perfecting, a new intoxicant he named gallinstan, alloyed from gallium, indium, and tin.

"I have always been curious, friend Klurl," Trapaucius began, "about the ancient mythology which holds [...]

---

Outline:

(00:20) Prologue:

(05:16) On Fleshling Capabilities (the First Debate between Klurl and Trapaucius):

(26:05) On Fleshling Motivations (the 2nd (and by Far Longest) Debate between Klurl and Trapaucius):

(36:32) On the Epistemology of Simplicitys Razor Applied to Fleshlings (the 2nd Part of their 2nd Debate, that is, its 2.2nd Part):

(51:36) On the Epistemology of Reasoning About Alien Optimizers and their Outputs (their 2.3rd Debate):

(01:08:46) On Considering the Outcome of a Succession of Filters (their 2.4th Debate):

(01:16:50) On the Purported Beneficial Influence of Complications (their 2.5th Debate):

(01:25:58) On the Comfortableness of All Reality (their 2.6th Debate):

(01:32:53) On the Way of Proceeding with the Discovered Fleshlings (their 3rd Debate):

(01:52:22) In which Klurl and Trapaucius Interrogate a Fleshling (that Being the 4th Part of their Sally):

(02:16:12) On the Storys End:

---

First published:
October 26th, 2025

Source:
https://www.lesswrong.com/posts/dHLdf8SB8oW5L27gg/on-fleshling-safety-a-debate-by-klurl-and-trapaucius

---



Narrated by TYPE III AUDIO.

If you want to understand a country, you should pick a similar country that you are already familiar with, research the differences between the two and there you go, you are now an expert.

But this approach doesn’t quite work for the European Union. You might start, for instance, by comparing it to the United States, assuming that EU member countries are roughly equivalent to U.S. states. But that analogy quickly breaks down. The deeper you dig, the more confused you become.

You try with other federal states. Germany. Switzerland. But it doesn’t work either.

Finally, you try with the United Nations. After all, the EU is an international organization, just like the UN. But again, the analogy does not work. The facts about the EU just don’t fit into your UN-shaped mental model.

Not getting anywhere, you decide to bite the bullet and learn about the EU the [...]

---

First published:
October 21st, 2025

Source:
https://www.lesswrong.com/posts/88CaT5RPZLqrCmFLL/eu-explained-in-10-minutes

---



Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I recently visited my girlfriend's parents in India. Here is what that experience taught me:

Yudkowsky has this facebook post where he makes some inferences about the economy after noticing two taxis stayed in the same place while he got his groceries. I had a few similar experiences while I was in India, though sadly I don't remember them in enough detail to illustrate them in as much detail as that post. Most of the thoughts relating to economics revolved around how labour in India is extremely cheap.

I knew in the abstract that India is not as rich as countries I had been in before, but it was very different seeing that in person. From the perspective of getting an intuitive feel for economics, it was very interesting to be thrown into a very different economy and seeing a lot of surprising facts and noticing how [...]

---

First published:
October 16th, 2025

Source:
https://www.lesswrong.com/posts/2xWC6FkQoRqTf9ZFL/cheap-labour-everywhere

---



Narrated by TYPE III AUDIO.