TL;DR: Models trained with outcome-based RL sometimes have reasoning traces that look very weird. In this paper, I evaluate 14 models and find that many of them often generate pretty illegible CoTs. I show that models seem to find this illegible text useful, with a model's accuracy dropping heavily when given only the legible parts of its CoT, and that legibility goes down when answering harder questions. However, when sampling many responses to the same questions, I find there's no real correlation between illegible reasoning and performance. From these results (and prior work), I think it's likely RL induces meaningful illegible reasoning, but that it may not be significantly more effective than legible reasoning.

Paper | Tweet thread | Streamlit | Code

Introduction

Reasoning models are LLMs that have been trained with RLVR (Reinforcement Learning from Verifiable Rewards), often to use extended reasoning in chain-of-thought to solve tasks. This could be pretty beneficial: if this reasoning is legible and faithful, then monitoring it would be very useful. There's a lot of prior work on faithfulness, but very little on legibility—which makes sense, until recently there haven’t been models with meaningfully illegible reasoning traces.

For some reason, in practice RLVR [...]

---

Outline:

(01:08) Introduction

(04:38) How useful are illegible CoTs?

(06:29) Discussion

(10:46) Acknowledgements

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

A defensive military coalition is a key frame for thinking about our international agreement aimed at forestalling the development of superintelligence. We introduce historical examples of former rivals or enemies forming defensive coalitions in response to an urgent mutual threat, and detail key aspects of our proposal which are analogous. We also contrast our proposal with the oft-employed analogy to nonproliferation agreements, and highlight shortcomings.

Introduction

We at the Machine Intelligence Research Institute's Technical Governance Team have proposed an illustrative international agreement (blog post) to halt the advancement of AI capabilities, in order to forestall the development of superintelligence until it can be done safely. For those who haven’t read it already, it is enough to know that we believe substantially delaying the arrival of superintelligence is warranted, as the world is not remotely prepared to manage this transition. A halt could give the world the time it needs, and this will require international coordination, likely led by the U.S. and China.

In thinking about our proposal (and ones like it), it is natural to seek for analogies and precedents. We ourselves have spent a lot of time comparing parts of the plan to precedent and have documented [...]

---

Outline:

(00:08) Summary

(00:37) Introduction

(02:38) Defensive Coalition

(11:38) Nonproliferation Agreement

(14:19) Conclusion

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

It's A Great Model, Sir

One might even say the best model. It is for now my default weapon of choice.

Google's official announcement of Gemini 3 Pro is full of big talk. Google tells us: Welcome to a new era of intelligence. Learn anything. Build anything. Plan anything. An agent-first development experience in Google Antigravity. Gemini Agent for your browser. It's terrific at everything. They even employed OpenAI-style vague posting.

In this case, they can (mostly) back up that talk.

Google CEO Sundar Pichai pitched that you can give it any scribble and have it turn that into a boardgame or even a full website, it can analyze your sports performance, create generative UI experiences and present new visual layouts.

He also pitched the new Gemini Agent mode (select the Tools icon in the app).

If what you want is raw intelligence, or what you want is to most often locate the right or best answer, Gemini 3 Pro looks like your pick.

If you want creative writing or humor, Gemini 3 Pro is definitely your pick.

If you want a teacher to help you learn known things, Gemini 3 [...]

---

Outline:

(00:10) It's A Great Model, Sir

(01:49) There Is A Catch

(03:28) Andrej Karpathy Cautions Us

(04:58) On Your Marks

(14:15) Defying Gravity

(15:40) The Efficient Market Hypothesis Is False

(18:17) The Product Of A Deranged Imagination

(22:41) Google Employee Hype

(26:37) Matt Shumer Is A Big Fan

(27:55) Roon Eventually Gains Access

(28:21) The Every Vibecheck

(29:49) Positive Reactions

(36:08) Embedding The App

(36:26) The Good, The Bad and The Unwillingness To Be Ugly

(40:03) Genuine People Personalities

(41:50) Game Recognize Game

(43:48) Negative Reactions

(47:36) Code Fails

(48:42) Hallucinations

(55:11) Early Janusworld Reports

(57:31) Where Do We Go From Here

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

You know what's sweeter than free money? Free money that you take from an organization of smart people who love prediction markets and having good values, by being better at prediction markets and values than they are.

Sadly, you can no longer turn this into actual U.S. legal tender. You can, however, still turn it into mana and bragging points.

The Best Of LessWrong Manifold Markets

Here's the setup.

Once a year, LessWrong picks ~50 posts from two years ago to be the Best Of posts. This is called the LessWrong Annual Review. (So, at the end of 2025 we'll be picking the best posts of 2024.) Any post can be nominated, including things that weren't actually posted to LessWrong, but LessWrong posts with 100+ karma are the default for consideration.

During the year, LessWrong has a bot. That bot looks for posts that get 100 karma, and it makes a market on Manifold Markets for whether that post will make the top fifty during the LessWrong annual review. Each market is initialized to 14% chance that the post reaches that lofty goal.

Manifold Markets is a prediction market platform. That platform shows market about whether something [...]

---

Outline:

(00:33) The Best Of LessWrong Manifold Markets

(02:26) Are You Smarter Than A Base Rate?

(04:41) Zero And One Are Not Probabilities

(07:49) Why Would Anyone Do This?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

NATO faces its gravest military disadvantage since 1949, as the balance of power has shifted decisively toward its adversaries in the Era of Drone Warfare. The speed and scale of NATO's relative military decline represents the most dramatic power shift since World War II—and the alliance appears dangerously unaware of its new vulnerability

NATO doctrine is obsolete. The Pax Americana is coming to its end.

• Failure to defeat Houthi Rebels: The US Navy's struggles against the Houthis' low-cost drone arsenal demonstrate how even non-state actors can challenge NATO air defense.  NATO forces found themselves hard-pressed to counter relatively primitive drone attacks.
• Economic logic of drone warfare: In Ukraine, drones account for over 70% of combat kills—a proportion likely to increase. The economics are devastating: $1,000 FPV drones routinely destroy $7 million Abrams tanks. Modern tanks are dangerously outdated and the entire armored warfare needs to be redesigned from the ground up to face the realities of modern drone warfare.
• Lack of military experience: NATO lacks both experienced drone operators and institutional knowledge of drone warfare. The alliance has no significant combat experience fighting against and integrating drones at scale.
• Lack of NATO anti-drone air defense: NATO's air [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Here I am on the plane on the way home from Inkhaven. Huge thanks to Ben Pace and the other organizers for inviting me. Lighthaven is a delightful venue and there sure are some brilliant writers taking part in this — both contributing writers and participants. With 40 posts published per day by participants (not counting those by organizers and contributing writers) it feels impossible to highlight samples of them. Fortunately that's been done for me: the Inkhaven spotlight. And just to semi-randomly pick a single post to highlight, for making me laugh out loud, I'll link to one by Rob Miles. There are also myriad poignant, insightful, informative, and otherwise delightful posts. And, sure, plenty that don't quite work yet. But that's the point: force yourself to keep publishing, ready or not, and trust that quality will follow.

Confession: I'm not at all happy with this post. Opening with "here I am on the plane"? Painful. I would've fixed that in an editing pass, but I'm going to leave it because it illustrates two Inkhaven lessons for me. First, that sometimes it's ok to hit publish before something is perfect. And second, that I in particular need to [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

TL;DR: Figure out what needs doing and do it, don't wait on approval from fellowships or jobs.

If you...

• Have short timelines
• Have been struggling to get into a position in AI safety
• Are able to self-motivate your efforts
• Have a sufficient financial safety net

... I would recommend changing your personal strategy entirely.

I started my full-time AI safety career transitioning process in March 2025. For the first 7 months or so, I heavily prioritized applying for jobs and fellowships. But like for many others trying to "break into the field" and get their "foot in the door", this became quite discouraging.

I'm not gonna get into the numbers here, but if you've been applying and getting rejected multiple times during the past year or so, you've probably noticed the number of applicants increasing at a preposterous rate. What this means in practice is that the "entry-level" positions are practically impossible for "entry-level" people to enter.

If you're like me and have short timelines, applying, getting better at applying, and applying again, becomes meaningless very fast. You're optimizing for signaling competence rather than actually being competent. Because if you a) have short timelines, and b) are [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Earlier this month, I was pretty desperately feeling the need for a vacation. So after a little googling, I booked a flight to New York city, a hotel, and four nights worth of tickets to a new immersive theater show called Masquerade.

Background: “Standard” Immersive Theater

To convey Masquerade, I find it easiest to compare against standard immersive theater.

It's weird to talk about “standard” immersive theater, because the standard is not that old and has not applied to that many shows at this point. Nonetheless, there is an unambiguous standard format, and the show which made that format standard is Sleep No More. I have not yet seen Sleep No More itself, but here's my understanding of it.

Sleep No More follows the story of Macbeth. Unlike Shakespearre's version, the story is not performed on a central stage, but rather spread out across five floors of a building, all of which is decked out as various locations from the story. Scene changes are not done by moving things on “stage”, but rather by walking to another area. The audience is free to wander the floors as they please, but must remain silent and wear a standard mask throughout [...]

---

Outline:

(00:24) Background: Standard Immersive Theater

(03:39) By Comparison, Masquerade

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

My understanding is that even though advocating a pause or massive slowdown in the development of superintelligence think we should get there eventually[1]. Something something this is necessary for humanity to reach its potential.

Perhaps so, but I'll be sad about it. Humanity has a lot of unsolved problems right now. Aging, death, disease, poverty, environmental degradation, abuse and oppression of the less powerful, conflicts, and insufficient resources such as energy and materials.

Even solving all the things that feel "negative", the active suffering, there's all this potential for us and the seemingly barren universe that could be filled with flourishing life. Reaching that potential will require a lot of engineering puzzles to be solved. Fusion reactors would be neat. Nanotechnology would be neat. Better gene editing and reproductive technology would be neat.

Superintelligence, with its superness, could solve these problems faster than humanity is on track to. Plausibly way way faster. With people dying every day, I see the case for it. Yet it also feels like the cheat code to solving all our problems. It's building an adult to take care of us, handing over the keys and steering wheel, and after that point our efforts [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

{early pause}... (you should have known this:)

YOU (not just them over there)

CAN (tho maybe you shouldn't?)

JUST (its weirdly easy)

DO (not talking/writing/planning/delaying)

THINGS (weird/hard/easy things... really ANYthing)

{late pause} (I'll make eye contract until you START)

-@jenniferRM, riffing on norvid_studies.

You should know this by now, but you can just do things. That you didn't know this is an indictment of your social environment, which taught you how to act.

You Can Just Do Things

Yes, you. All the activities you see other people do? You can do them, too. Whether or not you'll find it hard, you can do them.

The barriers you see to doing so are not iron-hard constraints. They are obstacles you can climb over, costs you've reified into infinitely high cliffs because constraining the action space, and thereby the solution space, can simplify life. But always remember, these are fake cliffs. If you actually want to solve a problem, if you feel but the barest moment of whimsy, you can topple them with but a thought and expand the action space however much you like.

If you are ruling out actions, make sure it is because it is [...]

---

Outline:

(00:48) You Can Just Do Things

(01:38) You Can Just Do Things

(02:20) You Can Just Do Things

(03:08) You Can Just Do Things

(04:02) You Can Just Do Things

(05:06) You Can Just Do Things

---

First published:

Source:

---

Narrated by TYPE III AUDIO.