This research was completed for LASR Labs 2025 by Benjamin Arnav, Pablo Bernabeu-Pérez, Nathan Helm-Burger, Tim Kostolansky and Hannes Whittingham. The team was supervised by Mary Phuong. Find out more about the program and express interest in upcoming iterations here. Read the full paper: "CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring."

Chain-of-thought (CoT) monitoring—where safety systems review a model's intermediate reasoning steps—is gaining traction at frontier labs like Google DeepMind and OpenAI as a safeguard against harmful AI actions. Despite intense interest, systematic evaluation has been limited. Our research examines the efficacy of this method and reveals a nuanced picture: CoT monitoring increases safety in situations where sabotage is subtle yet can be surprisingly ineffective against blatant harmful actions. We also discovered that a hybrid approach—using separate monitors for CoT and final actions and combining these suspicion scores—significantly improves sabotage detection compared to single-monitor strategies.

A summary of our [...]

---

Outline:

(01:51) Task Subtlety Matters

(03:12) A concerning vulnerability: unfaithful or misleading reasoning

(05:37) Hybrid protocol: improvement over action-only trusted monitoring

(06:17) Limitations and Future Directions

(07:01) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What's the main value proposition of romantic relationships?

Now, look, I know that when people drop that kind of question, they’re often about to present a hyper-cynical answer which totally ignores the main thing which is great and beautiful about relationships. And then they’re going to say something about how relationships are overrated or some such, making you as a reader just feel sad and/or enraged. That's not what this post is about.

So let me start with some more constructive motivations…

First Motivation: Noticing When The Thing Is Missing

I had a 10-year relationship. It had its ups and downs, but it was overall negative for me. And I now think a big part of the problem with that relationship was that it did not have the part which contributes most of the value in most relationships. But I did not know that at the time. Recently, I [...]

---

Outline:

(00:40) First Motivation: Noticing When The Thing Is Missing

(01:29) Second Motivation: Selecting For and Cultivating The Thing

(02:25) Some Pointers To The Thing

(03:17) How To Manufacture Relationships In The Lab

(04:53) Ace Aro Relationships

(08:04) Some Pointers To Willingness to Be Vulnerable

(12:33) Unfolding The Thing

(13:11) Play

(15:18) Emotional Support

(16:21) A Tiny High-Trust Community

(18:18) Communication

(21:28) The Obvious Caveat

(22:20) Summary

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Adapted from this twitter thread. See this as a quick take.

Mitigation Strategies

How to mitigate Scheming?

1. Architectural choices: ex-ante mitigation
2. Control systems: post-hoc containment
3. White box techniques: post-hoc detection
4. Black box techniques
5. Avoiding sandbagging

We can combine all of those mitigation via defense-in-depth system (like the Swiss Cheese model below)

I think that applying all of those strategies should divide the risk by at least 3.

1. Architectural Choices: Ex-ante Mitigation

Don't choose an architecture that uses neuraleses (i.e. hidden reasoning).

For example, don't train models to think in their latent spaces like Meta!

Chain of thought monitoring is a blessing for transparency and monitoring - let's use it as much as we can!

See: Why it's good for AI reasoning to be legible and faithful.

Similarly, I think text diffusion models are a bad idea for transparency. This is because you can no longer [...]

---

Outline:

(00:18) Mitigation Strategies

(01:20) 1. Architectural Choices: Ex-ante Mitigation

(03:26) 2. Control Systems: Post-hoc Containment

(04:00) 3. White Box Techniques: Post-hoc Detection

(05:16) 4. Black Box Techniques

(06:28) 5. Avoiding Sandbagging: Elicitation Technique

(06:52) Current State and Assessment

(07:06) Risk Reduction Estimates

(08:19) Potential Failure Modes

(09:16) The Real Bottleneck: Enforcement

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There have recently been various proposals for mitigations to "the intelligence curse" or "gradual disempowerment"—concerns that most humans would end up disempowered (or even dying) because their labor is no longer valuable. I'm currently skeptical that the typically highlighted prioritization and interventions are best and I have some alternative proposals for relatively targeted/differential interventions which I think would be more leveraged (as in, the payoff is higher relative to the difficulty of achieving them).

It's worth noting I doubt that these threats would result in huge casualty counts (due to e.g. starvation) or disempowerment of all humans (though substantial concentration of power among a smaller group of humans seems quite plausible).[1] I decided to put a bit of time into writing up my thoughts out of general cooperativeness (e.g., I would want someone in a symmetric position to do the same).

(This was a timeboxed effort of ~1.5 hr [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

On prioritizing orgs by theory of change, identifying effective giving opportunities, and how Manifund can help.

Epistemic status: I spent ~20h thinking about this. If I were to spend 100+ h thinking about this, I expect I’d write quite different things. I was surprised to find early GiveWell ‘learned in public’: perhaps this is worth trying.

The premise: EA was founded on cost-effectiveness analysis—why not try this for AI safety, aside from all the obvious reasons¹? A good thing about early GiveWell was its transparency. Some wish OpenPhil were more transparent today. That seems sometimes hard, due to strategic or personnel constraints. Can Manifund play GiveWell's role for AI safety—publishing rigorous, evidence-backed evaluations?

With that in mind, I set out to evaluate the cost-effectiveness of marginal donations to AI safety orgs². Since I was evaluating effective giving opportunities, I only looked at nonprofits³.

I couldn’t evaluate all 50+ orgs [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Letting kids be kids seems more and more important to me over time. Our safetyism and paranoia about children is catastrophic on way more levels than most people realize. I believe all these effects are very large:

This should be thought of as part of the Cost of Thriving Index discussion, and the fertility discussions as well. Before I return to the more general debate, I wanted to take care of this aspect first. It's [...]

---

Outline:

(02:31) Current Insanity Levels

(09:50) The Result of This

(12:15) Some Progress on Letting Kids Be Kids

(13:10) CPS Cares

(19:52) Panopticon

(22:37) They Even Close Playgrounds

(24:08) How Did Things Get This Bad?

(28:11) Let Kids be Adults

(30:12) Treat Kids Like People

(32:00) Lowering the Burden

(35:05) Screens

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In previous posts in this sequence, I laid out a case for why most AI governance research is too academic and too abstract to have much influence over the future. Politics is noisy and contested, so we can’t expect that good AI governance ideas will spread on their own – we need a large team of people who are actively promoting those ideas. Unfortunately, we currently have at least 3 researchers for every advocate, so many policy ideas have been “orphaned,” i.e., nobody is taking those ideas and showing them to decision-makers who have the power to implement them.

The best way of addressing this imbalance would be to shift funding and jobs from research to advocacy. However, as a practical matter, I don’t expect many funders to heed my arguments, nor do I expect many researchers to spontaneously quit and look for new jobs in other fields. So [...]

---

Outline:

(01:49) DRAFT ACTUAL POLICY DOCUMENTS

(04:43) MAKE YOUR WHITE PAPERS SPECIFIC

(08:31) CATALOG OF ORPHANED POLICIES

(08:57) Windfall Profits Clause

(09:48) Antitrust Waiver

(11:18) Strict Liability

(12:28) Visa Reform

(14:01) Insurance Requirements

(15:24) Public Grant Funding

(16:18) Global Crisis Hotline

(17:51) Compute Monitoring

(19:41) LAWS Boycott

(22:10) Industry Standards

(23:18) Structured Access to Research

(24:39) CONCLUSION

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Everyone around me has a notable lack of system prompt. And when they do have a system prompt, it's either the eigenprompt or some half-assed 3-paragraph attempt at telling the AI to “include less bullshit”.

I see no systematic attempts at making a good one anywhere.[1]

(For clarity, a system prompt is a bit of text—that's a subcategory of "preset" or "context"—that's included in every single message you send the AI.)

No one says “I have a conversation with Claude, then edit the system prompt based on what annoyed me about its responses, then I rinse and repeat”.

No one says “I figured out what phrasing most affects Claude's behavior, then used those to shape my system prompt".

I don't even see a “yeah I described what I liked and don't like about Claude TO Claude and then had it make a system prompt for [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There has been a flurry of recent papers proposing new RL methods that claim to improve the “reasoning abilities” in language models. The most recent ones, which show improvements with random or no external rewards have led to surprise, excitement and confusion.

We analyzed 7 popular LLM RL papers (100+ to 3000+ likes, 50k+ to 500k+ views on X) including “Spurious Rewards”, “RL from 1 example”, and 3 papers exploring “Intrinsic Confidence Rewards”. We found that in most of these papers the improvements could be a mirage due to various accidental issues in the evaluation setups (discussed below). As such, the baseline numbers of the pre-RL models are massively underreported compared to official numbers in the Qwen releases, or other standardized evaluations (for example in the Sober Reasoning paper). In several cases, the post-RL model performance was actually worse than the (correctly evaluated) pre-RL baseline [...]

---

First published:

Source:

Linkpost URL:
https://safe-lip-9a8.notion.site/Incorrect-Baseline-Evaluations-Call-into-Question-Recent-LLM-RL-Claims-2012f1fbf0ee8094ab8ded1953c15a37?pvs=4

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Hello from the Center for Applied Rationality!

Some of you may have attended our classic applied rationality workshops in the past; others of you may have wanted to attend a workshop but not yet had a chance to. It's been a while since we've last run public-facing workshops, but we wanted to say:

• We're not dead! (More on that perhaps in a later post)
   
• We have a new experimental mini-workshop coming up soon and hopefully more workshop content to follow after!
   

Our new workshop will be held after LessOnline at the upcoming Arbor Summer Camp program; classes will begin after lunch on Monday 6/2 and end just before lunch on Thursday 6/5. Some things to know about the upcoming workshop:

This workshop is "mini"

This workshop will last for about three days (two half-days on the edges, two full days in the middle), instead [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.