Sign up to save your podcasts
Or
Share “Extract-and-Evaluate Monitoring Can Significantly Enhance CoT Monitoring Performance (Research Note)” by Rauno Arike, RohanS, Shubhorup Biswas
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Extract-and-Evaluate Monitoring Can Significantly Enhance CoT Monitoring Performance (Research Note)” by Rauno Arike, RohanS, Shubhorup Biswas
Work produced at Aether. Thanks to Benjamin Arnav for providing us experimentation data and for helpful discussions, and to Francis Rhys Ward and Matt MacDermott for useful feedback.
Executive Summary
- We apply a two-step chain-of-thought (CoT) monitoring scheme to transcripts from Arnav et al. (2025) and find that this substantially outperforms simple CoT monitors: our extract-and-evaluate monitoring approach improves monitor catch rate at 1% FPR by 23.7 percentage points, averaged over all models, tasks, and attack policies.
- Extract-and-evaluate monitoring involves a lower-level monitor that reasons about the attacker's trajectory and picks out possibly suspicious quotes from it, and a higher-level monitor that only sees the original task and the quotes and assigns a suspicion score to the trajectory on that basis. After showing that this approach outperforms other monitors, we demonstrate small additional gains from using stronger models as the low-level and high-level monitors.
- We consider this as evidence there [...]
---
Outline:
(00:27) Executive Summary
(01:55) Motivation
(03:24) Experiment Setting
(04:57) Extract-and-Evaluate Monitoring
(08:30) Results: GPT 4.1-mini as both the Quote Extractor and the Judge
(10:33) Results: GPT 4.1-mini as the Quote Extractor, GPT 4.1 as the Judge
(15:08) Future Work
(17:49) Author Contributions Statement
(18:16) Appendix A: Details about the Experiment Setting
(19:15) Appendix B: CoT+action Monitor and Quote Extractor Prompt
(19:25) Appendix C: Judge Prompt
---
First published:
August 8th, 2025
Source:
https://www.lesswrong.com/posts/nRcKDYi2KfRTXdvDF/extract-and-evaluate-monitoring-can-significantly-enhance
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongWork produced at Aether. Thanks to Benjamin Arnav for providing us experimentation data and for helpful discussions, and to Francis Rhys Ward and Matt MacDermott for useful feedback.
Executive Summary
- We apply a two-step chain-of-thought (CoT) monitoring scheme to transcripts from Arnav et al. (2025) and find that this substantially outperforms simple CoT monitors: our extract-and-evaluate monitoring approach improves monitor catch rate at 1% FPR by 23.7 percentage points, averaged over all models, tasks, and attack policies.
- Extract-and-evaluate monitoring involves a lower-level monitor that reasons about the attacker's trajectory and picks out possibly suspicious quotes from it, and a higher-level monitor that only sees the original task and the quotes and assigns a suspicion score to the trajectory on that basis. After showing that this approach outperforms other monitors, we demonstrate small additional gains from using stronger models as the low-level and high-level monitors.
- We consider this as evidence there [...]
---
Outline:
(00:27) Executive Summary
(01:55) Motivation
(03:24) Experiment Setting
(04:57) Extract-and-Evaluate Monitoring
(08:30) Results: GPT 4.1-mini as both the Quote Extractor and the Judge
(10:33) Results: GPT 4.1-mini as the Quote Extractor, GPT 4.1 as the Judge
(15:08) Future Work
(17:49) Author Contributions Statement
(18:16) Appendix A: Details about the Experiment Setting
(19:15) Appendix B: CoT+action Monitor and Quote Extractor Prompt
(19:25) Appendix C: Judge Prompt
---
First published:
August 8th, 2025
Source:
https://www.lesswrong.com/posts/nRcKDYi2KfRTXdvDF/extract-and-evaluate-monitoring-can-significantly-enhance
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,328 Listeners
Conversations with Tyler
2,398 Listeners
The Peter Attia Drive
7,983 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,119 Listeners
ManifoldOne
90 Listeners
Your Undivided Attention
1,499 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,269 Listeners
Machine Learning Street Talk (MLST)
91 Listeners
Dwarkesh Podcast
425 Listeners
Hard Fork
5,462 Listeners
The Ezra Klein Show
15,410 Listeners
Moonshots with Peter Diamandis
508 Listeners
No Priors: Artificial Intelligence | Technology | Startups
124 Listeners
Latent Space: The AI Engineer Podcast
71 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
468 Listeners