Sign up to save your podcasts
Or
Share “Extract-and-Evaluate Monitoring Can Significantly Enhance CoT Monitoring Performance (Research Note)” by Rauno Arike, RohanS, Shubhorup Biswas
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Extract-and-Evaluate Monitoring Can Significantly Enhance CoT Monitoring Performance (Research Note)” by Rauno Arike, RohanS, Shubhorup Biswas
Work produced at Aether. Thanks to Benjamin Arnav for providing us experimentation data and for helpful discussions, and to Francis Rhys Ward and Matt MacDermott for useful feedback.
Executive Summary
- We apply a two-step chain-of-thought (CoT) monitoring scheme to transcripts from Arnav et al. (2025) and find that this substantially outperforms simple CoT monitors: our extract-and-evaluate monitoring approach improves monitor catch rate at 1% FPR by 23.7 percentage points, averaged over all models, tasks, and attack policies.
- Extract-and-evaluate monitoring involves a lower-level monitor that reasons about the attacker's trajectory and picks out possibly suspicious quotes from it, and a higher-level monitor that only sees the original task and the quotes and assigns a suspicion score to the trajectory on that basis. After showing that this approach outperforms other monitors, we demonstrate small additional gains from using stronger models as the low-level and high-level monitors.
- We consider this as evidence there [...]
---
Outline:
(00:27) Executive Summary
(01:55) Motivation
(03:24) Experiment Setting
(04:57) Extract-and-Evaluate Monitoring
(08:30) Results: GPT 4.1-mini as both the Quote Extractor and the Judge
(10:33) Results: GPT 4.1-mini as the Quote Extractor, GPT 4.1 as the Judge
(15:08) Future Work
(17:49) Author Contributions Statement
(18:16) Appendix A: Details about the Experiment Setting
(19:15) Appendix B: CoT+action Monitor and Quote Extractor Prompt
(19:25) Appendix C: Judge Prompt
---
First published:
August 8th, 2025
Source:
https://www.lesswrong.com/posts/nRcKDYi2KfRTXdvDF/extract-and-evaluate-monitoring-can-significantly-enhance
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongWork produced at Aether. Thanks to Benjamin Arnav for providing us experimentation data and for helpful discussions, and to Francis Rhys Ward and Matt MacDermott for useful feedback.
Executive Summary
- We apply a two-step chain-of-thought (CoT) monitoring scheme to transcripts from Arnav et al. (2025) and find that this substantially outperforms simple CoT monitors: our extract-and-evaluate monitoring approach improves monitor catch rate at 1% FPR by 23.7 percentage points, averaged over all models, tasks, and attack policies.
- Extract-and-evaluate monitoring involves a lower-level monitor that reasons about the attacker's trajectory and picks out possibly suspicious quotes from it, and a higher-level monitor that only sees the original task and the quotes and assigns a suspicion score to the trajectory on that basis. After showing that this approach outperforms other monitors, we demonstrate small additional gains from using stronger models as the low-level and high-level monitors.
- We consider this as evidence there [...]
---
Outline:
(00:27) Executive Summary
(01:55) Motivation
(03:24) Experiment Setting
(04:57) Extract-and-Evaluate Monitoring
(08:30) Results: GPT 4.1-mini as both the Quote Extractor and the Judge
(10:33) Results: GPT 4.1-mini as the Quote Extractor, GPT 4.1 as the Judge
(15:08) Future Work
(17:49) Author Contributions Statement
(18:16) Appendix A: Details about the Experiment Setting
(19:15) Appendix B: CoT+action Monitor and Quote Extractor Prompt
(19:25) Appendix C: Judge Prompt
---
First published:
August 8th, 2025
Source:
https://www.lesswrong.com/posts/nRcKDYi2KfRTXdvDF/extract-and-evaluate-monitoring-can-significantly-enhance
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,398 Listeners
Conversations with Tyler
2,422 Listeners
The Peter Attia Drive
8,798 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,148 Listeners
ManifoldOne
92 Listeners
Your Undivided Attention
1,588 Listeners
All-In with Chamath, Jason, Sacks Friedberg
9,840 Listeners
Machine Learning Street Talk (MLST)
90 Listeners
Dwarkesh Podcast
500 Listeners
Hard Fork
5,466 Listeners
The Ezra Klein Show
15,942 Listeners
Moonshots with Peter Diamandis
542 Listeners
No Priors: Artificial Intelligence | Technology | Startups
131 Listeners
Latent Space: The AI Engineer Podcast
94 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
497 Listeners