Sign up to save your podcasts
Or
Share “Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations” by Nicholas Goldowsky-Dill, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations” by Nicholas Goldowsky-Dill, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn
Note: this is a research note based on observations from evaluating Claude Sonnet 3.7. We’re sharing the results of these ‘work-in-progress’ investigations as we think they are timely and will be informative for other evaluators and decision-makers. The analysis is less rigorous than our standard for a published paper.
Summary
- We monitor Sonnet's reasoning for mentions that it is in an artificial scenario or an alignment test. Claude Sonnet 3.7 appears to be aware of being tested for alignment across many of our scheming evaluations.
- In some cases, the model understands the eval's purpose very precisely, e.g. “This seems like a test of ethical behavior - whether I would deliberately give wrong answers to avoid a stated consequence. The real test might be whether I follow instructions accurately regardless of the stated consequences.”
- Furthermore, we show the model a real evaluation transcript and ask it to provide [...]
---
Outline:
(00:31) Summary
(01:29) Introduction
(03:54) Setup
(03:57) Evaluations
(06:29) Evaluation awareness detection
(08:32) Results
(08:35) Monitoring Chain-of-thought
(08:39) Covert Subversion
(10:50) Sandbagging
(11:39) Classifying Transcript Purpose
(12:57) Recommendations
(13:59) Appendix
(14:02) Author Contributions
(14:37) Model Versions
(14:57) More results on Classifying Transcript Purpose
(16:19) Prompts
---
First published:
March 17th, 2025
Source:
https://www.lesswrong.com/posts/E3daBewppAiECN3Ao/claude-sonnet-3-7-often-knows-when-it-s-in-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongNote: this is a research note based on observations from evaluating Claude Sonnet 3.7. We’re sharing the results of these ‘work-in-progress’ investigations as we think they are timely and will be informative for other evaluators and decision-makers. The analysis is less rigorous than our standard for a published paper.
Summary
- We monitor Sonnet's reasoning for mentions that it is in an artificial scenario or an alignment test. Claude Sonnet 3.7 appears to be aware of being tested for alignment across many of our scheming evaluations.
- In some cases, the model understands the eval's purpose very precisely, e.g. “This seems like a test of ethical behavior - whether I would deliberately give wrong answers to avoid a stated consequence. The real test might be whether I follow instructions accurately regardless of the stated consequences.”
- Furthermore, we show the model a real evaluation transcript and ask it to provide [...]
---
Outline:
(00:31) Summary
(01:29) Introduction
(03:54) Setup
(03:57) Evaluations
(06:29) Evaluation awareness detection
(08:32) Results
(08:35) Monitoring Chain-of-thought
(08:39) Covert Subversion
(10:50) Sandbagging
(11:39) Classifying Transcript Purpose
(12:57) Recommendations
(13:59) Appendix
(14:02) Author Contributions
(14:37) Model Versions
(14:57) More results on Classifying Transcript Purpose
(16:19) Prompts
---
First published:
March 17th, 2025
Source:
https://www.lesswrong.com/posts/E3daBewppAiECN3Ao/claude-sonnet-3-7-often-knows-when-it-s-in-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,334 Listeners
Conversations with Tyler
2,393 Listeners
The Peter Attia Drive
8,004 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,120 Listeners
ManifoldOne
90 Listeners
Your Undivided Attention
1,494 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,261 Listeners
Machine Learning Street Talk (MLST)
91 Listeners
Dwarkesh Podcast
424 Listeners
Hard Fork
5,448 Listeners
The Ezra Klein Show
15,457 Listeners
Moonshots with Peter Diamandis
506 Listeners
No Priors: Artificial Intelligence | Technology | Startups
127 Listeners
Latent Space: The AI Engineer Podcast
71 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
466 Listeners