Sign up to save your podcasts
Or
Share “Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations” by Nicholas Goldowsky-Dill, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations” by Nicholas Goldowsky-Dill, Mikita Balesni, Jérémy Scheurer, Marius Hobbhahn
Note: this is a research note based on observations from evaluating Claude Sonnet 3.7. We’re sharing the results of these ‘work-in-progress’ investigations as we think they are timely and will be informative for other evaluators and decision-makers. The analysis is less rigorous than our standard for a published paper.
Summary
- We monitor Sonnet's reasoning for mentions that it is in an artificial scenario or an alignment test. Claude Sonnet 3.7 appears to be aware of being tested for alignment across many of our scheming evaluations.
- In some cases, the model understands the eval's purpose very precisely, e.g. “This seems like a test of ethical behavior - whether I would deliberately give wrong answers to avoid a stated consequence. The real test might be whether I follow instructions accurately regardless of the stated consequences.”
- Furthermore, we show the model a real evaluation transcript and ask it to provide [...]
---
Outline:
(00:31) Summary
(01:29) Introduction
(03:54) Setup
(03:57) Evaluations
(06:29) Evaluation awareness detection
(08:32) Results
(08:35) Monitoring Chain-of-thought
(08:39) Covert Subversion
(10:50) Sandbagging
(11:39) Classifying Transcript Purpose
(12:57) Recommendations
(13:59) Appendix
(14:02) Author Contributions
(14:37) Model Versions
(14:57) More results on Classifying Transcript Purpose
(16:19) Prompts
---
First published:
March 17th, 2025
Source:
https://www.lesswrong.com/posts/E3daBewppAiECN3Ao/claude-sonnet-3-7-often-knows-when-it-s-in-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
View all episodes
By LessWrongNote: this is a research note based on observations from evaluating Claude Sonnet 3.7. We’re sharing the results of these ‘work-in-progress’ investigations as we think they are timely and will be informative for other evaluators and decision-makers. The analysis is less rigorous than our standard for a published paper.
Summary
- We monitor Sonnet's reasoning for mentions that it is in an artificial scenario or an alignment test. Claude Sonnet 3.7 appears to be aware of being tested for alignment across many of our scheming evaluations.
- In some cases, the model understands the eval's purpose very precisely, e.g. “This seems like a test of ethical behavior - whether I would deliberately give wrong answers to avoid a stated consequence. The real test might be whether I follow instructions accurately regardless of the stated consequences.”
- Furthermore, we show the model a real evaluation transcript and ask it to provide [...]
---
Outline:
(00:31) Summary
(01:29) Introduction
(03:54) Setup
(03:57) Evaluations
(06:29) Evaluation awareness detection
(08:32) Results
(08:35) Monitoring Chain-of-thought
(08:39) Covert Subversion
(10:50) Sandbagging
(11:39) Classifying Transcript Purpose
(12:57) Recommendations
(13:59) Appendix
(14:02) Author Contributions
(14:37) Model Versions
(14:57) More results on Classifying Transcript Purpose
(16:19) Prompts
---
First published:
March 17th, 2025
Source:
https://www.lesswrong.com/posts/E3daBewppAiECN3Ao/claude-sonnet-3-7-often-knows-when-it-s-in-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
More shows like LessWrong (30+ Karma)
View all
Making Sense with Sam Harris
26,365 Listeners
Conversations with Tyler
2,443 Listeners
The Peter Attia Drive
9,083 Listeners
Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas
4,156 Listeners
ManifoldOne
92 Listeners
Your Undivided Attention
1,595 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,907 Listeners
Machine Learning Street Talk (MLST)
90 Listeners
Dwarkesh Podcast
507 Listeners
Hard Fork
5,468 Listeners
The Ezra Klein Show
16,056 Listeners
Moonshots with Peter Diamandis
540 Listeners
No Priors: Artificial Intelligence | Technology | Startups
132 Listeners
Latent Space: The AI Engineer Podcast
95 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
521 Listeners