Sign up to save your podcasts
Or
Share “Sonnet 4.5’s eval gaming seriously undermines alignment evals, and this seems caused by training on alignment evals” by Alexa Pan, ryan_greenblatt
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
“Sonnet 4.5’s eval gaming seriously undermines alignment evals, and this seems caused by training on alignment evals” by Alexa Pan, ryan_greenblatt
According to the Sonnet 4.5 system card, Sonnet 4.5 is much more likely than Sonnet 4 to mention in its chain-of-thought that it thinks it is being evaluated; this seems to meaningfully cause it to appear to behave better in alignment evaluations. So, Sonnet 4.5's behavioral improvements in these evaluations may partly be driven by growing tendency to notice and game evaluations rather than genuine alignment. This is an early example of a phenomenon that is going to get increasingly problematic: as evaluation gaming increases, alignment evaluations become harder to trust.[1]
To elaborate on the above: Sonnet 4.5 seems far more aware of being evaluated than previous models. In some of Anthropic's evaluations, Sonnet 4.5 explicitly mentions being tested between 80% and 100% of the time, compared with under 10% for Sonnet 4. Internal representations associated with evaluation awareness also became notably stronger over training.[2]
In addition, Sonnet 4.5 seems substantially less likely to behave poorly once it has noticed that it might be tested. As seen in the figure below, when Anthropic inhibited internal representations related to evaluation awareness[3], its misaligned behavior rate on synthetic honeypot tests increased from 0% to between 1% and 9%. This effect was [...]
---
Outline:
(06:58) Sonnet 4.5 is much more evaluation-aware than prior models
(10:00) Evaluation awareness seems to suppress misaligned behavior
(14:52) Anthropic's training plausibly caused Sonnet 4.5 to game evaluations
(16:28) Evaluation gaming is plausibly a large fraction of the effect of training against misaligned behaviors
(22:57) Suppressing evidence of misalignment in evaluation gamers is concerning
(25:25) What AI companies should do
(30:02) Appendix
The original text contained 21 footnotes which were omitted from this narration.
---
First published:
October 30th, 2025
Source:
https://www.lesswrong.com/posts/qgehQxiTXj53X49mM/sonnet-4-5-s-eval-gaming-seriously-undermines-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
To elaborate on the above: Sonnet 4.5 seems far more aware of being evaluated than previous models. In some of Anthropic's evaluations, Sonnet 4.5 explicitly mentions being tested between 80% and 100% of the time, compared with under 10% for Sonnet 4. Internal representations associated with evaluation awareness also became notably stronger over training.[2]
In addition, Sonnet 4.5 seems substantially less likely to behave poorly once it has noticed that it might be tested. As seen in the figure below, when Anthropic inhibited internal representations related to evaluation awareness[3], its misaligned behavior rate on synthetic honeypot tests increased from 0% to between 1% and 9%. This effect was [...]
---
Outline:
(06:58) Sonnet 4.5 is much more evaluation-aware than prior models
(10:00) Evaluation awareness seems to suppress misaligned behavior
(14:52) Anthropic's training plausibly caused Sonnet 4.5 to game evaluations
(16:28) Evaluation gaming is plausibly a large fraction of the effect of training against misaligned behaviors
(22:57) Suppressing evidence of misalignment in evaluation gamers is concerning
(25:25) What AI companies should do
(30:02) Appendix
The original text contained 21 footnotes which were omitted from this narration.
---
First published:
October 30th, 2025
Source:
https://www.lesswrong.com/posts/qgehQxiTXj53X49mM/sonnet-4-5-s-eval-gaming-seriously-undermines-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
View all episodes
By LessWrong
4.8
1212 ratings
According to the Sonnet 4.5 system card, Sonnet 4.5 is much more likely than Sonnet 4 to mention in its chain-of-thought that it thinks it is being evaluated; this seems to meaningfully cause it to appear to behave better in alignment evaluations. So, Sonnet 4.5's behavioral improvements in these evaluations may partly be driven by growing tendency to notice and game evaluations rather than genuine alignment. This is an early example of a phenomenon that is going to get increasingly problematic: as evaluation gaming increases, alignment evaluations become harder to trust.[1]
To elaborate on the above: Sonnet 4.5 seems far more aware of being evaluated than previous models. In some of Anthropic's evaluations, Sonnet 4.5 explicitly mentions being tested between 80% and 100% of the time, compared with under 10% for Sonnet 4. Internal representations associated with evaluation awareness also became notably stronger over training.[2]
In addition, Sonnet 4.5 seems substantially less likely to behave poorly once it has noticed that it might be tested. As seen in the figure below, when Anthropic inhibited internal representations related to evaluation awareness[3], its misaligned behavior rate on synthetic honeypot tests increased from 0% to between 1% and 9%. This effect was [...]
---
Outline:
(06:58) Sonnet 4.5 is much more evaluation-aware than prior models
(10:00) Evaluation awareness seems to suppress misaligned behavior
(14:52) Anthropic's training plausibly caused Sonnet 4.5 to game evaluations
(16:28) Evaluation gaming is plausibly a large fraction of the effect of training against misaligned behaviors
(22:57) Suppressing evidence of misalignment in evaluation gamers is concerning
(25:25) What AI companies should do
(30:02) Appendix
The original text contained 21 footnotes which were omitted from this narration.
---
First published:
October 30th, 2025
Source:
https://www.lesswrong.com/posts/qgehQxiTXj53X49mM/sonnet-4-5-s-eval-gaming-seriously-undermines-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
To elaborate on the above: Sonnet 4.5 seems far more aware of being evaluated than previous models. In some of Anthropic's evaluations, Sonnet 4.5 explicitly mentions being tested between 80% and 100% of the time, compared with under 10% for Sonnet 4. Internal representations associated with evaluation awareness also became notably stronger over training.[2]
In addition, Sonnet 4.5 seems substantially less likely to behave poorly once it has noticed that it might be tested. As seen in the figure below, when Anthropic inhibited internal representations related to evaluation awareness[3], its misaligned behavior rate on synthetic honeypot tests increased from 0% to between 1% and 9%. This effect was [...]
---
Outline:
(06:58) Sonnet 4.5 is much more evaluation-aware than prior models
(10:00) Evaluation awareness seems to suppress misaligned behavior
(14:52) Anthropic's training plausibly caused Sonnet 4.5 to game evaluations
(16:28) Evaluation gaming is plausibly a large fraction of the effect of training against misaligned behaviors
(22:57) Suppressing evidence of misalignment in evaluation gamers is concerning
(25:25) What AI companies should do
(30:02) Appendix
The original text contained 21 footnotes which were omitted from this narration.
---
First published:
October 30th, 2025
Source:
https://www.lesswrong.com/posts/qgehQxiTXj53X49mM/sonnet-4-5-s-eval-gaming-seriously-undermines-alignment
---
Narrated by TYPE III AUDIO.
---
Images from the article:
More shows like LessWrong (Curated & Popular)
View all
Macro Voices
3,068 Listeners
Odd Lots
1,936 Listeners
EconTalk
4,263 Listeners
Conversations with Tyler
2,453 Listeners
Philosophy Bites
1,546 Listeners
ChinaTalk
289 Listeners
ManifoldOne
93 Listeners
Machine Learning Street Talk (MLST)
95 Listeners
Dwarkesh Podcast
511 Listeners
Clearer Thinking with Spencer Greenberg
138 Listeners
Razib Khan's Unsupervised Learning
208 Listeners
"Econ 102" with Noah Smith and Erik Torenberg
152 Listeners
Money Stuff: The Podcast
394 Listeners
Complex Systems with Patrick McKenzie (patio11)
134 Listeners
The Marginal Revolution Podcast
95 Listeners