In the future, we will want to use powerful AIs on critical tasks such as doing AI safety and security R&D, dangerous capability evaluations, red-teaming safety protocols, or monitoring other powerful models. Since we care about models performing well on these tasks, we are worried about sandbagging: that if our models are misaligned [1], they will intentionally underperform.

Sandbagging is crucially different from many other situations with misalignment risk, because it involves models purposefully doing poorly on a task, rather than purposefully doing well. When people talk about risks from overoptimizing reward functions (e.g. as described in What failure looks like), the concern is that the model gets better performance (according to some metric) than an aligned model would have. And when they talk about scheming, the concern is mostly that the model gets performance that is as good as an aligned model despite not being aligned. When [...]

---

Outline:

(03:39) Sandbagging can cause a variety of problems

(07:50) Training makes sandbagging significantly harder for the AIs

(09:30) Training on high-quality data can remove sandbagging

(12:13) If off-policy data is low-quality, on-policy data might help

(14:22) Models might subvert training via exploration hacking

(18:31) Off-policy data could mitigate exploration hacking

(23:35) Quick takes on other countermeasures

(25:20) Conclusion and prognosis

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.