One of the most common (and comfortable) assumptions in AI safety discussions—especially outside of technical alignment circles—is that oversight will save us. Whether it's a human in the loop, a red team audit, or a governance committee reviewing deployments, oversight is invoked as the method by which we’ll prevent unacceptable outcomes.

It shows up everywhere: in policy frameworks, in corporate safety reports, and in standards documents. Sometimes it's explicit, like the EU AI Act saying that High-risk AI systems must be subject to human oversight, or stated as an assumption, as in a Deepmind paper also released yesterday, where they say that scheming won't happen because AI won't be able to evade oversight. Other times it's implicit, firms claiming that they are mitigating risk through regular audits and fallback procedures, or arguments that no-one will deploy unsafe systems in places without sufficient oversight.

But either [...]

---

First published:

Source:

Linkpost URL:
https://arxiv.org/abs/2507.03525

---

Narrated by TYPE III AUDIO.