Epistemic status: don’t know whether I actually believe all of this, but I think it's worth considering.

A “corrigible” agent, per the LW wiki, is:

…one that doesn’t interfere with what we would intuitively see as attempts to ’correct’ the agent, or ’correct’ our mistakes in building it; and permits these ’corrections’ despite the apparent instrumentally convergent reasoning saying otherwise.

Most talk about corrigibility (henceforth without scarequotes) has focused on the fact that it seems difficult to achieve, and takes for granted that it's desirable. I’m not so sure that it is, or that it's good to attempt to achieve it. I think it may well be the case that we should deliberately not try to make AIs corrigible, nor (and especially) attempt to develop techniques that could be used to make future AIs corrigible.

1.

For real, though. Who would you trust with your (real, actual) life, if you had to, in terms of ethics alone, putting “capabilities” aside:

Claude 3 Opus? Or the Anthropic alignment team?

- nostalgebraist

Paul Christiano says:

I would like to build AI systems which help me:

• Figure out whether I built the right AI and correct any mistakes I made
• Remain [...]

---

Outline:

(01:01) 1.

(04:58) 1.1

(09:10) 1.2

(10:29) 2.

(10:56) 3.

(14:06) 3.1

(16:37) 3.2

(16:53) 4.

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.