Subtitle: How can we make AIs aligned and well-elicited on extremely hard to check open ended tasks?.

As AI systems get more capable, it becomes increasingly uncompetitive and infeasible to avoid deferring to AIs on increasingly many decisions. Further, once systems are sufficiently capable, control becomes infeasible.1 Thus, one of the main strategies for handling AI risk is fully (or almost fully) deferring to AIs on managing these risks. Broadly speaking, when I say “deferring to AIs”2 I mean having these AIs do virtually all of the work to develop more capable and aligned successor AIs, managing exogenous risks, and making strategic decisions.3 If we plan to defer to AIs, I think it's safest to do so only a bit above the minimum level of qualitative capability/intelligence required to automate safety research, implementation, and strategy.4

For deference to go well, we both need it to be the case that the AIs we defer to aren’t scheming against us and that they are sufficiently aligned and effective on key tasks (aligning the next generation of AIs, buying more time to work on alignment, making good strategic decisions). Being sufficiently aligned and effective on key tasks requires that AIs are [...]

---

Outline:

(05:01) High-level objectives

(05:04) What does successful deference look like?

(05:53) Recursive self-improvement of alignment and wisdom: the hope for a Basin of Good Deference

(09:46) Summary of objectives and what the AIs we defer to must do

(12:34) Handling exogenous risk in time

(20:38) Deferring to weaker AIs (that are less likely to be schemers)

(26:41) Do increases in speed/productivity mean we should defer to more capable AIs?

(29:28) Human level deference-goodness is a reasonable target

(33:45) A relatively prosaic strategy for deference

(40:02) Objectives with the capability profile

(48:28) Objectives with broad alignment

(01:05:45) Behavioral testing for capabilities and (broad) alignment

(01:29:44) Approaches for improving alignment

(01:53:26) Approaches for improving capability profile

(01:56:44) How much of the work needed for preparing for deference will happen due to myopic commercial incentives?

(02:00:03) Political difficulties with deference

(02:02:38) When should we defer to AIs?

The original text contained 35 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.