If AIs became strategically competent enough, they may realize that RSI is too dangerous because they're not good enough at alignment or philosophy or strategy, and potentially convince, help, or work with humans to implement an AI pause. This presents an alternative "victory condition" that someone could pursue (e.g. by working on AI strategic competence) if they were relatively confident about the alignment of near-human-level AIs but concerned about the AI transition as a whole, for example because they're worried about alignment of ASI, or worried about correctly solving other philosophical problems that would arise during the transition. (But note that if the near-human-level AIs are not aligned, then this effort could backfire by letting them apply better strategy to take over more easily.)

Strategic vs Philosophical Competence

The previous "victory path" I've been focused on was to improve AI philosophical competence, under the theory that if the AIs are aligned, they'll want to help us align the next generation of AIs and otherwise help guide us through the AI transition. I think by default they will be too incompetent at philosophical reasoning to do a good enough job at this, hence the proposal to improve such competence. However [...]

---

Outline:

(00:55) Strategic vs Philosophical Competence

(01:48) Unilateral Refusal vs AI Assistance for Pausing AI

---

First published:

Source:

---

Narrated by TYPE III AUDIO.