This post benefitted greatly from comments, suggestions, and ongoing discussions with David Duvenaud, David Krueger, and Jan Kulveit. All errors are my own.

A few months ago, I and my coauthors published Gradual Disempowerment (GD hereafter). It was mostly about how things might go wrong, but naturally a lot of the resulting interest has been about solutions.

We have some more formal followup work coming: in the meantime, this is my 80/20 for ‘what would I do if I had way more time’ / ‘what would I find it helpful if someone else had done well’. This document is very much breadth over depth, and still missing a lot of details; I hope it is nonetheless helpful. For many of these, I expect even a pretty motivated and smart undergraduate could make useful progress in 10-20 hours.

I would be excited about people doing good work on [...]

---

Outline:

(01:15) Conceptual / High-Level

(01:19) Interaction with other x-risk concerns

(02:37) Responding to counterarguments

(04:10) Beyond competitive pressures

(06:12) Clarifying the goal

(07:42) Social Science-y

(07:45) Robustness of societal fundamentals

(08:59) Studying Historical Parallels

(10:12) Indicators and Policy

(11:32) Technical / Mathematical

(11:36) Simulating entire civilizations

(12:46) AI cognition and agency

(13:26) Civilisational Alignment / Hierarchical Agency

(16:41) Differential Progress / Differential Empowerment

(17:38) AI Complementarity

(18:48) Concluding thoughts

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.