I’d like to thank my mentors David Duvenaud, Raymond Douglas, David Krueger and Jan Kulveit for providing helpful ideas, comments and discussions. The views expressed in this, and any mistakes, are solely my own.

1: Introduction

There is a small but growing literature focused on “Gradual Disempowerment” threat models, where disempowerment occurs due to the integration of more advanced AI systems into politics, economy and culture. These scenarios posit that, even without a system with a decisive advantage deliberately taking over, competitive dynamics and influence-seeking behaviour within social, political and cultural systems will eventually lead to the erosion of human influence, and at the extreme, the permanent disempowerment of humanity. I define permanent disempowerment as a state of affairs where humanity loses the ability to meaningfully exert any influence over the state and direction of civilisation.

This post is a summary of an early draft of a paper I [...]

---

Outline:

(00:27) 1: Introduction

(04:29) 2: Principal Driven Pathways

(04:34) 2.1: Power concentrated in specific principals

(05:32) 2.2: Principals Voluntarily Hand Over Power to the AIs

(08:30) 3: Alignment Driven Pathways

(08:35) 3.1: Misaligned, powerseeking AIs take over

(10:09) 3.2: Legal Lock In

(12:04) 3.3: AI-driven culture causes value drift

(13:17) 4: System-Driven Pathways

(13:22) 4.1: The cost of shutdown is too high

(14:48) 4.2: Human veto is uncompetitive

(15:37) 4.3: Coordination Ability is never good enough

(16:49) 5: The interaction between pathways

---

First published:

Source:

---

Narrated by TYPE III AUDIO.