(Part 2 of the CAST sequence)

As a reminder, here's how I’ve been defining “corrigible” when introducing the concept: an agent is corrigible when it robustly acts opposite of the trope of "be careful what you wish for" by cautiously reflecting on itself as a flawed tool and focusing on empowering the principal to fix its flaws and mistakes.

This definition is vague, imprecise, and hides a lot of nuance. What do we mean by “flaws,” for example? Even the parts that may seem most solid, such as the notion of there being a principal and an agent, may seem philosophically confused to a sufficiently advanced mind. We’ll get into trying to precisely formalize corrigibility later on, but part of the point of corrigibility is to work even when it's only loosely understood. I’m more interested [...]

---

Outline:

(04:00) Emergent Desiderata

(04:04) Communication

(05:01) Low-Impact

(05:32) Reversibility

(06:31) Efficiency

(07:17) Relevance

(07:56) Transparency

(08:21) Obedience

(09:19) Mild-Optimization

(10:07) Protectiveness

(10:57) Local Scope

(11:32) Simple Self-Protectiveness

(11:50) Stop Button

(12:53) Graceful Shutdown

(13:25) Configurable Verbosity

(14:27) Disambiguation/Concreteness

(15:17) Honesty

(16:02) Handling Antagonists

(17:20) Straightforwardness

(18:03) Proactive Reflection

(18:44) Cognitive Legibility

(19:53) Infohazard Caution

(20:42) Resource Accumulation

(21:46) Non-Manipulation

(23:25) Sub-Agent Stability

(24:22) Principal-Looping

(25:01) Graceful Obsolescence

(26:00) Handling Trolley-Tradeoffs

(26:46) Handling Time-Pressure

(28:32) Expandable Concerns

(29:21) Navigating Conflict

(29:32) Simple Conflict

(30:15) Violent Conflict

(30:45) Authority Conflict

(31:18) Shutdown Conflict

(31:55) Emergent Downsides

(31:59) Intrusiveness

(32:45) Indifference

(33:31) Rigidity

(34:17) Immorality

(34:52) Irresponsibility

(35:28) Myopia

(36:02) Incorrigible Counter-Examples

(36:24) Honesty

(36:41) Protectiveness

(37:05) Proactive Benevolence

(37:22) Kindness

(37:40) Human-In-Loop

(38:02) Moral Learning

(38:14) Balancing Needs

(38:37) Broad Perspective

(38:55) Top-Level-Goal Focus

(39:13) Nearby Concepts that Aren’t Synonyms for Corrigible

(39:43) Correctability

(41:58) “The Thing Frontier Labs Are Currently Aiming For”

(44:32) Preference Satisfaction

(48:07) Empowerment (in general)

(50:20) Caution

(51:47) Servility

(54:30) Tool/Task-ishness

(56:21) Discussion

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.