Topic of the post: I list potential things to work on other than keeping AI under human control.

Motivation

The EA community has long been worried about AI safety. Most of the efforts going into AI safety are focused on making sure humans are able to control AI. Regardless of whether we succeed at this, I think there's a lot of additional value on the line.

First of all, if we succeed at keeping AI under human control, there are still a lot of things that can go wrong. My perception is that this has recently gotten more attention, for example here, here, here, and at least indirectly here (I haven’t read all these posts. and have chosen them to illustrate that others have made this point purely based on how easily I could find them). Why controlling AI doesn’t solve everything is not the main topic of this [...]

---

Outline:

(00:14) Motivation

(04:03) Other works and where this list fits in

(05:44) The list

(05:47) Making AI-powered humanity wiser, making AIs wiser

(06:35) More detailed definition

(08:15) Preparing AIs for reasoning about philosophically complex topics with high stakes

(09:25) Improve epistemics during an early AI period

(10:35) Metacognition for areas where it is better for you to avoid information

(13:27) Improve decision theoretical reasoning and anthropic beliefs

(17:34) Compatibility of earth-originating AI with other intelligent life

(19:16) More detailed definition

(20:55) How to make progress on this

(24:46) Surrogate goals and other Safe Pareto improvements

(25:54) More detailed definition

(27:28) How to make progress on this

(28:39) AI personality profiling and avoiding the worst AI personality traits

(30:20) How to make progress on this

(31:46) Avoiding harm from how we train AIs: Niceness, near miss, and sign flip

(32:10) More detailed definition

(35:07) How to make progress on this

(37:56) Reducing human malevolence

(38:32) How to make progress on this

(39:15) Hot take: I want more surveys

(40:35) Acknowledgements

---

First published:

Source:

Narrated by TYPE III AUDIO.