Caspar Oesterheld came up with two of the most important concepts in my field of work: Evidential Cooperation in Large Worlds and Safe Pareto Improvements. He also came up with a potential implementation of evidential decision theory in boundedly rational agents called decision auctions, wrote a comprehensive review of anthropics and how it interacts with decision theory which most of my anthropics discussions built on, and independently decided to work on AI some time late 2009 or early 2010.

Needless to say, I have a lot of respect for Caspar's work. I’ve often felt very confused about what to do in my attempts at conceptual research, so I decided to ask Caspar how he did his research. Below is my writeup from the resulting conversation.

How Caspar came up with surrogate goals

The process

• Caspar had spent six months FTE thinking about a specific bargaining problem [...]

---

Outline:

(00:54) How Caspar came up with surrogate goals

(00:59) The process

(02:44) Caspar's reflections on what was important during the process

(04:33) How Caspar came up with ECL

(04:38) The process

(07:02) Caspar's reflections on what was important during the process

(07:24) How Caspar came up with decision auctions

(07:29) The process

(09:44) How Caspar decided to work on superhuman AI in late 2009 or early 2010

(10:25) The process

(13:09) Caspar's reflections on what was important during the process

(14:30) General notes on his approach to research

(14:34) What does research concretely look like in his case?

(15:17) Research immersion

(16:36) Goal orientation vs. curiosity orientation

---

First published:

Source:

Narrated by TYPE III AUDIO.