Epistemic status: Speculation on agent foundations research culture (which I am pretty deeply engaged with) and whether "we are confused about agency" which I am not sure about. I will take for granted that this is a common refrain, which should be familiar to anyone who is part of the relevant scene.

The phrase "We are confused about agency," often with variations such as "way too," "deeply," or "dangerously" is a common membership signal for a certain AI safety research culture. Roughly speaking, this is the culture of the agent foundations research program that accreted around MIRI.[1] The phrase is usually supplied as an argument for delaying the development of A.I. until certain mathematical research (particularly in learning/decision theory) has been carried out.

I find the phrase uncomfortable on various levels.

As Cultural Signal

Since I claim that the phrase (apart from its literal meaning) functions as an in-group signal, it is natural to wonder what "we" refers to here.

I believe the intention is "we = everyone." I think that it requires a pretty serious level of scholarship to confidently claim that everyone is confused about agency. One general frustration I have with rationalist culture [...]

---

Outline:

(00:59) As Cultural Signal

(02:48) Confused in what Sense?

(05:10) Confused about what?

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.