This dialogue is still in progress, but due to other commitments we don't have much time to continue it. We think the content is interesting, so we decided to publish it unfinished. We will maybe very slowly continue adding to it in the future, but can't commit to doing so.

Context: I wrote a shortform about conditioning on high capability AI, and the ways I think my past self messed that up. In particular, I claimed that conditioning on high capability means that the AI will be goal-directed.

Matt responded:

Distinguish two notions of "goal-directedness":

1. The system has a fixed goal that it capably works towards across all contexts.
2. The system is able to capably work towards goals, but which it does, if any, may depend on the context.

My sense is that a high level of capability implies (2) but not (1). And that (1) [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.