Audio note: this article contains 172 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

This post was written during Alex Altair's agent foundations fellowship program, funded by LTFF. Thank you Alex Altair, Alfred Harwood, Daniel C for feedback and comments.

Introduction

The selection theorems agenda aims to prove statements of the following form: "agents selected under criteria _X_ provably has property _Y_," where _Y_ are things such as world models, general purpose search, modularity, etc. We're going to focus on world models.

But what is the intuition that makes us expect to be able to prove such things in the first place? Why expect world models?

Because: assuming the world is a Causal Bayesian Network with the agent's actions corresponding to the _D_ (decision) node, if its actions can robustly control the [...]

---

Outline:

(00:30) Introduction

(03:07) Basic Setup

(03:11) World

(03:42) Agent as a Policy Oracle

(05:14) Robustness as _\\delta_\-optimality under interventions

(10:27) Assumptions

(11:30) Main Theorem

(12:31) High-level argument

(16:34) Discussion

(17:08) Policy oracle is not a good model of an agent.

(18:03) The Causal Good Regulator Theorem isnt a structural theorem.

(20:51) Conclusion

The original text contained 1 footnote which was omitted from this narration.

The original text contained 11 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.