A common element in many AI control schemes is monitoring – using some model to review actions taken by an untrusted model in order to catch dangerous actions if they occur. Monitoring can serve two different goals. The first is detection: identifying misbehavior so you can understand it and prevent similar actions from happening in the future. For instance, some monitoring at AI companies is already used to identify behaviors like complying with misuse requests, reward hacking, and hallucination. The second goal is prevention: identifying misbehavior in order to mitigate the harmful effects of the particular action. For instance, constitutional classifiers are used by Anthropic to detect and prevent jailbreaks, and Claude Code auto mode prevents destructive mistakes in real time.

The latency requirements for monitoring are different depending on whether the goal is detection or prevention. If the goal is detection, it's fine for there to be substantial latency in the monitor. But if the goal is prevention, then the monitor needs to trigger quickly, so that damage can be prevented or recovered from. In the extreme, monitors can run synchronously – after an action is generated by an LLM but before it is executed. Both constitutional classifiers [...]

---

Outline:

(01:59) When is monitor latency important for prevention?

(05:13) Blocking with synchronous monitors

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.