Introduction

I’m excited by deception probes. When I mention this, I’m sometimes asked “Do deception probes work?”

But I think there are many applications of deception probes, and each application will require probes with different properties, i.e. whether a deception probe works will depend on what you’re using it for. Furthermore, whether one deception probe works better than another will also depend on what you’re using them for. This remark sounds a bit trivial, but I didn’t appreciate it initially.

In this document, I'll enumerate 18 different applications of deception probes, and what properties each application requires. This are just my very rough guesses, just to kickstart some public conversation.

Applications of deception probes

1. Monitor current models

Problem Statement: AIs at the current capability level may be important for future safety work, e.g. trusted monitoring. However, current LLMs engage in strategic deception. This is despite being post-trained to [...]

---

Outline:

(00:10) Introduction

(00:56) Applications of deception probes

(01:00) 1. Monitor current models

(03:31) 2. Discard unsafe actions

(05:22) 3. Inform trusted monitor

(07:43) 4. Allocate the auditing budget

(10:35) 5. Allocate the careful feedback budget

(14:24) 6. Augment AI Debate

(17:18) 7. Elicit Latent (External) Knowledge

(19:57) 8. Elicit Latent (Self) Knowledge

(23:29) 9. Elicit Latent (Activity) Knowledge

(27:09) 10. Compare deceptiveness of models

(28:38) 11. Compare deceptiveness of topics

(31:00) 12. Increase honesty by resampling

(33:55) 13. Increase honesty by modifying activations

(36:22) 14. Increase honesty by finetuning

(38:22) 15. Understand deception mechanistically

(40:24) 16. Motivate compliance to bargained agreements

(42:47) 17. Complement Chain-of-Thought monitoring

(44:36) 18. Safely pass the buck

(46:50) Conclusion

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.