Summary

When discussing the possibility that LLMs will cease to reason in transparent natural language with other AI safety researchers, we have sometimes noticed that we talk past each other: e.g., when discussing ‘neuralese reasoning’, some people have indefinitely long chains of recurrent activations in mind, while others think of an alien language in token-space. This post will propose terminology for distinguishing different types of hidden reasoning to create a shared language for such conversations.

In short, our taxonomy categorizes types of hidden reasoning as follows:

• Neuralese: Arbitrary-depth reasoning in high-dimensional latent space, enabled by recurrence.
• Hidden parallelized reasoning: Important steps are computed in activation-space in parallel without appearing in the chain-of-thought (CoT).
  • Subcategories: partial verbalization, hidden utilization of test-time compute, epiphenomenal reasoning
• Encoded reasoning: The AI either encodes the CoT itself in such a way that the monitor cannot decode, or encodes additional information inside seemingly legible [...]

---

Outline:

(00:10) Summary

(02:47) Taxonomy

(04:26) Recurrent neuralese reasoning

(04:55) Hidden parallelized reasoning

(07:51) Steganography

(10:21) Linguistic drift

(11:38) Dazzling

(12:20) Examples in the Wild

(12:24) Neuralese

(13:05) Hidden parallelized reasoning

(15:20) Steganography

(15:23) Semantic steganography

(17:15) Lexical steganography through steganographically embedded ciphers

(18:21) Lexical steganography through internal token layering

(19:52) Linguistic drift

(21:26) Dazzling

(21:29) Post-hoc justifications

(22:35) Manipulation

(23:04) Some additional categories

(25:09) Canary string

(25:25) Acknowledgements

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.