Over the past year, I have talked to several people about whether they expect frontier AI companies to transition away from the current paradigm of transformer LLMs toward models that reason in neuralese within the next few years. This post summarizes 13 common arguments I’ve heard, six in favor and seven against a transition to neuralese AIs. The following table provides a summary:

---

Outline:

(00:49) What do I mean by neuralese?

(02:07) Six arguments in favor of a transition to neuralese AIs

(02:13) 1) A lot of information is lost in a text bottleneck

(03:42) 2) The increasing importance of post-training

(04:37) 3) Active research on recurrent LLMs

(05:50) 4) Analogy with human thinking

(08:03) 5) SGD inductive biases

(08:34) 6) The limit of capabilities

(08:54) Seven arguments against a transition to neuralese AIs

(09:00) 1) The natural language sweet spot

(10:59) 2) The parallelism trade-off

(12:11) 3) Business value of visible reasoning traces

(12:55) 4) Analogy with human thinking

(13:47) 5) Evidence from past attempts to build recurrent LLMs

(14:53) 6) The depth-latency trade-off

(16:09) 7) Safety value of visible reasoning traces

(16:38) Conclusion

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.