Short Summary

LLMs may be fundamentally incapable of fully general reasoning, and if so, short timelines are less plausible.

Longer summary

There is ML research suggesting that LLMs fail badly on attempts at general reasoning, such as planning problems, scheduling, and attempts to solve novel visual puzzles. This post provides a brief introduction to that research, and asks:

• Whether this limitation is illusory or actually exists.
• If it exists, whether it will be solved by scaling or is a problem fundamental to LLMs.
• If fundamental, whether it can be overcome by scaffolding & tooling.

If this is a real and fundamental limitation that can't be fully overcome by scaffolding, we should be skeptical of arguments like Leopold Aschenbrenner's (in his recent 'Situational Awareness') that we can just 'follow straight lines on graphs' and expect AGI in [...]

---

Outline:

(00:09) Short Summary

(00:19) Longer summary

(01:11) Introduction

(02:31) What is general reasoning?

(03:44) How general are LLMs?

(06:04) Evidence for generality

(07:15) Evidence against generality

(07:33) Block world

(08:26) Scheduling

(09:02) ARC-AGI

(09:29) Will scaling solve this problem?

(10:16) Will scaffolding or tooling solve this problem?

(11:54) Why does this matter?

(13:17) OK, but what do you think?

The original text contained 10 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.