[Subtitle.] Automating R&D is not sufficient for superintelligence

This is a crosspost for The Goodhart Singularity by Tom Reed, which was originally published on Thomas's Substack on 7 May 2026.

Buyer beware: bombastic claims advanced, often flippantly.

AI systems are starting to build themselves. Because each generation of model will be better at building its successor than the last, it seems plausible that the full automation of AI R&D could rapidly lead to an exponential growth in overall AI capabilities. A natural inference is that domain-general superintelligence arrives shortly after AI research is automated.

This natural inference is, indeed, almost the default story of AI progress, as well as the strategy of those pursuing it. Where the AI industry once distinguished itself by the search for general intelligence, it has in recent years converged on a narrower path: the targeted pursuit of coding and AI research. For Sam Altman, “perhaps nothing is quite as significant as the fact that we can now use [AI] to do faster AI research”.

To many, the (ostensibly imminent) automation of AI research looks sufficient for the rapid subsequent development of other world-changing capabilities. It would constitute, in Jack Clark's [...]

---

First published:

Source:

Linkpost URL:
https://meagreprotestanthistory.substack.com/p/the-goodhart-singularity

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.