How large of a breakthrough is necessary for dangerous AI?

In order to cause a catastrophe, an AI system would need to be very competent at agentic tasks[1]. The best metric of general agentic capabilities is METR's time horizon. The time horizon measures the length of well-specified software tasks AI systems can do, and is grounded in human baselines, which means AI performance can be closely compared to human performance.

Causing a catastrophe[2] is very difficult. It would likely take many decades, or even centuries, of skilled human labor. Let's use one year of human labor as a lower bound on how difficult it is. This means that AI systems will need to at least have a time horizon of one work-year (2000 hours) in order to cause a catastrophe.

Current AIs have a time horizon of 2 hours, which means it's 1000x lower than the time horizon necessary to cause a catastrophe. This presents a pretty large buffer.

Currently, the time horizon is doubling roughly every half-year. That means that a 1000x increase would take roughly 5 years at the current rate of progress. So, in order for AI to reach a time horizon of 1 work-year within [...]

---

Outline:

(00:11) How large of a breakthrough is necessary for dangerous AI?

(02:04) AI breakthroughs of the recent past

(02:27) Case 1: Transformers

(03:54) Case 2: AlphaFold

(04:30) What is the probability of 1-year time horizons in the next 6 months?

(05:10) Narrowly superhuman AI leading to generally competent AI

(06:27) Would we notice a massive capabilities increase?

(07:44) Conclusion

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.