AI risk discussion often seems to assume that the AI we most want to prepare for will emerge in a “normal” world — one that hasn’t really been transformed by earlier AI systems.

I think betting on this assumption could be a big mistake. If it turns out to be wrong, most of our preparation for advanced AI could end up ~worthless, or at least far less effective than it could have been. We might find ourselves wishing that we’d laid the groundwork for leveraging enormous political will, prepared for government inadequacy, figured out how to run large-scale automated research projects, and so on. Moreover, if earlier systems do change the background situation, influencing how that plays out could be one of our best levers for dealing with AI challenges overall, as it could leave us in a much better situation for the “boss-battle” AI (an opportunity we’ll squander if we focus exclusively on the endgame).

So I really want us to invest in improving our views on which AI changes we should expect to see first. Progress here seems at least as important as improving our AI timelines (which get significantly more attention). [...]

---

First published:

Source:

Linkpost URL:
https://www.forethought.org/research/the-first-type-of-transformative-ai#improving-how-earlier-ai-transformations-play-out

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.