There's many debates about "what counts as AGI" or "what counts as superintelligence?".

Some people might consider those arguments "goalpost moving." Some people were using "superintelligence" to mean "overwhelmingly smarter than humanity", and using it to mean "spikily good at some coding tasks while still not really successfully generalizing or maintaining focus" feels like watering it down.

I think there's just actually a wide range of concepts that need to get talked about. And, right now, most of the AIs that people will wanna talk about are kinda general and kinda superintelligent and kinda aligned.

If you have an specific concept you wanna protect, I think it's better to just give it a clunky name that people don't want to use in casual conversation, rather than pumping against entropy to defend a simple term that could be defined to mean other things.

Previously OpenPhil had used "Transformative AI" to mean "AI that is, you know, powerful enough to radically transform society, somehow." I think that's a useful term. But, it's not exactly what If Anyone Builds It is cautioning about.

The type of AI I'm most directly worried about is "overwhelmingly superhuman compared to humanity." (And, AIs that [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.