In this essay, I introduce two Points of No Return (PNR):

• The Hard PNR. The moment where we have AI systems powerful and intelligent enough that they can prevent humanity from turning them off.
• The Soft PNR. The moment where we have AI systems that we will not want to turn off. For instance, because they are too intertwined with the economy, military systems, geopolitics, or romantic relationships.

I believe the Soft PNR is underrated (especially in rationalist communities).

I also believe that it is roughly as irrecoverable as the Hard PNR, and that it is quite likely to come first.

Cheers!

---

First published:

Source:

Linkpost URL:
https://cognition.cafe/p/ai-timelines-and-points-of-no-return

---

Narrated by TYPE III AUDIO.