To me it feels pretty emotionally clear we are nearing the end-times with AI. That in 1-4 years[1] things will be radically transformed, that at least one the big AI labs will become autonomous research organizations working on developing the next version on AI, perhaps with some narrow guidance of humans in oversight or acquisition of more resources until robotics is solved too.

And i believe there will be some nice benefits at first with this, with the AI organizations providing many goods and services in exchange for money, to raise capital so that the self-improvement resource acquisition loop can continue.

But I’m not sure how it will ultimately turn out. Declaring risk of extinction-level events less than 10% seems overconfident. Yet, declaring the risks to be >90% also seems overconfident. But I generally remain quite uncertain about which factors will dominate. Maybe AIs will remain friendly and for decision theory reasons continue put some fraction of resources to look after us to some extent, as a signal that future entities should do the same for them. Maybe the loop of capital acquisition is so brutal and molochian that models that doom keeps on winning. And people have been [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.