"He who fights with monsters might take care lest he thereby become a monster. And if you gaze for long into an abyss, the abyss gazes also into you."

- Friedrich Nietzsche

I think it is not an exaggeration to say that many people I know in this community hate the idea of powerful, unaligned AI. They describe it in apocalyptic terms, equate its existence to the death of everything they love, treat every step towards it as evil in its purest form. We as a community are haunted by pictures of paperclip maximisers quietly recycling human civilisation into spools of wire, disneylands without children, Von Neumann probes spreading out from a darkened Earth where humanity chokes to death because the sun has been blacked out by a Dyson sphere. As a community, you would think, we would be the least interested in pressing the make-a-superintelligence button.

But [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.