Is consciousness the last moral world?

Imagine trying to explain to a virus why suffering matters.

A virus is a simple self-replicating molecule: unsophisticated and arguably not even alive. It has no experience. It just copies itself according to chemical laws. From its “perspective” (it doesn’t have one), the universe is just physics: particles following rules. If you could somehow tell it that certain arrangements of matter are good and others are bad, it wouldn’t disagree with you. It does not have the concepts to agree or disagree. Might as well ask a stone what it thinks of war.

Are we that virus, relative to what the future could hold?

Photo by Kiril Dobrev on Unsplash

I. The Three Worlds

Today I want to discuss the possibility of further moral goods: further axes of moral value as yet inaccessible to us, that are qualitatively not just quantitatively different from anything we’ve observed to date.

For background, I think normal, secular humans navigate three conceptually distinct but overlapping worlds:

1. The physical world. Matter, energy, atoms, stars, cells. If you were a detached observer of our universe, you might think this is all there is.
2. The mathematical world. Logic, abstract structure, rationality [...]

---

Outline:

(01:05) I. The Three Worlds

(04:16) II. Pinpointing the Ineffable

(05:50) III. Reasons for optimism

(07:46) IV. Implications and Future Work

(09:00) The case for AI catastrophe, in four steps

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.