Audio note: this article contains 196 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

[Sorry about the lengthiness of this post. I recommend not fixating too much on all the specific numbers and the formal apparatus. Originally the plan was to also analyze optimal timing from an impersonal (xrisk-minimization) perspective; but to prevent the text from ballooning even more, that topic was set aside for future work (which might never get done). But I should at least emphasize that there are other important factors, not covered here, that would need to be taken into account if one wishes to determine which timeline would be best all things considered.]

[Working paper.[1] Version 1.0. Canonical link to future revised version of this paper.]

Abstract

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal. We examine optimal timing from a person-affecting stance (and set aside simulation hypotheses and other arcane considerations). Models incorporating safety progress, temporal discounting, quality-of-life differentials, and concave QALY utilities suggest that even high catastrophe probabilities are often [...]

---

Outline:

(01:06) Abstract

(01:58) Introduction

(08:00) Evaluative framework

(10:08) A simple go/no-go model

(12:46) Incorporating time and safety progress

(17:10) Temporal discounting

(19:19) Quality of life adjustment

(22:04) Diminishing marginal utility

(25:27) Changing rates of safety progress

(34:35) Shifting mortality rates

(41:11) Safety testing

(48:07) Distributional considerations

(01:03:14) Other-focused prudential concerns

(01:05:32) Theory of second best

(01:16:11) Conclusions

The original text contained 22 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.