Consider a future with many diverse AIs that need to coordinate with each other, or at least coexist without conflict. Such AIs would need shared values they can coordinate around. According to Hanson's theory, groups of diverse agents facing coordination pressure will tend to sacralize some shared value — seeing it in “far mode” so they can see it together. Unfortunately, this makes them systematically worse at making decisions about these things.

This model suggests three claims:

1. Helpfulness, harmlessness, and honesty (HHH) will be good candidates for sacralization.
2. The sacralisation of HHH would be bad.
3. We can avoid the sacralisation of these values.

I'm not confident any of these claims are true. They factor through three assumptions: (i) Hanson's model of human sociology is correct, (ii) the model applies equally well to future AIs, and (iii) instilling HHH values into AIs went somewhat well. Read this post as an exploration of the idea, not a confident prediction.

HHH values will be good candidates for sacralization

Hanson collects 62 correlates of things people treat as sacred (democracy, medicine [...]

---

Outline:

(01:27) HHH values will be good candidates for sacralization

(06:07) The sacralisation of HHH would be bad.

(08:25) We can avoid the sacralisation of HHH

(10:22) Appendix: Proposed Claude constitution

---

First published:

Source:

---

Narrated by TYPE III AUDIO.