Crosspost from my blog. PDF version. berkeleygenomics.org.

Crosspost from my blog. PDF version. berkeleygenomics.org.

Context

Previously, in "HIA and X-risk part 1: Why it helps", I laid out the reasons I think human intelligence amplification would decrease existential risk from AGI. Here I'll give all the reasons I can currently think of that HIA might plausibly increase AGI X-risk.

Questions for the reader

Caveats

The world is very complicated and chaotic and I can't plausibly predict even important questions like "what actual effect would such and such have". I can't even plausibly resolve much uncertainty, and the world is full of agents who will adaptively do surprising things. So the actual search procedure is something like: What is a way, or reason to think, that HIA might increase [...]

---

Outline:

(00:26) Context

(00:46) Questions for the reader

(01:15) Caveats

(02:31) What is HIA?

(03:18) Vague definitions of intelligence and HIA

(05:37) HIA as a general access good

(07:39) HIA and reprogenetics

(08:21) AGI X-risk

(08:24) Background assumptions

(09:29) Red vs. Blue AGI capabilities research

(11:03) An ontology of effects of interventions on world processes

(12:21) The meaning of acceleration

(13:58) Effects of HIA on a single process

(17:24) Effects of HIA involving multiple processes

(19:00) Processes

(20:20) Some plausible bad effects of HIA on processes

(20:34) Speeding up Blue research

(22:27) Speeding up Red research

(24:56) Less speeding up legal and social regulation

(26:21) Nonlinear / race-condition regulatory escape

(26:58) Alignment loses the race anyway

(27:37) Intrinsic regulatory escape

(28:34) Disrupting regulatory systems

(31:02) Social values favor following local incentives

(32:44) Less speeding up change towards better values

(34:03) Alignment harnesses added brainpower much less effectively than capabilities research does

(39:32) HIA people may tend to be transgressive

(41:59) Other arguments

(42:08) Concentration of power

(43:06) HIA is unpredictable and therefore risky

(43:35) More capable but not wiser

(44:38) Acknowledgements

---

First published:

Source:

---

Narrated by TYPE III AUDIO.