Introduction

There are lots of good, neglected, and pretty concrete projects people could set up to make the transition to superintelligence go better. This document describes some that readers might not have thought much about before. They are ordered roughly by how excited we are about them.[1] Of these, Forethought is actively working on AI character evaluation and space governance, and we are very interested in automating macrostrategy.

Summary

AI character evaluation. Start an independent org to evaluate and stress-test AI character traits (epistemic integrity, prosociality, appropriate refusals), hold developers accountable against their own model specs / constitutions, and suggest and incentivise improvements to the specs.

Automated macrostrategy. Create evaluations and benchmarks, collect human-generated training data, and build scaffolds to improve AI competence at big-picture strategic and philosophical reasoning.

AI security assessment. Start an independent org that evaluates AI models for sabotage and backdoors, and makes recommendations about AI constitutions.

Enabling deals. Start an independent organisation to broker deals with potentially misaligned AI models in order to incentivise early schemers to disclose misalignment and cooperate with alignment efforts.

AI for improving collective epistemics. E.g. build an AI chief of staff that helps users act in [...]

---

Outline:

(00:14) Introduction

(00:43) Summary

(02:31) AI character evaluation

(04:40) Automated macrostrategy

(08:09) AI security evaluations

(09:11) Enabling deals with AIs

(11:14) Tools for collective epistemics

(13:11) Epistemic tools for strategic awareness

(14:26) Tools for coordination

(16:16) Space governance institute

(19:39) Coalition of concerned ML scientists

---

First published:

Source:

Linkpost URL:
https://www.forethought.org/research/concrete-projects-in-agi-preparedness

---

Narrated by TYPE III AUDIO.