If we get AI safety research wrong, we may not get a second chance. But despite the stakes being so high, there has been no effort to systematically review and verify empirical AI safety papers. I would like to change that.

Today I sent in funding applications to found a team of researchers dedicated to replicating AI safety work. But what exactly should we aim to accomplish? What should AI safety replications even look like? After 1-2 months of consideration and 50+ hours of conversation, this document outlines principles that will guide our future team.

I. Meta-science doesn’t vindicate anyone

Researchers appear to agree that some share of AI safety work is low-quality, false, or misleading. However, everyone seems to disagree on which share of papers are the problematic ones.

When I expressed interest in starting a group that does AI safety replications, I suspect some assumed I would be “exposing” the papers that they don’t approve of. This is a trap and it is especially important for us, as the replicators, not to fall into it. If our replications tend to confirm our beliefs, that probably says more about our priors than the papers we are studying.

[...]

---

Outline:

(00:48) I. Meta-science doesn't vindicate anyone

(01:27) II. Searching for bad papers is like searching for haunted houses

(02:22) III. Research doesn't regulate itself

(03:29) IV. Replications are more than repeating the experiments

(04:23) V. The replication is just as dubious as the paper itself

(05:11) VI. Why not do this in a more decentralized way?

(06:07) VII. We are all adults here

(06:46) VIII. Feedback is everything

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.