Some thoughts on open-source AI (copied over from a recent twitter thread):

1. We should have a strong prior favoring open source. It's been a huge success driving tech progress over many decades. We forget how counterintuitive it was originally, and shouldn’t take it for granted.

2. Open source has also been very valuable for alignment. It's key to progress on interpretability, as outlined here.

3. I am concerned, however, that offense will heavily outpace defense in the long term. As AI accelerates science, many new WMDs will emerge. Even if defense of infrastructure keeps up with offense, human bodies are a roughly fixed and very vulnerable attack surface.

4. A central concern about open source AI: it’ll allow terrorists to build bioweapons. This shouldn’t be dismissed, but IMO it's easy to be disproportionately scared of terrorism. More central risks are eg “North Korea becomes [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.