Last week, Thinking Machines announced Tinker. It's an API for running fine-tuning and inference on open-source LLMs that works in a unique way. I think it has some immediate practical implications for AI safety research: I suspect that it will make RL experiments substantially easier, and increase the number of safety papers that involve RL on big models.

But it's also interesting to me for another reason: the design of this API makes it possible to do many types of ML research without direct access to the model you’re working with. APIs like this might allow AI companies to reduce how many of their researchers (either human or AI) have access to sensitive model weights, which is good for reducing the probability of weight exfiltration and other rogue deployments.

(Thinking Machines gave us early access to the product; in exchange, we gave them bug reports and let them mention [...]

---

Outline:

(01:23) How the Tinker API is different

(03:20) Why this is good for AI security and control

(07:19) A lot of ML research can be done without direct access to dangerous model weights

(10:19) Conclusions

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.