Executive Summary

• Refusing harmful requests is not a novel behavior learned in chat fine-tuning, as pre-trained base models will also refuse requests (48% of all harmful requests, 3% of harmless) just at a lower rate than chat models (90% harmful, 3% harmless)
• Further, for both Qwen 1.5 0.5B and Gemma 2 9B, chat fine-tuning reinforces the existing mechanisms. In both the chat and base models it is mediated by the refusal direction described in Arditi et al.
  • We can both induce and bypass refusal in a pre-trained model, using a steering vector transferred from the chat model's activations
  • On the contrary, in LLaMA 1 7B (which was trained on data from before November 2022 and so can't have had ChatGPT outputs in the pre-training data), we find evidence that chat fine-tuning learns additional / different refusal representations and mechanisms.
• We open source our code at https://github.com/ckkissane/base-models-refuse

---

Outline:

(00:06) Executive Summary

(01:25) Introduction

(03:23) Background and methodology

(06:30) Results

(06:33) Base models refuse harmful requests

(08:21) Eliciting more base model refusals with steering vectors

(11:00) Bypassing refusal in base models

(12:22) Investigating (pre-ChatGPT model) LLaMA 1 7B

(16:32) Related work

(17:44) Conclusion

(19:50) Citing this work

(20:04) Author contributions statement

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.