Reward models are important for aligning language models with human preferences, but they can be exploited by the model. Training an ensemble of reward models can mitigate this issue, but it does not completely eliminate reward hacking.

https://arxiv.org/abs//2312.09244

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers