The paper surveys existing methods for extending the context length of large language models and introduces a new truncation strategy. The authors test these methods on various evaluation tasks and find that linear scaling is the most effective method. They also release new long-context models and code for replication.

https://arxiv.org/abs//2308.10882

YouTube: https://www.youtube.com/@ArxivPapers

PODCASTS:

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers