A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens.<a href="https://www.microsoft.com/en-us/research/publication/longrope-extending-llm-context-window-beyond-2-million-tokens/">Read the paper</a><a href="https://github.com/microsoft/LongRoPE">Get the code</a>

A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens. Read the paper Get the code

A lack of appropriate data, decreased model performance, and other obstacles have made it difficult to expand the input language models can receive. Li Lyna Zhang introduces LongRoPE, a method capable of extending content windows to more than 2 million tokens.<a href="https://www.microsoft.com/en-us/research/publication/longrope-extending-llm-context-window-beyond-2-million-tokens/" rel="noopener noreferrer">Read the paper</a><a href="https://github.com/microsoft/LongRoPE" rel="noopener noreferrer">Get the code</a>

Abstracts: July 29, 2024

An ongoing series of conversations bringing you right up to the cutting edge of Microsoft Research.

Share Abstracts: July 29, 2024

Sign up to save your podcasts

Abstracts: July 29, 2024

Abstracts: July 29, 2024

More shows like Microsoft Research Podcast

The Daily

Dwarkesh Podcast

Hard Fork

No Priors: Artificial Intelligence | Technology | Startups