Sign up to save your podcasts
Or
Share Ep19. Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Ep19. Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
This research explores the challenges and opportunities of using long-context Large Language Models (LLMs) in Retrieval-Augmented Generation (RAG) systems. The authors find that while increasing the number of retrieved passages initially improves performance, it can lead to a decline due to the detrimental impact of irrelevant passages, known as "hard negatives." To overcome this challenge, the paper proposes three solutions: training-free retrieval reordering, RAG-specific implicit LLM fine-tuning, and RAG-oriented LLM fine-tuning with intermediate reasoning. The paper concludes with a systematic analysis of the training-based methods, examining the effects of data distribution, retriever for training, and training context length.
View all episodes
By The Daily MLThis research explores the challenges and opportunities of using long-context Large Language Models (LLMs) in Retrieval-Augmented Generation (RAG) systems. The authors find that while increasing the number of retrieved passages initially improves performance, it can lead to a decline due to the detrimental impact of irrelevant passages, known as "hard negatives." To overcome this challenge, the paper proposes three solutions: training-free retrieval reordering, RAG-specific implicit LLM fine-tuning, and RAG-oriented LLM fine-tuning with intermediate reasoning. The paper concludes with a systematic analysis of the training-based methods, examining the effects of data distribution, retriever for training, and training context length.