Artificial Discourse

CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation Generation


Listen Later

CORAL, a novel benchmark dataset for evaluating Retrieval-Augmented Generation (RAG) systems in a multi-turn conversational setting. The authors highlight the limitations of existing datasets in assessing conversational RAG and detail CORAL's unique features, including open-domain coverage, knowledge intensity, free-form responses, topic shifts, and citation labeling. They explain how CORAL is derived from Wikipedia, automatically converting its content into conversational formats, and outline the three core tasks it supports: conversational passage retrieval, response generation, and citation labeling. The authors present a unified framework for evaluating conversational RAG methods and report on experiments conducted on CORAL, showcasing the performance of different conversational search and generation models.

...more
View all episodesView all episodes
Download on the App Store

Artificial DiscourseBy Kenpachi