The paper examines the data curation process for large language models and investigates how different filters affect webpages based on social and geographic dimensions. The study highlights implicit preferences in data curation and calls for further research on the social implications of pretraining data curation.

https://arxiv.org/abs//2401.06408

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers