This August 2025 paper from Google DeepMind, titled "On the Theoretical Limitations of Embedding-Based Retrieval," explores the fundamental constraints of vector embedding models in information retrieval. The authors demonstrate that the number of relevant document combinations an embedding can represent is inherently limited by its dimension. Through empirical "free embedding" experiments and the introduction of a new dataset called LIMIT, they show that even state-of-the-art models struggle with simple queries designed to stress these theoretical boundaries. The research concludes that for complex, instruction-following queries, alternative retrieval approaches like cross-encoders or multi-vector models may be necessary to overcome these inherent limitations.

Source:

https://arxiv.org/pdf/2508.21038