The text introduces the **Retrieval Embedding Benchmark (RTEB)**, a new standard designed to accurately evaluate the **retrieval accuracy of embedding models** for real-world applications. The authors argue that existing benchmarks fail due to a **generalization gap** and misalignment with **modern enterprise AI applications**, often leading to inflated scores from models that are "teaching to the test." RTEB addresses this with a **hybrid strategy** using both transparent open datasets and impartial private datasets to measure true generalization. Emphasizing **multilingual and domain-specific enterprise use cases** like law and finance, RTEB aims to be a reliable, community-trusted standard, using **NDCG@10** as its primary evaluation metric.

Source:

https://huggingface.co/blog/rteb