On October 20, 2025 Hugging Face released MTEB v2, a significant refactoring of the Massive Text Embedding Benchmark, which was originally designed for evaluating text embedding models across various tasks like classification and retrieval. The update addresses package bloating and the need for broader support by introducing a more consistent interface, better typing, and improved documentation. Key new features include support for multimodal evaluation (text, images, and audio), unified retrieval and reranking tasks, and an easier evaluation process using the new `mteb.evaluate` function and `ResultCache` for managing results. The article also provides detailed instructions for upgrading from MTEB v1, including how to convert old models and datasets to the new v2 format. Source: https://huggingface.co/blog/isaacchung/mteb-v2