Neural Search Talks — Zeta Alpha

Zeta-Alpha-E5-Mistral: Finetuning LLMs for Retrieval (with Arthur Câmara)

Listen Later

In the 30th episode of Neural Search Talks, we have our very own Arthur Câmara, Senior Research Engineer at Zeta Alpha, presenting a 20-minute guide on how we fine-tune Large Language Models for effective text retrieval. Arthur discusses the common issues with embedding models in a general-purpose RAG pipeline, how to tackle the lack of retrieval-oriented data for fine-tuning with InPars, and how we adapted E5-Mistral to rank in the top 10 on the BEIR benchmark.


## Sources

InPars

  • https://github.com/zetaalphavector/InPars
  • https://dl.acm.org/doi/10.1145/3477495.3531863
  • https://arxiv.org/abs/2301.01820
  • https://arxiv.org/abs/2307.04601


    • Zeta-Alpha-E5-Mistral

    • https://zeta-alpha.com/post/fine-tuning-an-llm-for-state-of-the-art-retrieval-zeta-alpha-s-top-10-submission-to-the-the-mteb-be
    • https://huggingface.co/zeta-alpha-ai/Zeta-Alpha-E5-Mistral

      • NanoBEIR

      • https://huggingface.co/collections/zeta-alpha-ai/nanobeir-66e1a0af21dfd93e620cd9f6
        • ...more
        View all episodesView all episodes
        Download on the App Store
        Neural Search Talks — Zeta AlphaBy Zeta Alpha