Data Science at Home

Efficiently Retraining Language Models: How to Level Up Without Breaking the Bank (Ep. 227)


Listen Later

Get ready for an eye-opening episode! 🎙️

In our latest podcast episode, we dive deep into the world of LoRa (Low-Rank Adaptation) for large language models (LLMs). This groundbreaking technique is revolutionizing the way we approach language model training by leveraging low-rank approximations.

Join us as we unravel the mysteries of LoRa and discover how it enables us to retrain LLMs with minimal expenditure of money and resources. We'll explore the ingenious strategies and practical methods that empower you to fine-tune your language models without breaking the bank.

Whether you're a researcher, developer, or language model enthusiast, this episode is packed with invaluable insights. Learn how to unlock the potential of LLMs without draining your resources.

Tune in and join the conversation as we unravel the secrets of LoRa low-rank adaptation and show you how to retrain LLMs on a budget.

Listen to the full episode now on your favorite podcast platform! 🎧✨

 
References
  • LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685
  • Low-rank approximation https://en.wikipedia.org/wiki/Low-rank_approximation
  • Attention is all you need https://arxiv.org/pdf/1706.03762.pdf
  •  

    ...more
    View all episodesView all episodes
    Download on the App Store

    Data Science at HomeBy Francesco Gadaleta

    • 4.2
    • 4.2
    • 4.2
    • 4.2
    • 4.2

    4.2

    72 ratings


    More shows like Data Science at Home

    View all
    On Point with Meghna Chakrabarti by WBUR

    On Point with Meghna Chakrabarti

    4,022 Listeners

    Making Sense with Sam Harris by Sam Harris

    Making Sense with Sam Harris

    26,380 Listeners

    Nature Podcast by Springer Nature Limited

    Nature Podcast

    756 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    626 Listeners

    Science Vs by Spotify Studios

    Science Vs

    12,130 Listeners

    Science Friday by Science Friday and WNYC Studios

    Science Friday

    6,467 Listeners

    Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

    Super Data Science: ML & AI Podcast with Jon Krohn

    306 Listeners

    The Daily by The New York Times

    The Daily

    113,121 Listeners

    Up First from NPR by NPR

    Up First from NPR

    56,944 Listeners

    The Atlantic Interview by The Atlantic

    The Atlantic Interview

    14 Listeners

    Modern Wisdom by Chris Williamson

    Modern Wisdom

    4,025 Listeners

    The Peter Attia Drive by Peter Attia, MD

    The Peter Attia Drive

    8,043 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    212 Listeners

    Consider This from NPR by NPR

    Consider This from NPR

    6,462 Listeners

    The Ezra Klein Show by New York Times Opinion

    The Ezra Klein Show

    16,525 Listeners