Data Science at Home

Efficiently Retraining Language Models: How to Level Up Without Breaking the Bank (Ep. 227)


Listen Later

Get ready for an eye-opening episode! 🎙️

In our latest podcast episode, we dive deep into the world of LoRa (Low-Rank Adaptation) for large language models (LLMs). This groundbreaking technique is revolutionizing the way we approach language model training by leveraging low-rank approximations.

Join us as we unravel the mysteries of LoRa and discover how it enables us to retrain LLMs with minimal expenditure of money and resources. We'll explore the ingenious strategies and practical methods that empower you to fine-tune your language models without breaking the bank.

Whether you're a researcher, developer, or language model enthusiast, this episode is packed with invaluable insights. Learn how to unlock the potential of LLMs without draining your resources.

Tune in and join the conversation as we unravel the secrets of LoRa low-rank adaptation and show you how to retrain LLMs on a budget.

Listen to the full episode now on your favorite podcast platform! 🎧✨

 
References
  • LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685
  • Low-rank approximation https://en.wikipedia.org/wiki/Low-rank_approximation
  • Attention is all you need https://arxiv.org/pdf/1706.03762.pdf
  •  

    ...more
    View all episodesView all episodes
    Download on the App Store

    Data Science at HomeBy Francesco Gadaleta

    • 4.2
    • 4.2
    • 4.2
    • 4.2
    • 4.2

    4.2

    71 ratings


    More shows like Data Science at Home

    View all
    This Week in Startups by Jason Calacanis

    This Week in Startups

    1,284 Listeners

    The McKinsey Podcast by McKinsey & Company

    The McKinsey Podcast

    388 Listeners

    Data Skeptic by Kyle Polich

    Data Skeptic

    476 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    625 Listeners

    HBR IdeaCast by Harvard Business Review

    HBR IdeaCast

    1,825 Listeners

    Soft Skills Engineering by Jamison Dance and Dave Smith

    Soft Skills Engineering

    285 Listeners

    Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

    Super Data Science: ML & AI Podcast with Jon Krohn

    302 Listeners

    NVIDIA AI Podcast by NVIDIA

    NVIDIA AI Podcast

    339 Listeners

    Data Engineering Podcast by Tobias Macey

    Data Engineering Podcast

    145 Listeners

    DataFramed by DataCamp

    DataFramed

    268 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    212 Listeners

    Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

    Machine Learning Street Talk (MLST)

    89 Listeners

    Latent Space: The AI Engineer Podcast by swyx + Alessio

    Latent Space: The AI Engineer Podcast

    96 Listeners

    This Day in AI Podcast by Michael Sharkey, Chris Sharkey

    This Day in AI Podcast

    209 Listeners

    The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

    The AI Daily Brief: Artificial Intelligence News and Analysis

    558 Listeners