Data Science at Home

Efficiently Retraining Language Models: How to Level Up Without Breaking the Bank (Ep. 227)


Listen Later

Get ready for an eye-opening episode! 🎙️

In our latest podcast episode, we dive deep into the world of LoRa (Low-Rank Adaptation) for large language models (LLMs). This groundbreaking technique is revolutionizing the way we approach language model training by leveraging low-rank approximations.

Join us as we unravel the mysteries of LoRa and discover how it enables us to retrain LLMs with minimal expenditure of money and resources. We'll explore the ingenious strategies and practical methods that empower you to fine-tune your language models without breaking the bank.

Whether you're a researcher, developer, or language model enthusiast, this episode is packed with invaluable insights. Learn how to unlock the potential of LLMs without draining your resources.

Tune in and join the conversation as we unravel the secrets of LoRa low-rank adaptation and show you how to retrain LLMs on a budget.

Listen to the full episode now on your favorite podcast platform! 🎧✨

 
References
  • LoRA: Low-Rank Adaptation of Large Language Models https://arxiv.org/abs/2106.09685
  • Low-rank approximation https://en.wikipedia.org/wiki/Low-rank_approximation
  • Attention is all you need https://arxiv.org/pdf/1706.03762.pdf
  •  

    ...more
    View all episodesView all episodes
    Download on the App Store

    Data Science at HomeBy Francesco Gadaleta

    • 4.2
    • 4.2
    • 4.2
    • 4.2
    • 4.2

    4.2

    72 ratings


    More shows like Data Science at Home

    View all
    Freakonomics Radio by Freakonomics Radio + Stitcher

    Freakonomics Radio

    31,960 Listeners

    Global News Podcast by BBC World Service

    Global News Podcast

    7,585 Listeners

    WSJ Your Money Briefing by The Wall Street Journal

    WSJ Your Money Briefing

    1,704 Listeners

    The a16z Show by Andreessen Horowitz

    The a16z Show

    1,091 Listeners

    Software Engineering Daily by Software Engineering Daily

    Software Engineering Daily

    622 Listeners

    Talk Python To Me by Michael Kennedy

    Talk Python To Me

    584 Listeners

    Science Magazine Podcast by Science Magazine

    Science Magazine Podcast

    823 Listeners

    Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

    Super Data Science: ML & AI Podcast with Jon Krohn

    301 Listeners

    FT Tech Tonic by Financial Times

    FT Tech Tonic

    99 Listeners

    Worklife with Adam Grant by TED

    Worklife with Adam Grant

    9,157 Listeners

    Practical AI by Practical AI LLC

    Practical AI

    206 Listeners

    Last Week in AI by Skynet Today

    Last Week in AI

    306 Listeners

    Hard Fork by The New York Times

    Hard Fork

    5,506 Listeners

    This Day in AI Podcast by Michael Sharkey, Chris Sharkey

    This Day in AI Podcast

    228 Listeners

    The Last Invention by Longview

    The Last Invention

    1,103 Listeners