Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA). Additional materials:<a href="https://www.superdatascience.com/672"> </a><a href="https://www.superdatascience.com/674">www.superdatascience.com/674</a> Interested in sponsoring a SuperDataScience Podcast episode? Visit <a href="https://www.jonkrohn.com/podcast">JonKrohn.com/podcast</a> for sponsorship information.

Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA). Additional materials: www.superdatascience.com/674 Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.

Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, Jon explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA). Additional materials:<a href="https://www.superdatascience.com/672" rel="noopener noreferrer"> </a><a href="https://www.superdatascience.com/674" rel="noopener noreferrer">www.superdatascience.com/674</a> Interested in sponsoring a SuperDataScience Podcast episode? Visit <a href="https://www.jonkrohn.com/podcast" rel="noopener noreferrer">JonKrohn.com/podcast</a> for sponsorship information.

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

The latest machine learning, A.I., and data career topics from across both academia and industry are brought to you by host Dr. Jon Krohn on the Super Data Science Podcast. As the quantity of data on our planet doubles every couple of years and with this trend set to continue for decades to come, there's an unprecedented opportunity for you to make a meaningful impact in your lifetime. In conversation with the biggest names in the data science industry, Jon cuts through hype to fuel that professional impact.

Whether you're curious about getting started in a data career or you're a deep technical expert, whether you'd like to understand what A.I. is or you'd like to integrate more data-driven processes into your business, we have inspiring guests and lighthearted conversation for you to enjoy.

We cover tools, techniques, and implementation tricks across data collection, databases, analytics, predictive modeling, visualization, software engineering, real-world applications, commercialization, and entrepreneurship − everything you need to crush it with data science.

Science

Technology

The latest machine learning, A.I., and data career topics from across both academia and industry are brought to you by host Dr. Jon Krohn on the Super Data Science Podcast. As the quantity of data on our planet doubles every couple of years and with this trend set to continue for decades to come, there's an unprecedented opportunity for you to make a meaningful impact in your lifetime. In conversation with the biggest names in the data science industry, Jon cuts through hype to fuel that professional impact. Whether you're curious about getting started in a data career or you're a deep technical expert, whether you'd like to understand what A.I. is or you'd like to integrate more data-driven processes into your business, we have inspiring guests and lighthearted conversation for you to enjoy. We cover tools, techniques, and implementation tricks across data collection, databases, analytics, predictive modeling, visualization, software engineering, real-world applications, commercialization, and entrepreneurship − everything you need to crush it with data science.

Share 674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

Sign up to save your podcasts

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation)

More shows like Super Data Science: ML & AI Podcast with Jon Krohn

Data Skeptic

a16z Podcast

NVIDIA AI Podcast

Machine Learning Guide

AI Today Podcast

DataFramed

Practical AI

The Real Python Podcast

Machine Learning Street Talk (MLST)

No Priors: Artificial Intelligence | Technology | Startups

AI Chat: ChatGPT & AI News, Artificial Intelligence, OpenAI, Machine Learning

This Day in AI Podcast

The AI Daily Brief: Artificial Intelligence News and Analysis

AI For Humans: Making Artificial Intelligence Fun Practical

AI Hustle: Make Money from AI and ChatGPT, Midjourney, NVIDIA, Anthropic, OpenAI