Sign up to save your podcasts
Or
Share Weights and Biases on Fine-Tuning LLMs - Weaviate Podcast #68!
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Weights and Biases on Fine-Tuning LLMs - Weaviate Podcast #68!
Hey everyone! Thank you so much for watching the 68th episode of the Weaviate Podcast! We are super excited to welcome Morgan McGuire, Darek Kleczek, and Thomas Capelle! This was such a fun discussion beginning with generally how see the space of fine-tuning from why you would want to do it, to the available tooling, intersection with RAG and more!
Check out W&B Prompts! https://wandb.ai/site/prompts
Check out the W&B Tiny Llama Report! https://wandb.ai/capecape/llamac/reports/Training-Tiny-Llamas-for-Fun-and-Science--Vmlldzo1MDM2MDg0
Chapters
0:00 Tiny Llamas!
1:53 Welcome!
2:22 LLM Fine-Tuning
5:25 Tooling for Fine-Tuning
7:55 Why Fine-Tune?
9:55 RAG vs. Fine-Tuning
12:25 Knowledge Distillation
14:40 Gorilla LLMs
18:25 Open-Source LLMs
22:48 Jonathan Frankle on W&B
23:45 Data Quality for LLM Training
25:55 W&B for Data Versioning
27:25 Curriculum Learning
29:28 GPU Rich and Data Quality
30:30 Vector DBs and Data Quality
32:50 Tuning Training with Weights & Biases
35:47 Training Reports
42:28 HF Collections and W&B Sweeps
44:50 Exciting Directions for AI
View all episodes
By Weaviate
4
44 ratings
Hey everyone! Thank you so much for watching the 68th episode of the Weaviate Podcast! We are super excited to welcome Morgan McGuire, Darek Kleczek, and Thomas Capelle! This was such a fun discussion beginning with generally how see the space of fine-tuning from why you would want to do it, to the available tooling, intersection with RAG and more!
Check out W&B Prompts! https://wandb.ai/site/prompts
Check out the W&B Tiny Llama Report! https://wandb.ai/capecape/llamac/reports/Training-Tiny-Llamas-for-Fun-and-Science--Vmlldzo1MDM2MDg0
Chapters
0:00 Tiny Llamas!
1:53 Welcome!
2:22 LLM Fine-Tuning
5:25 Tooling for Fine-Tuning
7:55 Why Fine-Tune?
9:55 RAG vs. Fine-Tuning
12:25 Knowledge Distillation
14:40 Gorilla LLMs
18:25 Open-Source LLMs
22:48 Jonathan Frankle on W&B
23:45 Data Quality for LLM Training
25:55 W&B for Data Versioning
27:25 Curriculum Learning
29:28 GPU Rich and Data Quality
30:30 Vector DBs and Data Quality
32:50 Tuning Training with Weights & Biases
35:47 Training Reports
42:28 HF Collections and W&B Sweeps
44:50 Exciting Directions for AI
More shows like Weaviate Podcast
View all
Practical AI
204 Listeners
AWS Podcast
205 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,968 Listeners
Dwarkesh Podcast
517 Listeners
No Priors: Artificial Intelligence | Technology | Startups
130 Listeners
Latent Space: The AI Engineer Podcast
92 Listeners
Interconnects
9 Listeners
OpenAI Podcast
52 Listeners