Sign up to save your podcasts
Or
Share The Unreasonable Ineffectiveness of the Deeper Layers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
The Unreasonable Ineffectiveness of the Deeper Layers
Study explores layer pruning in pretrained LLMs, finding minimal performance drop until half of layers removed. Optimal block identified for pruning, followed by finetuning with PEFT methods for efficiency and improved inference.
https://arxiv.org/abs//2403.17887
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
View all episodes
By Igor Melnyk
5
33 ratings
Study explores layer pruning in pretrained LLMs, finding minimal performance drop until half of layers removed. Optimal block identified for pruning, followed by finetuning with PEFT methods for efficiency and improved inference.
https://arxiv.org/abs//2403.17887
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
More shows like Arxiv Papers
View all
Exchanges
956 Listeners
Odd Lots
1,943 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
438 Listeners
The Daily
111,863 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
9,997 Listeners
Hard Fork
5,522 Listeners
UnHerd with Freddie Sayers
212 Listeners
Unsupervised Learning with Jacob Effron
51 Listeners
Latent Space: The AI Engineer Podcast
92 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
473 Listeners