Sign up to save your podcasts
Or
Share Arxiv Papers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Arxiv Papers
By Igor MelnykScience
Arxiv Papers episodes
- [QA] Beyond position: how rotary embeddings shape representations and memory in autoregressive transfomers
This paper explores how Rotary Positional Embeddings (RoPE) affect Transformer model dynamics, introducing phase shifts that influence embeddings, information retention, and attention through oscillatory behaviors and frequency components.
https://arxiv.org/abs//2410.18067
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
9 min - Beyond position: how rotary embeddings shape representations and memory in autoregressive transfomers
This paper explores how Rotary Positional Embeddings (RoPE) affect Transformer model dynamics, introducing phase shifts that influence embeddings, information retention, and attention through oscillatory behaviors and frequency components.
https://arxiv.org/abs//2410.18067
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
21 min - [QA] ALTA: Compiler-Based Analysis of Transformers
ALTA is a new programming language and compiler that maps programs to Transformer weights, enabling loop expression and improved algorithm representation, while providing tools for analyzing training challenges.
https://arxiv.org/abs//2410.18077
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
8 min - ALTA: Compiler-Based Analysis of Transformers
ALTA is a new programming language and compiler that maps programs to Transformer weights, enabling loop expression and improved algorithm representation, while providing tools for analyzing training challenges.
https://arxiv.org/abs//2410.18077
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
23 min - [QA] UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs
This paper introduces UNSTAR, a novel unlearning method for large language models using anti-samples to efficiently and selectively reverse learned associations, enhancing privacy and model modification capabilities.
https://arxiv.org/abs//2410.17050
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
9 min - UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMs
This paper introduces UNSTAR, a novel unlearning method for large language models using anti-samples to efficiently and selectively reverse learned associations, enhancing privacy and model modification capabilities.
https://arxiv.org/abs//2410.17050
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
17 min - [QA] Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing
This paper explores how Knowledge Editing algorithms can unintentionally distort model representations, leading to decreased factual recall and reasoning abilities, a phenomenon termed "representation shattering."
https://arxiv.org/abs//2410.17194
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
8 min - Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing
This paper explores how Knowledge Editing algorithms can unintentionally distort model representations, leading to decreased factual recall and reasoning abilities, a phenomenon termed "representation shattering."
https://arxiv.org/abs//2410.17194
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
19 min - [QA] Generative Reward Models
The paper proposes GenRM, a hybrid approach combining RLHF and RLAIF, improving synthetic preference labels' quality and outperforming existing models in both in-distribution and out-of-distribution tasks.
https://arxiv.org/abs//2410.12832
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
8 min - Generative Reward Models
The paper proposes GenRM, a hybrid approach combining RLHF and RLAIF, improving synthetic preference labels' quality and outperforming existing models in both in-distribution and out-of-distribution tasks.
https://arxiv.org/abs//2410.12832
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
13 min
About Arxiv Papers
From the publisher's feed
Running out of time to catch up with new arXiv papers? We take the most impactful papers and present them as convenient podcasts. If you're a visual learner, we offer these papers in an engaging video…
format. Our service fills the gap between overly brief paper summaries and time-consuming full paper reads. You gain academic insights in a time-efficient, digestible format. Code behind this work: https://github.com/imelnyk/ArxivPapers
More shows like Arxiv Papers
Exchanges
966 Listeners
Odd Lots
1,963 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
436 Listeners
The Daily
111,727 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
10,106 Listeners
Hard Fork
5,565 Listeners
UnHerd with Freddie Sayers
216 Listeners
Unsupervised Learning with Jacob Effron
51 Listeners
Latent Space: The AI Engineer Podcast
102 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
477 Listeners