Sign up to save your podcasts
Or
Share Arxiv Papers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Arxiv Papers
By Igor MelnykScience
Arxiv Papers episodes
- Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention
Efficiently scale Transformer-based Large Language Models to infinitely long inputs with bounded memory using Infini-attention, demonstrated on long-context language modeling benchmarks.
https://arxiv.org/abs//2404.07143
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
11 min - [QA] LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
LLM2Vec transforms decoder-only LLMs into powerful text encoders through bidirectional attention, masked token prediction, and contrastive learning, achieving state-of-the-art performance on text embedding tasks.
https://arxiv.org/abs//2404.05961
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
13 min - [short] LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
LLM2Vec transforms decoder-only LLMs into powerful text encoders through bidirectional attention, masked token prediction, and contrastive learning, achieving state-of-the-art performance on text embedding tasks.
https://arxiv.org/abs//2404.05961
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
2 min - LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
LLM2Vec transforms decoder-only LLMs into powerful text encoders through bidirectional attention, masked token prediction, and contrastive learning, achieving state-of-the-art performance on text embedding tasks.
https://arxiv.org/abs//2404.05961
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
16 min - [QA] Simultaneous linear connectivity of neural networks modulo permutation
The paper explores permutation symmetries in neural networks, proposing three claims of increasing strength. Evidence suggests the possibility of strong linear connectivity, reducing loss barriers between trained networks.
https://arxiv.org/abs//2404.06498
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
14 min - [short] Simultaneous linear connectivity of neural networks modulo permutation
The paper explores permutation symmetries in neural networks, proposing three claims of increasing strength. Evidence suggests the possibility of strong linear connectivity, reducing loss barriers between trained networks.
https://arxiv.org/abs//2404.06498
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
3 min - Simultaneous linear connectivity of neural networks modulo permutation
The paper explores permutation symmetries in neural networks, proposing three claims of increasing strength. Evidence suggests the possibility of strong linear connectivity, reducing loss barriers between trained networks.
https://arxiv.org/abs//2404.06498
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
19 min - [QA] Does Transformer Interpretability Transfer to RNNs?
Recent advancements in RNN architectures like Mamba and RWKV rival transformers in language tasks. This paper explores adapting interpretability methods from transformers to enhance RNN performance.
https://arxiv.org/abs//2404.05971
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
12 min - [short] Does Transformer Interpretability Transfer to RNNs?
Recent advancements in RNN architectures like Mamba and RWKV rival transformers in language tasks. This paper explores adapting interpretability methods from transformers to enhance RNN performance.
https://arxiv.org/abs//2404.05971
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
2 min - Does Transformer Interpretability Transfer to RNNs?
Recent advancements in RNN architectures like Mamba and RWKV rival transformers in language tasks. This paper explores adapting interpretability methods from transformers to enhance RNN performance.
https://arxiv.org/abs//2404.05971
YouTube: https://www.youtube.com/@ArxivPapers
TikTok: https://www.tiktok.com/@arxiv_papers
Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016
Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers
8 min
About Arxiv Papers
From the publisher's feed
Running out of time to catch up with new arXiv papers? We take the most impactful papers and present them as convenient podcasts. If you're a visual learner, we offer these papers in an engaging video…
format. Our service fills the gap between overly brief paper summaries and time-consuming full paper reads. You gain academic insights in a time-efficient, digestible format. Code behind this work: https://github.com/imelnyk/ArxivPapers
More shows like Arxiv Papers
Exchanges
968 Listeners
Odd Lots
1,960 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
436 Listeners
The Daily
111,708 Listeners
All-In with Chamath, Jason, Sacks & Friedberg
10,107 Listeners
Hard Fork
5,564 Listeners
UnHerd with Freddie Sayers
216 Listeners
Unsupervised Learning with Jacob Effron
51 Listeners
Latent Space: The AI Engineer Podcast
101 Listeners
BG2Pod with Brad Gerstner and Bill Gurley
477 Listeners