Sign up to save your podcasts
Or
Share Episode 52: Why Most LLM Products Break at Retrieval (And How to Fix Them)
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Episode 52: Why Most LLM Products Break at Retrieval (And How to Fix Them)
Most LLM-powered features do not break at the model. They break at the context. So how do you retrieve the right information to get useful results, even under vague or messy user queries?
In this episode, we hear from Eric Ma, who leads data science research in the Data Science and AI group at Moderna. He shares what it takes to move beyond toy demos and ship LLM features that actually help people do their jobs.
We cover:
• How to align retrieval with user intent and why cosine similarity is not the answer
• How a dumb YAML-based system outperformed so-called smart retrieval pipelines
• Why vague queries like “what is this all about” expose real weaknesses in most systems
• When vibe checks are enough and when formal evaluation is worth the effort
• How retrieval workflows can evolve alongside your product and user needs
If you are building LLM-powered systems and care about how they work, not just whether they work, this one is for you.
LINKS
🎓 Learn more:
📺 Watch the video version on YouTube: YouTube link
View all episodes
By Hugo Bowne-Anderson
5
1111 ratings
Most LLM-powered features do not break at the model. They break at the context. So how do you retrieve the right information to get useful results, even under vague or messy user queries?
In this episode, we hear from Eric Ma, who leads data science research in the Data Science and AI group at Moderna. He shares what it takes to move beyond toy demos and ship LLM features that actually help people do their jobs.
We cover:
• How to align retrieval with user intent and why cosine similarity is not the answer
• How a dumb YAML-based system outperformed so-called smart retrieval pipelines
• Why vague queries like “what is this all about” expose real weaknesses in most systems
• When vibe checks are enough and when formal evaluation is worth the effort
• How retrieval workflows can evolve alongside your product and user needs
If you are building LLM-powered systems and care about how they work, not just whether they work, this one is for you.
LINKS
🎓 Learn more:
📺 Watch the video version on YouTube: YouTube link
More shows like Vanishing Gradients
View all
a16z Podcast
1,032 Listeners
Data Skeptic
480 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
441 Listeners
Super Data Science: ML & AI Podcast with Jon Krohn
298 Listeners
NVIDIA AI Podcast
322 Listeners
DataFramed
267 Listeners
Practical AI
192 Listeners
Google DeepMind: The Podcast
198 Listeners
Machine Learning Street Talk (MLST)
88 Listeners
Dwarkesh Podcast
408 Listeners
No Priors: Artificial Intelligence | Technology | Startups
121 Listeners
Latent Space: The AI Engineer Podcast
75 Listeners
AI + a16z
31 Listeners
High Signal: Data Science | Career | AI
4 Listeners
OpenAI Podcast
28 Listeners