Sign up to save your podcasts
Or
Share Episode 55: From Frittatas to Production LLMs: Breakfast at SciPy
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Episode 55: From Frittatas to Production LLMs: Breakfast at SciPy
Traditional software expects 100% passing tests. In LLM-powered systems, that’s not just unrealistic — it’s a feature, not a bug. Eric Ma leads research data science in Moderna’s data science and AI group, and over breakfast at SciPy we explored why AI products break the old rules, what skills different personas bring (and miss), and how to keep systems alive after the launch hype fades.
You’ll hear the clink of coffee cups, the murmur of SciPy in the background, and the occasional bite of frittata as we talk (hopefully also a feature, not a bug!)
We talk through:
• The three personas — and the blind spots each has when shipping AI systems
• Why “perfect” tests can be a sign you’re testing the wrong thing
• Development vs. production observability loops — and why you need both
• How curiosity about failing data separates good builders from great ones
• Ways large organizations can create space for experimentation without losing delivery focus
If you want to build AI products that thrive in the messy real world, this episode will help you embrace the chaos — and make it work for you.
LINKS
🎓 Learn more:
View all episodes
By Hugo Bowne-Anderson
5
1111 ratings
Traditional software expects 100% passing tests. In LLM-powered systems, that’s not just unrealistic — it’s a feature, not a bug. Eric Ma leads research data science in Moderna’s data science and AI group, and over breakfast at SciPy we explored why AI products break the old rules, what skills different personas bring (and miss), and how to keep systems alive after the launch hype fades.
You’ll hear the clink of coffee cups, the murmur of SciPy in the background, and the occasional bite of frittata as we talk (hopefully also a feature, not a bug!)
We talk through:
• The three personas — and the blind spots each has when shipping AI systems
• Why “perfect” tests can be a sign you’re testing the wrong thing
• Development vs. production observability loops — and why you need both
• How curiosity about failing data separates good builders from great ones
• Ways large organizations can create space for experimentation without losing delivery focus
If you want to build AI products that thrive in the messy real world, this episode will help you embrace the chaos — and make it work for you.
LINKS
🎓 Learn more:
More shows like Vanishing Gradients
View all
Data Skeptic
477 Listeners
a16z Podcast
1,083 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
434 Listeners
Super Data Science: ML & AI Podcast with Jon Krohn
301 Listeners
NVIDIA AI Podcast
341 Listeners
DataFramed
268 Listeners
Practical AI
210 Listeners
Google DeepMind: The Podcast
194 Listeners
Machine Learning Street Talk (MLST)
89 Listeners
Dwarkesh Podcast
489 Listeners
No Priors: Artificial Intelligence | Technology | Startups
133 Listeners
Latent Space: The AI Engineer Podcast
97 Listeners
AI + a16z
33 Listeners
High Signal: Data Science | Career | AI
18 Listeners
OpenAI Podcast
52 Listeners