Super Data Science: ML & AI Podcast with Jon Krohn

689: Observing LLMs in Production to Automatically Catch Issues


Listen Later

Arize's Amber Roberts and Xander Song join Jon Krohn this week, sharing invaluable insights into ML Observability, drift detection, retraining strategies, and the crucial task of ensuring fairness and ethical considerations in AI development.

This episode is brought to you by Posit, the open-source data science company, by AWS Inferentia, and by Anaconda, the world's most popular Python distribution. Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.

In this episode you will learn:
• What is ML Observability [05:07]
• What is Drift [08:18]
• The different kinds of model drift [15:31]
• How frequently production models should be retrained? [25:15]
• Arize's open-source product, Phoenix [30:49]
• How ML Observability relates to discovering model biases [50:30]
• Arize case studies [57:13]
• What is a developer advocate [1:04:51]

Additional materials: www.superdatascience.com/689

...more
View all episodesView all episodes
Download on the App Store

Super Data Science: ML & AI Podcast with Jon KrohnBy Jon Krohn

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

285 ratings


More shows like Super Data Science: ML & AI Podcast with Jon Krohn

View all
Data Skeptic by Kyle Polich

Data Skeptic

470 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

586 Listeners

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

435 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

324 Listeners

Data Engineering Podcast by Tobias Macey

Data Engineering Podcast

140 Listeners

AI Today Podcast: Artificial Intelligence Insights, Experts, and Opinion by AI & Data Today

AI Today Podcast: Artificial Intelligence Insights, Experts, and Opinion

143 Listeners

DataFramed by DataCamp

DataFramed

269 Listeners

Practical AI by Practical AI LLC

Practical AI

190 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

136 Listeners

Last Week in AI by Skynet Today

Last Week in AI

282 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

87 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

189 Listeners

Latent Space: The AI Engineer Podcast by swyx + Alessio

Latent Space: The AI Engineer Podcast

63 Listeners

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief (Formerly The AI Breakdown): Artificial Intelligence News and Analysis

421 Listeners

AI For Humans: Making Artificial Intelligence Fun & Practical by Kevin Pereira & Gavin Purcell

AI For Humans: Making Artificial Intelligence Fun & Practical

233 Listeners