Super Data Science: ML & AI Podcast with Jon Krohn

689: Observing LLMs in Production to Automatically Catch Issues


Listen Later

Arize's Amber Roberts and Xander Song join Jon Krohn this week, sharing invaluable insights into ML Observability, drift detection, retraining strategies, and the crucial task of ensuring fairness and ethical considerations in AI development.

This episode is brought to you by Posit, the open-source data science company, by AWS Inferentia, and by Anaconda, the world's most popular Python distribution. Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.

In this episode you will learn:
• What is ML Observability [05:07]
• What is Drift [08:18]
• The different kinds of model drift [15:31]
• How frequently production models should be retrained? [25:15]
• Arize's open-source product, Phoenix [30:49]
• How ML Observability relates to discovering model biases [50:30]
• Arize case studies [57:13]
• What is a developer advocate [1:04:51]

Additional materials: www.superdatascience.com/689

...more
View all episodesView all episodes
Download on the App Store

Super Data Science: ML & AI Podcast with Jon KrohnBy Jon Krohn

  • 4.6
  • 4.6
  • 4.6
  • 4.6
  • 4.6

4.6

295 ratings


More shows like Super Data Science: ML & AI Podcast with Jon Krohn

View all
Data Skeptic by Kyle Polich

Data Skeptic

480 Listeners

Software Engineering Daily by Software Engineering Daily

Software Engineering Daily

623 Listeners

Talk Python To Me by Michael Kennedy

Talk Python To Me

585 Listeners

NVIDIA AI Podcast by NVIDIA

NVIDIA AI Podcast

334 Listeners

AI Today Podcast by AI & Data Today

AI Today Podcast

152 Listeners

DataFramed by DataCamp

DataFramed

269 Listeners

Practical AI by Practical AI LLC

Practical AI

207 Listeners

The Real Python Podcast by Real Python

The Real Python Podcast

142 Listeners

Machine Learning Street Talk (MLST) by Machine Learning Street Talk (MLST)

Machine Learning Street Talk (MLST)

95 Listeners

No Priors: Artificial Intelligence | Technology | Startups by Conviction

No Priors: Artificial Intelligence | Technology | Startups

131 Listeners

AI Chat: ChatGPT, AI News, Artificial Intelligence, OpenAI, Machine Learning by Jaeden Schafer

AI Chat: ChatGPT, AI News, Artificial Intelligence, OpenAI, Machine Learning

154 Listeners

This Day in AI Podcast by Michael Sharkey, Chris Sharkey

This Day in AI Podcast

227 Listeners

The AI Daily Brief: Artificial Intelligence News and Analysis by Nathaniel Whittemore

The AI Daily Brief: Artificial Intelligence News and Analysis

608 Listeners

AI For Humans: Making Artificial Intelligence Fun & Practical by Kevin Pereira & Gavin Purcell

AI For Humans: Making Artificial Intelligence Fun & Practical

275 Listeners

Training Data by Sequoia Capital

Training Data

40 Listeners