Sign up to save your podcasts
Or
Share 903: LLM Benchmarks Are Lying to You (And What to Do Instead), with Sinan Ozdemir
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
903: LLM Benchmarks Are Lying to You (And What to Do Instead), with Sinan Ozdemir
Has AI benchmarking reached its limit, and what do we have to fill this gap? Sinan Ozdemir speaks to Jon Krohn about the lack of transparency in training data and the necessity of human-led quality assurance to detect AI hallucinations, when and why to be skeptical of AI benchmarks, and the future of benchmarking agentic and multimodal models.
Additional materials: www.superdatascience.com/903
This episode is brought to you by Trainium2, the latest AI chip from AWS, by Adverity, the conversational analytics platform and by the Dell AI Factory with NVIDIA.
Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.
In this episode you will learn:
View all episodes
By Jon Krohn
4.6
294294 ratings
Has AI benchmarking reached its limit, and what do we have to fill this gap? Sinan Ozdemir speaks to Jon Krohn about the lack of transparency in training data and the necessity of human-led quality assurance to detect AI hallucinations, when and why to be skeptical of AI benchmarks, and the future of benchmarking agentic and multimodal models.
Additional materials: www.superdatascience.com/903
This episode is brought to you by Trainium2, the latest AI chip from AWS, by Adverity, the conversational analytics platform and by the Dell AI Factory with NVIDIA.
Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.
In this episode you will learn:
More shows like Super Data Science: ML & AI Podcast with Jon Krohn
View all
Data Skeptic
478 Listeners
Talk Python To Me
588 Listeners
The AI in Business Podcast
169 Listeners
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
433 Listeners
NVIDIA AI Podcast
341 Listeners
Data Engineering Podcast
146 Listeners
Machine Learning Guide
768 Listeners
DataFramed
268 Listeners
Practical AI
211 Listeners
The Real Python Podcast
142 Listeners
Machine Learning Street Talk (MLST)
88 Listeners
No Priors: Artificial Intelligence | Technology | Startups
133 Listeners
AI Chat: ChatGPT & AI News, Artificial Intelligence, OpenAI, Machine Learning
150 Listeners
This Day in AI Podcast
209 Listeners
The AI Daily Brief: Artificial Intelligence News and Analysis
557 Listeners