// Abstract

Demetrios explores common themes in ML model testing with insights from Erica Greene (Yahoo News), Matar Haller (ActiveFence), Mohamed Elgendy (Kolena), and Catherine Nelson (Freelance Data Scientist). They discuss tiered test cases, functional testing for hate speech, differences between AI and traditional software testing, and the complexities of evaluating LLMs. Demetrios wraps up by inviting feedback and promoting an upcoming virtual conference on data engineering for AI and ML.

// Bio

At the moment Demetrios is immersing himself in Machine Learning by interviewing experts from around the world in the weekly MLOps Community Podcasts. Demetrios is constantly learning and engaging in new activities to get uncomfortable and learn from his mistakes. He tries to bring creativity into every aspect of his life, whether that be analyzing the best paths forward, overcoming obstacles, or building lego houses with his daughter.

// MLOps Jobs board

https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Balancing Speed and Safety // Panel // AIQCON - https://youtu.be/c81puRgu3Kw

AI For Good - Detecting Harmful Content at Scale // Matar Haller // MLOps Podcast #246 - https://youtu.be/wLKlZ6yHg1k

What is AI Quality? // Mohamed Elgendy // MLOps Podcast #229 - https://youtu.be/-Jdmq4DiOew

All Data Scientists Should Learn Software Engineering Principles // Catherine Nelson // Podcast #245 - https://youtu.be/yP6Eyny7p20

--------------- ✌️Connect With Us ✌️ -------------

Join our slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Timestamps:

[00:00] Exploring common themes in MLOps community

[00:49] Common patterns about model output and testing

[01:34] Tiered test case strategy

[03:05] Functional testing for models

[05:24] Testing coverage and quality

[07:47] Evaluating LLMs challenges

[08:35] Please like, share, leave a review, and subscribe to our MLOps channels!