The MLOps Podcast

Dean Pleban and Liron Itzhakhi Allerhand explore what it really takes to move LLMs into production. They cover how to define clear requirements, prep data for RAG, engineer effective prompts, and evaluate model performance using concrete metrics. The conversation dives into managing sensitive data, avoiding leakage, and why crisp outputs and clear user intent matter. Plus: future trends like in-context learning and the decoupling of foundation models from vertical apps.

Join our Discord community:

https://discord.gg/tEYvqxwhah

---

Timestamps:

• 00:00 Introduction
• 01:48 Phases of LLM Project Development
• 03:32 Defining the Problem
• 09:35 Data Preparation and Understanding
• 23:59 Multimodal RAG
• 26:28 Prompt Engineering & Model Selection
• 27:58 Model Fine-tuning & Customization
• 33:18 LLM as a Judge
• 38:58 Evaluating Model Performance and Handling Hallucinations
• 41:02 Using LLMs with sensitive data
• 45:24 Other ideas for model evaluation and guardrails
• 49:28 Recommendations for the audience

➡️ Liron Itzhaki Allerhand on LinkedIn – https://www.linkedin.com/in/liron-izhaki-allerhand-16579b4/

🌐 Check Out Our Website! https://dagshub.com Social

Links:

➡️ LinkedIn: https://www.linkedin.com/company/dagshub

➡️ Twitter: https://x.com/TheRealDAGsHub

➡️ Dean Pleban: https://x.com/DeanPlbn

In this episode, Dean speaks with Jeremie Dreyfuss, Head of AI Research and Development at Intel, about the evolving role of AI in the enterprise. Jeremie shares insights into scaling machine learning solutions, the challenges of building AI infrastructure, and the future of AI-driven innovation in large organizations. Learn how enterprises are leveraging AI for efficiency, the latest advancements in AI research, and the strategies for staying competitive in a rapidly changing landscape.

In this episode, Dean speaks with Federico Bacci, a data scientist and ML engineer at Bol, the largest e-commerce company in the Netherlands and Belgium. Federico shares valuable insights into the intricacies of deploying machine learning models in production, particularly for forecasting problems. He discusses the challenges of model explainability, the importance of feature engineering over model complexity, and the critical role of stakeholder feedback in improving ML systems. Federico also offers a compelling perspective on why LLMs aren't always the answer in AI applications, emphasizing the need for tailored solutions. This conversation provides a wealth of practical knowledge for data scientists and ML engineers looking to enhance their understanding of real-world ML operations and challenges in e-commerce.

In this episode, Dean speaks with Michał Oleszak, an ML engineering manager at Solera. Michał shares insights into how his team is using machine learning to transform the automotive claims process, from recognizing vehicle damages in images to estimating repair costs. The conversation covers the challenges of deploying ML pipelines in production, managing data quality for computer vision tasks, and balancing technical implementation with business needs. Michał also discusses his approach to model evaluation, the benefits of monorepo architecture, and his views on exciting developments in self-supervised learning for computer vision.

In this episode, I chat with Ljubomir Buturovic, VP of ML and Informatics at Inflammatix. We discuss using ML to diagnose infections and blood tests in the emergency room. We dive into the challenges of building diagnostic (classification) and prognostic (predictive) modes, with takeaways related to building datasets for production use cases.

In this episode, Idan Gazit, Senior Director of Research at GitHub Next, discusses his role in exploring strategic technologies and incubating long bet projects. He explains how the GitHub Next team chooses research projects and the process of exploration and theme selection. Idan also shares insights into the ML focus at GitHub Next and the challenges of evaluating the impact of AI products. He reflects on his journey into the AI space and provides advice for testing AI products in smaller organizations. Finally, he shares his thoughts on the future of AI interfaces.

In this episode, I chatted with Uri Goren, founder and CEO of Argmax, about Machine Learning and the future of digital advertising in a world moving away from cookies due to privacy laws like GDPR and CCPA. We chat about challenges in maintaining personalized ads while respecting user privacy, and new methods like probabilistic models and contextual features to cover some of the gap left by removing cookies.

In this episode, I speak with Han-Chung Lee, a machine learning engineer with a lot of interesting takes on ML and AI. We dive into the buzz around natural language processing and the big waves in generative AI. They chat about how newcomers are racing through NLP’s history, mixing old school and new tech, and the shift towards smarter databases. Han-Chung breaks it down with his straightforward takes, making complex AI trends feel like coffee chat topics. It’s a perfect listen for anyone keen on where AI’s headed, minus the jargon.