MLOps podcast #192 with Chris Van Pelt, CISO and co-founder of Weights & Biases, Enterprises Using MLOps, the Changing LLM Landscape, MLOps Pipelines sponsored by Weights & Biases.

// Abstract

Chris provides insights into his machine learning (ML) journey, emphasizing the significance of ML evaluation processes and the evolving landscape of MLOps. The conversation covers effective evaluation metrics, demo-driven development nuances, and the complexities of ML Ops pipelines. Chris reflects on his experience with Crowdflower, detailing its transition to Weights and Biases and stressing the early integration of security measures. The discussion extends to the transformative impact of ML on the tech industry, challenges in detecting subtle bugs, and the potential of open-source models and multimodal capabilities.

// Bio

Chris Van Pelt is a co-founder of Weights & Biases, a developer MLOps platform. In 2009, Chris founded Figure Eight/CrowdFlower. Over the past 12 years, Chris has dedicated his career to optimizing ML workflows and teaching ML practitioners, making machine learning more accessible to all. Chris has worked as a studio artist, computer scientist, and web engineer. He studied both art and computer science at Hope College.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://wandb.ai/site

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Chris on LinkedIn: https://www.linkedin.com/in/chrisvanpelt/

Timestamps:

[00:00] Chris' preferred coffee

[00:33] Takeaways

[03:50] Huge shout-out to Weights & Biases for sponsoring this episode!

[04:15] Please like, share, and subscribe to our MLOps channels!

[04:25] CrowdFlower

[07:02] Difference between CrowdFlower and Trajectory

[09:13] Transition from CrowdFlower to Weights & Biases

[13:05] Excel spreadsheets being passed around via email

[15:45] Evolution of Weights & Biases

[19:24] CISO role

[22:23] Advise for easy wins

[25:32] Transition into LLMs

[27:36] Prompt injection risks on data

[29:42] LLMs for New Personas

[34:42] Iterative Value Evaluation Process

[36:36] Iterating on New Release

[39:31] Evaluation survey

[43:21] Landscape of LLMs and its evolution

[45:40] Conan O'Brien

[46:48] Wrap up