Agentic Conversations (formally mlops.community)

MLOps Coffee Sessions #166 with Roy Hasson & Santona Tuli, Eliminating Garbage In/Garbage Out for Analytics and ML.

// Abstract

Shift left data quality ownership and observability that makes it easy for users to catch bad data at the source and stop it from entering your analytics/ML stack.

// Bio

Santona Tuli

Santona Tuli, Ph.D., began her data journey through fundamental physics—searching through massive event data from particle collisions at CERN to detect rare particles. She’s since extended her machine learning engineering to natural language processing, before switching focus to product and data engineering for data workflow authoring frameworks. As a Python engineer, she started with the programmatic data orchestration tool, Airflow, helping improve its developer experience for data science and machine learning pipelines. Currently, at Upsolver, she leads data engineering and science, driving developer research and engagement for the declarative workflow authoring framework in SQL. Dr. Tuli is passionate about building, as well as empowering others to build, end-to-end data and ML pipelines, scalably.

Roy Hasson

Roy is the head of product at Upsolver, helping companies deliver high-quality data to their analytics and ML tools. Previously, Roy led product management for AWS Glue and AWS Lake Formation.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://royondata.substack.com/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community:

https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup:

https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more:

https://mlops.community/

Connect with Demetrios on LinkedIn:

https://www.linkedin.com/in/dpbrinkm/

Connect with Roy on LinkedIn:

https://www.linkedin.com/in/royhasson/

Connect with Santona on LinkedIn:

https://www.linkedin.com/in/santona-tuli/

Timestamps:

[00:00] Santona's and Roy's preferred coffee

[01:05] Santona's and Roy's background

[03:33] Takeaways

[05:49] Please like, share, and subscribe to our MLOps channels!

[06:42] Back story of having Santona and Roy on the podcast

[09:51] Santona's story

[11:37] Optimal tag teamwork

[16:53] Dealing with stakeholder needs

[26:25] Having mechanisms in place

[27:30] Building for data Engineers vs building for data scientists

[34:50] Creating solutions for users

[38:55] User experience holistic point of view

[41:11] Tooling sprawl is real

[42:00] LLMs' reliability

[45:00] Things I would have loved to learn five years ago

[49:46] Wrap up

MLOps Coffee Sessions #165 with Sammy Sidhu, Python Power: How Daft Embeds Models and Revolutionizes Data Processing.

// Abstract

Sammy shares his fascinating journey in the autonomous vehicle industry, highlighting his involvement in two successful startup acquisitions by Tesla and Toyota. He emphasizes his expertise in optimizing and distilling models for efficient machine learning, which he has incorporated into his new company, Eventual. The company's open-source offering, daf, focuses on tackling the challenges of unstructured and complex data. Sammy discusses the future of MLOps, machine learning, and data storage, particularly in relation to the retrieval and processing of unstructured data. The Eventual team is developing Daft, an open-source query engine that aims to provide efficient data storage solutions for unstructured data, offering features like governance, schema evolution, and time travel. The conversation sheds light on the innovative developments in the field and the potential impact on various industries.

// Bio

Sammy is a Deep Learning and systems veteran, holding over a dozen publications and patents in the space. Sammy graduated from the University of California, Berkeley, where he did research in Deep Learning and High Performance Computing. He then joined DeepScale as the Chief Architect and led the development of perception technologies for autonomous vehicles. During this time, DeepScale grew rapidly and was subsequently acquired by Tesla in 2019. Staying in Autonomous Vehicles, Sammy joined Lyft Level 5 as a Senior Staff Software Engineer, building out core perception algorithms as well as infrastructure for machine learning and embedded systems. Level 5 was then acquired by Toyota in 2021, adopting much of its work. Sammy is now CEO and Co-Founder at Eventual Building Daft, an open-source query engine that specializes in multimodal data.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://sammysidhu.com/

Check out Daft, our open-source query engine for multimodal data! https://www.getdaft.io/

Here are some talks/shows we have given about it:- PyData Global (Dec 2022): Large-scale image processing: https://www.youtube.com/watch?v=ol6IQUbyeDo&ab_channel=PyData

- Ray Meetup (March 2023): Distributed ML preprocessing + training on Ray https://www.youtube.com/watch?v=1MpEYlIlu7w&t=2972s&ab_channel=Anyscale

- The Data Stack Show (April 2023): Self-Driving Technology and Data Infrastructure with Sammy Sidhu https://datastackshow.com/podcast/the-prql-self-driving-technology-and-data-infrastructure-with-sammy-sidhu-co-founder-and-ceo-of-eventual/

Chain of Thought for LLMs: https://cobusgreyling.medium.com/chain-of-thought-prompting-in-llms-1077164edf97

Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes: https://arxiv.org/abs/2305.02301

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Sammy on LinkedIn: https://www.linkedin.com/in/sammy-sidhu/

MLOps Coffee Sessions #164 with Rob Hirschfeld, Open Source and Fast Decision Making. This episode is brought to you by.

// Abstract

Rob Hirschfeld, the CEO and co-founder of Rack N, discusses his extensive experience in the DevOps movement. He shares his notable achievement of coining the term "the cloud" and obtaining patents for infrastructure management and API provision. Rob highlights the stagnant progress in operations and the persistent challenges in security and access controls within the industry. The absence of standardization in areas such as Kubernetes and single sign-on complicates the development of robust solutions. To address these issues, Rob underscores the significance of open-source practices, automation, and version control in achieving operational independence and resilience in infrastructure management.

// Bio

Rob is the CEO and Co-founder of RackN, an Austin-based start-up that develops software to help automate data centers, which they call Digital Rebar. This platform helps connect all the different pieces and tools that people use to manage infrastructure into workflow pipelines through seamless multi-component automation across the different pieces and parts needed to bring up IT systems, platforms, and applications. Rob has a background in Scale Computing, Mechanical and Systems Engineering, and specializes in large-scale complex systems that are integrated with the physical environment. He has founded companies and been in the cloud and infrastructure space for nearly 25 years, and has done everything from building the first Clouds using ESXi betas to serving four terms on the OpenStack Foundation Board. Rob was trained as an Industrial Engineer and holds degrees from Duke University and Louisiana State University.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

https://rackn.com/

https://robhirschfeld.com/about/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Rob on LinkedIn: https://www.linkedin.com/in/rhirschfeld/

Timestamps:

[00:00] Rob's preferred coffee

[00:17] Rob Hirschfeld's background

[01:42] Takeaways

[02:36] Please like, share, and subscribe to this channel!

[03:09] Creation of Cloud

[08:38] Changes in Cloud after 25 Years

[10:54] Pros and cons of microservices

[13:06] Secure Access Provisioning

[15:46] Parallelism with ads

[18:08] Redfish protocol

[20:21] Impact of using open source vs using a SAS provider

[26:15] Automation

[32:39] Embrace Operational Flexibility

[35:08] Automating infrastructure inefficiently

[41:26] Legacy code and resiliency

[43:39] Collection of metadata

[45:50] RackN

[51:23] Granular Cloud Preferences

[54:35] Reframing of perceived complexity

[57:32] Generative DevOps

[58:50] Wrap up

MLOps Coffee Sessions #163 with Yujian Tang, Democratizing AI, co-hosted by Abi Aryan.

// Abstract

The popularity of ChatGPT has brought large language model (LLMs) apps and their supporting technologies to the forefront. One of the supporting technologies is vector databases. Yujian shares how vector databases like Milvus are used in production and how they solve one of the biggest problems in LLM app building - data issues. They also discuss how Zilliz is democratizing vector databases through education, expanding access to technologies, and technical evangelism.

// Bio

Yujian Tang is a Developer Advocate at Zilliz. He has a background as a software engineer working on AutoML at Amazon. Yujian studied Computer Science, Statistics, and Neuroscience with research papers published at conferences, including IEEE Big Data. He enjoys drinking bubble tea, spending time with family, and being near water.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Yujian on LinkedIn: https://www.linkedin.com/in/yujiantang

Timestamps:

[00:00] Yujian's preferred coffee

[02:40] Takeaways

[05:14] Please share this episode with your friends!

[06:39] Vector databases trajectory

[09:00] 2 start-up companies created by Yujian

[09:39] Uninitiated Vector Databases

[12:20] Vector Databases trade-off

[14:16] Difficulties in training LLMs

[23:30] Enterprise use cases

[27:38] Process/rules not to use LLMs unless necessary

[32:14] Setting up returns

[33:13] When not to use Vector Databases

[35:30] Elastic search

[36:07] Generative AI apps' common pitfalls

[39:35] Knowing your data

[41:50] Milvus

[48:28] Actual Enterprise use cases

[49:32] Horror stories

[50:31] Data mesh

[51:06] GPTCash

[52:10] Shout out to the Seattle Community!

[53:44] Wrap up

MLOps Coffee Sessions #162 with Soham Chatterjee, From LLMs to TinyML: The Dynamic Spectrum of MLOps, co-hosted by Abi Aryan.

// Abstract

Explore the spectrum of MLOps from large language models (LLMs) to TinyML. Soham highlights the difficulties of scaling machine learning models and cautions against relying exclusively on OpenAI's API due to its limitations. Soham is particularly interested in the effective deployment of models and the integration of IoT with deep learning. He offers insights into the challenges and strategies involved in deploying models in constrained environments, such as remote areas with limited power, and utilizing small devices like Arduino Nano.

// Bio

Soham leads the machine learning team at Sleek, where he builds tools for automated accounting and back-office management. As an electrical engineer, Soham has a passion for the intersection of machine learning and electronics, specifically TinyML/Edge Computing. He has several courses on MLOps and TinyMLOps available on Udacity and LinkedIn, with more courses in the works.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Soham on LinkedIn: https://www.linkedin.com/in/soham-chatterjee

Timestamps:

[00:00] Soham's preferred coffee

[01:49] Takeaways

[05:33] Please share this episode with

[07:02] Soham's background

[09:00] From electrical engineering to Machine Learning

[10:40] Deep learning, Edge Computing, and Quantum Computing

[11:34] Tiny ML

[13:29] Favorite area in Tiny ML chain

[14:03] Applications explored

[16:56] Operational challenges transformation

[18:49] Building with Large Language Models

[25:44] Most Optimal Model

[26:33] LLMs path

[29:19] Prompt engineering

[33:17] Migrating infrastructures to a new product

[37:20] Your success where others failed

[38:26] API Accessibility

[39:02] Reality about LLMs

[40:39] The Compression angle adds to the bias

[43:28] Wrap up

MLOps Coffee Sessions #161 with Tuhin Srivastava, The Long Tail of ML Deployment, co-hosted by Abi Aryan. This episode is brought to you by QuantumBlack.

// Abstract

Baseten is an engineer-first platform designed to alleviate the engineering burden for machine learning and data engineers. Tuhin's perspective, based on research with Stanford students, emphasizes the importance of engineers embracing the engineering aspects and considering them from a reproductive perspective.

// Bio

Tuhin Srivastava is the co-founder and CEO of Baseten. Tuhin has spent the better part of the last decade building machine learning-powered products and is currently working on empowering engineers to build production-grade services with machine learning.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

QuantumBlack: https://www.mckinsey.com/capabilities/quantumblack/contact-us

Baseten: https://www.baseten.co/

Baseten Careers: https://www.baseten.co/careers

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Tuhin on LinkedIn: https://www.linkedin.com/in/tuhin-srivastava-60601114/

Timestamps:

[00:00] Partnership with QuantumBlack

[00:16] Nayur Khan presenting QuantumBlack

[03:35] QuantumBlack is hiring!

[03:47] Tuhin's preferred coffee

[05:03] Takeaways

[07:00] Please share this episode with a friend!

[07:12] Comments/Reviews

[08:49] Tuhin's background

[10:13] Finance and Law common complaint culture

[11:40] Doing Machine Learning in 2010 - 2011

[14:31] Gum broad or the next company shape?

[16:33] Engineers need to learn machine learning

[20:18] Software engineers need to dig deeper

[24:49] Cambrian Explosion

[27:53] The Holy Trifecta

[28:54] Objective truth and prompting

[31:23] Limitations of LLMs

[35:37] Documentation challenges

[38:25] Baseten creating valuable models

[40:37] Advocate for Microservices or API-based solution

[42:54] Learning Git pains

[44:16] Baseten back ups

[48:00] Baseten is hiring!

[49:32] Wrap up

MLOps Coffee Sessions #160 with Matt Sharp, Data Developer at Shopify, Clean Code for Data Scientists, co-hosted by Abi Aryan.

// Abstract

Let's delve into Shopify's real-time serving platform, Merlin, which enables features like recommender systems, inbox classification, and fraud detection. Matt shares his insights on clean coding and the new book he is writing about LLMs in production.

// Bio

Matt is a Chemical Engineer turned Data scientist turned Data Engineer. Self-described "Recovering Data Scientist", Matt got tired of all the inefficiencies he faced as a Data Scientist and made the switch to Data Engineering. At Matt's last job, he ended up building the entire MLOps platform from scratch for a fintech startup called MX. Matt gives tips to data scientists on LinkedIn on how to level up their careers and has started to be known for my clean code tips in particular. Matt recently started a new job at Shopify.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Matt on LinkedIn: https://www.linkedin.com/in/matthewsharp/

Timestamps:

[00:00] Matt's preferred drink

[00:54] Takeaways

[03:04] Watch out for Matt's LLMs in Production book coming up!

[03:29] Please like, share, subscribe, and join the upcoming LLMs in Production Conference Part 2!

[05:07] Raising awareness about the fundamental problems of writing clean code

[07:57] Definition of clean code[09:46] Communicable clean code

[13:52] Getting out of Jupyter notebooks at the end of their life

[17:21] Exploratory data analysis

[21:22] Most popular post on LinkedIn

[26:41] Zilliz Ad

[27:44] Best practices on production-level software engineering

[29:41] Merlin

[33:51] Upcoming Shopify projects

[39:10] Matt's upcoming LLMs in Production book

[45:06] LLMs in Production book Early Access

[46:00] Wrap up

MLOps Coffee Sessions #159 with Maria Vechtomova, Lead ML engineer, and Basak Eskili, Machine Learning Engineer, at Ahold Delhaize. Why is MLOps Hard in an Enterprise? co-hosted by Abi Aryan.

// Abstract

MLOps is particularly challenging to implement in enterprise organizations due to the complexity of the data ecosystem, the need for collaboration across multiple teams, and the lack of standardization in ML tooling and infrastructure. In addition to these challenges, at Ahold Delhaize, there is a requirement for the reusability of models as our brands seek to have similar data science products, such as personalized offers, demand forecasts, and cross-sell.

// Bio

Maria Vechtomova

Maria is a Machine Learning Engineer at Ahold Delhaize. Maria is bridging the gap between data scientists, infra, and IT teams at different brands and focuses on standardization of machine learning operations across all the brands within Ahold Delhaize. During nine years in Data&Analytics, Maria tried herself in different roles, from data scientist to machine learning engineer, was part of teams in various domains, and has built broad knowledge. Maria believes that a model only starts living when it is in production. For this reason, last six years, her focus has been on the automation and standardization of processes related to machine learning.

Basak Eskili

Basak Eskili is a Machine Learning Engineer at Ahold Delhaize. She is working on creating new tools and infrastructure that enable data scientists to quickly operationalize algorithms. She is bridging the space between data scientists and platform engineers while improving the way of working in accordance with MLOps principles. In her previous role, she was responsible for bringing models to production. She focused on NLP projects and building data processing pipelines. Basak also implemented new solutions by using cloud services for existing applications and databases to improve time and efficiency.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related LinksMLOps Maturity Assessment Blog: https://mlops.community/mlops-maturity-assessment/

The Minimum Set of Must-Haves for MLOps Blog: https://mlops.community/the-minimum-set-of-must-haves-for-mlops/

Traceability & Reproducibility Blog: https://mlops.community/traceability-reproducibility/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Maria on LinkedIn: https://www.linkedin.com/in/maria-vechtomova/

Connect with Basak on LinkedIn: https://www.linkedin.com/in/ba%C5%9Fak-tu%C4%9F%C3%A7e-eskili-61511b58/

Timestamps:

[00:00] Maria & Basak's preferred coffee

[00:59] LLMs in Production Conference Part 2 coming up on June 15-16!

[02:08] Maria & Basak's background

[02:47] Takeaways

[04:52] A colorful history

[06:59] 4 levels of evolution

[08:15] Standardization and Model Registry Evolution

[11:52] Ahold Delhaize Standard task

[15:05] Ahold Delhaize Workflow

[25:19] Avoiding tooling sprawl

[28:10] Guardrails

[29:50] Secret sharing and credential sharing sloppy processes

[32:23] Distrust between DevOps engineers and data scientists

[33:29] MLOps vs DevOps

[35:31] Monitoring pieces heroes

[38:32] Future accumulative cost issues

[40:09] Exploratory phase in notebooks

MLOps Coffee Sessions #158 with Nils Reimer, MLOps Build or Buy, Large Language Model at Scale, co-hosted by Abi Aryan.

// Abstract

Large Language Models with billions of parameters have the possibility to change how we work with textual data. However, running them on scale at potentially hundreds of millions of texts a day is a massive challenge. Nils talks about finding the right model size for respective tasks, model distillation, and promising new ways of transferring knowledge from large to smaller models.

// Bio

Nils Reimers is highly recognized throughout the AI community for creating and maintaining the now-famous Sentence Transformers library (www.SBERT.net) used to develop, train, and use state-of-the-art LLMs. The project has 900+ stars on GitHub and 30M+ installations. Nils is currently the Director of Machine Learning at Cohere, where he leads the team that develops and trains Large Language Models (LLMs) with billions of parameters. Prior to Cohere, Nils created and led the science team for Neural Search at HuggingFace. Nils holds a Ph.D. in Computer Science from UKP in Darmstadt.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

(www.SBERT.net)

https://www.nils-reimers.de/

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Abi on LinkedIn: https://www.linkedin.com/in/goabiaryan/

Connect with Nils on LinkedIn: https://www.linkedin.com/in/reimersnils/

Timestamps:

[00:00] Nils' preferred coffee

[00:45] Nils' background

[01:30] Takeaways

[06:47] Subscribe to our Newsletters and IRL Meetups, and leave your reviews!

[07:32] Nils' history

[10:39] From IT Security to Machine Learning

[13:22] Tangibility of IT and Security

[14:46] NLP transition

[15:55] Bad augmentation to new capabilities of LLMs

[16:59] Nils' concern during his PH.D.

[19:55] Making Money from Machine Learning

[22:06] Catastrophic forgetting

[26:34] Updating solutions

[28:42] Neural search space and building adaptive models

[31:23] Filtering models

[32:32] Latency issues

[36:53] Models running in parallel

[37:54] Generative models problems

[38:43] Nils' role at Cohere

[41:41] To build or not to build API

[43:00] Search models

[45:38] Large use cases

[46:43] Open source discussion within Cohere

[50:48] Competitive Edge

[55:27] Future world of API

[58:14] LLMs in Production Conference Part 2 announcement!

[1:00:17] Hopeful direction of Cohere's future

[1:02:33] Data silos

[1:04:34] Where to update the database and code

[1:05:24] Nils' focus

[1:08:49] Make money or save money

[1:10:30] Cohere's upcoming project

[1:12:37] Time spent red teaming the models

[1:14:05] Wrap up