Agentic Conversations (formally mlops.community)

Jonathan Frankle works as Chief Scientist (Neural Networks) at MosaicML (recently acquired by Databricks), a startup dedicated to making it easy and cost-effective for anyone to train large-scale, state-of-the-art neural networks. He leads the research team.

MLOps podcast #205 with Jonathan Frankle, Chief Scientist (Neural Networks) at Databricks, The Myth of AI Breakthroughs, co-hosted by Denny Lee, brought to us by our Premium Brand Partner, Databricks.

// Abstract

Jonathan takes us behind the scenes of the rigorous work they undertake to test new knowledge in AI and to create effective and efficient model training tools. With a knack for cutting through the hype, Jonathan focuses on the realities and usefulness of AI and its application. We delve into issues such as face recognition systems, the 'lottery ticket hypothesis,' and robust decision-making protocols for training models. Our discussion extends into Jonathan's interesting move into the world of law as an adjunct professor, the need for healthy scientific discourse, his experience with GPUs, and the amusing claim of a revolutionary algorithm called Qstar.

// Bio

Jonathan Frankle is Chief Scientist (Neural Networks) at Databricks, where he leads the research team toward the goal of developing more efficient algorithms for training neural networks. He arrived via Databricks’ $1.3B acquisition of MosaicML as part of the founding team. He recently completed his PhD at MIT, where he empirically studied deep learning with Prof. Michael Carbin, specifically the properties of sparse networks that allow them to train effectively (his "Lottery Ticket Hypothesis" - ICLR 2019 Best Paper). In addition to his technical work, he is actively involved in policymaking around challenges related to machine learning. He earned his BSE and MSE in computer science at Princeton and has previously spent time at Google Brain and Facebook AI Research as an intern and at Georgetown Law as an Adjunct Professor of Law.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: www.jfrankle.com

Facial recognition: perpetuallineup.org

The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks by Jonathan Frankle and Michael Carbin paper: https://arxiv.org/abs/1803.03635

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Denny on LinkedIn: https://linkedin.com/in/dennyglee

Connect with Jonathan on LinkedIn: https://www.linkedin.com/in/jfrankle/

Timestamps:

[00:00] Jonathan's preferred coffee

[01:16] Takeaways

[07:19] LM Avalanche Panel Surprise

[10:07] Adjunct Professor of Law

[12:59] Low facial recognition accuracy

[14:22] Automated decision making, human in the loop argument

[16:09] Control vs. Outsourcing Concerns

[18:02] perpetuallineup.org

[23:41] Face Recognition Challenges

[26:18] The lottery ticket hypothesis

[29:20] Mosaic Role: Model Expertise

[31:40] Expertise Integration in Training

[38:19] SLURM opinions

[41:30] GPU Affinity

[45:04] Breakthroughs with QStar

[49:52] Deciphering the noise advice

[53:07] Real Conversations

[55:47] How to cut through the noise

[1:00:12] Research Iterations and Timelines

[1:02:30] User Interests, Model Limits

[1:06:18] Debugability

[1:08:00] Wrap up

Patrick Barker is the Founder / CTO of Kentauros AI.

Farhood Etaati is a Software Engineer at Yektanet.

MLOps podcast #204 with Patrick Barker, CTO of Kentauros AI and Farhood Etaati, MLOps/Platform Team Lead at AIMedic, MLOps at the Crossroads.

// Abstract

MLOps is at a crossroads. The ever-increasing excitement for LLMs' ability to solve some interesting real-world problems has made many people interested in applying these models in new applications which comes with its own challenges, that have upstarted the term "LLMLOps". But how much of those challenges are not a newer representation of what older-gen ML models had to deal with in the production, and the question arises whether developing "new" specialized tools to address these applications actually provides any substantial value for the sustainability of the field in general. Tools are coming and going at a rate that makes many technical people skeptical of adopting newer tools. What can we do as a community to alleviate these issues? Why OSS MLOps is lacking behind and how VC money is contributing to that?

// Bio

Farhood Etaati

MLOps engineer at AIMedic. Studied EE at Uni of Tehran, started out as a data scientist, and pivoted to software engineering. Currently working on on-premise MLOps platform development suitable for Iran's infrastructure.

Patrick Barker

When Patrick is not occupied with building his AI company, he enjoys spending time with his wonderful kids or exploring the hills of Boulder.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Websites: https://github.com/pbarker

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Patrick on LinkedIn: https://www.linkedin.com/in/patrickbarkerco/

Connect with Farhood on LinkedIn: www.linkedin.com/in/farhood-etaati

Timestamps:

[00:00] Farhood's and Patrick's preferred coffee

[01:13] Takeaways

[04:00] Please like, share, and subscribe to our MLOps channels!

[05:26] Strong feelings

[10:21] MLOps vs DevOps Challenges

[13:44] Medical setting, ML tools, NLP, model building

[16:23] MLOps vs Data Engineering

[20:45] MLOps Boosts LLM Development

[23:54] Longtail Use Cases

[31:00] Tech Roles Distinctions

[34:42] Did He Say That?

[37:04] Fine-tuning AI Models

[38:57] ML 2.0 Advancements Explained

[41:11] Generative AI in MLOps

[45:04] ML Reproducibility Challenges

[48:03] Wrap up

Aleksa Gordić is an ex-Google DeepMind / Microsoft ML engineer currently working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project and YugoGPT.

MLOps podcast #203 with Aleksa Gordić, Founder of OrtusAI, Pioneering AI Models for Regional Languages.

// Abstract

Dive deep into Aleksa's work with the YugoGPT, a language model serving Serbian, Croatian, Bosnian, and Montenegrin dialects - emphasizing the need for multilingual AI developments. Explore the unique language dynamics in the Balkans and Eastern Europe, the potential business opportunities around multilingual models, and the challenges in deploying large language models. Aleksa shares his experience with vision and image models, his collaborations with key tech players, and his use of advanced technologies. Hear about Aleksa Gordić's journey of being active and visible in the tech community and his insights into the world of machine learning and AI. Prepare to have your thinking challenged and horizons widened as we converse about the intriguing and complex world of MLOps.

// Bio

Working on non-English LLMs at OrtusAI, open-sourcing Meta's NLLB (no language left behind) project. Worked at DeepMind on the Flamingo project as a research engineer. Worked at Microsoft on the HoloLens 2 project & next-gen mixed reality glasses.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://gordicaleksa.com/https://github.com/gordicaleksa

- I build stuff :) https://discord.com/invite/peBrCpheKE

- active AI Discord server (~6000) I bring the best AI researchers in the world to give talks (James Betker, DALL-E 3 author, Tri Dao (Flash Attention), etc.)

https://gordicaleksa.medium.com/how-i-got-a-job-at-deepmind-as-a-research-engineer-without-a-machine-learning-degree-1a45f2a781de

- how I landed a job at DeepMind (and a couple more potentially interesting writings)Aleksa Gordić The AI Epiphany Youtube Channel: https://www.youtube.com/channel/UCj8shE7aIn4Yawwbo2FceCQ/videos

W&B AI Academy: http://wandb.me/mlops_com_llm_course

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Aleksa on LinkedIn: https://www.linkedin.com/in/aleksagordic/

Timestamps:

[00:00] Aleksa's preferred coffee

[00:17] Takeaways

[02:51] Humming the GPU's

[06:23] Built a Chrome extension for communicating with videos

[08:04] Rig Doubles Throughput Time

[09:32] Vector databases advise

[10:38] Learning from experts, connecting, and gathering insights.

[13:47] Zero to Hero for MLOps

[15:37] Serendipitous moments

[17:52] Depth Over Breaking News

[19:50] Trust in GPT Content

[22:22] Exam Challenges and AI

[26:53] YugoGPT

[31:41] WandB Ad

[33:33] Linguistic Mysteries

[34:52] No Language Left Behind project (NLLB project)

[36:53] YugoGPT Development Overview

[37:49] NLLB vs YugoGPT

[39:35] Yugo GPT parameters

[41:16] Opportunities for unsupported languages

[43:08] Diffusion model

[44:39] Generative AI with image generation models

[47:45] AI Challenges and Excitement

[50:32] Challenges with different alphabet characters

[52:10] Need a co-founder

[56:05] Career transition and entrepreneurial mindset

[1:00:20] Big Tech salary misconceptions

[1:03:02] Inspiring wrap-up

Prof. Dr. Hannes Mühleisen is a creator of the DuckDB database management system and Co-founder and CEO of DuckDB Labs.

Jordan Tigani is co-founder and chief duck-herder at MotherDuck, a startup building a serverless analytics platform based on DuckDB.

MLOps podcast #202 with Hannes Mühleisen, Co-Founder & CEO of DuckDB Labs and Jordan Tigani, Chief Duck-Herder at MotherDuck, Small Data, Big Impact: The Story Behind DuckDB.

// Abstract

Navigate the intricacies of data management with Jordan Tagani and Hannes Mühleisen, the creative geniuses behind DuckDB and MotherDuck. This deep dive unravels the game-changing principles behind DuckDB's creation, tackling the prevailing wisdom to passionately fill the gap for smaller data set management. Let's also discover MotherDuck's unique focus on providing an unprecedented developer experience and its innovative edge in visualization and data delivery. This episode is teeming with enlightening discussions about managing community feedback, funding, and future possibilities that should not be missed for any tech enthusiasts and data management practitioners.

// Bio

Hannes Mühleisen

Prof. Dr. Hannes Mühleisen is a creator of the DuckDB database management system and Co-founder and CEO of DuckDB Labs, a consulting company providing services around DuckDB. Hannes is also Professor of Data Engineering at Radboud Universiteit Nijmegen. His main interest is analytical data management systems.

Jordan Tigani

Jordan is co-founder and chief duck-herder at MotherDuck, a startup building a serverless analytics platform based on DuckDB. He spent a decade working on Google BigQuery as a founding engineer, book author, engineering leader, and product leader. More recently, as SingleStore’s Chief Product Officer, Jordan helped them build a cloud-native SaaS business. Jordan has also worked at Microsoft Research, the Windows Kernel team, and at a handful of star-crossed startups. His biggest claim to fame is predicting World Cup matches using machine learning with a better record than Paul the Octopus.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Websites: https://duckdb.org/https://motherduck.com/

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Hannes on LinkedIn: https://www.linkedin.com/in/hfmuehleisen/

Connect with Jordan on LinkedIn: https://www.linkedin.com/in/jordantigani/

Timestamps:

[00:00] Hannes and Jordan's preferred coffee

[01:30] Takeaways

[03:43] Swaggers in the house!

[07:13] Duck DB's inception

[09:38] Jordan's background

[12:28] Simplify Developer Experience

[17:54] Big Data Shift

[26:01] Creation of MotherDuck

[30:58] Duck DB and MotherDuck Partnership

[31:57] Incentive Alignment Concerns

[37:46] Building an incredible developer experience

[43:38] User Testing Lab

[47:18] Setting a higher standard

[49:22] The moments before the moment

[52:18] Gathering feedback and talking to the community

[54:30] MotherDuck Features

[1:00:19] Cloud Innovation for MotherDuck

[1:02:41] ML Engineers and DuckDB

[1:08:03] Wrap up

Paco Nathan is the Managing Partner at Derwen, Inc., and author of Latent Space, along with other books, plus popular videos and tutorials about machine learning, natural language, graph technologies, and related topics.

MLOps podcast #201 with Paco Nathan, Managing Partner at Derwen, Inc., Language, Graphs, and AI in Industry.

// Abstract

Let's talk about key findings from these conferences, specifically summarizing teams that have ROI on machine learning in production: what are the things in common they're doing, and what are the most important caveats they urge other teams to consider when getting started? Because these key takeaways aren't found in the current AI news cycle.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

AI Conference: https://aiconference.com/

K1st World: https://www.k1st.world/

Corunna Innovation Summit: https://corunna.dataspartan.com/

"Cloud Computing on Amazon AWS EC2" UC Berkeley EECS guest lecture (2009)

https://vimeo.com/manage/videos/3616394

"Hardware - Software - Process: Data Science in a Post-Moore’s Law World"

https://www.nvidia.com/en-us/ai-data-science/resources/hardware-software-process-book/

“LLMs in Production: Learning from Experience” by Waleed Kadous @ Anyscale

https://www.youtube.com/watch?v=xa7k9MUeIdk

"Supercharging Industrial Operations with Problem-Solving GenAI & Domain Knowledge" by Christopher Nguyen @ Aitomatic

https://www.k1st.world/2023-program/supercharging-industrial-operations-with-problem-solving-genai-domain-knowledge

“The Next Million AI Systems” by Mark Huang @ Gradient: https://www.youtube.com/watch?v=lA0Npe4PqFw

"AI in a Box" by Useful Sensorshttps://usefulsensors.com/#products

"Opportunities in AI - 2023" by Andrew Ng

https://www.youtube.com/watch?v=5p248yoa3oE

"Advancing the Marine Industry Through the Harmony of Fishermen's Knowledge and AI" by Akinori Kasai @ Furuno

https://www.k1st.world/2023-program/advancing-the-marine-industry-through-the-harmony-of-fishermen-knowledge-and-al

Macy conferences (1941-1960)

https://en.wikipedia.org/wiki/Macy_conferenceshttps://www.asc-cybernetics.org/foundations/history/MacySummary.htm

https://press.uchicago.edu/ucp/books/book/distributed/C/bo23348570.htmlsecond-order cybernetics

https://pangaro.com/designconversation/wp-content/uploads/dubberly-pangaro-chk-journal-2015.pdf

https://en.wikipedia.org/wiki/Second-order_cyberneticsProject Cybersyn

https://jacobin.com/2015/04/allende-chile-beer-medina-cybersyn/

https://thereader.mitpress.mit.edu/project-cybersyn-chiles-radical-experiment-in-cybernetic-socialism/

https://99percentinvisible.org/episode/project-cybersyn/

https://medium.com/@rjog/project-cybersyn-an-early-attempt-at-iot-governance-and-how-we-can-apply-its-learnings-5164be850413

https://www.sustema.com/post/project-cybersyn-how-a-chilean-government-almost-controlled-the-economy-from-a-control-room

https://transform-social.org/en/texts/cybersyn/

Humberto Maturana, Francisco Varela: Autopoeisis "De Maquinas y Seres Vivos""Everything said is said by an observer"

https://proyectos.yura.website/wp-content/uploads/2021/06/de_maquinas_y_seres_vivos_-_maturana.pdf

https://en.wikipedia.org/wiki/Autopoiesis_and_Cognition:_The_Realization_of_the_Living

Fernando Flores(led Project Cybersyn, imprisoned, later worked with Prof. Terry Winograd @ Stanford, the grad advisor for what became Google)

https://lorenabarba.com/gallery/prof-barba-gave-keynote-at-pycon-2016/

https://conversationsforaction.com/fernando-flores

"Navigating the Risk Landscape: A Deep Dive into Generative AI" by Ben Lorica and Andrew Burt

https://thedataexchange.media/mitigating-generative-ai-risks/

"SpanMarker" by Tom Aarsen @ Hugging Face

https://tomaarsen.github.io/SpanMarkerNER/

Examples of "the math catching up with the machine learning": Guy Van den Broeck @ UCLA

https://web.cs.ucla.edu/~guyvdb/talks/

Charles Martin @ Calculations Consulting

https://weightwatcher.ai/

Mihail Eric is an engineer, researcher, and educator who has helped start teams at innovative organizations such as Amazon Alexa and RideOS.

Mihail is a cofounder of Storia AI, where they build an AI-powered creative assistant for fast and delightful image and video generation.

MLOps podcast #200 with Mihail Eric, Co-founder of Storia AI, Founding, Funding, and the Future of MLOps.

// Abstract

Demetrios and Mihail journey deep into the significance of human sentiment in an increasingly AI-driven era, the perils and promises of conversational AI, and the evolution and impact of image generation models.

Delve into the world of MLOps versus LLMOps, offering clarifying perspectives on how the core concerns and technology persist, even amidst an evolving tech landscape with new buzzwords making waves. Mihail generously provides an inside look at his AI tool and its wide range of applications across various industries, offering insights into interesting niche-specific verticals and unexpected use cases.

// Bio

Mihail is a co-CEO of Storia AI, an early-stage startup building an AI-powered creative assistant for video production. He has over a decade of experience researching and engineering AI systems at scale. Previously, he built the first deep-learning dialogue systems at the Stanford NLP group. He was also a founding member of Amazon Alexa’s first special projects team, where he built the organization’s earliest large language models. Mihail is a serial entrepreneur who previously founded Confetti AI, a machine-learning education company that he led until its acquisition in 2022.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: www.mihaileric.com

https://www.storia.ai/⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/

Timestamps:

[00:00] Spotify playlist

[02:08] How to live a longer, healthier life

[03:43] Absurd sweater collection

[06:03] The catch-up episode

[08:10] MLOps versus LLMOps

[13:27] AI apps mixing visuals and text

[16:00] Founder dating

[16:54] Stable diffusion and difficulties with Mid Journey

[23:12] Stripe developer experience

[25:04] APIs and Model Providers

[27:33] Host stable diffusion on AWS

[34:07] AI Creativity: Prompt Experimentation

[35:45] AI Challenges and Solutions

[39:51] AI Hype Frustration

[44:31] AI Impact on Hollywood

[48:11] AI Impact on Filmmaking

[51:48] Generalizable Tool for Verticals

[52:49] MLOps versus LLMOps

[56:42] Wrap up

Nathan Ryan Frank is the Machine Learning Operations and Platform Director of Grainger. Former Astrophysicist turned data scientist and machine learning engineer with a proven history of delivering results into production across a wide variety of domains while leading projects with international, cross-functional teams.

MLOps podcast #199 with Nathan Ryan Frank, Director, Machine Learning Platform & Operations at WW Grainger, Challenges Operationalizing Machine Learning (And Some Solutions).

// Abstract

This talk details some common challenges and pitfalls when attempting to operationalize machine learning systems and discusses some simple solutions. We dive into the machine learning development workflow and cover topics such as team dynamics, communication issues between roles that don't share a common language, and approaching MLOps from an SRE/DevOps perspective. Similarly, the talk highlights some challenges unique to operationalizing machine learning, drawing distinctions where necessary to highlight a large amount of similarity. Finally, the talk offers some simple and practical guidance for those new to MLOps who want to understand where to start and how to adopt best practices in an evolving field.

// Bio

Nathan Frank is currently the Director of Machine Learning Platform and Operations at Grainger, where he is building a team to support the Technology Group's expanding machine learning efforts. Prior to joining Grainger, Nathan led machine learning engineering efforts at Strong Analytics, a boutique data science and machine learning consulting firm, as well as machine learning platform and development teams at Stats Perform, a leader in sports data and technology. Nathan holds bachelor's and master's degrees in Astrophysics from UC-Santa Cruz and UNC-Chapel Hill, respectively. When not building machine learning systems, Nathan spends as much time as possible with his favorite person in the world, his wife, as well as their four kids and two dogs, and enjoys getting outside to hike or garden and baking bread.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://nrfrank.github.io/

Bisi: https://bisi.gitbook.io/bisi/⁠

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Nathan on LinkedIn: https://www.linkedin.com/in/nrfrank

Timestamps:

[00:00] Nathan's preferred coffee

[00:40] Takeaways

[02:00] Please leave a review in our comment sections! Please like, share, and subscribe to our MLOps channels!

[03:00] Telescope for gamma-ray burst

[07:31] Transition into ML

[11:23] Stats-heavy US sports commentary

[14:25] Building a machine learning systems approach

[20:02] ML Workflow Must-Haves

[26:50] Love for tests

[33:10] Test Writing Importance

[34:37] Bridging Stakeholder Language Gap

[43:04] Shared Language, Team Collaboration

[47:28] Rapid-fire questions

[51:20] Wrap up

Nick Hasty is the Director of Product, Discovery & Machine Learning at Giphy, an animated-gif search engine that allows users to search, share, and discover GIFs.

MLOps podcast #198, Inferring Creativity.

// Abstract

Generative AI models have captured our imaginations with their ability to produce new "creative" works such as visually striking images, poems, stories, etc, and their outputs often rival or excel what most humans can do. I believe that these developments should make us rethink the nature of creativity itself, and through identifying parallels and differences between generative models and the human brain, we can establish a framework to talk about creativity and its relationship to intelligence that should hold up against future revelations in ML and neuroscience.

// Bio

Nick Hasty is a technologist & entrepreneur with a background in the creative arts. He was the founding engineer for GIPHY, where he’s worked for the last 10 years and now leads ML/AI product initiatives. He also serves as a consultant helping early-stage startups scale their product and engineering teams. Before GIPHY, he worked with arts+cultural organizations such as Rhizome.org and the Alan Lomax archives. He got his graduate degree from NYU’s ITP program.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: http://jnhasty.com/

Previous talks:https://engineering.giphy.com/giphy2vec-natural-language-processing-giphy/https://changelog.com/practicalai/38

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Nick on LinkedIn: https://www.linkedin.com/in/nickhasty/

Timestamps:

[00:00] Nick's preferred coffee

[00:15] Takeaways

[06:58] Nick's background in ML

[12:15] Nick's GIPHY journey

[17:39] Nick's Success Factors

[20:50] The trajectory of AI

[28:09] Identifying as a product engineer

[32:42] Evaluate LLMs vs. Traditional Models

[35:03] AI Product: Intuition vs Data

[38:53] Giphy AI Product Development

[45:25] Startups and Venture Assistance

[52:30] AI Funding Landscape Shift

[54:00] Wrap up

MLOps podcast #197 with Niels Bantilan, Chief Machine Learning Engineer at Union, The Role of Infrastructure in ML Leveraging Open Source, brought to us by Union.

// Abstract

When we start out building and deploying models in a new organization, life is simple: all I need to do is grab some data, iterate on a model that fits the data well, and perform reasonably well on some held-out test set. Then, if you’re fortunate enough to get to the point where you want to deploy it, it’s fairly straightforward to wrap it in an app framework and host it on a cloud server. However, once you get past this stage, you’re likely to find yourself needing: More scalable data processing frameworkExperiment tracking for modelsHeavier duty CPU/GPU hardwareVersioning tools to link models, data, code, and resource requirementsMonitoring tools for tracking data and model qualityThere’s a rich ecosystem of open-source tools that solves each of these problems and more: but how do you unify all of them together into a single view? This is where orchestration tools like Flyte can help. Flyte not only allows you to compose data and ML pipelines, but it also serves as “infrastructure as code” so that you can leverage the open-source ecosystem and unify purpose-built tools for different parts of the ML lifecycle on a single platform. ML systems are not just models: they are the models, data, and infrastructure combined.

// Bio

Niels is the Chief Machine Learning Engineer at Union.ai, and core maintainer of Flyte, an open-source workflow orchestration tool, author of UnionML, an MLOps framework for machine learning microservices, and creator of Pandera, a statistical typing and data testing tool for scientific data containers. His mission is to help data science and machine learning practitioners be more productive. He has a Master's in Public Health with a specialization in sociomedical science and public health informatics, and prior to that, a background in developmental biology and immunology. His research interests include reinforcement learning, AutoML, creative machine learning, and fairness, accountability, and transparency in automated systems.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://github.com/cosmicBboy, https://union.ai/

Flyte: https://flyte.org/

MLOps vs ML Orchestration // Ketan Umare // MLOps Podcast #183 - https://youtu.be/k2QRNJXyzFg

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Niels on LinkedIn: https://www.linkedin.com/in/nbantilan/

Timestamps:

[00:00] Niels' preferred coffee

[00:17] Takeaways

[03:45] Shout out to our Premium Brand Partner, Union!

[04:30] Pandera

[08:12] Creating a company

[14:22] Injecting ML for Data

[17:30] ML for Infrastructure Optimization

[22:17] AI Implementation Challenges

[24:25] Generative DevOps movement

[28:27] Pushing Limits: Code Responsibility

[29:46] Orchestration in OpenAI's Dev Day

[34:27] MLOps Stack: Layers & Challenges

[42:45] Mature Companies Embrace Kubernetes

[45:29] Horizon Challenges

[47:24] Flexible Integration for Resources

[49:10] MLOps Reproducibility Challenges

[53:14] MLOps Maturity Spectrum

[57:48] First-Class Citizens in Design

[1:00:16] Delegating for Efficient Collaboration

[1:04:55] Wrap up

Laurel Orr is a Principal Engineer at Numbers Station, a startup that applies Foundation Model technology to the enterprise data stack. Venky Orr is SVP of Product & Engineering.

MLOps podcast #196 with Numbers Station's Venky Ganti, SVP, Product & Engineering, and Principal Engineer, Laurel Orr, LLMs in Focus: From One-Size Fits All to Verticalized Solutions.

// Abstract

Dive into the realm of large language models (LLMs) as we explore the merits and limitations of 'one-size fits all' LLMs and their role in data analytics. Through customer stories, we showcase real-world applications and contrast general LLMs with verticalized, enterprise-centric models. We address the significance of ownership structures, with a focus on open-source vs proprietary impacts on transparency and trustworthiness. Delving into the NSQL foundation models, we emphasize the importance of diverse, quality training data, especially with enterprise challenges. Lastly, we speculate on the future of LLMs, highlighting hosting solutions and the evolution towards specialized challenges.

// Bio

Laurel Orr

Laurel Orr is a Principal Engineer at Numbers Station, a startup that applies Foundation Model technology to the enterprise data stack. Her research interests include how to use FMs to solve classically hard data-wrangling tasks and how to put FM technology into deployment. Before Numbers Station, Laurel was a postdoc at Stanford, advised by Chris Re as part of the Hazy Research Labs working in the intersection of AI and data management. She graduated with a PhD in database systems from the University of Washington.

Venky Orr

Venky brings over two decades of experience in software engineering and technical leadership to Numbers Station as SVP of Product & Engineering. Most recently, he served as General Manager, leading several initiatives on query understanding and commerce in the ads product area at Google. Before that, he was CEO and co-founder of Mesh Dynamics, the API test automation company, which was acquired by Google in 2021. Prior to Mesh Dynamics, Venky was CTO and co-founder of Alation, the enterprise data catalog company, where he led technology and helped create the new data catalog product category.

// MLOps Jobs board

jobs.mlops.community

// MLOps Swag/Merch

https://mlops-community.myshopify.com/

// Related Links

Website: https://www.numbersstation.ai/

⁠--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity

Sign up for the next meetup: https://go.mlops.community/register

Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/

Connect with Laurel on LinkedIn: https://www.linkedin.com/in/laurel-orr/

Connect with Venky on LinkedIn: https://www.linkedin.com/in/venky-ganti-2679a2/

Timestamps:

[00:00] Venky's and Laurel's preferred coffee

[00:36] Takeaways

[03:15] Please like, share, and subscribe to our MLOps channels!

[04:38] Venky's background

[07:47] Laurel's in the background

[09:38] Data wrangling

[13:45] Sequel query

[19:25] One-size-fits-all LLMs vs Verticalized and Specific LLMs

[23:42] Model Choice Trade-offs

[30:18] NSQL Foundational Models

[37:26] LLM Trends in 12 Months

[40:09] Data recipes being democratized

[45:16] Claude and 100,000 Context

[48:02] Exploring Varieties of LLMs

[50:02] AI Gateway

[51:07] Text-to-SQL Model Evaluation

[54:00] Wrap up