Workflow orchestration has always been a pain for data scientists, but this is exacerbated in these AI hype days by agentic workflows executing arbitrary (not pre-defined) workflows with a variety of failure modes. Adam from Prefect joins us to talk through their open source Python library for orchestration and visibility into python-based pipelines. Along the way, he introduces us to things like Marvin, their AI engineering framework, and ControlFlow, their agent workflow system.

Sponsors:

• WorkOS – A platform that gives developers a set of building blocks for quickly adding enterprise-ready features to their application. Add Single Sign-On (Okta, Azure, Google, Microsoft OAuth), sync users from any SCIM directory, HRIS integration, audit trails (SIEM), free magic link sign-in. WorkOS is designed for developers and offers a single, elegant interface that abstracts dozens of enterprise integrations. Learn more and get started at WorkOS.com
• Shopify – Sign up for a $1/month trial period at shopify.com/practicalai
• Notion – Notion is a place where any team can write, plan, organize, and rediscover the joy of play. It’s a workspace designed not just for making progress, but getting inspired. Notion is for everyone — whether you’re a Fortune 500 company or freelance designer, starting a new startup or a student juggling classes and clubs. 

Featuring:

• Adam Azzam – LinkedIn, X
• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Prefect
• Marvin
• ControlFlow

Upcoming Events: 

• Register for upcoming webinars here!

As Argilla puts it: “Data quality is what makes or breaks AI.” However, what exactly does this mean and how can AI team probably collaborate with domain experts towards improved data quality? David Berenstein & Ben Burtenshaw, who are building Argilla & Distilabel at Hugging Face, join us to dig into these topics along with synthetic data generation & AI-generated labeling / feedback.

Sponsors:

• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. 
• WorkOS – A platform that gives developers a set of building blocks for quickly adding enterprise-ready features to their application. Add Single Sign-On (Okta, Azure, Google, Microsoft OAuth), sync users from any SCIM directory, HRIS integration, audit trails (SIEM), free magic link sign-in. WorkOS is designed for developers and offers a single, elegant interface that abstracts dozens of enterprise integrations. Learn more and get started at WorkOS.com
• Eight Sleep – Take your sleep and recovery to the next level. Go to eightsleep.com/PRACTICALAI and use the code PRACTICALAI to get $350 off your very own Pod 4 Ultra. You can try it for free for 30 days - but we’re confident you will not want to return it. Once you experience AI-optimized sleep, you’ll wonder how you ever slept without it. Currently shipping to: United States, Canada, United Kingdom, Europe, and Australia. 

Featuring:

• Ben Burtenshaw – GitHub, LinkedIn, X
• David Berenstein – GitHub, LinkedIn, X
• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Argilla
• Distilabel
• Synthetic Data Generator UI
• Hugging Face and Argilla meetups

Upcoming Events: 

• Register for upcoming webinars here!

We are constantly hearing about disillusionment as it relates to AI. Some of that is probably valid, but Mike Lewis, an AI architect from Cincinnati, has proven that he can consistently get LLM and GenAI apps to the point of real enterprise value (even with the Big Cos of the world). In this episode, Mike joins us to share some stories from the AI trenches & highlight what it takes (practically) to show what is possible, doable & scalable with AI.

Sponsors:

• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. 
• WorkOS – A platform that gives developers a set of building blocks for quickly adding enterprise-ready features to their application. Add Single Sign-On (Okta, Azure, Google, Microsoft OAuth), sync users from any SCIM directory, HRIS integration, audit trails (SIEM), free magic link sign-in. WorkOS is designed for developers and offers a single, elegant interface that abstracts dozens of enterprise integrations. Learn more and get started at WorkOS.com
• Eight Sleep – Take your sleep and recovery to the next level. Go to eightsleep.com/PRACTICALAI and use the code PRACTICALAI to get $350 off your very own Pod 4 Ultra. You can try it for free for 30 days - but we’re confident you will not want to return it. Once you experience AI-optimized sleep, you’ll wonder how you ever slept without it. Currently shipping to: United States, Canada, United Kingdom, Europe, and Australia. 

Featuring:

• Mike Lewis – LinkedIn
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Synthoni

Upcoming Events: 

• Register for upcoming webinars here!

Seems like we are hearing a lot about GraphRAG these days, but there are lots of questions: what is it, is it hype, what is practical? One of our all time favorite podcast friends, Prashanth Rao, joins us to dig into this topic beyond the hype. Prashanth gives us a bit of background and practical use cases for GraphRAG and graph data.

Sponsors:

• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. 
• Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. 
• Speakeasy – Production-ready, enterprise-resilient, best-in-class SDKs crafted in minutes. Speakeasy takes care of the entire SDK workflow to save you significant time, delivering SDKs to your customers in minutes with just a few clicks! Create your first SDK for free!

Featuring:

• Prashanth Rao – Website, GitHub, LinkedIn, X
• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Kùzu: A highly scalable, extremely fast, easy-to-use embeddable, open source graph database: GitHub repo
• The goals and vision of Kùzu: Blog post
• Kùzu YouTube channel
• Graph RAG strategies with Kùzu: GitHub repo
• Prashanth Rao’s blog: thedataquarry.com

Upcoming Events: 

• Register for upcoming webinars here!

Recently the company stewarding the open source library scikit-learn announced their seed funding. Also, OpenAI released “o1” with new behavior in which it pauses to “think” about complex tasks. Chris and Daniel take some time to do their own thinking about o1 and the contrast to the scikit-learn ecosystem, which has the goal to promote “data science that you own.”

Sponsors:

• Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. 
• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. 
• Speakeasy – Production-ready, enterprise-resilient, best-in-class SDKs crafted in minutes. Speakeasy takes care of the entire SDK workflow to save you significant time, delivering SDKs to your customers in minutes with just a few clicks! Create your first SDK for free!

Featuring:

• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Probabl seed funding announcement
• OpenAI o1 announcement
• Purdue Data4Good competition
• MLOps Community homepage and Slack workspace
• Latent Space homepage and Discord server

Upcoming Events: 

• Register for upcoming webinars here!

Dinis Cruz drops by to chat about cybersecurity for generative AI and large language models. In addition to discussing The Cyber Boardroom, Dinis also delves into cybersecurity efforts at OWASP and that organization’s Top 10 for LLMs and Generative AI Apps.

Sponsors:

• Speakeasy – Production-ready, enterprise-resilient, best-in-class SDKs crafted in minutes. Speakeasy takes care of the entire SDK workflow to save you significant time, delivering SDKs to your customers in minutes with just a few clicks! Create your first SDK for free!
• Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes. 
• Shopify – Sign up for a $1/month trial period at shopify.com/practicalai

Featuring:

• Dinis Cruz – Website, GitHub, LinkedIn, X
• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• The Cyber Boardroom
• OWASP - Top 10 for LLMs and Generative AI Apps
• OWASP - Generative AI

Upcoming Events: 

• Register for upcoming webinars here!

GenAI is often what people think of when someone mentions AI. However, AI is much more. In this episode, Daniel breaks down a history of developments in data science, machine learning, AI, and GenAI in this episode to give listeners a better mental model. Don’t miss this one if you are wanting to understand the AI ecosystem holistically and how models, embeddings, data, prompts, etc. all fit together.

Sponsors:

• Speakeasy – Production-ready, enterprise-resilient, best-in-class SDKs crafted in minutes. Speakeasy takes care of the entire SDK workflow to save you significant time, delivering SDKs to your customers in minutes with just a few clicks! Create your first SDK for free!

Featuring:

• Daniel Whitenack – Website, GitHub, X

Show Notes:

• scikit-learn
• AutoML in Google Cloud

Upcoming Events: 

• Register for upcoming webinars here!

How do you systematically measure, optimize, and improve the performance of LLM applications (like those powered by RAG or tool use)? Ragas is an open source effort that has been trying to answer this question comprehensively, and they are promoting a “Metrics Driven Development” approach. Shahul from Ragas joins us to discuss Ragas in this episode, and we dig into specific metrics, the difference between benchmarking models and evaluating LLM apps, generating synthetic test data and more.

Sponsors:

• Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. 

Featuring:

• Shahul Es – GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• Ragas

Upcoming Events: 

• Register for upcoming webinars here!

If you have questions at the intersection of Cybersecurity and AI, you need to know Donato at WithSecure! Donato has been threat modeling AI applications and seriously applying those models in his day-to-day work. He joins us in this episode to discuss his LLM application security canvas, prompt injections, alignment, and more.

Sponsors:

• Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. 
• Porkbun – Go to porkbun.com to get .app, .dev, or .foo domain names at Porkbun for only $1 for the first year! 
• Changelog News – A podcast+newsletter combo that’s brief, entertaining & always on-point. Subscribe today. 

Featuring:

• Donato Capitella – Website
• Daniel Whitenack – Website, GitHub, X

Show Notes:

• The Security Canvas for LLM Applications
• LLM Security Chronicles Playlist on YouTube
• Should you let ChatGPT control your browser?
• When your AI assistant has an evil twin
• Pliny the Liberator - jailbreak guru

Upcoming Events: 

• Register for upcoming webinars here!

You might have heard that “AI is only as good as the data.” What does that mean and what data are we talking about? Chris and Daniel dig into that topic in the episode exploring the categories of data that you might encounter working in AI (for training, testing, fine-tuning, benchmarks, etc.). They also discuss the latest developments in AI regulation with the EU’s AI Act coming into force.

Sponsors:

• Assembly AI – Turn voice data into summaries with AssemblyAI’s leading Speech AI models. Built by AI experts, their Speech AI models include accurate speech-to-text for voice data (such as calls, virtual meetings, and podcasts), speaker detection, sentiment analysis, chapter detection, PII redaction, and more. 
• Changelog News – A podcast+newsletter combo that’s brief, entertaining & always on-point. Subscribe today. 

Featuring:

• Chris Benson – Website, GitHub, LinkedIn, X
• Daniel Whitenack – Website, GitHub, X

Upcoming Events: 

• Register for upcoming webinars here!