Agentic Conversations (formally mlops.community)

Streaming Ecosystem Complexities and Cost Management // MLOps Podcast #302 with Rohit Agrawal, Director of Engineering at Tecton.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Demetrios talks with Rohit Agrawal, Director of Engineering at Tecton, about the challenges and future of streaming data in ML. Rohit shares his path at Tecton and insights on managing real-time and batch systems. They cover tool fragmentation (Kafka, Flink, etc.), infrastructure costs, managed services, and trends like using S3 for storage and Iceberg as the GitHub for data. The episode wraps with thoughts on BYOC solutions and evolving data architectures.

// Bio

Rohit Agrawal is an Engineering Manager at Tecton, leading the Real-Time Execution team. Before Tecton, Rohit was a Lead Software Engineer at Salesforce, where he focused on transaction processing and storage in OLTP relational databases. He holds a Master’s Degree in Computer Systems from Carnegie Mellon University and a Bachelor’s Degree in Electrical Engineering from the Biria Institute of Technology and Science in Pilani, India.

// Related Links

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Rohit on LinkedIn: /agrawalrohit10

Timestamps:

[00:00] Rohit's preferred coffee

[00:34] Takeaways

[01:25] Data Streaming

[06:50] Optimizing for Speed

[09:17] DuckDB vs Spark Flink

[13:09] Optimizing Feature Engineering Best Practices

[16:42] Checkpointing Frequency Guide

[23:08] Streaming Tips and Tricks

[30:00] Cloud Costs vs Human Costs

[32:09] Race to Learn

[39:24] Right-Sized Engineering Practices

[42:06] Streaming Simplicity vs Complexity

[47:02] Wrap up

Building Trust Through Technology: Responsible AI in Practice // MLOps Podcast #301 with Rafael Sandroni, Founder and CEO of GardionAI.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Rafael Sandroni shares key insights on securing AI systems, tackling fraud, and implementing robust guardrails. From prompt injection attacks to AI-driven fraud detection, we explore the challenges and best practices for building safer AI.

// Bio

Entrepreneur and problem solver.

// Related Links

GardionAI LinkedIn: https://www.linkedin.com/company/guardionai/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Rafael on LinkedIn: /rafaelsandroni

Timestamps:

[00:00] Rafael's preferred coffee

[00:16] Takeaways

[01:03] AI Assistant Best Practices

[03:48] Siri vs In-App AI

[08:44] AI Security Exploration

[11:55] Zero Trust for LLMS

[18:02] Indirect Prompt Injection Risks

[22:42] WhatsApp Banking Risks

[26:27] Traditional vs New Age Fraud

[29:12] AI Fraud Mitigation Patterns

[32:50] Agent Access Control Risks

[34:31] Red Teaming and Pentesting

[39:40] Data Security Paradox[

40:48] Wrap up

Beyond the Matrix: AI and the Future of Human Creativity // MLOps Podcast #300 with Fausto Albers, AI Engineer & Community Lead at AI Builders Club.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Fausto Albers discusses the intersection of AI and human creativity. He explores AI’s role in job interviews, personalized AI assistants, and the evolving nature of human-computer interaction. Key topics include AI-driven self-analysis, context-aware AI systems, and the impact of AI on optimizing human decision-making. The conversation highlights how AI can enhance creativity, collaboration, and efficiency by reducing cognitive load and making intelligent suggestions in real time.

// Bio

Fausto Albers is a relentless explorer of the unconventional—a techno-optimist with a foundation in sociology and behavioral economics, always connecting seemingly absurd ideas that, upon closer inspection, turn out to be the missing pieces of a bigger puzzle. He thrives in paradox: he overcomplicates the simple, oversimplifies the complex, and yet somehow lands on solutions that feel inevitable in hindsight. He believes that true innovation exists in the tension between chaos and structure—too much of either, and you’re stuck. His career has been anything but linear. He’s owned and operated successful restaurants, served high-stakes cocktails while juggling bottles on London’s bar tops, and later traded spirits for code—designing digital waiters, recommender systems, and AI-driven accounting tools. Now, he leads the AI Builders Club Amsterdam, a fast-growing community where AI engineers, researchers, and founders push the boundaries of intelligent systems. Ask him about RAG, and he’ll insist on specificity—because, as he puts it, discussing retrieval-augmented generation without clear definitions is as useful as declaring that “AI will have an impact on the world.” An engaging communicator, a sharp systems thinker, and a builder of both technology and communities, Fausto is here to challenge perspectives, deconstruct assumptions, and remix the future of AI.

// Related Links

Website: aibuilders.club

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Fausto on LinkedIn: /stepintoliquid

Timestamps:

[00:00] Fausto's preferred coffee

[01:11] Takeaways

[02:03] Audio to Text GPT

[05:55] AI Contextual Profiling

[10:13] Vertex AI Trade-Offs

[13:15] AI Context in Communities

[21:35] Conversation Matchmaking for Meetups

[27:00] AI UX and Cognitive Load

[38:42] Chunks and Reasoning in RAG

[42:00] Context Iteration and Reasoning

[45:32] RAG System and Embeddings

[50:40] RAG System for Engineering

[54:07] Wrap up

Building Trust Through Technology: Responsible AI in Practice // MLOps Podcast #299 with Animesh Singh, Executive Director, AI Platform and Infrastructure of LinkedIn.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Animesh discusses LLMs at scale, GPU infrastructure, and optimization strategies. He highlights LinkedIn's use of LLMs for features like profile summarization and hiring assistants, the rising cost of GPUs, and the trade-offs in model deployment. Animesh also touches on real-time training, inference efficiency, and balancing infrastructure costs with AI advancements. The conversation explores the evolving AI landscape, compliance challenges, and simplifying architecture to enhance scalability and talent acquisition.

// Bio

Executive Director, AI and ML Platform at LinkedIn | Ex IBM Senior Director and Distinguished Engineer, Watson AI and Data | Founder at Kubeflow | Ex LFAI Trusted AI NA Chair

Animesh is the Executive Director leading the next-generation AI and ML Platform at LinkedIn, enabling the creation of the AI Foundation Models Platform, serving the needs of 930+ million members of LinkedIn. Building Distributed Training Platforms, Machine Learning Pipelines, Feature Pipelines, Metadata engines, etc. Leading the creation of the LinkedIn GAI platform for fine-tuning, experimentation, and inference needs. Animesh has more than 20 patents and 50+ publications.

Past IBM Watson AI and Data Open Tech CTO, Senior Director, and Distinguished Engineer, with 20+ years of experience in the Software industry, and 15+ years in AI, Data, and Cloud Platform. Led globally dispersed teams, managed globally distributed projects, and served as a trusted adviser to Fortune 500 firms. Played a leadership role in creating, designing, and implementing Data and AI engines for AI and ML platforms, led Trusted AI efforts, and drove the strategy and execution for Kubeflow, OpenDataHub, and execution in products like Watson OpenScale and Watson Machine Learning.

// Related Links

Composable Memory for GPU Optimization // Bernie Wu // Pod #270 - https://youtu.be/ccaDEFoKwko

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Animesh on LinkedIn: /animeshsingh1

Timestamps:

[00:00] Animesh's preferred coffee

[00:16] Takeaways

[02:12] What is working?

[07:00] What's not working?

[13:40] LLM vs Rexis Efficiency

[21:49] GPU Utilization and Architecture

[27:32] GPU reliability concerns

[36:50] Memory Bottleneck in AI

[41:06] Optimizing LLM Checkpointing

[46:51] Checkpoint Offloading and Platform Design

[54:55] Workflow Divergence Points

[58:41] Wrap up

Building Trust Through Technology: Responsible AI in Practice // MLOps Podcast #298 with Allegra Guinan, Co-founder of Lumiera.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Allegra joins the podcast to discuss how Responsible AI (RAI) extends beyond traditional pillars like transparency and privacy. While these foundational elements are crucial, true RAI success requires deeply embedding responsible practices into organizational culture and decision-making processes. Drawing from Lumiera's comprehensive approach, Allegra shares how organizations can move from checkbox compliance to genuine RAI integration that drives innovation and sustainable AI adoption.

// Bio

Allegra is a technical leader with a background in managing data and enterprise engineering portfolios. Having built her career bridging technical teams and business stakeholders, she's seen the ins and outs of how decisions are made across organizations. She combines her understanding of data value chains, passion for responsible technology, and practical experience guiding teams through complex implementations into her role as co-founder and CTO of Lumiera.

// Related Links

Website: https://www.lumiera.ai/

Weekly newsletter: https://lumiera.beehiiv.com/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Allegra on LinkedIn: /allegraguinan

Timestamps:

[00:00] Allegra's preferred coffee

[00:14] Takeaways

[01:11] Responsible AI principles

[03:13] Shades of Transparency

[07:56] Effective questioning for clarity

[11:17] Managing stakeholder input effectively

[14:06] Business to Tech Translation

[19:30] Responsible AI challenges

[23:59] Successful plan vs Retroactive responsibility

[28:38] AI product robustness explained

[30:44] AI transparency vs Engagement

[34:10] Efficient interaction preferences

[37:57] Preserving human essence

[39:51] Conflict and growth in life

[46:02] Subscribe to Allegra's Weekly Newsletter!

I Let An AI Play Pokémon! - Claude plays Pokémon Creator // MLOps Podcast #297 with David Hershey, Member of Technical Staff at Anthropic.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Demetrios chats with David Hershey from Anthropic's Applied AI team about his agent-powered Pokémon project using Claude. They explore agent frameworks, prompt optimization vs. fine-tuning, and AI's growing role in software, legal, and accounting fields. David highlights how managed AI platforms simplify deployment, making advanced AI more accessible.

// Bio

David Hershey devoted most of his career to machine learning infrastructure and trying to abstract away the hairy systems complexity that gets in the way of people building amazing ML applications.

// Related Links

Website: https://www.davidhershey.com/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with David on LinkedIn: /david-hershey-458ab081

Timestamps:

[00:00] David's preferred coffee

[00:07] Takeaways

[00:52] Claude Plays Pokémon insights

[07:07] AI Agent Framework design

[12:45] Fine-tuning vs prompting

[17:25] Model Cost vs Inference Cost

[22:20] Prompt vs Fine-Tuning

[32:34] Model Updates and Prompting

[36:09] AI Evolution and Reflection

[40:11] Cognitive Load in UX

[44:35] Outsourcing ML in Finance

[46:38] Wrap up

From Rules to Reasoning Engines // MLOps Podcast #297 with George Mathew, Managing Director at Insight Partners.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

George Mathew (Insight Partners) joins Demetrios to break down how AI and ML have evolved over the past few years and where they’re headed. He reflects on the major shifts since his last chat with Demetrios, especially how models like ChatGPT have changed the game.

George dives into "generational outcomes"—building companies with lasting impact—and the move from rule-based software to AI-driven reasoning engines. He sees AI becoming a core part of all software, fundamentally changing business operations.

The chat covers the rise of agent-based systems, the importance of high-quality data, and recent breakthroughs like Deep SEQ, which push AI reasoning further. They also explore AI’s future—its role in software, enterprise adoption, and everyday life.

// Bio

George Mathew is a Managing Director at Insight Partners focused on venture stage investments in AI, ML, Analytics, and Data companies as they are establishing product/market fit.

He brings 20+ years of experience developing high-growth technology startups, including most recently being CEO of Kespry. Prior to Kespry, George was President & COO of Alteryx, where he scaled the company through its IPO (AYX). Previously, he held senior leadership positions at SAP and Salesforce.com. He has driven company strategy, led product management and development, and built sales and marketing teams.

George holds a Bachelor of Science in Neurobiology from Cornell University and a Master's in Business Administration from Duke University, where he was a Fuqua Scholar.

// Related Links

Website: https://www.insightpartners.com/

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with George on LinkedIn: /gmathew

Timestamps:

[00:00] George's preferred coffee

[00:17] Takeaways

[00:58] AI and Evolution

[04:41] Beyond Chat AI

[09:52] Mechanistic vs Humane AI

[14:27] Legacy Systems vs Agents

[20:43] Human Patience and AI Reasoning

[22:36] Creative Hallucination Pipeline

[26:28] DeepSeek Impact on Providers

[31:33] Model Investment Uncertainty

[33:12] Mistral IPO Surprise

[43:37] Language-based Software Interaction

[47:41] Spark and Databricks Collaboration

[52:08] Generational AI Companies

[58:13] AI as a Companion

[1:04:56] Wrap up

GenAI Traffic: Why API Infrastructure Must Evolve... Again // MLOps Podcast #296 with Erica Hughberg, Community Advocate at Tetrate.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

The way we handle API traffic is broken for GenAI. We've spent years optimizing for microservices—fast, stateless, and lightweight API calls. But GenAI changes everything. Requests are slower, heavier, and more complex, requiring long-lived connections, massive payloads, and streaming responses. Suddenly, traditional API gateways are struggling—timeout limits are too short, rate limiting models don’t fit, and payload constraints are blocking innovation.

In this episode, we unpack the new challenges of GenAI traffic and why infrastructure must evolve—again. We look back at previous API shifts, from the C10K problem to the monolith-to-microservices revolution, and how they reshaped networking. Now, AI-driven workloads demand a new kind of API gateway—one that handles token-based rate limiting, cost-aware request shaping, and scalable AI inference traffic.

// Bio

Erica Hughberg is a technical leader and community advocate passionate about helping engineering teams build scalable, secure, and human-centric application platforms. With a background in software engineering and a deep understanding of cloud-native technologies, she specializes in driving the adoption of open-source projects like Envoy Gateway, Istio, and Kubernetes Gateway API, which enable organizations to simplify traffic management, security, and API distribution.

As a maintainer of Envoy AI Gateway, she plays a key role in shaping the future of API infrastructure. She focuses on features to ensure organizations can securely and efficiently integrate AI-powered services while simplifying traffic management, security, and API distribution. In the Envoy community, she drives collaboration, mentorship, and contributions that advance the project and its adoption.

Lastly, as a believer in the power of storytelling, Erica enjoys translating complex technical concepts into engaging, accessible narratives in the form of social media posts, conference talks, podcasts, and educational content.

// Related Links

Efficient Deployment of Models at the Edge // Krishna Sridhar // MLOps Podcast #284 - https://youtu.be/sFqm7GTeulg

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Erica on LinkedIn: /ericahughberg

Timestamps:

[00:00] Erica's preferred coffee

[00:30] Takeaways

[01:50] Evolving Web Gateways

[14:35] Microservices to LLM Shift

[17:42] Intelligence Privacy Model

[22:26] Infrastructure for AI Creativity

[25:25] AI Gateway Networking Challenges

[30:37] Streamlit MVP to Production

[43:03] AI Model Scaling Challenges

[47:48] Tech Advocacy and Skills

[53:17] Optimizing Edge AI Performance

[56:43] Product Management Insights

[1:00:02] Navigating Evolving Tech Challenges

[1:04:35] Wrap up

The Unbearable Lightness of Data // MLOps Podcast #295 with Rohit Krishnan, Chief Product Officer at bodo.ai.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Rohit Krishnan, Chief Product Officer at Bodo.AI, joins Demetrios to discuss AI's evolving landscape. They explore interactive reasoning models, AI's impact on jobs, scalability challenges, and the path to AGI. Rohit also shares insights on Bodo.AI’s open-source move and its impact on data science.

// Bio

Building products, writing, messing around with AI pretty much everywhere

// Related Links

Website: www.strangeloopcanon.com

In life, my kids. Professionally, https://github.com/bodo-ai/Bodo ... Otherwise, personally, it's writing every single day at strangeloopcanon.com!

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Rohit on LinkedIn: /rkris

Timestamps:

[00:00] Rohit's preferred coffee

[00:12] Takeaways

[00:46] DeepSeek UI Insight

[07:07] Signal vs. Noise Feedback

[09:24] Report refinement process

[11:54] Cognitive load and model control

[15:27] Future AI possibilities

[18:47] AI data labeling jobs

[21:27] AI noise and overhead

[26:12] Information Overload & Attention

[29:52] AGI Progress Vectors Explained

[37:28] GPU Memory Bottlenecks in AI

[46:23] AI vs ML Engineering

[50:22] Parsing PDFs with Metadata

[53:24] Wrap up

Kubernetes, AI Gateways, and the Future of MLOps // MLOps Podcast #294 with Alexa Griffith, Senior Software Engineer at Bloomberg.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Alexa shares her journey into software engineering, from early struggles with Airflow and Kubernetes to leading open-source projects like the Envoy AI Gateway. She and Demetrios discuss AI model deployment, tooling differences across tech roles, and the importance of abstraction. They highlight aligning technical work with business goals and improving cross-team communication, offering key insights into MLOps and AI infrastructure.

// Bio

Alexa Griffith is a Senior Software Engineer at Bloomberg, where she builds scalable inference platforms for machine learning workflows and contributes to open-source projects like KServe. She began her career at Bluecore, working in data science infrastructure, and holds an honors degree in Chemistry from the University of Tennessee, Knoxville. She shares her insights through her podcast, Alexa’s Input (AI), technical blogs, and active engagement with the tech community at conferences and meetups.

// Related Links

Website: https://alexagriffith.com/

Kubecon Keynote about Envoy AI Gateway https://www.youtube.com/watch?v=do1viOk8nok

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity][https://x.com/mlopscommunity] or LinkedIn [https://go.mlops.community/linkedin]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Alexa on LinkedIn: /alexa-griffith

Timestamps:

[00:00] Alexa's preferred coffee

[00:13] Takeaways

[02:39] Airflow Kubernetes Pain Points

[07:35] Pipelining Tools Landscape

[12:11] KubeCon AI Gateway Keynote

[14:21] Envoy vs Envoy AI Gateway

[20:53] AI Model Routing Flexibility

[25:15] Celery and Airflow Integration

[26:25] Open Source Contribution Tips

[32:03] ML Platform Adoption Tips

[37:20] Simplicity Over Complexity

[40:42] Engaging Non-Technical Stakeholders

[44:19] ML vs LLM Platform Divergence

[48:27] LLM vs ML Models

[49:47] Wrap up