Sign up to save your podcasts
Or
Share The AI Tug-of-War: Bridging the Divide Between Platform Engineering and Data Science
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
The AI Tug-of-War: Bridging the Divide Between Platform Engineering and Data Science
Keith Maddox, co-lead of the Kubernetes AI Working Group, breaks down the architectural shifts and security challenges required to run enterprise AI agents at scale.
In this The Kubernetes AI Show episode, we chat with Keith Maddox, senior principal software engineer lead at Microsoft and Istio maintainer, who shares his perspective on the convergence of data science, AI agents, and platform engineering on Kubernetes AI workflows. He details the organizational dissonance between traditional platform stacks and data science workflows and how the Kubernetes AI working group is working to create a seamless migration path. We cover advanced model specialization techniques like Low Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG), which are crucial for enterprise use cases driven by data privacy and liability concerns.
Maddox also provides advice for platform owners, including the technical and non-technical strategies for LLM token spend management—recommending an egress gateway to centralize policy—and the importance of customer empathy with application developers. A major focus is the AI agent identity security gap, which falls between traditional human and machine identities. He strongly advocates for a zero trust AI mindset and immediate mitigation through agent sandboxing (using technologies like gVisor, KVM, or Wazet) and short-lived, ephemeral machine identities to manage the non-deterministic nature of LLMs.
Read the blog post: www.buoyant.io/ai-kubernetes-episode/the-ai-tug-of-war-bridging-the-divide-between-platform-engineering-and-data-science
Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/
Key Learnings
✓ The core conflict is a "tug of war" over tech stacks between platform and data science teams.
✓ Model specialization is necessary due to the high cost and lack of specificity of foundational models for enterprise applications.
✓ Managing LLM costs requires centralizing policy through an egress gateway and open communication with development teams.
✓ AI agents pose a new security challenge, requiring a move toward short-lived, ephemeral machine identities and agent sandboxing.
✓ A "Zero Trust" mindset is the recommended security approach for non-deterministic AI agents and workflows.
If you're building, deploying, or securing AI workflows, hit the Like button and subscribe for more deep-dive technical content!
Let us know in the comments: What is the biggest challenge your team is facing with AI agent identity and security today?
#PlatformEngineering #Kubernetes #AIAgents #LLMs #ZeroTrustAI #KubeCon #DataScience #TechSecurity #DevOps
View all episodes
By The AI Kubernetes ShowKeith Maddox, co-lead of the Kubernetes AI Working Group, breaks down the architectural shifts and security challenges required to run enterprise AI agents at scale.
In this The Kubernetes AI Show episode, we chat with Keith Maddox, senior principal software engineer lead at Microsoft and Istio maintainer, who shares his perspective on the convergence of data science, AI agents, and platform engineering on Kubernetes AI workflows. He details the organizational dissonance between traditional platform stacks and data science workflows and how the Kubernetes AI working group is working to create a seamless migration path. We cover advanced model specialization techniques like Low Rank Adaptation (LoRA) and Retrieval-Augmented Generation (RAG), which are crucial for enterprise use cases driven by data privacy and liability concerns.
Maddox also provides advice for platform owners, including the technical and non-technical strategies for LLM token spend management—recommending an egress gateway to centralize policy—and the importance of customer empathy with application developers. A major focus is the AI agent identity security gap, which falls between traditional human and machine identities. He strongly advocates for a zero trust AI mindset and immediate mitigation through agent sandboxing (using technologies like gVisor, KVM, or Wazet) and short-lived, ephemeral machine identities to manage the non-deterministic nature of LLMs.
Read the blog post: www.buoyant.io/ai-kubernetes-episode/the-ai-tug-of-war-bridging-the-divide-between-platform-engineering-and-data-science
Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/
Key Learnings
✓ The core conflict is a "tug of war" over tech stacks between platform and data science teams.
✓ Model specialization is necessary due to the high cost and lack of specificity of foundational models for enterprise applications.
✓ Managing LLM costs requires centralizing policy through an egress gateway and open communication with development teams.
✓ AI agents pose a new security challenge, requiring a move toward short-lived, ephemeral machine identities and agent sandboxing.
✓ A "Zero Trust" mindset is the recommended security approach for non-deterministic AI agents and workflows.
If you're building, deploying, or securing AI workflows, hit the Like button and subscribe for more deep-dive technical content!
Let us know in the comments: What is the biggest challenge your team is facing with AI agent identity and security today?
#PlatformEngineering #Kubernetes #AIAgents #LLMs #ZeroTrustAI #KubeCon #DataScience #TechSecurity #DevOps