Sign up to save your podcasts
Or
Share Agentic AI Security on Kubernetes: Understanding Payload Routing and Defense-in-Depth
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Agentic AI Security on Kubernetes: Understanding Payload Routing and Defense-in-Depth
Is your Kubernetes cluster ready for the next generation of AI workloads? Join Buoyant Tech Evangelist Flynn and Red Hat's Shane Utt from the AI Gateway Working Group as they reveal the critical shift from header-based to payload-based Kubernetes Networking for Agentic AI.
In this episode of The Kubernetes AI Show, we dive deep into the challenges of running AI workloads on cloud native infrastructure with AI Gateway leaders, Flynn and Shane. They discuss how the AI Gateway Working Group evolved from the Gateway API inference extension (GIE) to focus on the average Kubernetes user who is often performing inference via egress outside the cluster. A key challenge is that AI models are fundamentally different from microservices—they are not interchangeable, making advanced routing a must.
The conversation highlights a critical shift: networking infrastructure must now be retrofitted for payload processing for routing to ensure security, compliance (like GDPR), and proper model selection. We explore parallels between AI’s hardware demands and High Performance Computing (HPC), revealing the problem of low LLM hardware utilization and the talent gap between data scientists and network engineers. Finally, the co-leads detail the security implications of Agentic AI's autonomy, advocating for a defense-in-depth security strategy that includes multi-cluster routing and Model Context Protocol (MCP) servers with 'elicitation' to prevent accidental, destructive actions.
Read the blog post: www.buoyant.io/ai-kubernetes-episode/agentic-ai-security-on-kubernetes-understanding-payload-routing-and-defense-in-depth
Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/
Key Learnings
✓ Learn why AI models require advanced, non-interchangeable routing logic compared to standard microservices.
✓ Discover how the shift to payload processing is essential for semantic routing, security, and GDPR compliance.
✓ Understand why AI’s low hardware utilization (20-30%) is a known problem from High Performance Computing (HPC).
✓ See how Agentic AI introduces a "deeply terrifying" security risk that demands a defense-in-depth security approach like MCP elicitation.
If you're building or securing AI workloads on Kubernetes, you can't afford to miss this discussion! Subscribe to The Kubernetes AI Show for more foundational insights, like this one from the AI Gateway leaders.
Hit the like button and let us know in the comments: What is the most critical challenge you are facing when building Kubernetes Networking for your Agentic AI applications?
#KubernetesNetworking #AIGateway #AgenticAI #CloudNative #GatewayAPI #LLMs #K8s #Microservices #TechEvangelist
View all episodes
By The AI Kubernetes ShowIs your Kubernetes cluster ready for the next generation of AI workloads? Join Buoyant Tech Evangelist Flynn and Red Hat's Shane Utt from the AI Gateway Working Group as they reveal the critical shift from header-based to payload-based Kubernetes Networking for Agentic AI.
In this episode of The Kubernetes AI Show, we dive deep into the challenges of running AI workloads on cloud native infrastructure with AI Gateway leaders, Flynn and Shane. They discuss how the AI Gateway Working Group evolved from the Gateway API inference extension (GIE) to focus on the average Kubernetes user who is often performing inference via egress outside the cluster. A key challenge is that AI models are fundamentally different from microservices—they are not interchangeable, making advanced routing a must.
The conversation highlights a critical shift: networking infrastructure must now be retrofitted for payload processing for routing to ensure security, compliance (like GDPR), and proper model selection. We explore parallels between AI’s hardware demands and High Performance Computing (HPC), revealing the problem of low LLM hardware utilization and the talent gap between data scientists and network engineers. Finally, the co-leads detail the security implications of Agentic AI's autonomy, advocating for a defense-in-depth security strategy that includes multi-cluster routing and Model Context Protocol (MCP) servers with 'elicitation' to prevent accidental, destructive actions.
Read the blog post: www.buoyant.io/ai-kubernetes-episode/agentic-ai-security-on-kubernetes-understanding-payload-routing-and-defense-in-depth
Follow us on LinkedIn: https://www.linkedin.com/company/the-ai-kubernetes-show/
Key Learnings
✓ Learn why AI models require advanced, non-interchangeable routing logic compared to standard microservices.
✓ Discover how the shift to payload processing is essential for semantic routing, security, and GDPR compliance.
✓ Understand why AI’s low hardware utilization (20-30%) is a known problem from High Performance Computing (HPC).
✓ See how Agentic AI introduces a "deeply terrifying" security risk that demands a defense-in-depth security approach like MCP elicitation.
If you're building or securing AI workloads on Kubernetes, you can't afford to miss this discussion! Subscribe to The Kubernetes AI Show for more foundational insights, like this one from the AI Gateway leaders.
Hit the like button and let us know in the comments: What is the most critical challenge you are facing when building Kubernetes Networking for your Agentic AI applications?
#KubernetesNetworking #AIGateway #AgenticAI #CloudNative #GatewayAPI #LLMs #K8s #Microservices #TechEvangelist