As AI models grow more powerful, the question of where they run is becoming just as important as what they do. In this episode, Brandon Weng, Co-Founder and CEO of Fluid Inference, unpacks what it takes to move AI from massive data centers to everyday devices—and why that shift matters.

Brandon shares the story behind Fluid Inference, a company focused on making it easier for developers to deploy large AI models like transformers on consumer hardware. From pivoting away from his previous project, Slipbox, to the technical and philosophical choices that shaped Fluid's direction, he walks us through the thinking behind local-first AI. We explore the tradeoffs between cloud-based and on-device inference—touching on privacy, cost, control, and performance—and the hardware breakthroughs that are making edge AI more viable, including integrated NPUs in devices like Intel's AI PCs.

#EdgeAI #OnDeviceInference #AIOptimization #PrivacyFirst #OpenSourceAI #LocalAI