Join us as Du'An digs into the real mechanics of running AI locally and in production - from GPU memory math to multi-agent architectures, observability, and the economics of self-hosted inference.

Du'An walks through how model weights and KV cache compete for GPU memory, why continuous batching matters when you have more than a handful of users, and how agent architectures like single-agent, workflow, graph, swarm, and supervisor patterns each solve different problems. You will learn how to instrument your agents with Langfuse for observability and cost tracking, when to use Ollama versus vLLM, how prompt caching can cut provider costs by up to 75%, and why GPUs should never sit idle. Episode two of three - the next episode covers deploying at scale.

Timestamps

0:00 Welcome & Introduction

1:47 Du'An's New Role at Akamai Cloud

3:10 Data Privacy and the Case for Self-Hosted AI

7:21 Anthropic and OpenAI as the New Cloud Layer

• 12:48 Local Models for Specific Use Cases - Cancer Detection Example

How to find Du'An:

https://duanlightfoot.com

https://www.linkedin.com/in/duanlightfoot/

Links from the show:

https://langfuse.com/

https://github.com/akamai-developers/akamai-workshop-solution-architect-agent

https://amzn.to/4bvHn1p

https://vllm.ai/