
Sign up to save your podcasts
Or


Can a model spend more time reasoning without generating more tokens?
In the first Architecting Reasoning talk, we explore looped transformers: how they reuse layers for iterative computation, how they differ from standard transformers, and what recurrent depth could mean for reasoning and test-time compute. We also discuss ideas from Universal Transformers, Looped Transformers as Programmable Computers, and recent work on latent reasoning.
Before you can understand the KV cache, you need to understand how LLMs build context. That's what this episode is about.
In Part 2A of Architecting LLM Inference, we build the foundation for everything that follows — starting with what context means inside a language model, and ending with exactly why the KV cache exists.
What we cover:
The mental model that makes it click:
Queries ask. Keys match. Values carry the information.
In Part 2B, we go fully into the KV cache — what it stores, why Queries are not cached, how memory grows with sequence length, and why KV cache becomes the #1 bottleneck in production LLM inference.
▶️ Watch the full video on YouTube: https://youtu.be/gMKNWM7_U4Y
Subscribe to follow the complete Architecting LLM Inference series.
Part of the Architecting Intelligence Flashcards series — short visual notes on AI, LLM systems, and ML infrastructure.
From the publisher's feed