Architecting Intelligence

Architecting Intelligence

By Pawan K JhaTechnology
Download on the App Store

Architecting Intelligence episodes

  • Looped Transformers: Can Models Think Longer Without Generating More Tokens?

    Can a model spend more time reasoning without generating more tokens?

    In the first Architecting Reasoning talk, we explore looped transformers: how they reuse layers for iterative computation, how they differ from standard transformers, and what recurrent depth could mean for reasoning and test-time compute. We also discuss ideas from Universal Transformers, Looped Transformers as Programmable Computers, and recent work on latent reasoning.

    35 min
  • Self-Attention, QKV, and the Foundation of KV Cache | Architecting LLM Inference — Part 2A

    Before you can understand the KV cache, you need to understand how LLMs build context. That's what this episode is about.

    In Part 2A of Architecting LLM Inference, we build the foundation for everything that follows — starting with what context means inside a language model, and ending with exactly why the KV cache exists.

    What we cover:

    • What context and contextual representation mean in practice
    • How RNNs, CNNs, attention, state-space models, and GNNs each represent context differently
    • How self-attention builds contextual representations
    • Causal attention and causal masking
    • Query, Key, and Value — explained through a real-world analogy
    • How attention scores are calculated
    • Why autoregressive decoding is sequential across tokens
    • How attention leads directly to the KV cache

    The mental model that makes it click:
    Queries ask. Keys match. Values carry the information.

    In Part 2B, we go fully into the KV cache — what it stores, why Queries are not cached, how memory grows with sequence length, and why KV cache becomes the #1 bottleneck in production LLM inference.

    ▶️ Watch the full video on YouTube: https://youtu.be/gMKNWM7_U4Y

    Subscribe to follow the complete Architecting LLM Inference series.

    17 min

About Architecting Intelligence

From the publisher's feed

AI systems, LLM engineering, ML infrastructure, and production AI architecture.