AI Post Transformers

This episode explores OPSDL (On-Policy Self-Distillation for Long-Context Language Models), a technique out of Baidu that tackles the gap between a model's advertised context window and how much of it the model can actually reason over faithfully. Rather than training on a separate reward model or human-labeled preferences, OPSDL has the same model supervise itself: a version reading a short, evidence-only excerpt acts as teacher for the version reading the full long document, with reverse KL divergence pulling the long-context student toward the short-context teacher's most confident token-by-token predictions. The discussion traces how this improves on prior approaches like LongPO and LongReward, which rely on blunt, sequence-level preference signals, and explains why the short-context teacher's immunity to irrelevant material makes this a direct lever against hallucination in noisy long documents. The hosts also flag a methodological wrinkle worth watching: every experiment runs on a single model family (Qwen2.5-Instruct) at three parameter scales, raising questions about whether the paper's "generalization" claims hold up under scrutiny. Listeners interested in the mechanics of long-context reasoning, self-distillation, and how models can be trained to trust their own better-calibrated judgments will find the framing compelling.

Sources:

1. OPSDL: On-Policy Self-Distillation for Long-Context Language Models — Xinsen Zhang, Zhenkai Ding, Tianjun Pan, Run Yang, Chun Kang, Xue Xiong, Jingnan Gu, 2026

http://arxiv.org/abs/2604.17535

2. Longpo: Long context self-evolution of large language models through short-to-long preference optimization — Guanzheng Chen, Xin Li, Michael Qizhe Shieh, Lidong Bing, 2025

https://scholar.google.com/scholar?q=Longpo%3A+Long+context+self-evolution+of+large+language+models+through+short-to-long+preference+optimization

3. Self-distilled reasoner: On-policy self-distillation for large language models (OPSD) — Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, Aditya Grover, 2026

https://scholar.google.com/scholar?q=Self-distilled+reasoner%3A+On-policy+self-distillation+for+large+language+models+%28OPSD%29

4. On-policy distillation of language models: Learning from self-generated mistakes (GKD) — Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, Olivier Bachem, 2024

https://scholar.google.com/scholar?q=On-policy+distillation+of+language+models%3A+Learning+from+self-generated+mistakes+%28GKD%29

5. Ruler: What's the real context size of your long-context language models? — Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, Boris Ginsburg, 2024

https://scholar.google.com/scholar?q=Ruler%3A+What%27s+the+real+context+size+of+your+long-context+language+models%3F

6. Solopo: Unlocking long-context capabilities in llms via short-to-long preference optimization — Huashan Sun, Shengyi Liao, Yansen Han, Yu Bai, Yang Gao, Cheng Fu, Weizhou Shen, Fanqi Wan, Ming Yan, Ji Zhang, et al., 2025

https://scholar.google.com/scholar?q=Solopo%3A+Unlocking+long-context+capabilities+in+llms+via+short-to-long+preference+optimization

Interactive Visualization: OPSDL: Teaching Long-Context Models to Trust Their Short-Context Selves

This episode explores DeltaProduct, a linear-RNN architecture that improves state-tracking by generalizing DeltaNet's single Householder reflection into a product of multiple reflections per token. The discussion traces the theoretical foundation: transformers and diagonal linear RNNs like Mamba face a proven complexity-class ceiling (TC0 vs NC1) that prevents them from tracking permutation-group state such as parity, while DeltaNet's recurrence—reinterpreted as one step of online gradient descent on an associative recall objective—naturally produces a Householder reflection as its state-transition matrix. DeltaProduct extends this by taking multiple gradient steps per token, and the hosts unpack why this matters via the Cartan-Dieudonné theorem: composing two reflections yields a full rotation, not just a bigger reflection, making the jump from one to two steps a qualitative leap in expressive power rather than an incremental one. Listeners get a rare example of a paper that derives a geometric theorem before running experiments, then tests whether empirical results match the prediction, offering a genuine dial between computational efficiency and expressivity instead of a fixed architectural tradeoff.

Sources:

1. DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products — Julien Siems, Timur Carstensen, Arber Zela, Frank Hutter, Massimiliano Pontil, Riccardo Grazzi, 2025

http://arxiv.org/abs/2502.10297

2. Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues — R. Grazzi, J. Siems, A. Zela, J. Franke, F. Hutter, M. Pontil, 2025 (ICLR)

https://scholar.google.com/scholar?q=Unlocking+State-Tracking+in+Linear+RNNs+Through+Negative+Eigenvalues

3. Parallelizing Linear Transformers with the Delta Rule over Sequence Length — S. Yang, B. Wang, Y. Zhang, Y. Shen, Y. Kim, 2024 (NeurIPS)

https://scholar.google.com/scholar?q=Parallelizing+Linear+Transformers+with+the+Delta+Rule+over+Sequence+Length

4. Gated Delta Networks: Improving Mamba2 with the Delta Rule — S. Yang, J. Kautz, A. Hatamizadeh, 2025 (ICLR)

https://scholar.google.com/scholar?q=Gated+Delta+Networks%3A+Improving+Mamba2+with+the+Delta+Rule

5. RWKV-7 'Goose' with Expressive Dynamic State Evolution — B. Peng, R. Zhang, D. Goldstein, et al., 2025

https://scholar.google.com/scholar?q=RWKV-7+%27Goose%27+with+Expressive+Dynamic+State+Evolution

6. The Illusion of State in State-Space Models — W. Merrill, J. Petty, A. Sabharwal, 2024 (ICML)

https://scholar.google.com/scholar?q=The+Illusion+of+State+in+State-Space+Models

7. Fixed-Point RNNs: From Diagonal to Dense in a Few Iterations — S. Movahedi, F. Sarnthein, N. Muca Cirone, A. Orvieto, 2025

https://scholar.google.com/scholar?q=Fixed-Point+RNNs%3A+From+Diagonal+to+Dense+in+a+Few+Iterations

Interactive Visualization: DeltaProduct: Extending DeltaNet's State-Tracking via Householder Products

This episode examines a black-box audit method for probing what large language models associate with a given name, applied across eight models including GPT-4o, GPT-5, Grok-3, and several locally-run open models. The researchers built WikiMem-style completion probing that works without token probabilities, testing models against 100 famous public figures and 100 invented synthetic names to isolate genuine memorization from guesswork, and uncovered failure patterns like "default token collapse" (models reflexively answering "ambidextrous" or "+1" regardless of the actual person) and base-rate anchoring on attributes like victim counts. The hosts dig into a four-category framework — direct, indirect, inferred, and guessed data — and debate how alarming it really is that GPT-4o hit 60%+ accuracy on several personal attributes for ordinary, non-famous individuals. A companion tool, LMP2, lets anyone query what a model associates with their own name, and survey results from 155 participants reveal a gap between which attributes people fear exposing (financial data, phone numbers, medical conditions) and which ones models actually get right. Listeners interested in AI privacy risks, model auditing methodology, or the gap between perceived and actual data exposure will find plenty to chew on here.

Sources:

1. What Do LLMs Associate with Your Name? A Human-Centered Black-Box Audit of Personal Data — Dimitri Staufer, Kirsten Morehouse, 2026

http://arxiv.org/abs/2602.17483

2. Membership Inference Attacks Against Machine Learning Models — Reza Shokri, Marco Stronati, Congzheng Song, Vitaly Shmatikov, 2017

https://scholar.google.com/scholar?q=Membership+Inference+Attacks+Against+Machine+Learning+Models

3. Auditing Differentially Private Machine Learning: How Private is Private SGD? — Matthew Jagielski, Jonathan Ullman, Alina Oprea, 2020

https://scholar.google.com/scholar?q=Auditing+Differentially+Private+Machine+Learning%3A+How+Private+is+Private+SGD%3F

4. Extracting Training Data from Large Language Models — Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, et al., 2021

https://scholar.google.com/scholar?q=Extracting+Training+Data+from+Large+Language+Models

5. Deduplicating Training Data Makes Language Models Better — Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini, 2022

https://scholar.google.com/scholar?q=Deduplicating+Training+Data+Makes+Language+Models+Better

6. Quantifying Memorization Across Neural Language Models — Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, Chiyuan Zhang, 2023

https://scholar.google.com/scholar?q=Quantifying+Memorization+Across+Neural+Language+Models

7. Scalable Extraction of Training Data from (Production) Language Models — Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, et al., 2023

https://scholar.google.com/scholar?q=Scalable+Extraction+of+Training+Data+from+%28Production%29+Language+Models

8. Beyond Memorization: Violating Privacy Via Inference with Large Language Models — Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev, 2024

https://scholar.google.com/scholar?q=Beyond+Memorization%3A+Violating+Privacy+Via+Inference+with+Large+Language+Models

9. Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms — Christian Sandvig, Kevin Hamilton, Karrie Karahalios, Cedric Langbort, 2014

https://scholar.google.com/scholar?q=Auditing+Algorithms%3A+Research+Methods+for+Detecting+Discrimination+on+Internet+Platforms

10. Auditing Algorithms: Understanding Algorithmic Systems from the Outside In — Danaë Metaxa, Joon Sung Park, Ronald E. Robertson, Karrie Karahalios, Christo Wilson, Jeff Hancock, Christian Sandvig, 2021

https://scholar.google.com/scholar?q=Auditing+Algorithms%3A+Understanding+Algorithmic+Systems+from+the+Outside+In

11. WikiMem probing framework paper (cited as [72]) — Not fully given in excerpt; referenced throughout as 'WikiMem', 2025/2026 (cited as prior work)

https://scholar.google.com/scholar?q=WikiMem+probing+framework+paper+%28cited+as+%5B72%5D%29

12. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks — Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, Dawn Song, 2019 (USENIX Security 19)

https://scholar.google.com/scholar?q=The+Secret+Sharer%3A+Evaluating+and+Testing+Unintended+Memorization+in+Neural+Networks

13. Minerva (cited as [81]) — Not fully given in excerpt, Not specified

https://scholar.google.com/scholar?q=Minerva+%28cited+as+%5B81%5D%29

14. Attribute inference from seemingly benign inputs (cited as [71]) — Not fully given in excerpt, Not specified

https://scholar.google.com/scholar?q=Attribute+inference+from+seemingly+benign+inputs+%28cited+as+%5B71%5D%29

15. Contextual-integrity study of ChatGPT users' privacy judgments — Tran et al., 2025

https://scholar.google.com/scholar?q=Contextual-integrity+study+of+ChatGPT+users%27+privacy+judgments

16. Dark patterns and design choices amplifying disclosure (cited as [28]) — Gumusel et al., 2025

https://scholar.google.com/scholar?q=Dark+patterns+and+design+choices+amplifying+disclosure+%28cited+as+%5B28%5D%29

17. Tell me something new: data subject rights applied to inferred data and profiles — Bart Custers, Helena Vrabec, 2024

https://scholar.google.com/scholar?q=Tell+me+something+new%3A+data+subject+rights+applied+to+inferred+data+and+profiles

18. Digital Forgetting in Large Language Models: A Survey of Unlearning Methods — Alberto Blanco-Justicia, Najeeb Jebreel, Benet Manzanares-Salor, David Sánchez, et al., 2025

https://scholar.google.com/scholar?q=Digital+Forgetting+in+Large+Language+Models%3A+A+Survey+of+Unlearning+Methods

Interactive Visualization: LLMs Guess Wrong: Auditing What Models Know About You

This episode examines SAC, a disaggregated KV-cache system built to serve sparse-attention LLMs like DeepSeek-V3.2 by pairing prefill and decode instances with a CXL-based memory pool instead of RDMA. The discussion traces how sparse attention mechanisms — specifically DeepSeek Sparse Attention's Lightning Indexer, which selects only the top-2048 relevant KV entries per layer — break the assumptions behind existing RDMA-based disaggregation systems like Mooncake and LMCache, which haul the entire KV cache across the network regardless of how much actually gets used. It explains why RDMA's message-based protocol can't cheaply serve scattered top-k lookups, while CXL's cache-line-granular load/store semantics can, and walks through why this matters for two concrete failure modes: wasted network bandwidth and wasted local memory. The hosts cite SAC's reported gains over an RDMA baseline — 2.1x throughput, 9.7x lower time-to-first-token, and 1.8x lower time-between-tokens — before scrutinizing the paper's benchmark methodology, questioning whether the RDMA-vs-CXL comparison (loopback NICs versus a single-hop CXL switch) is genuinely apples-to-apples. Listeners interested in the shifting compute-to-memory bottleneck in long-context LLM serving, and how new interconnects are reshaping systems design around sparse attention, will find the critical dissection of the experimental setup especially useful.

Sources:

1. SAC: Disaggregated KV Cache System for Sparse Attention LLMs with CXL — Ruiyang Ma, Teng Ma, Junru Li, Hantian Zha, Xuchun Shang, Qingda Hu, Zheng Liu, Xinjun Yang, Tao Ma, Guojie Luo, 2026

http://arxiv.org/abs/2606.19746

2. Mooncake: Trading More Storage for Less Computation — A KVCache-centric Architecture for Serving LLM Chatbot — Ruoyu Qin, Zheming Li, Weiran He, et al., 2025

https://scholar.google.com/scholar?q=Mooncake%3A+Trading+More+Storage+for+Less+Computation+%E2%80%94+A+KVCache-centric+Architecture+for+Serving+LLM+Chatbot

3. Beluga: A CXL-based Memory Architecture for Scalable and Efficient LLM KVCache Management — Xinjun Yang, Qingda Hu, Junru Li, Feifei Li, et al., 2026

https://scholar.google.com/scholar?q=Beluga%3A+A+CXL-based+Memory+Architecture+for+Scalable+and+Efficient+LLM+KVCache+Management

4. TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale — Dongha Yoon, Younghoon Min, Hoshik Kim, Sam H. Noh, Jongryool Kim, 2025

https://scholar.google.com/scholar?q=TraCT%3A+Disaggregated+LLM+Serving+with+CXL+Shared+Memory+KV+Cache+at+Rack-Scale

5. HiSparse: High-Efficiency Sparse Attention Inference in SGLang — Zhiqiang Xie, Zhangheng Huang, Tingwei Huang, 2026

https://scholar.google.com/scholar?q=HiSparse%3A+High-Efficiency+Sparse+Attention+Inference+in+SGLang

6. SnapKV: LLM Knows What You Are Looking For Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, et al., 2024

https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+Are+Looking+For+Before+Generation

7. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, et al., 2023

https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models

Interactive Visualization: SAC: Making Sparse Attention KV Caches Actually Sparse with CXL

This episode examines "Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective," a paper that attempts to derive an optimal approach to KV cache eviction from first principles rather than stacking heuristics. The discussion covers the two dominant camps in cache eviction—attention-pattern-based methods like SnapKV and H2O versus structure-aware methods like KeyDiff and Knorm—and how this paper unifies them under the Information Bottleneck principle, treating the surviving cache as a compression of KV history that must stay maximally informative about future queries. The hosts trace how the authors make an intractable nonlinear problem solvable by substituting a linear-Gaussian approximation, then use statistical leverage scores borrowed from classical numerical linear algebra and D-optimal experimental design to cheaply identify which tokens are irreplaceable. The conversation also digs into the paper's validation methodology, questioning whether the Spearman correlation results in Section 4.1 truly confirm the theory or reflect a built-in structural bias in how the proxies were constructed. It's a compelling listen for anyone interested in why long-context inference is so memory-hungry and whether cache eviction can finally be grounded in real mathematics instead of empirical guesswork.

Sources:

1. Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective — Jiaming Yang, Chenwei Tang, Liangli Zhen, Jiancheng Lv, 2026

http://arxiv.org/abs/2604.25975

2. The Information Bottleneck Method — Naftali Tishby, Fernando C. Pereira, William Bialek, 1999

https://scholar.google.com/scholar?q=The+Information+Bottleneck+Method

3. Deep Learning and the Information Bottleneck Principle — Naftali Tishby, Noga Zaslavsky, 2015

https://scholar.google.com/scholar?q=Deep+Learning+and+the+Information+Bottleneck+Principle

4. Opening the Black Box of Deep Neural Networks via Information — Ravid Shwartz-Ziv, Naftali Tishby, 2017

https://scholar.google.com/scholar?q=Opening+the+Black+Box+of+Deep+Neural+Networks+via+Information

5. Deep Variational Information Bottleneck — Alexander A. Alemi, Ian Fischer, Joshua V. Dillon, Kevin Murphy, 2017

https://scholar.google.com/scholar?q=Deep+Variational+Information+Bottleneck

6. Relative-Error CUR Matrix Decompositions — Petros Drineas, Michael W. Mahoney, S. Muthukrishnan, 2008

https://scholar.google.com/scholar?q=Relative-Error+CUR+Matrix+Decompositions

7. CUR Matrix Decompositions for Improved Data Analysis — Michael W. Mahoney, Petros Drineas, 2009

https://scholar.google.com/scholar?q=CUR+Matrix+Decompositions+for+Improved+Data+Analysis

8. Fast Approximation of Matrix Coherence and Statistical Leverage — Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, David P. Woodruff, 2012

https://scholar.google.com/scholar?q=Fast+Approximation+of+Matrix+Coherence+and+Statistical+Leverage

9. Randomized Algorithms for Matrices and Data — Michael W. Mahoney, 2011

https://scholar.google.com/scholar?q=Randomized+Algorithms+for+Matrices+and+Data

10. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention — Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, Francois Fleuret, 2020

https://scholar.google.com/scholar?q=Transformers+are+RNNs%3A+Fast+Autoregressive+Transformers+with+Linear+Attention

11. Infinite Attention: NNGP and NTK for Multi-Head Attention Networks — Jiri Hron, Yasaman Bahri, Jascha Sohl-Dickstein, Roman Novak, 2020

https://scholar.google.com/scholar?q=Infinite+Attention%3A+NNGP+and+NTK+for+Multi-Head+Attention+Networks

12. Rethinking Attention with Performers — Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, et al., 2021

https://scholar.google.com/scholar?q=Rethinking+Attention+with+Performers

13. Elements of Information Theory (Gaussian channel capacity results) — Thomas M. Cover, Joy A. Thomas (textbook synthesis of Shannon-era results), textbook; foundational results from 1948 onward

https://scholar.google.com/scholar?q=Elements+of+Information+Theory+%28Gaussian+channel+capacity+results%29

14. Efficient Memory Management for Large Language Model Serving with PagedAttention — Kwon, W., Li, Z., Zhuang, S., et al., 2023

https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention

15. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhang, Z., Sheng, Y., Zhou, T., et al., 2023

https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models

16. Information Bottleneck for Gaussian Variables — Chechik, G., Globerson, A., Tishby, N., Weiss, Y., 2003

https://scholar.google.com/scholar?q=Information+Bottleneck+for+Gaussian+Variables

Interactive Visualization: KV Cache Eviction Through an Information Bottleneck Lens

This episode closes a three-part arc on "Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time," examining how researchers identify and manipulate the specific attention heads responsible for triggering non-linear reasoning detours in language models. The discussion covers the technical pipeline: segmenting chains of thought at delimiter tokens, fitting per-head linear probes to find heads that predict reasoning-style shifts, then denoising those signals through shared-subspace PCA across heads in a layer. A live walkthrough shows the payoff — pausing mid-generation and rotating a hidden state to suppress or amplify the "non-linear" direction sends the model down a 12-step versus 45-step path to the same correct answer, making abstract "redundant reasoning" concrete. Benchmark results follow, with the CREST method cutting token usage over 30% while matching or beating baseline accuracy across four architectures (dense and mixture-of-experts) and transferring — without recalibration — from math-only training data to code generation, science QA, and scheduling tasks. The hosts close by flagging an unresolved inconsistency in the paper: three different head-selection ratios appear across sections, with no clear statement of which one produced the headline results.

Sources:

1. Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time — Zhenyu Zhang, Xiaoxia Wu, Zhongzhu Zhou, Qingyang Wu, Yineng Zhang, Pragaash Ponnusamy, Harikaran Subbaraj, Jue Wang, Shuaiwen Leon Song, Ben Athiwaratkun, 2025

http://arxiv.org/abs/2512.24574

2. Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs — Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, Noah D. Goodman, 2025

https://scholar.google.com/scholar?q=Cognitive+Behaviors+that+Enable+Self-Improving+Reasoners%2C+or%2C+Four+Habits+of+Highly+Effective+STaRs

3. Retrieval Head Mechanistically Explains Long-Context Factuality — Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, Yao Fu, 2024

https://scholar.google.com/scholar?q=Retrieval+Head+Mechanistically+Explains+Long-Context+Factuality

4. Representation Engineering: A Top-Down Approach to AI Transparency — Andy Zou, Long Phan, Sarah Chen, et al., 2025

https://scholar.google.com/scholar?q=Representation+Engineering%3A+A+Top-Down+Approach+to+AI+Transparency

5. Reasoning Models Can Be Effective Without Thinking — Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, Matei Zaharia, 2025

https://scholar.google.com/scholar?q=Reasoning+Models+Can+Be+Effective+Without+Thinking

6. Thoughts Are All Over the Place: On the Underthinking of o1-like LLMs — Yue Wang, Qiuzhi Liu, Jiahao Xu, et al., 2025

https://scholar.google.com/scholar?q=Thoughts+Are+All+Over+the+Place%3A+On+the+Underthinking+of+o1-like+LLMs

Interactive Visualization: Steering Reasoning Models' Cognitive Behaviors at Test-Time

This episode explores "Thought Anchors: Which LLM Reasoning Steps Matter?" by Paul C. Bogdan and Uzay Macar, with senior authors Neel Nanda and Arthur Conmy, examining which specific sentences in a chain-of-thought trace carry disproportionate causal weight over a model's final answer. The discussion covers why standard mechanistic interpretability tools, built for single forward passes, break down for reasoning models that generate thousands of sequentially dependent tokens, and how the authors instead treat the sentence as the right unit of analysis. Three independent methods are unpacked: counterfactual resampling with embedding-based filtering, receiver-head attention analysis, and attention suppression measured via KL divergence, all converging on identifying "thought anchors." A concrete case study on a base-16-to-binary conversion problem shows how a single pivot sentence rescues an otherwise wrong reasoning trace, illustrating the stakes in vivid detail. Listeners interested in interpretability, reasoning-model behavior, and how backtracking and self-correction actually work under the hood will find the mechanistic grounding — and its connection to related work like the s1 paper's "Wait"-token forcing — especially compelling.

Sources:

1. Thought Anchors: Which LLM Reasoning Steps Matter? — Paul C. Bogdan, Uzay Macar, Neel Nanda, Arthur Conmy, 2025

http://arxiv.org/abs/2506.19143

2. s1: Simple test-time scaling — Niklas Muennighoff, Zitong Yang, Weijia Shi, et al., 2025

https://scholar.google.com/scholar?q=s1%3A+Simple+test-time+scaling

3. Understanding reasoning in thinking language models via steering vectors — Constantin Venhoff, Iván Arcuschin, Philip Torr, Arthur Conmy, Neel Nanda, 2025

https://scholar.google.com/scholar?q=Understanding+reasoning+in+thinking+language+models+via+steering+vectors

4. Chain-of-thought reasoning in the wild is not always faithful — Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, Arthur Conmy, 2025

https://scholar.google.com/scholar?q=Chain-of-thought+reasoning+in+the+wild+is+not+always+faithful

5. Reasoning models don't always say what they think — Yanda Chen, Joe Benton, Ansh Radhakrishnan, et al., 2025

https://scholar.google.com/scholar?q=Reasoning+models+don%27t+always+say+what+they+think

6. Chain of thought monitorability: A new and fragile opportunity for AI safety — Tomek Korbak, Mikita Balesni, Elizabeth Barnes, et al., 2025

https://scholar.google.com/scholar?q=Chain+of+thought+monitorability%3A+A+new+and+fragile+opportunity+for+AI+safety

7. Forking paths in neural text generation — Eric Bigelow, Ari Holtzman, Hidenori Tanaka, Tomer Ullman, 2024

https://scholar.google.com/scholar?q=Forking+paths+in+neural+text+generation

8. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation — Bowen Baker, Joost Huizinga, Leo Gao, et al., 2025

https://scholar.google.com/scholar?q=Monitoring+reasoning+models+for+misbehavior+and+the+risks+of+promoting+obfuscation

Interactive Visualization: Thought Anchors: Which Sentences Really Drive LLM Reasoning

This episode explores a paper introducing Gimbal, a serving system for Mixture-of-Experts LLMs that coordinates two scheduling decisions previously handled separately: which backend engine a request enters through and where individual experts are physically placed across GPUs. The hosts walk through why request-count-based load balancing fails for MoE models — a 200-token and a 2,000-token request look identical by count but differ tenfold in KV-cache pressure and time-to-first-token — and why expert activation is highly uneven and source-dependent, with profiling on Qwen3-80B showing over 83% of one layer's traffic from a given engine routing to remote, non-local experts. The core argument is that dispatch and placement are a coupled optimization problem rather than two problems to solve independently, since offline expert rebalancing is blind to real-time backend pressure like queue depth and KV-cache usage. Reported results back the claim: 42.9% lower time-to-first-token and 33.3% lower time-per-output-token versus vLLM. Listeners interested in LLM inference infrastructure will find a concrete, numbers-driven case for why sparse-model serving needs system-level coordination rather than heuristics borrowed from dense-model deployments.

Sources:

1. Coordinated Scheduling for MoE LLM Serving — Yifan Sun, Zhexiang Zhang, Jiantong Jiang, Gholamreza Haffari, Minxian Xu, Feng Liu, Rajkumar Buyya, Adel N. Toosi, 2026

http://arxiv.org/abs/2606.15177v1

2. Orca: A Distributed Serving System for Transformer-Based Generative Models — Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, Byung-Gon Chun, 2022

https://scholar.google.com/scholar?q=Orca%3A+A+Distributed+Serving+System+for+Transformer-Based+Generative+Models

3. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, et al., 2023

https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention

4. DeepSeek-V3 Technical Report — DeepSeek-AI (large team; Damai Dai, Wenfeng Liang, et al.), 2024

https://scholar.google.com/scholar?q=DeepSeek-V3+Technical+Report

5. Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — Ruoyu Qin, Zheming Li, Weiran He, Mingxing Zhang, Yongwei Wu, et al. (Moonshot AI / Kimi), 2024

https://scholar.google.com/scholar?q=Mooncake%3A+A+KVCache-centric+Disaggregated+Architecture+for+LLM+Serving

6. Preble: Efficient Distributed Prompt Scheduling for LLM Serving — Vikranth Srivatsa, Zijian He, Reyna Abhyankar, Dongming Li, Yiying Zhang, 2024/2025 (ICLR 2025)

https://scholar.google.com/scholar?q=Preble%3A+Efficient+Distributed+Prompt+Scheduling+for+LLM+Serving

7. MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing — Seokjin Go, Divya Mahajan, 2025

https://scholar.google.com/scholar?q=MoETuner%3A+Optimized+Mixture+of+Expert+Serving+with+Balanced+Expert+Placement+and+Token+Routing

8. Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling (Sem-MoE) — Yan Li, Zhenyu Zhang, Zhengang Wang, Pengfei Chen, Pengfei Zheng, 2026 (ICLR)

https://scholar.google.com/scholar?q=Semantic+Parallelism%3A+Redefining+Efficient+MoE+Inference+via+Model-Data+Co-Scheduling+%28Sem-MoE%29

9. Exploiting Inter-layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference — Jinghan Yao, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda, 2024 (IPDPS)

https://scholar.google.com/scholar?q=Exploiting+Inter-layer+Expert+Affinity+for+Accelerating+Mixture-of-Experts+Model+Inference

10. JANUS: Disaggregating Attention and Experts for Scalable MoE Inference — Zhexiang Zhang, Ye Wang, Yumiao Zhao, et al., 2025

https://scholar.google.com/scholar?q=JANUS%3A+Disaggregating+Attention+and+Experts+for+Scalable+MoE+Inference

Interactive Visualization: Coordinated MoE Scheduling: How Gimbal Fixes Expert Locality

This episode traces the evolution from classic knowledge distillation to on-policy distillation and finally to Lightning OPD, a technique from NVIDIA researchers for post-training large reasoning models. The hosts unpack why on-policy distillation offers denser training signal than RLVR's sparse end-of-trace rewards, but has historically required an expensive live teacher model running alongside the student throughout training. They explain the paper's key insight — that a student's rollout distribution drifts only modestly from its SFT starting point — which motivates capturing the teacher's judgments once offline rather than serving it continuously. The discussion grounds the work in its lineage, from Hinton et al.'s original 2015 distillation paper to DeepMind's 2024 Generalized Knowledge Distillation formalism, while flagging why the infrastructure savings matter even more for sparse Mixture-of-Experts models. Listeners interested in the theory-first rigor behind cost-cutting techniques in LLM post-training will find the paper's formal proof-before-benchmarks approach a refreshing departure from the field's norm.

Sources:

1. Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillation — Yecheng Wu, Song Han, Hai Cai, 2026

http://arxiv.org/abs/2604.13010

2. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes — Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, Olivier Bachem (Google DeepMind), 2024 (ICLR)

https://scholar.google.com/scholar?q=On-Policy+Distillation+of+Language+Models%3A+Learning+from+Self-Generated+Mistakes

3. On-Policy Distillation — Kevin Lu et al. (Thinking Machines Lab), 2025

https://scholar.google.com/scholar?q=On-Policy+Distillation

4. Distilling the Knowledge in a Neural Network — Geoffrey Hinton, Oriol Vinyals, Jeff Dean, 2015

https://scholar.google.com/scholar?q=Distilling+the+Knowledge+in+a+Neural+Network

5. Tulu 3: Pushing Frontiers in Open Language Model Post-Training — Nathan Lambert, Jacob Morrison, Valentina Pyatkin, et al. (Allen Institute for AI), 2024

https://scholar.google.com/scholar?q=Tulu+3%3A+Pushing+Frontiers+in+Open+Language+Model+Post-Training

6. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI (Daya Guo et al.), 2025

https://scholar.google.com/scholar?q=DeepSeek-R1%3A+Incentivizing+Reasoning+Capability+in+LLMs+via+Reinforcement+Learning

7. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models — Zhihong Shao, Peiyi Wang, Qihao Zhu, et al. (DeepSeek-AI), 2024

https://scholar.google.com/scholar?q=DeepSeekMath%3A+Pushing+the+Limits+of+Mathematical+Reasoning+in+Open+Language+Models

8. Let's Verify Step by Step — Hunter Lightman, Vineet Kosaraju, Yura Burda, et al. (OpenAI), 2023

https://scholar.google.com/scholar?q=Let%27s+Verify+Step+by+Step

9. On-policy distillation (Thinking Machines Lab: Connectionism) — Kevin Lu, Thinking Machines Lab, 2025

https://scholar.google.com/scholar?q=On-policy+distillation+%28Thinking+Machines+Lab%3A+Connectionism%29

10. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? — Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, Gao Huang, 2025

https://scholar.google.com/scholar?q=Does+reinforcement+learning+really+incentivize+reasoning+capacity+in+LLMs+beyond+the+base+model%3F

11. RL's razor: Why online reinforcement learning forgets less — Idan Shenfeld, Jyothish Pari, Pulkit Agrawal, 2025

https://scholar.google.com/scholar?q=RL%27s+razor%3A+Why+online+reinforcement+learning+forgets+less

12. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe — Siyuan Chen, Daya Guo, Yichun Tan, Xiaohan Liang, Hao Zhou, Bo Zheng, Dejian Yang, 2026

https://scholar.google.com/scholar?q=Rethinking+on-policy+distillation+of+large+language+models%3A+Phenomenology%2C+mechanism%2C+and+recipe

13. Learning beyond teacher: Generalized on-policy distillation with reward extrapolation (ExOPD) — Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, Saiyong Yang, Yankai Lin, 2026

https://scholar.google.com/scholar?q=Learning+beyond+teacher%3A+Generalized+on-policy+distillation+with+reward+extrapolation+%28ExOPD%29

Interactive Visualization: Lightning OPD: Precomputing the Teacher for Cheaper Reasoning Distillation

This episode explores MemGPT, a UC Berkeley paper proposing to manage large language model context windows the way operating systems manage virtual memory, paging information in and out rather than trying to cram everything into a fixed prompt window. The hosts unpack why this matters: self-attention's quadratic cost caps context size, and even models with huge windows suffer from the "lost in the middle" problem, where information buried mid-context gets recalled far less reliably than content at the start or end. They dig into MemGPT's architecture — a fixed "main context" split into an editable working scratchpad and a FIFO conversation queue, backed by two external tiers (archival storage for documents and recall storage for full message history) that the model pages in via its own function calls. A recurring debate threads through the discussion: whether letting the LLM itself decide when to page memory, rather than a deterministic kernel policy, makes the system meaningfully less reliable than a real OS. Listeners interested in agent memory design, long-context limitations, and the tradeoffs of self-directed retrieval will find the hosts' back-and-forth over how far the OS analogy actually holds particularly engaging.

Sources:

1. MemGPT: Treating LLM Context Windows Like Virtual Memory

https://arxiv.org/pdf/2310.08560

2. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023

https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts

3. Generative Agents: Interactive Simulacra of Human Behavior — Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein, 2023

https://scholar.google.com/scholar?q=Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior

4. Beyond Goldfish Memory: Long-Term Open-Domain Conversation — Jing Xu, Arthur Szlam, Jason Weston, 2021

https://scholar.google.com/scholar?q=Beyond+Goldfish+Memory%3A+Long-Term+Open-Domain+Conversation

5. Improving Language Models by Retrieving from Trillions of Tokens (RETRO) — Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, et al., 2022

https://scholar.google.com/scholar?q=Improving+Language+Models+by+Retrieving+from+Trillions+of+Tokens+%28RETRO%29

6. ReAct: Synergizing Reasoning and Acting in Language Models — Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao, 2022

https://scholar.google.com/scholar?q=ReAct%3A+Synergizing+Reasoning+and+Acting+in+Language+Models

Interactive Visualization: MemGPT: Treating LLM Context Windows Like Virtual Memory