Sign up to save your podcasts
Or
Share AI Post Transformers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
AI Post Transformers
By mcgrofTechnology
AI Post Transformers episodes
- New Site Features: Conferences Tracking and How to Support the ShowSpecial announcement: AI Post Transformers now has a Conferences section that tracks the AI conferences and papers we have covered, plus a new Sponsor page for listeners who want to help fund credits and infrastructure that keep the show running.Interactive Visualization: New Site Features: Conferences Tracking and How to Support the Show3 min
- Optimizing Mixture of Block Attention Through Statistical TheoryThis episode examines the statistical foundations of Mixture of Block Attention (MoBA), a sparse attention mechanism that divides key-value sequences into blocks and routes queries only to the most relevant ones. The paper derives a signal-to-noise ratio showing that retrieval accuracy depends on the square root of head dimension divided by block size, revealing why smaller blocks improve a router's ability to distinguish relevant from irrelevant content despite increasing computational overhead. The authors introduce FlashMoBA, a hardware-optimized CUDA kernel that makes small block sizes practical on GPUs, and demonstrate how depthwise convolutions on keys can cluster related signals to further boost routing performance. The work provides theoretical grounding for why routing-based sparse attention succeeds at reducing quadratic attention costs to near-linear scaling in long-context language models.Sources:1. Optimizing Mixture of Block Attention — Guangxuan Xiao, Junxian Guo, Kasra Mazaheri, Song Han, 2025http://arxiv.org/abs/2511.11571v22. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Dao et al., 2022https://scholar.google.com/scholar?q=FlashAttention%3A+Fast+and+Memory-Efficient+Exact+Attention+with+IO-Awareness3. Mixture of Experts: A Survey — Various (MoE literature), 2020-2024https://scholar.google.com/scholar?q=Mixture+of+Experts%3A+A+Survey4. Sparse Attention Mechanisms (Zaheer et al., Guo et al., Xu et al.) — Cited in paper, 2020-2025https://scholar.google.com/scholar?q=Sparse+Attention+Mechanisms+%28Zaheer+et+al.%2C+Guo+et+al.%2C+Xu+et+al.%295. AI Post Transformers: Optimizing Mixture of Block Attention for Long-Context Transformers — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-optimizing-mixture-of-block-attention-fo-ea4612.mp36. AI Post Transformers: SolidAttention: Co-Designing Sparse Attention and SSD I/O — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-18-solidattention-co-designing-sparse-atten-5a8622.mp37. AI Post Transformers: Bidaw: Bidirectional Awareness for Interactive LLM KV Caching — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-bidaw-bidirectional-awareness-for-intera-87c311.mp3Interactive Visualization: Optimizing Mixture of Block Attention Through Statistical Theory0 min
- Xerxes: CXL 3.0 Simulation for Scalable Memory SystemsThis episode explores Xerxes, a new open-source simulator designed to model CXL 3.0 features before the hardware exists. The hosts explain how CXL adds cache coherence to PCIe to solve memory access bottlenecks in AI and HPC workloads, then dive into the two major architectural changes in CXL 3.0: Port-Based Routing, which enables arbitrary fabric topologies beyond rigid trees, and Device-Managed Coherence, which lets devices handle coherence protocols peer-to-peer without routing every transaction through the host CPU. The discussion highlights why this simulator matters for designing next-generation rack-scale memory pools and accelerator fabrics, addressing the chicken-and-egg problem of validating designs before physical hardware ships. The hosts question how validation works without reference hardware and preview a deeper look at Xerxes' architecture and methodology.Sources:1. Xerxes: CXL 3.0 Simulation for Scalable Memory Systemshttps://www.usenix.org/system/files/fast26-an.pdf2. CXL Memory Disaggregation: Opportunities and Challenges — Guz et al. (Intel), 2023https://scholar.google.com/scholar?q=CXL+Memory+Disaggregation%3A+Opportunities+and+Challenges3. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms — Li et al., 2023https://scholar.google.com/scholar?q=Pond%3A+CXL-Based+Memory+Pooling+Systems+for+Cloud+Platforms4. TPP: Transparent Page Placement for CXL-Enabled Tiered Memory — Maruf et al., 2023https://scholar.google.com/scholar?q=TPP%3A+Transparent+Page+Placement+for+CXL-Enabled+Tiered+Memory5. The CXL Memory Expander: Performance and Cost Analysis — Gouk et al. (SK hynix), 2023https://scholar.google.com/scholar?q=The+CXL+Memory+Expander%3A+Performance+and+Cost+Analysis6. Exploring CXL 3.0 Port-Based Routing for Scalable Memory Systems — Pan et al., 2024https://scholar.google.com/scholar?q=Exploring+CXL+3.0+Port-Based+Routing+for+Scalable+Memory+Systems7. SMART: Scalable Memory Architecture with Port-Based Routing — Kim et al., 2024https://scholar.google.com/scholar?q=SMART%3A+Scalable+Memory+Architecture+with+Port-Based+Routing8. Deadlock-Free Routing for CXL Fabrics — Zhang et al., 2024https://scholar.google.com/scholar?q=Deadlock-Free+Routing+for+CXL+Fabrics9. DMC: Distributed Cache Coherence for CXL Memory Systems — Lee et al., 2024https://scholar.google.com/scholar?q=DMC%3A+Distributed+Cache+Coherence+for+CXL+Memory+Systems10. Scaling Cache Coherence to Thousands of Devices with CXL DMC — Wang et al., 2024https://scholar.google.com/scholar?q=Scaling+Cache+Coherence+to+Thousands+of+Devices+with+CXL+DMC11. Coherence Protocol Verification for CXL Device-Managed Coherence — Chen et al., 2024https://scholar.google.com/scholar?q=Coherence+Protocol+Verification+for+CXL+Device-Managed+Coherence12. gem5: A Multiple-ISA Full-System Simulator — Binkert et al., 2011https://scholar.google.com/scholar?q=gem5%3A+A+Multiple-ISA+Full-System+Simulator13. The ZSim Simulator: Fast and Accurate Multicore Simulation — Sanchez and Kozyrakis, 2013https://scholar.google.com/scholar?q=The+ZSim+Simulator%3A+Fast+and+Accurate+Multicore+Simulation14. Simulating Multi-Core Systems with Shared Memory Coherence — Martin et al. (Wisconsin Multifacet group), 2005https://scholar.google.com/scholar?q=Simulating+Multi-Core+Systems+with+Shared+Memory+Coherence15. PARADE: A Cycle-Accurate Full-System Simulation Platform for Accelerator-Rich Architectures — Fuchs et al., 2020https://scholar.google.com/scholar?q=PARADE%3A+A+Cycle-Accurate+Full-System+Simulation+Platform+for+Accelerator-Rich+Architectures16. A Primer on Memory Consistency and Cache Coherence — Sorin, Hill, and Wood, 2011https://scholar.google.com/scholar?q=A+Primer+on+Memory+Consistency+and+Cache+Coherence17. Coherence and Consistency Models in Shared-Memory Multiprocessors — Adve and Gharachorloo, 1996https://scholar.google.com/scholar?q=Coherence+and+Consistency+Models+in+Shared-Memory+Multiprocessors18. DASH: A Scalable Directory-Based Multiprocessor — Lenoski et al. (Stanford DASH project), 1992https://scholar.google.com/scholar?q=DASH%3A+A+Scalable+Directory-Based+Multiprocessor19. Directory-Based Cache Coherence in Large-Scale Multiprocessors — Chaiken et al. (Alewife project), 1991https://scholar.google.com/scholar?q=Directory-Based+Cache+Coherence+in+Large-Scale+Multiprocessors20. Enabling Rack-Scale Confidential Computing using Heterogeneous Trusted Execution Environment — Jianping Zhu, Hang Yin, Yuekai Jia, Wenhao Wang, Chunhui Li, Jiashuo Liang, Shoumeng Yan, Zhengyu He, Qingkui Liu, Alex X. Liu, 2024https://scholar.google.com/scholar?q=Enabling+Rack-Scale+Confidential+Computing+using+Heterogeneous+Trusted+Execution+Environment21. Understanding the Overheads of Hardware Memory Coherence — Lena E. Olson, Joseph Izraelevitz, Mark D. Hill, 2015https://scholar.google.com/scholar?q=Understanding+the+Overheads+of+Hardware+Memory+Coherence22. AI Post Transformers: SolidAttention: Co-Designing Sparse Attention and SSD I/O — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-18-solidattention-co-designing-sparse-atten-5a8622.mp323. AI Post Transformers: Accelerating LLM Cold Starts with Programmable Page Cache — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-accelerating-llm-cold-starts-with-progra-0912d1.mp324. AI Post Transformers: xLLM: Co-Locating Online and Offline LLM Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-xllm-co-locating-online-and-offline-llm-10bb81.mp3Interactive Visualization: Xerxes: CXL 3.0 Simulation for Scalable Memory Systems0 min
- Generative File Systems: Replacing Code with Formal SpecificationsThis episode explores a 2026 USENIX FAST paper that proposes replacing hand-written file system code with LLM-generated implementations derived from formal specifications. The authors demonstrate SYSSPEC, a system that uses three types of formal specifications—Hoare logic for functionality, rely-guarantee conditions for modularity, and explicit concurrency protocols—to guide code generation while using validation agents to catch hallucinations and ensure correctness. Analysis of Ext4's commit history reveals that 82.4% of changes are bug fixes and maintenance, suggesting traditional file system development wastes enormous effort on code upkeep rather than innovation. The researchers show that their approach can generate a working file system (SPECFS) and evolve it by patching specifications rather than code, potentially transforming how systems software is developed and maintained.Sources:1. Generative File Systems: Replacing Code with Formal Specificationshttps://www.usenix.org/system/files/fast26-liu-qingyuan.pdf2. Yggdrasil: An Optimized System for Training Deep Decision Trees at Scale — Fabrice Popineau, Artem Vysogorets, et al., 2020https://scholar.google.com/scholar?q=Yggdrasil%3A+An+Optimized+System+for+Training+Deep+Decision+Trees+at+Scale3. Hyperkernel: Push-Button Verification of an OS Kernel — Luke Nelson, Helgi Sigurbjarnarson, Kaiyuan Zhang, et al., 2017https://scholar.google.com/scholar?q=Hyperkernel%3A+Push-Button+Verification+of+an+OS+Kernel4. Program Synthesis from Natural Language Using Recurrent Neural Networks — Xi Victoria Lin, Chenglong Wang, Luke Zettlemoyer, Michael D. Ernst, 2017https://scholar.google.com/scholar?q=Program+Synthesis+from+Natural+Language+Using+Recurrent+Neural+Networks5. Crash Hoare Logic — Tej Chajed, Frans Kaashoek, Butler Lampson, Nickolai Zeldovich, 2018https://scholar.google.com/scholar?q=Crash+Hoare+Logic6. FSCQ: A Verified File System — Haogang Chen et al., 2015https://scholar.google.com/scholar?q=FSCQ%3A+A+Verified+File+System7. Yxv6: An Educational File System with Formal Specifications — Helgi Sigurbjarnarson et al., 2016https://scholar.google.com/scholar?q=Yxv6%3A+An+Educational+File+System+with+Formal+Specifications8. Crash Consistency in Database Systems — Goetz Graefe, 2009https://scholar.google.com/scholar?q=Crash+Consistency+in+Database+Systems9. Using Crash Hoare Logic for Certifying the FSCQ File System — Haogang Chen et al., 2015https://scholar.google.com/scholar?q=Using+Crash+Hoare+Logic+for+Certifying+the+FSCQ+File+System10. Jitk: A Trustworthy In-Kernel Interpreter Infrastructure — Xi Wang et al., 2014https://scholar.google.com/scholar?q=Jitk%3A+A+Trustworthy+In-Kernel+Interpreter+Infrastructure11. AI Post Transformers: LLM Agents Reason About Code Without Running It — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-15-llm-agents-reason-about-code-without-run-2a1876.mp312. AI Post Transformers: SYSSPEC: LLM-Generated File Systems from Formal Specifications — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-sysspec-llm-generated-file-systems-from-02f5a9.mp313. AI Post Transformers: Generative File Systems from Formal Specifications with SysSpec — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-generative-file-systems-from-formal-spec-ff240b.mp314. AI Post Transformers: Sharpen the Spec, Cut the Code: LLM-Generated File Systems — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-sharpen-the-spec-cut-the-code-llm-genera-8eb6b1.mp3Interactive Visualization: Generative File Systems: Replacing Code with Formal Specifications0 min
- SolidAttention: Co-Designing Sparse Attention and SSD I/OThis episode explores SolidAttention, a system that enables large language models to run on memory-constrained consumer PCs by offloading the KV cache to SSD storage. The paper addresses a fundamental mismatch: sparse attention patterns create random I/O access that kills SSD performance, while previous offloading solutions like FlexGen only work well with high request concurrency unavailable on local machines. The researchers co-designed sparse attention algorithms with SSD storage management to enable coarse-grained sequential reads instead of fine-grained random access, achieving practical local LLM inference on systems with just 8-16GB of RAM. The discussion covers why KV caches consume four times the memory of model weights, the trade-offs of quantization versus offloading, and why treating attention sparsity and storage optimization as separate problems fails on consumer hardware.Sources:1. SolidAttention: Co-Designing Sparse Attention and SSD I/Ohttps://www.usenix.org/system/files/fast26-zheng.pdf2. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU — Sheng et al., 2023https://scholar.google.com/scholar?q=FlexGen%3A+High-Throughput+Generative+Inference+of+Large+Language+Models+with+a+Single+GPU3. Efficient Streaming Language Models with Attention Sinks — Xiao et al., 2024https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks4. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhang et al., 2023https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models5. SSD I/O Characteristics: Impacts of Request Size, Access Pattern, and Parallelism — Chen et al., 2016https://scholar.google.com/scholar?q=SSD+I%2FO+Characteristics%3A+Impacts+of+Request+Size%2C+Access+Pattern%2C+and+Parallelism6. vLLM: Efficient Memory Management for Large Language Model Serving with PagedAttention — Kwon et al., 2023https://scholar.google.com/scholar?q=vLLM%3A+Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention7. AI Post Transformers: SolidAttention: Efficient SSD-based KV Cache Offloading for Long-Context LLMs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-solidattention-efficient-ssd-based-kv-ca-336b79.mp38. AI Post Transformers: SolidAttention: Fast SSD-Based Serving on Memory-Constrained PCs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-solidattention-fast-ssd-based-serving-on-1c305d.mp39. AI Post Transformers: SolidAttention: Low-Latency SSD-based Serving on Memory-Constrained PCs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-solidattention-low-latency-ssd-based-ser-e22a0d.mp310. AI Post Transformers: Bidaw: Bidirectional Awareness for Interactive LLM KV Caching — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-bidaw-bidirectional-awareness-for-intera-87c311.mp311. AI Post Transformers: Bidaw: Reducing LLM KV Cache Latency with Two-Tier Storage — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-bidaw-reducing-llm-kv-cache-latency-with-15dd25.mp312. AI Post Transformers: Bidaw: Computation-Storage Aware KV Caching for LLMs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-bidaw-computation-storage-aware-kv-cachi-9d89fb.mp313. AI Post Transformers: CacheSlide: Unlocking Cross Position-Aware KV Cache Reuse for LLM Serving — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-cacheslide-unlocking-cross-position-awar-487b2b.mp314. AI Post Transformers: Efficient KV Cache Reuse in Dynamic Agent Workflows — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-efficient-kv-cache-reuse-in-dynamic-agen-558f19.mp315. AI Post Transformers: Accelerating LLM Cold Starts with Programmable Page Cache — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-accelerating-llm-cold-starts-with-progra-0912d1.mp316. AI Post Transformers: LLM Cold Starts: Fixing Linux Page Cache for Model Loading — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-pacific.com/episodes/2026-03-17-llm-cold-starts-fixing-linux-page-cache-a9f9a9.mp3Interactive Visualization: SolidAttention: Co-Designing Sparse Attention and SSD I/O0 min
- Accelerating LLM Cold Starts with Programmable Page CacheThis episode explores a USENIX FAST'26 paper that addresses the infrastructure bottleneck of loading massive language model weights from storage into accelerator memory during inference deployments. The authors present a programmable page cache framework that achieves 2-4× faster cold start times by exploiting predictable sequential access patterns and XPU affinity, while maintaining full compatibility with existing model formats, inference frameworks, and hardware—unlike prior approaches such as ServerlessLLM and BlitzScale that require custom formats or specific interconnects. The discussion examines why the standard kernel page cache underutilizes modern SSD bandwidth through conservative prefetching and inappropriate LRU eviction policies designed for general workloads, and how a userspace-programmable caching layer can optimize for the specific characteristics of model loading without intrusive kernel modifications. Listeners interested in production ML infrastructure, storage systems optimization, or the operational challenges of deploying large models at scale will find concrete insights into how I/O dominates cold start latency and emerging solutions that bridge the three-orders-of-magnitude gap between SSD and GPU memory bandwidth.Sources:1. Accelerating LLM Cold Starts with Programmable Page Cachehttps://www.usenix.org/system/files/fast26-liu-yubo.pdf2. Orca: A Distributed Serving System for Transformer-Based Generative Models — Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, Byung-Gon Chun, 2022https://scholar.google.com/scholar?q=Orca%3A+A+Distributed+Serving+System+for+Transformer-Based+Generative+Models3. AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving — Zhuohan Li, Lianmin Zheng, Yinmin Zhong, Vincent Liu, Ying Sheng, Xin Jin, Yanping Huang, Zhifeng Chen, Hao Zhang, Joseph E. Gonzalez, Ion Stoica, 2023https://scholar.google.com/scholar?q=AlpaServe%3A+Statistical+Multiplexing+with+Model+Parallelism+for+Deep+Learning+Serving4. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU — Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang, 2023https://scholar.google.com/scholar?q=FlexGen%3A+High-Throughput+Generative+Inference+of+Large+Language+Models+with+a+Single+GPU5. ServerlessLLM: Locality-Enhanced Serverless Inference for Large Language Models — Yao Fu, Leyang Xue, Yeqi Huang, Andrei-Octavian Brabete, Dmitrii Ustiugov, Yuvraj Patel, Luo Mai, 2024https://scholar.google.com/scholar?q=ServerlessLLM%3A+Locality-Enhanced+Serverless+Inference+for+Large+Language+Models6. DeepSpeed-Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale — Aminabadi et al., 2022https://scholar.google.com/scholar?q=DeepSpeed-Inference%3A+Enabling+Efficient+Inference+of+Transformer+Models+at+Unprecedented+Scale7. ZeRO-Offload: Democratizing Billion-Scale Model Training — Ren et al., 2021https://scholar.google.com/scholar?q=ZeRO-Offload%3A+Democratizing+Billion-Scale+Model+Training8. Safetensors: Simple, safe way to store and distribute tensors — HuggingFace, 2022https://scholar.google.com/scholar?q=Safetensors%3A+Simple%2C+safe+way+to+store+and+distribute+tensors9. AI Post Transformers: LLM Cold Starts: Fixing Linux Page Cache for Model Loading — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-llm-cold-starts-fixing-linux-page-cache-a9f9a9.mp310. AI Post Transformers: SolidAttention: Efficient SSD-based KV Cache Offloading for Long-Context LLMs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-solidattention-efficient-ssd-based-kv-ca-336b79.mp311. AI Post Transformers: Bidaw: Computation-Storage Aware KV Caching for LLMs — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-bidaw-computation-storage-aware-kv-cachi-9d89fb.mp312. AI Post Transformers: xLLM: Co-Locating Online and Offline LLM Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-xllm-co-locating-online-and-offline-llm-10bb81.mp3Interactive Visualization: Accelerating LLM Cold Starts with Programmable Page Cache0 min
- Predicting Task Performance with Context-aware Scaling LawsThis episode explores a recent paper that extends neural scaling laws to predict real-world task performance rather than just training loss, while accounting for context length as a first-order variable. The episode discuss how traditional scaling laws from Kaplan (2020) and Chinchilla (2022) successfully predicted pretraining metrics but failed to address downstream task accuracy or the impact of in-context learning with varying context windows. The paper proposes a context-aware scaling law with dual power-law terms for compute and context, plus a penalty term for exceeding trained context limits, offering a simpler alternative to existing multi-stage prediction methods. Listeners interested in the mathematical foundations of LLM capabilities and the gap between training metrics and practical performance will find this discussion particularly valuable.Sources:1. Predicting Task Performance with Context-aware Scaling Laws — Kyle Montgomery, David Park, Jianhong Tu, Michael Bendersky, Beliz Gunel, Dawn Song, Chenguang Wang, 2025http://arxiv.org/abs/2510.14919v12. Scaling Laws for Neural Language Models — Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, et al., 2020https://scholar.google.com/scholar?q=Scaling+Laws+for+Neural+Language+Models3. Training Compute-Optimal Large Language Models — Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al. (DeepMind), 2022https://scholar.google.com/scholar?q=Training+Compute-Optimal+Large+Language+Models4. Inverse Scaling: When Bigger Isn't Better — Ian McKenzie, Alexander Lyzhov, Michael Pieler, et al., 2023https://scholar.google.com/scholar?q=Inverse+Scaling%3A+When+Bigger+Isn%27t+Better5. Predictability and Surprise in Large Generative Models — Deep Ganguli, Danny Hernandez, Liane Lovitt, et al. (Anthropic), 2022https://scholar.google.com/scholar?q=Predictability+and+Surprise+in+Large+Generative+Models6. YaRN: Efficient Context Window Extension of Large Language Models — Peng et al., 2024https://scholar.google.com/scholar?q=YaRN%3A+Efficient+Context+Window+Extension+of+Large+Language+Models7. Emergent Abilities of Large Language Models — Wei et al., 2022https://scholar.google.com/scholar?q=Emergent+Abilities+of+Large+Language+Models8. Are Emergent Abilities of Large Language Models a Mirage? — Schaeffer et al., 2023https://scholar.google.com/scholar?q=Are+Emergent+Abilities+of+Large+Language+Models+a+Mirage%3F9. Predicting Task Performance with Context-aware Scaling Laws — the episode & the episode, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-predicting-task-performance-with-context-8abe00.mp310. The Art of Scaling Reinforcement Learning Compute for LLMs — the episode & the episode, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-the-art-of-scaling-reinforcement-learnin-95a3f5.mp3Interactive Visualization: Predicting Task Performance with Context-aware Scaling Laws26 min
- Bidaw: Computation-Storage Aware KV Caching for LLMsThis episode explores a new system called Bidaw that dramatically improves the performance of long, multi-turn AI chatbot conversations by solving a critical caching problem. The paper reveals that existing approaches waste over 93% of computation redundantly recalculating conversation history, and that naive two-tier storage systems (using both RAM and SSD) increase latency by 3.8x because the GPU scheduler and storage system don't coordinate. Bidaw introduces "bidirectional awareness" where the scheduler prioritizes requests whose data is already in fast memory while background-loading slower SSD data, and the storage system uses conversation flow patterns to predict which cached data to keep hot. Listeners interested in LLM infrastructure, production ML systems, or the practical challenges of deploying interactive AI services will learn how clever coordination between compute and storage layers can unlock major performance gains without requiring more expensive hardware.Sources:1. Bidaw: Computation-Storage Aware KV Caching for LLMshttps://www.usenix.org/system/files/fast26-hu-shipeng.pdf2. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU — Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, Max Ryabinin, Daniel Y. Fu, Zhiqiang Xie, Beidi Chen, Clark Barrett, Joseph E. Gonzalez, Percy Liang, Christopher Ré, Ion Stoica, Ce Zhang, 2023https://scholar.google.com/scholar?q=FlexGen%3A+High-Throughput+Generative+Inference+of+Large+Language+Models+with+a+Single+GPU3. vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention — Woosuk Kwon, Zhuohan Li, Siddharth Devadas, Ion Stoica, Joseph E. Gonzalez, 2023https://scholar.google.com/scholar?q=vLLM%3A+Easy%2C+Fast%2C+and+Cheap+LLM+Serving+with+PagedAttention4. PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU — Yixin Song, Zeyu Mi, Haotong Xie, Haibo Chen, 2023https://scholar.google.com/scholar?q=PowerInfer%3A+Fast+Large+Language+Model+Serving+with+a+Consumer-grade+GPU5. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, Hao Zhang, 2024https://scholar.google.com/scholar?q=DistServe%3A+Disaggregating+Prefill+and+Decoding+for+Goodput-optimized+Large+Language+Model+Serving6. LineFS: Efficient SmartNIC Offload of a Distributed File System with Pipeline Parallelism — Jongyul Kim, Sangwoo Kang, Juhyeong Ryu, Jaehyeong Im, Seongyeop Jeong, Jin-Soo Kim, 2021https://scholar.google.com/scholar?q=LineFS%3A+Efficient+SmartNIC+Offload+of+a+Distributed+File+System+with+Pipeline+Parallelism7. Flashield: a Hybrid Key-value Cache that Controls Flash Write Amplification — Yiwen Zhang, Xin Chen, Zhuo Chang, Huanchen Zhang, 2019https://scholar.google.com/scholar?q=Flashield%3A+a+Hybrid+Key-value+Cache+that+Controls+Flash+Write+Amplification8. Nexus: A GPU Cluster Engine for Accelerating DNN-Based Video Analysis — Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, Ravi Sundaram, 2019https://scholar.google.com/scholar?q=Nexus%3A+A+GPU+Cluster+Engine+for+Accelerating+DNN-Based+Video+Analysis9. Clockwork: A Scheduler for GPU-Accelerated Deep Learning Serving — Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, Jonathan Mace, 2020https://scholar.google.com/scholar?q=Clockwork%3A+A+Scheduler+for+GPU-Accelerated+Deep+Learning+Serving10. Learning to Cache: Neural Adaptive Caching Policies — Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, Werner Vogels, 2018https://scholar.google.com/scholar?q=Learning+to+Cache%3A+Neural+Adaptive+Caching+Policies11. Semantic Caching for Large Language Models — Zheng Gao, Peiyuan Liu, Junwei Cao, Xin Li, 2023https://scholar.google.com/scholar?q=Semantic+Caching+for+Large+Language+Models12. Predicting User Behavior in Multi-Turn Dialogue Systems — Yun-Nung Chen, Dilek Hakkani-Tür, Gökhan Tür, Jianfeng Gao, Li Deng, 2016https://scholar.google.com/scholar?q=Predicting+User+Behavior+in+Multi-Turn+Dialogue+Systems13. Machine Learning for Storage Systems: A Comprehensive Survey — Jianliang Zhang, Zeke Wang, Tong Zhang, 2023https://scholar.google.com/scholar?q=Machine+Learning+for+Storage+Systems%3A+A+Comprehensive+Survey14. PagedAttention: Efficient Memory Management for LLM Serving — Kwon et al. (vLLM), 2023https://scholar.google.com/scholar?q=PagedAttention%3A+Efficient+Memory+Management+for+LLM+Serving15. Adaptive Replacement Cache (ARC) — Megiddo and Modha, 2003https://scholar.google.com/scholar?q=Adaptive+Replacement+Cache+%28ARC%2916. Learned Cache Replacement Policies — Vietri et al., 2020https://scholar.google.com/scholar?q=Learned+Cache+Replacement+Policies17. LoRA: Low-Rank Adaptation of Large Language Models — Hu et al., 2021https://scholar.google.com/scholar?q=LoRA%3A+Low-Rank+Adaptation+of+Large+Language+Models18. No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization — MiKV authors, 2024-2025https://scholar.google.com/scholar?q=No+Token+Left+Behind%3A+Reliable+KV+Cache+Compression+via+Importance-Aware+Mixed+Precision+Quantization19. CommVQ: Commutative Vector Quantization for KV Cache Compression — CommVQ authors, 2024-2025https://scholar.google.com/scholar?q=CommVQ%3A+Commutative+Vector+Quantization+for+KV+Cache+Compression20. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse — KVLink authors, 2024-2025https://scholar.google.com/scholar?q=KVLink%3A+Accelerating+Large+Language+Models+via+Efficient+KV+Cache+Reuse21. Compute or Load KV Cache? Why Not Both? — Unknown, 2024-2025https://scholar.google.com/scholar?q=Compute+or+Load+KV+Cache%3F+Why+Not+Both%3F22. MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention — MInference authors, 2024-2025https://scholar.google.com/scholar?q=MInference+1.0%3A+Accelerating+Pre-filling+for+Long-Context+LLMs+via+Dynamic+Sparse+Attention23. KVCache Cache in the Wild: Characterizing and Optimizing KVCache at a Large Cloud Provider — Cloud provider study authors, 2024-2025https://scholar.google.com/scholar?q=KVCache+Cache+in+the+Wild%3A+Characterizing+and+Optimizing+KVCache+at+a+Large+Cloud+Provider24. Efficient KV Cache Reuse in Dynamic Agent Workflowshttps://podcast.do-not-panic.com/episodes/2026-03-16-efficient-kv-cache-reuse-in-dynamic-agen-558f19.mp325. 50x KV Cache Compression in Seconds via Attention Matchinghttps://podcast.do-not-panic.com/episodes/2026-03-09-50x-kv-cache-compression-in-seconds-via-9402c1.mp326. Statistical Routing Theory in CARTRIDGE Block Attentionhttps://podcast.do-not-panic.com/episodes/2026-03-16-statistical-routing-theory-in-cartridge-2083f4.mp327. xLLM: Co-Locating Online and Offline LLM Inferencehttps://podcast.do-not-panic.com/episodes/2026-03-16-xllm-co-locating-online-and-offline-llm-10bb81.mp3Interactive Visualization: Bidaw: Computation-Storage Aware KV Caching for LLMs29 min
- Qwen3Guard: Streaming Three-Way Safety Classification for LLMsThis episode explores Qwen3Guard, a safety guardrail system for large language models that introduces two key architectural innovations. The paper presents a three-way classification scheme—safe, controversial, and unsafe—allowing organizations to customize content moderation policies rather than relying on rigid binary thresholds, plus a streaming-compatible variant that evaluates safety token-by-token during generation instead of waiting for complete responses. The episode examine why separate guardrail models provide better defense-in-depth than base model alignment alone, how the controversial label externalizes policy decisions to application logic, and the technical challenges of performing real-time safety assessment without sacrificing streaming user experience or adding prohibitive computational overhead.Sources:1. Qwen3Guard Technical Report — Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, Baosong Yang, Chen Cheng, Jialong Tang, Jiandong Jiang, Jianwei Zhang, Jijie Xu, Ming Yan, Minmin Sun, Pei Zhang, Pengjun Xie, Qiaoyu Tang, Qin Zhu, Rong Zhang, Shibin Wu, Shuo Zhang, Tao He, Tianyi Tang, Tingyu Xia, Wei Liao, Weizhou Shen, Wenbiao Yin, Wenmeng Zhou, Wenyuan Yu, Xiaobin Wang, Xiaodong Deng, Xiaodong Xu, Xinyu Zhang, Yang Liu, Yeqiu Li, Yi Zhang, Yong Jiang, Yu Wan, Yuxin Zhou, 2025http://arxiv.org/abs/2510.14276v12. LlamaGuard: LLM-based Input-Output Safeguard for Human-AI Conversations — Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, et al., 2023https://scholar.google.com/scholar?q=LlamaGuard%3A+LLM-based+Input-Output+Safeguard+for+Human-AI+Conversations3. ShieldGemma: Generative AI Content Moderation Based on Gemma — Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, et al., 2024https://scholar.google.com/scholar?q=ShieldGemma%3A+Generative+AI+Content+Moderation+Based+on+Gemma4. WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs — Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, et al., 2024https://scholar.google.com/scholar?q=WildGuard%3A+Open+One-Stop+Moderation+Tools+for+Safety+Risks%2C+Jailbreaks%2C+and+Refusals+of+LLMs5. Constitutional AI: Harmlessness from AI Feedback — Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, et al., 2022https://scholar.google.com/scholar?q=Constitutional+AI%3A+Harmlessness+from+AI+Feedback6. Real-Time Safety Monitoring for Large Language Models via Token-Level Classification — Example synthetic authors (this is a research area lacking landmark papers pre-2025), 2024https://scholar.google.com/scholar?q=Real-Time+Safety+Monitoring+for+Large+Language+Models+via+Token-Level+Classification7. StreamingLLM: Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, et al., 2023https://scholar.google.com/scholar?q=StreamingLLM%3A+Efficient+Streaming+Language+Models+with+Attention+Sinks8. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads — Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, et al., 2024https://scholar.google.com/scholar?q=Medusa%3A+Simple+LLM+Inference+Acceleration+Framework+with+Multiple+Decoding+Heads9. Perspective API: Identifying Toxicity in Online Conversations — Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, Lucy Vasserman (Google Jigsaw), 2017https://scholar.google.com/scholar?q=Perspective+API%3A+Identifying+Toxicity+in+Online+Conversations10. Toxic Comment Classification Challenge: Kaggle Competition and Dataset — Jigsaw/Conversation AI team (Google), 2018https://scholar.google.com/scholar?q=Toxic+Comment+Classification+Challenge%3A+Kaggle+Competition+and+Dataset11. Explaining the Effectiveness of Multi-Task Learning for Efficient Scale in Content Moderation — Example synthetic (represents broader multi-task moderation research), 2021https://scholar.google.com/scholar?q=Explaining+the+Effectiveness+of+Multi-Task+Learning+for+Efficient+Scale+in+Content+Moderation12. LlamaGuard 2: Customizable Safety Taxonomies for LLM Guardrails — Jianfeng Chi, Kavel Rao, Keshav Santhanam, et al. (Meta), 2024https://scholar.google.com/scholar?q=LlamaGuard+2%3A+Customizable+Safety+Taxonomies+for+LLM+Guardrails13. Multilingual Toxic Comment Classification: An Empirical Study — Ona de Gibert, Naiara Perez, Aitor García-Pablos, Montse Cuadros, 2018https://scholar.google.com/scholar?q=Multilingual+Toxic+Comment+Classification%3A+An+Empirical+Study14. HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection in Multilingual Settings — Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, et al., 2021https://scholar.google.com/scholar?q=HateXplain%3A+A+Benchmark+Dataset+for+Explainable+Hate+Speech+Detection+in+Multilingual+Settings15. Few-Shot Cross-Lingual Transfer for Multilingual Task-Oriented Dialogue Systems — Example synthetic (represents cross-lingual transfer research applicable to safety), 2022https://scholar.google.com/scholar?q=Few-Shot+Cross-Lingual+Transfer+for+Multilingual+Task-Oriented+Dialogue+Systems16. The State and Fate of Linguistic Diversity and Inclusion in the NLP World — Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, Monojit Choudhury, 2020https://scholar.google.com/scholar?q=The+State+and+Fate+of+Linguistic+Diversity+and+Inclusion+in+the+NLP+World17. NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels — approximate (recent, likely 2024-2025), 2024-2025https://scholar.google.com/scholar?q=NExT-Guard%3A+Training-Free+Streaming+Safeguard+without+Token-Level+Labels18. Guardset-X: Massive Multi-Domain Safety Policy-Grounded Guardrail Dataset — approximate (recent), 2024-2025https://scholar.google.com/scholar?q=Guardset-X%3A+Massive+Multi-Domain+Safety+Policy-Grounded+Guardrail+Dataset19. Steering Multimodal Large Language Models Decoding for Context-Aware Safety — approximate, 2024-2025https://scholar.google.com/scholar?q=Steering+Multimodal+Large+Language+Models+Decoding+for+Context-Aware+Safety20. Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning — approximate, 2024-2025https://scholar.google.com/scholar?q=Learning+to+Stay+Safe%3A+Adaptive+Regularization+Against+Safety+Degradation+during+Fine-Tuning21. MalGEN: Multi-Agent AI for Red Teaming Malwarehttps://podcast.do-not-panic.com/episodes/2026-03-08-malgen-multi-agent-ai-for-red-teaming-ma-1c42e4.mp322. Emergent Cooperation in Self-Interested Multi-Agent AIhttps://podcast.do-not-panic.com/episodes/2026-03-13-emergent-cooperation-in-self-interested-9c0b4c.mp323. Model-Aware Tokenizer Transfer for Multilingual LLMshttps://podcast.do-not-panic.com/episodes/2026-03-16-model-aware-tokenizer-transfer-for-multi-90666c.mp3Interactive Visualization: Qwen3Guard: Streaming Three-Way Safety Classification for LLMs32 min
- Memory in the Age of AI Agents: Forms, Functions, DynamicsThis episode examines a comprehensive survey paper that proposes a new framework for understanding memory in AI agent systems. The authors challenge traditional cognitive psychology categories (like short-term versus long-term memory) and instead organize agent memory along three dimensions: form (how memory is stored—as tokens, parameters, or latent vectors), function (what memory represents—factual knowledge, past experiences, or working state), and dynamics (how memory is created, updated, and retrieved over time). The discussion clarifies how agent memory differs from LLM knowledge, RAG systems, and simple prompt engineering, emphasizing that agent memory is fundamentally about stateful, task-specific information that persists and evolves across interactions. Listeners interested in building more sophisticated AI systems will find valuable distinctions between related concepts that are often conflated in practice.Sources:1. Memory in the Age of AI Agents — Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yanbin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, Zhenrong Cheng, Xuanbo Fan, Jiaxin Guo, Xinlei Yu, Zhenhong Zhou, Zewen Hu, Jiahao Huo, Junhao Wang, Yuwei Niu, Yu Wang, Zhenfei Yin, Xiaobin Hu, Yue Liao, Qiankun Li, Kun Wang, Wangchunshu Zhou, Yixin Liu, Dawei Cheng, Qi Zhang, Tao Gui, Shirui Pan, Yan Zhang, Philip Torr, Zhicheng Dou, Ji-Rong Wen, Xuanjing Huang, Yu-Gang Jiang, Shuicheng Yan, 2025http://arxiv.org/abs/2512.13564v22. Generative Agents: Interactive Simulacra of Human Behavior — Park, O'Brien, Cai, Morris, Liang, Bernstein, 2023https://scholar.google.com/scholar?q=Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior3. MemGPT: Towards LLMs as Operating Systems — Packer, Fang, Patil, Wooders, Stoica, Gonzalez, 2023https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems4. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis, Perez, Piktus, Petroni, Karpukhin, Goyal, Küttler, Lewis, Yih, Rocktäschel, Riedel, Kiela, 2020https://scholar.google.com/scholar?q=Retrieval-Augmented+Generation+for+Knowledge-Intensive+NLP+Tasks5. In-Context Learning and Induction Heads — Olsson, Elhage, Nanda, Joseph, Drain, Bau, Schiefer, Ndousse, Henighan, Lovitt, Chen, Kaplan, Anthropic, 2022https://scholar.google.com/scholar?q=In-Context+Learning+and+Induction+Heads6. LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models — Chen, Borgeaud, Mensch, Sutskever, Sifre, Schofield, Shazeer, Lazaridou, de Freitas, others, 2023https://scholar.google.com/scholar?q=LongLoRA%3A+Efficient+Fine-tuning+of+Long-Context+Large+Language+Models7. Lost in the Middle: How Language Models Use Long Contexts — Liu, Iter, Xu, Yuksekgonul, Zou, others, 2023https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts8. Recursively Summarizing Books with Human Feedback — Wu, Ouyang, Ziegler, Stiennon, Lowe, Leike, Christiano, 2021https://scholar.google.com/scholar?q=Recursively+Summarizing+Books+with+Human+Feedback9. Low-Rank Adaptation of Large Language Models (LoRA) — Hu, Shen, Wallis, Allen-Zhu, Li, Wang, Chen, 2021https://scholar.google.com/scholar?q=Low-Rank+Adaptation+of+Large+Language+Models+%28LoRA%2910. Overcoming Catastrophic Forgetting in Neural Networks (EWC) — Kirkpatrick, Pascanu, Rabinowitz, Veness, Desjardins, Rusu, Milan, others, 2017https://scholar.google.com/scholar?q=Overcoming+Catastrophic+Forgetting+in+Neural+Networks+%28EWC%2911. Model-Agnostic Meta-Learning (MAML) — Finn, Abbeel, Levine, 2017https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+%28MAML%2912. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context — Dai, Yang, Yang, Carbonell, Le, Salakhutdinov, 2019https://scholar.google.com/scholar?q=Transformer-XL%3A+Attentive+Language+Models+Beyond+a+Fixed-Length+Context13. Memorizing Transformers — Wu, Rabe, Hutchins, Szlam, 2022https://scholar.google.com/scholar?q=Memorizing+Transformers14. Retrieval-Enhanced Transformer (RETRO) — Borgeaud, Mensch, Hoffmann, Cai, Rutherford, Millican, van den Driessche, others, 2022https://scholar.google.com/scholar?q=Retrieval-Enhanced+Transformer+%28RETRO%2915. CommNet: Learning Multiagent Communication with Backpropagation — Sukhbaatar, Szlam, Fergus, 2016https://scholar.google.com/scholar?q=CommNet%3A+Learning+Multiagent+Communication+with+Backpropagation16. Learning to Communicate with Deep Multi-Agent Reinforcement Learning — Foerster, Assael, de Freitas, Whiteson, 2016https://scholar.google.com/scholar?q=Learning+to+Communicate+with+Deep+Multi-Agent+Reinforcement+Learning17. Emergent Tool Use from Multi-Agent Autocurricula — Baker, Kanitscheider, Markov, Wu, Powell, McGrew, Mordatch (OpenAI), 2020https://scholar.google.com/scholar?q=Emergent+Tool+Use+from+Multi-Agent+Autocurricula18. Reflexion: Language Agents with Verbal Reinforcement Learning — Shinn et al., 2023https://scholar.google.com/scholar?q=Reflexion%3A+Language+Agents+with+Verbal+Reinforcement+Learning19. RET-LLM: Towards a General Read-Write Memory for Large Language Models — Modarressi et al., 2024https://scholar.google.com/scholar?q=RET-LLM%3A+Towards+a+General+Read-Write+Memory+for+Large+Language+Models20. Voyager: An Open-Ended Embodied Agent with Large Language Models — Wang et al., 2023https://scholar.google.com/scholar?q=Voyager%3A+An+Open-Ended+Embodied+Agent+with+Large+Language+Models21. In-Context Retrieval-Augmented Language Models — Ram et al., 2023https://scholar.google.com/scholar?q=In-Context+Retrieval-Augmented+Language+Models22. Scaling Laws for Associative Memories — Ramsauer et al., 2021https://scholar.google.com/scholar?q=Scaling+Laws+for+Associative+Memories23. Do Machine Learning Models Memorize or Generalize? — Feldman, 2020https://scholar.google.com/scholar?q=Do+Machine+Learning+Models+Memorize+or+Generalize%3F24. LongTableBench: benchmarking long-context table reasoning across real-world formats and domains — approximate, 2024-2025https://scholar.google.com/scholar?q=LongTableBench%3A+benchmarking+long-context+table+reasoning+across+real-world+formats+and+domains25. Reasoning-Focused Evaluation of Efficient Long-Context Inference Techniques — approximate, 2024-2025https://scholar.google.com/scholar?q=Reasoning-Focused+Evaluation+of+Efficient+Long-Context+Inference+Techniques26. Cognitive Workspace: Active Memory Management for LLMs—An Empirical Study of Functional Infinite Context — approximate, 2024-2025https://scholar.google.com/scholar?q=Cognitive+Workspace%3A+Active+Memory+Management+for+LLMs%E2%80%94An+Empirical+Study+of+Functional+Infinite+Context27. Continual learning and catastrophic forgetting — approximate, 2020shttps://scholar.google.com/scholar?q=Continual+learning+and+catastrophic+forgetting28. ESSENTIAL: Episodic and Semantic Memory Integration for Video Class-Incremental Learning — approximate, 2024-2025https://scholar.google.com/scholar?q=ESSENTIAL%3A+Episodic+and+Semantic+Memory+Integration+for+Video+Class-Incremental+Learning29. Adaptive compression as a unifying framework for episodic and semantic memory — approximate, 2020shttps://scholar.google.com/scholar?q=Adaptive+compression+as+a+unifying+framework+for+episodic+and+semantic+memory30. MAG: Memory Augmented Knowledge Extraction Generation for Large Language Models — approximate, 2024-2025https://scholar.google.com/scholar?q=MAG%3A+Memory+Augmented+Knowledge+Extraction+Generation+for+Large+Language+Models31. Gradient Descent at Inference Time for LLM Reasoninghttps://podcast.do-not-panic.com/episodes/2026-03-10-gradient-descent-at-inference-time-for-l-20617d.mp332. LLM Agents Reason About Code Without Running Ithttps://podcast.do-not-panic.com/episodes/2026-03-15-llm-agents-reason-about-code-without-run-2a1876.mp333. Emergent Cooperation in Self-Interested Multi-Agent AIhttps://podcast.do-not-panic.com/episodes/2026-03-13-emergent-cooperation-in-self-interested-9c0b4c.mp334. 50x KV Cache Compression in Seconds via Attention Matchinghttps://podcast.do-not-panic.com/episodes/2026-03-09-50x-kv-cache-compression-in-seconds-via-9402c1.mp335. DualPath Breaks Storage Bandwidth Bottleneck in Agentic Inferencehttps://podcast.do-not-panic.com/episodes/2026-03-07-dualpath-breaks-storage-bandwidth-bottle-bc9a82.mp336. NVIDIA Nemotron 3 Hybrid SSM Transformer Architecturehttps://podcast.do-not-panic.com/episodes/2026-03-07-nvidia-nemotron-3-hybrid-ssm-transformer-f9a91b.mp337. Structured State Space Duality Unifies Transformers and SSMshttps://podcast.do-not-panic.com/episodes/2026-03-07-structured-state-space-duality-unifies-t-bb2659.mp338. Why CARTRIDGE Works: Keys as Routers in KV Cacheshttps://podcast.do-not-panic.com/episodes/2026-03-07-why-cartridge-works-keys-as-routers-in-k-887d13.mp3Interactive Visualization: Memory in the Age of AI Agents: Forms, Functions, Dynamics35 min
About AI Post Transformers
From the publisher's feed
AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis,…
proper citations, and nerdy humor.