Sign up to save your podcasts
Or
Share AI Post Transformers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
AI Post Transformers
By mcgrofTechnology
AI Post Transformers episodes
- The Abstraction Fallacy and AI ConsciousnessThis episode explores a philosophical challenge to computational functionalism through Alexander Lerchner’s “The Abstraction Fallacy,” which argues that software can simulate conscious behavior without ever producing real subjective experience. It examines whether computation is an objective physical process or an interpretation imposed on physical systems, contrasting Lerchner’s “mapmaker” idea with functionalist views from Putnam, Fodor, Chalmers, and mechanistic accounts of computation. The discussion also connects the debate to current AI policy, questioning whether popular consciousness indicators and AI welfare arguments rest on assumptions about computation that may be weaker than they appear. Listeners would find it interesting because it moves beyond “are today’s models conscious?” to a deeper claim about whether computation alone could ever make any machine conscious at all.Sources:1. The Abstraction Fallacy and AI Consciousnesshttps://philpapers.org/archive/LERTAF.pdf2. Consciousness in Artificial Intelligence: Insights from the Science of Consciousness — Robert Long, Patrick Butlin, Yoshua Bengio, Jonathan Birch, Eric Elmoznino, Matthew Crosby, and others, 2024https://scholar.google.com/scholar?q=Consciousness+in+Artificial+Intelligence%3A+Insights+from+the+Science+of+Consciousness3. Could a Large Language Model Be Conscious? — David J. Chalmers, 2023https://scholar.google.com/scholar?q=Could+a+Large+Language+Model+Be+Conscious%3F4. The Conscious Mind: In Search of a Fundamental Theory — David J. Chalmers, 1996https://scholar.google.com/scholar?q=The+Conscious+Mind%3A+In+Search+of+a+Fundamental+Theory5. The Hidden Spring: A Journey to the Source of Consciousness — Mark Solms, 2021https://scholar.google.com/scholar?q=The+Hidden+Spring%3A+A+Journey+to+the+Source+of+Consciousness6. The Nature of Mental States — Hilary Putnam, 1967https://scholar.google.com/scholar?q=The+Nature+of+Mental+States7. Psychological Predicates — Jerry A. Fodor, 1968https://scholar.google.com/scholar?q=Psychological+Predicates8. Minds, Brains, and Programs — John R. Searle, 1980https://scholar.google.com/scholar?q=Minds%2C+Brains%2C+and+Programs9. Consciousness Explained — Daniel C. Dennett, 1991https://scholar.google.com/scholar?q=Consciousness+Explained10. Why the Mind Is Not a Computer — John R. Searle, 1990https://scholar.google.com/scholar?q=Why+the+Mind+Is+Not+a+Computer11. Computation and Cognition: Why Minds are not Machines — Zenon W. Pylyshyn, 1980https://scholar.google.com/scholar?q=Computation+and+Cognition%3A+Why+Minds+are+not+Machines12. Computation, Causation, and Computational Explanation — Gualtiero Piccinini, 2007https://scholar.google.com/scholar?q=Computation%2C+Causation%2C+and+Computational+Explanation13. Physical Computation: A Mechanistic Account — Gualtiero Piccinini, 2015https://scholar.google.com/scholar?q=Physical+Computation%3A+A+Mechanistic+Account14. Troubles with Functionalism — Ned Block, 1978https://scholar.google.com/scholar?q=Troubles+with+Functionalism15. A Computer Simulation of the Cerebral Cortex: Forthcoming or Impossible? — Stevan Harnad, 1994https://scholar.google.com/scholar?q=A+Computer+Simulation+of+the+Cerebral+Cortex%3A+Forthcoming+or+Impossible%3F16. The Emperor's New Mind — Roger Penrose, 1989https://scholar.google.com/scholar?q=The+Emperor%27s+New+Mind17. Taking AI Welfare Seriously — Robert Long and Patrick Butlin, 2025https://scholar.google.com/scholar?q=Taking+AI+Welfare+Seriously18. The Moral Status of Artificial Intelligence — David J. Gunkel, 2020https://scholar.google.com/scholar?q=The+Moral+Status+of+Artificial+Intelligence19. Moral Consideration for Artificial Entities — Thomas Metzinger, 2021https://scholar.google.com/scholar?q=Moral+Consideration+for+Artificial+Entities20. A Computational Foundation for the Study of Cognition — David J. Chalmers, 1996https://scholar.google.com/scholar?q=A+Computational+Foundation+for+the+Study+of+Cognition21. Computationalism about Consciousness — Hilary Putnam, 1988https://scholar.google.com/scholar?q=Computationalism+about+Consciousness22. The Biological Turn in Consciousness Science — Anil Seth, 2025https://scholar.google.com/scholar?q=The+Biological+Turn+in+Consciousness+Science23. Why Computers Can't Be Conscious — Ned Block, 2025https://scholar.google.com/scholar?q=Why+Computers+Can%27t+Be+Conscious24. The Bitter Lesson — Rich Sutton, 2019https://scholar.google.com/scholar?q=The+Bitter+Lesson25. Language Models are Few-Shot Learners — Tom B. Brown et al., 2020https://scholar.google.com/scholar?q=Language+Models+are+Few-Shot+Learners26. Sparks of Artificial General Intelligence: Early Experiments with GPT-4 — Sébastien Bubeck et al., 2023https://scholar.google.com/scholar?q=Sparks+of+Artificial+General+Intelligence%3A+Early+Experiments+with+GPT-427. An idealised account of mechanistic computation — approx. Gualtiero Piccinini / mechanistic-computation literature, recenthttps://scholar.google.com/scholar?q=An+idealised+account+of+mechanistic+computation28. Accounts of Computation in Physical Systems — approx. philosophy of computation authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Accounts+of+Computation+in+Physical+Systems29. Physical computing: a category theoretic perspective on physical computation and system compositionality — approx. recent philosophy/computation authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Physical+computing%3A+a+category+theoretic+perspective+on+physical+computation+and+system+compositionality30. Beyond computational functionalism: the behavioral inference principle for machine consciousness — approx. recent machine-consciousness authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Beyond+computational+functionalism%3A+the+behavioral+inference+principle+for+machine+consciousness31. Computation+ x: A constraint-based approach to investigating artificial consciousness — approx. recent artificial consciousness authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Computation%2B+x%3A+A+constraint-based+approach+to+investigating+artificial+consciousness32. Simulated Minds, Artificial Consciousness, and the Theory-Relative Simulation Hypothesis — approx. recent philosophy of mind / simulation authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Simulated+Minds%2C+Artificial+Consciousness%2C+and+the+Theory-Relative+Simulation+Hypothesis33. Substrate Independence — approx. recent philosophy of mind authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Substrate+Independence34. Assessing AI Consciousness: A Substrate-Independent Framework — approx. recent AI consciousness authors; exact authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Assessing+AI+Consciousness%3A+A+Substrate-Independent+Framework35. AI Post Transformers: Neural Computers as Learned Latent Runtimes — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-11-neural-computers-as-learned-latent-runti-9fa282.mp336. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp337. AI Post Transformers: Neural Chameleons and Evading Activation Monitors — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-14-neural-chameleons-and-evading-activation-bc470e.mp338. AI Post Transformers: Dragon Hatchling: Brain-Inspired AI Architecture — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/dragon-hatchling-brain-inspired-ai-architecture/39. AI Post Transformers: Learning Latent Action World Models from Video — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-09-learning-latent-action-world-models-from-1570a4.mp3Interactive Visualization: The Abstraction Fallacy and AI Consciousness0 min
- Prefill-as-a-Service for Cross-Datacenter KV CacheThis episode explores whether newer hybrid-attention language models make prefill-decode disaggregation practical across clusters or even datacenters by shrinking the KV cache enough to move it over ordinary Ethernet. It explains why the real production bottleneck is not request routing but transferring the attention state between prefill and decode, and contrasts dense transformers—where KV cache grows heavily with context across many layers—with hybrid designs that use fewer full-attention layers and more bounded-state alternatives. The discussion highlights the paper’s central claim that smaller KV footprints could enable remote, compute-dense prefill clusters and local decode clusters, especially for long, uncached prompts, while also questioning how broadly that conclusion generalizes given the evidence comes from a single internal 1-trillion-parameter model. Listeners would find it interesting for its concrete systems view of where disaggregated inference actually breaks, and for its argument that model architecture—not just serving software—may determine whether cross-cluster AI serving is viable.Sources:1. Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter — Ruoyu Qin, Weiran He, Yaoyu Wang, Zheming Li, Xinran Xu, Yongwei Wu, Weimin Zheng, Mingxing Zhang, 2026http://arxiv.org/abs/2604.150392. Mooncake: Trading More Storage for Less Computation — KVCache-centric Architecture for Serving LLM Chatbot — Ruoyu Qin, Zhenheng Tang, Weian Zhao, Zeming Chen, and collaborators, 2024https://scholar.google.com/scholar?q=Mooncake%3A+Trading+More+Storage+for+Less+Computation+%E2%80%94+KVCache-centric+Architecture+for+Serving+LLM+Chatbot3. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — Ying Sheng, Yilun Du, and collaborators, 2024https://scholar.google.com/scholar?q=DistServe%3A+Disaggregating+Prefill+and+Decoding+for+Goodput-optimized+Large+Language+Model+Serving4. TetriInfer: Reconfigurable Inference Serving for Long-Context LLMs with KV Cache Dynamics — Authors vary by version; commonly cited as a systems paper from 2024 on long-context serving, 2024https://scholar.google.com/scholar?q=TetriInfer%3A+Reconfigurable+Inference+Serving+for+Long-Context+LLMs+with+KV+Cache+Dynamics5. Prefill-as-a-Service: KVCache of Next-Generation Models Could Go Cross-Datacenter — Ruoyu Qin, Weiran He, Yaoyu Wang, Zheming Li, Xinran Xu, Yongwei Wu, Weimin Zheng, Mingxing Zhang, 2026https://scholar.google.com/scholar?q=Prefill-as-a-Service%3A+KVCache+of+Next-Generation+Models+Could+Go+Cross-Datacenter6. SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills — Akhil Agrawal and collaborators, 2024https://scholar.google.com/scholar?q=SARATHI%3A+Efficient+LLM+Inference+by+Piggybacking+Decodes+with+Chunked+Prefills7. Splitwise: Efficient Generative LLM Inference Using Phase Splitting — Authors commonly cited from the phase-splitting serving literature in 2024, 2024https://scholar.google.com/scholar?q=Splitwise%3A+Efficient+Generative+LLM+Inference+Using+Phase+Splitting8. Mooncake — Referenced as [22] in the paper, Unknown from excerpthttps://scholar.google.com/scholar?q=Mooncake9. vLLM — Kwon et al. and collaborators, 2023https://scholar.google.com/scholar?q=vLLM10. SGLang — Referenced as [7] in the paper, Likely 2024https://scholar.google.com/scholar?q=SGLang11. Dynamo — Referenced as [20] in the paper, Likely 2024 or 2025https://scholar.google.com/scholar?q=Dynamo12. Kimi Linear — Referenced as [26] in the paper, Likely 2025 or 2026https://scholar.google.com/scholar?q=Kimi+Linear13. Ring-2.5-1T — Referenced as [3] in the paper, Likely 2025 or 2026https://scholar.google.com/scholar?q=Ring-2.5-1T14. Lightning — Referenced as [23] in the paper, Unknown from excerpthttps://scholar.google.com/scholar?q=Lightning15. Multi-Head Latent Attention (MLA) — Referenced as [12] in the paper, Unknown from excerpthttps://scholar.google.com/scholar?q=Multi-Head+Latent+Attention+%28MLA%2916. LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference — approx. enterprise systems / LLM serving authors, 2024/2025https://scholar.google.com/scholar?q=LMCache%3A+An+Efficient+KV+Cache+Layer+for+Enterprise-Scale+LLM+Inference17. HotPrefix: Hotness-Aware KV Cache Scheduling for Efficient Prefix Sharing in LLM Inference Systems — approx. systems authors, 2024/2025https://scholar.google.com/scholar?q=HotPrefix%3A+Hotness-Aware+KV+Cache+Scheduling+for+Efficient+Prefix+Sharing+in+LLM+Inference+Systems18. KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse — approx. systems authors, 2024/2025https://scholar.google.com/scholar?q=KVShare%3A+An+LLM+Service+System+with+Efficient+and+Effective+Multi-Tenant+KV+Cache+Reuse19. Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference — approx. security/systems authors, 2024/2025https://scholar.google.com/scholar?q=Selective+KV-Cache+Sharing+to+Mitigate+Timing+Side-Channels+in+LLM+Inference20. CacheSolidarity: Preventing Prefix Caching Side Channels in Multi-tenant LLM Serving Systems — approx. security authors, 2024/2025https://scholar.google.com/scholar?q=CacheSolidarity%3A+Preventing+Prefix+Caching+Side+Channels+in+Multi-tenant+LLM+Serving+Systems21. WindServe: Efficient Phase-Disaggregated LLM Serving with Stream-Based Dynamic Scheduling — approx. systems authors, 2024/2025https://scholar.google.com/scholar?q=WindServe%3A+Efficient+Phase-Disaggregated+LLM+Serving+with+Stream-Based+Dynamic+Scheduling22. SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving — approx. edge/serving authors, 2024/2025https://scholar.google.com/scholar?q=SLED%3A+A+Speculative+LLM+Decoding+Framework+for+Efficient+Edge+Serving23. LServe: Efficient Long-Sequence LLM Serving with Unified Sparse Attention — approx. systems/ML authors, 2024/2025https://scholar.google.com/scholar?q=LServe%3A+Efficient+Long-Sequence+LLM+Serving+with+Unified+Sparse+Attention24. On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention for Long-Context LLM Serving — approx. ML systems authors, 2024/2025https://scholar.google.com/scholar?q=On-the-Fly+Adaptive+Distillation+of+Transformer+to+Dual-State+Linear+Attention+for+Long-Context+LLM+Serving25. AI Post Transformers: Splitwise: Phase-Split LLM Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-26-splitwise-phase-split-llm-inference-e8945b.mp326. AI Post Transformers: From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-22-from-prefix-cache-to-fusion-rag-9c5d39.mp327. AI Post Transformers: FengHuang for Rack-Scale LLM Inference Memory — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-12-fenghuang-for-rack-scale-llm-inference-m-62708e.mp328. AI Post Transformers: CXL-SpecKV: Bridging the LLM Memory Wall with Speculative FPGA Disaggregation — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/cxl-speckv-bridging-the-llm-memory-wall-with-speculative-fpga-disaggregation/29. AI Post Transformers: KVSwap for Disk-Aware Long-Context On-Device Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-kvswap-for-disk-aware-long-context-on-de-f3c15e.mp330. AI Post Transformers: FAST26: Bidaw: Enhancing Key-Value Caching for Interactive LLM Serving via Bidirectional — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/fast26-bidaw-enhancing-key-value-caching-for-interactive-llm-serving-via-bidirec/31. AI Post Transformers: Computation-Bandwidth-Memory Trade-offs for AI Infrastructure — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-09-computation-bandwidth-memory-trade-offs-a83f2b.mp332. AI Post Transformers: Mamba-3 for Efficient Sequence Modeling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-mamba-3-for-efficient-sequence-modeling-97a22a.mp3Interactive Visualization: Prefill-as-a-Service for Cross-Datacenter KV Cache20 min
- Gated Delta Networks for Long-Context RetrievalThis episode explores Gated Delta Networks, a sequence-modeling approach that combines Mamba-style gating with DeltaNet-style selective memory updates to improve long-context and retrieval-heavy performance. It explains how linear attention and state-space models compress the past into a fixed recurrent state, why that makes them hardware-efficient, and where they often fail: memory collisions that blur stored associations and weaken retrieval. The discussion argues that gating is useful for broad forgetting while delta updates enable targeted overwrites, making their combination a promising way to preserve retrieval quality without the quadratic costs of standard attention. Listeners would find it interesting for its clear framing of the tradeoff between efficiency and memory fidelity, and for its practical focus on whether these architectures can move beyond elegant theory into GPU-friendly, real-world use.Sources:1. Gated Delta Networks: Improving Mamba2 with Delta Rule — Songlin Yang, Jan Kautz, Ali Hatamizadeh, 2024http://arxiv.org/abs/2412.064642. Linear Transformers Are Secretly Fast Weight Programmers — Michael Schlag, Kazuki Irie, Jürgen Schmidhuber, 2021https://scholar.google.com/scholar?q=Linear+Transformers+Are+Secretly+Fast+Weight+Programmers3. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention — Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François Fleuret, 2020https://scholar.google.com/scholar?q=Transformers+are+RNNs%3A+Fast+Autoregressive+Transformers+with+Linear+Attention4. The Delta Transformer: A Highly Efficient and Effective Transformer Architecture for Sequence Modeling — Songlin Yang, Bailin Wang, Yikang Shen, Yoon Kim, others, 2024https://scholar.google.com/scholar?q=The+Delta+Transformer%3A+A+Highly+Efficient+and+Effective+Transformer+Architecture+for+Sequence+Modeling5. Gated Linear Attention Transformers with Hardware-Efficient Training — Songlin Yang, others, 2024https://scholar.google.com/scholar?q=Gated+Linear+Attention+Transformers+with+Hardware-Efficient+Training6. The Delta Transformer: A Highly Effective Transformer Layer for Sequence Modeling — David E. Rumelhart? / not this one; cited lineage centers on Schlag et al., 2021https://scholar.google.com/scholar?q=The+Delta+Transformer%3A+A+Highly+Effective+Transformer+Layer+for+Sequence+Modeling7. Titans? / DeltaNet follow-up by Yang et al. 2024b — Songlin Yang and collaborators, 2024https://scholar.google.com/scholar?q=Titans%3F+%2F+DeltaNet+follow-up+by+Yang+et+al.+2024b8. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality — Tri Dao, Albert Gu, 2024https://scholar.google.com/scholar?q=Transformers+are+SSMs%3A+Generalized+Models+and+Efficient+Algorithms+Through+Structured+State+Space+Duality9. Linear Transformers as Associative Memories — Michael Schlag, Kazuki Irie, Jürgen Schmidhuber, 2021https://scholar.google.com/scholar?q=Linear+Transformers+as+Associative+Memories10. Learning to (Learn at Test Time): RNNs with Expressive Hidden States — Akyürek et al., 2024https://scholar.google.com/scholar?q=Learning+to+%28Learn+at+Test+Time%29%3A+RNNs+with+Expressive+Hidden+States11. From Neural Network Learning to Associative Memories — Bernard Widrow and collaborators, 1960https://scholar.google.com/scholar?q=From+Neural+Network+Learning+to+Associative+Memories12. Householder and WY Representations / WY Representation for Products of Householder Matrices — C. Bischof, Charles Van Loan, 1985https://scholar.google.com/scholar?q=Householder+and+WY+Representations+%2F+WY+Representation+for+Products+of+Householder+Matrices13. Hungry Hungry Hippos: Towards Language Modeling with State Space Models — Hua et al., 2022https://scholar.google.com/scholar?q=Hungry+Hungry+Hippos%3A+Towards+Language+Modeling+with+State+Space+Models14. SILA: Enhancing Long-Context Retrieval Capability of Linear Attention via Selective Ignoring — approx. recent linear-attention/long-context authors, 2025https://scholar.google.com/scholar?q=SILA%3A+Enhancing+Long-Context+Retrieval+Capability+of+Linear+Attention+via+Selective+Ignoring15. Simple linear attention language models balance the recall-throughput tradeoff — approx. recent linear-attention LM authors, 2025https://scholar.google.com/scholar?q=Simple+linear+attention+language+models+balance+the+recall-throughput+tradeoff16. A systematic analysis of hybrid linear attention — approx. recent hybrid-attention authors, 2025https://scholar.google.com/scholar?q=A+systematic+analysis+of+hybrid+linear+attention17. Understanding transformer from the perspective of associative memory — approx. recent associative-memory authors, 2024 or 2025https://scholar.google.com/scholar?q=Understanding+transformer+from+the+perspective+of+associative+memory18. Bayesian Optimality of In-Context Learning with Selective State Spaces — approx. recent selective-SSM theory authors, 2025https://scholar.google.com/scholar?q=Bayesian+Optimality+of+In-Context+Learning+with+Selective+State+Spaces19. Sliding window attention training for efficient large language models — approx. SWAT authors, 2025https://scholar.google.com/scholar?q=Sliding+window+attention+training+for+efficient+large+language+models20. SWAA: Sliding Window Attention Adaptation for Efficient Long-Context LLMs Without Pretraining — approx. SWAA authors, 2025https://scholar.google.com/scholar?q=SWAA%3A+Sliding+Window+Attention+Adaptation+for+Efficient+Long-Context+LLMs+Without+Pretraining21. Short window attention enables long-term memorization — approx. recent long-context attention authors, 2025https://scholar.google.com/scholar?q=Short+window+attention+enables+long-term+memorization22. AI Post Transformers: Kimi Linear: Efficient Expressive Attention Architecture — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/kimi-linear-efficient-expressive-attention-architecture/23. AI Post Transformers: Mamba-3 for Efficient Sequence Modeling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-mamba-3-for-efficient-sequence-modeling-97a22a.mp324. AI Post Transformers: Jet-Nemotron and PostNAS for Faster Long Context — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-24-jet-nemotron-and-postnas-for-faster-long-436381.mp325. AI Post Transformers: Longformer: A Transformer for Long Documents — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/longformer-a-transformer-for-long-documents/26. AI Post Transformers: Optimizing Mixture of Block Attention Through Statistical Theory — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-18-optimizing-mixture-of-block-attention-th-214f91.mp3Interactive Visualization: Gated Delta Networks for Long-Context Retrieval19 min
- Defining AGI as Adaptive Artificial ScientistThis episode explores a provocative 2025 paper that argues AGI has become too vague to be useful and should instead be defined as broad adaptive competence under limited knowledge and resources. It examines the paper’s proposal to treat AGI as an “artificial scientist” capable of forming hypotheses, testing models, and improving understanding across domains, while also debating whether that framing is genuinely measurable or just a more sophisticated metaphor. The discussion compares this view with major intelligence frameworks from Legg and Hutter, Chollet, and Pei Wang, and highlights the paper’s central critique of “computational dualism” — the mistake of judging intelligence as software alone while ignoring hardware, embodiment, latency, and energy constraints. Listeners would find it interesting because it connects abstract AGI debates to concrete technical ideas like search, approximation, scaling, and hardware-aware design, offering a sharper lens for thinking about what advanced AI systems should actually be able to do.Sources:1. What the F*ck Is Artificial General Intelligence? — Michael Timothy Bennett, 2025http://arxiv.org/abs/2503.239232. Artificial General Intelligence: Concept, State of the Art, and Future Prospects — Ben Goertzel, 2014https://scholar.google.com/scholar?q=Artificial+General+Intelligence%3A+Concept%2C+State+of+the+Art%2C+and+Future+Prospects3. On the Measure of Intelligence — Shane Legg and Marcus Hutter, 2007https://scholar.google.com/scholar?q=On+the+Measure+of+Intelligence4. On the Nature of Intelligence — Pei Wang, 1995https://scholar.google.com/scholar?q=On+the+Nature+of+Intelligence5. The Bitter Lesson — Richard Sutton, 2019https://scholar.google.com/scholar?q=The+Bitter+Lesson6. Mastering the Game of Go with Deep Neural Networks and Tree Search — David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, et al., 2016https://scholar.google.com/scholar?q=Mastering+the+Game+of+Go+with+Deep+Neural+Networks+and+Tree+Search7. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm — David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, et al., 2018https://scholar.google.com/scholar?q=Mastering+Chess+and+Shogi+by+Self-Play+with+a+General+Reinforcement+Learning+Algorithm8. Planning with Large Language Models for Code Generation — Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, Mike Lewis, 2023https://scholar.google.com/scholar?q=Planning+with+Large+Language+Models+for+Code+Generation9. The Unreasonable Effectiveness of Data — Alon Halevy, Peter Norvig, Fernando Pereira, 2009https://scholar.google.com/scholar?q=The+Unreasonable+Effectiveness+of+Data10. Deep Learning — Yann LeCun, Yoshua Bengio, Geoffrey Hinton, 2015https://scholar.google.com/scholar?q=Deep+Learning11. Scaling Laws for Neural Language Models — Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, et al., 2020https://scholar.google.com/scholar?q=Scaling+Laws+for+Neural+Language+Models12. Emergent Abilities of Large Language Models — Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, et al., 2022https://scholar.google.com/scholar?q=Emergent+Abilities+of+Large+Language+Models13. A Definition of Artificial Intelligence — Shane Legg and Marcus Hutter, 2007https://scholar.google.com/scholar?q=A+Definition+of+Artificial+Intelligence14. Can Deep Learning Lead to Artificial General Intelligence? — Melanie Mitchell, 2021https://scholar.google.com/scholar?q=Can+Deep+Learning+Lead+to+Artificial+General+Intelligence%3F15. AIXI — Marcus Hutter, 2005https://scholar.google.com/scholar?q=AIXI16. Reinforcement Learning with A* and a Deep Heuristic — Various AERA/NARS/related authors depending on exact citation context, varieshttps://scholar.google.com/scholar?q=Reinforcement+Learning+with+A%2A+and+a+Deep+Heuristic17. Levels of AGI for Operationalizing Progress on the Path to AGI — approx. authors include Google DeepMind/OpenAI-affiliated researchers; likely Morris, Brundage, et al., 2024https://scholar.google.com/scholar?q=Levels+of+AGI+for+Operationalizing+Progress+on+the+Path+to+AGI18. Position: Levels of AGI for operationalizing progress on the path to AGI — approx. same author group as the paper above, 2024https://scholar.google.com/scholar?q=Position%3A+Levels+of+AGI+for+operationalizing+progress+on+the+path+to+AGI19. On the timescales of embodied intelligence for autonomous adaptive systems — approximate; authors unclear from snippet, recent, likely 2024 or 2025https://scholar.google.com/scholar?q=On+the+timescales+of+embodied+intelligence+for+autonomous+adaptive+systems20. The concept of embodied human intelligence: power and limits — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=The+concept+of+embodied+human+intelligence%3A+power+and+limits21. Neurosymbolic AI: towards sound reasoning and causal learning and the road to AGI — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Neurosymbolic+AI%3A+towards+sound+reasoning+and+causal+learning+and+the+road+to+AGI22. Enhancing Cognitive Functions in Large Language Models Towards AGI: A State-of-the-Art Survey and Exploratory Review — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Enhancing+Cognitive+Functions+in+Large+Language+Models+Towards+AGI%3A+A+State-of-the-Art+Survey+and+Exploratory+Review23. Towards AGI? Evaluating Current Limitations of Foundation Models in Reasoning Tasks — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Towards+AGI%3F+Evaluating+Current+Limitations+of+Foundation+Models+in+Reasoning+Tasks24. Efficient reasoning models: A survey — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Efficient+reasoning+models%3A+A+survey25. Efficient inference for large reasoning models: A survey — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=Efficient+inference+for+large+reasoning+models%3A+A+survey26. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond — approximate; authors unclear from snippet, recenthttps://scholar.google.com/scholar?q=A+survey+of+efficient+reasoning+for+large+reasoning+models%3A+Language%2C+multimodality%2C+and+beyond27. AI Post Transformers: Kosmos AI Scientist for Autonomous Discovery — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-kosmos-ai-scientist-for-autonomous-disco-311775.mp328. AI Post Transformers: Agentic AI and the Next Intelligence Explosion — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-28-agentic-ai-and-the-next-intelligence-exp-d06561.mp329. AI Post Transformers: MAML and the Basics of Meta-Learning — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-29-maml-and-the-basics-of-meta-learning-7d449f.mp330. AI Post Transformers: Procgen Benchmark: Measuring Generalization in Reinforcement Learning — Hal Turing & Dr. Ada Shannon, Fri,https://podcast.do-not-panic.com/episodes/procgen-benchmark-measuring-generalization-in-reinforcement-learning/31. AI Post Transformers: Memory in the Age of AI Agents: Forms, Functions, Dynamics — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-16-memory-in-the-age-of-ai-agents-forms-fun-5abc60.mp332. AI Post Transformers: Experimental Comparison of Agentic and Enhanced RAG — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-14-experimental-comparison-of-agentic-and-e-37d8bc.mp333. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp334. AI Post Transformers: Neural Computers as Learned Latent Runtimes — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-11-neural-computers-as-learned-latent-runti-9fa282.mp335. AI Post Transformers: SkillsBench for Evaluating Agent Skills — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-14-skillsbench-for-evaluating-agent-skills-58bb1e.mp3Interactive Visualization: Defining AGI as Adaptive Artificial Scientist20 min
- Gated Linear Attention for Efficient Long SequencesThis episode explores a paper on Gated Linear Attention Transformers that aims to make long-sequence modeling both higher quality and genuinely faster on modern GPUs. It explains how GLA replaces standard softmax attention with a gated, recurrent-style memory update that can better decide what information to keep, decay, or overwrite, positioning it between classic linear attention, RetNet-style decay models, and state-space approaches like Mamba. The discussion argues that earlier linear-attention methods often failed twice—underperforming on model quality and losing to optimized softmax baselines such as FlashAttention-2—so the real test is hardware efficiency, not just better asymptotic complexity. Listeners would find it interesting for its clear breakdown of why memory traffic, chunked training, and on-chip SRAM usage may determine whether linear attention becomes a practical alternative for long-context AI systems.Sources:1. Gated Linear Attention Transformers with Hardware-Efficient Training — Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, Yoon Kim, 2023http://arxiv.org/abs/2312.066352. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention — Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, François Fleuret, 2020https://scholar.google.com/scholar?q=Transformers+are+RNNs%3A+Fast+Autoregressive+Transformers+with+Linear+Attention3. Rethinking Attention with Performers — Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos and others, 2021https://scholar.google.com/scholar?q=Rethinking+Attention+with+Performers4. Retentive Network: A Successor to Transformer for Large Language Models — Yutao Sun, Li Dong, Shaohan Huang, Shuming Ma, Yuqing Xia, Jilong Xue and others, 2023https://scholar.google.com/scholar?q=Retentive+Network%3A+A+Successor+to+Transformer+for+Large+Language+Models5. Gated Linear Attention Transformers with Hardware-Efficient Training — Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, Yoon Kim, 2024https://scholar.google.com/scholar?q=Gated+Linear+Attention+Transformers+with+Hardware-Efficient+Training6. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning — Tri Dao, 2023https://scholar.google.com/scholar?q=FlashAttention-2%3A+Faster+Attention+with+Better+Parallelism+and+Work+Partitioning7. Finetuned Language Models Are Zero-Shot Learners? / A Study of Linear Attention and State Space Models for Language Modeling — Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, Noah A. Smith, 2021https://scholar.google.com/scholar?q=Finetuned+Language+Models+Are+Zero-Shot+Learners%3F+%2F+A+Study+of+Linear+Attention+and+State+Space+Models+for+Language+Modeling8. Mamba: Linear-Time Sequence Modeling with Selective State Spaces — Albert Gu, Tri Dao, 2023https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces9. TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer — Yushui Qin, Zihao Sun, Xiaoyu Li, et al., 2023https://scholar.google.com/scholar?q=TransNormerLLM%3A+A+Faster+and+Better+Large+Language+Model+with+Improved+TransNormer10. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré, 2022https://scholar.google.com/scholar?q=FlashAttention%3A+Fast+and+Memory-Efficient+Exact+Attention+with+IO-Awareness11. Blockwise Parallel Transformer for Large Context Models — Yunlong Hua, Tianyu Gao, et al., 2022https://scholar.google.com/scholar?q=Blockwise+Parallel+Transformer+for+Large+Context+Models12. Were RNNs All We Needed? — Jaap van der Westhuizen, Joan Lasenby, 2018https://scholar.google.com/scholar?q=Were+RNNs+All+We+Needed%3F13. On the Expressiveness of Softmax Attention: A Recurrent Neural Network Perspective — approx. recent theory paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=On+the+Expressiveness+of+Softmax+Attention%3A+A+Recurrent+Neural+Network+Perspective14. The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry — approx. recent linear-attention paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=The+Hedgehog+%26+the+Porcupine%3A+Expressive+Linear+Attentions+with+Softmax+Mimicry15. Agent Attention: On the Integration of Softmax and Linear Attention — approx. recent attention paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=Agent+Attention%3A+On+the+Integration+of+Softmax+and+Linear+Attention16. Transformer Based Linear Attention with Optimized GPU Kernel Implementation — approx. recent systems paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=Transformer+Based+Linear+Attention+with+Optimized+GPU+Kernel+Implementation17. FlexLinearAttention: Compiling a Unified Abstraction into Scalable Kernels for Linear Attention — approx. recent systems/compiler paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=FlexLinearAttention%3A+Compiling+a+Unified+Abstraction+into+Scalable+Kernels+for+Linear+Attention18. PyramidInfer: Pyramid KV Cache Compression for High-Throughput LLM Inference — approx. recent inference paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=PyramidInfer%3A+Pyramid+KV+Cache+Compression+for+High-Throughput+LLM+Inference19. Inference-time Hyper-Scaling with KV Cache Compression — approx. recent inference paper; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=Inference-time+Hyper-Scaling+with+KV+Cache+Compression20. KV Cache Compression for Inference Efficiency in LLMs: A Review — approx. recent review; exact authors not recoverable from snippet, 2024/2025https://scholar.google.com/scholar?q=KV+Cache+Compression+for+Inference+Efficiency+in+LLMs%3A+A+Review21. AI Post Transformers: Mamba-3 for Efficient Sequence Modeling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-mamba-3-for-efficient-sequence-modeling-97a22a.mp322. AI Post Transformers: Kimi Linear: Efficient Expressive Attention Architecture — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/kimi-linear-efficient-expressive-attention-architecture/23. AI Post Transformers: FlatAttention for Tile-Based Accelerator Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-flatattention-for-tile-based-accelerator-56e6ca.mp324. AI Post Transformers: Memory Sparse Attention for 100M-Token Scaling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-07-memory-sparse-attention-for-100m-token-s-377cff.mp325. AI Post Transformers: KVSwap for Disk-Aware Long-Context On-Device Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-kvswap-for-disk-aware-long-context-on-de-f3c15e.mp326. AI Post Transformers: TriAttention for Efficient Long-Context KV Compression — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-07-triattention-for-efficient-long-context-6c08ee.mp3Interactive Visualization: Gated Linear Attention for Efficient Long Sequences19 min
- Parallelizing DeltaNet Linear Transformers over Sequence LengthThis episode explores a June 2024 paper that redesigns DeltaNet-style linear attention so it can train efficiently in parallel across sequence length, making it practical at language-model scale rather than just theoretically appealing. It explains how the work builds on the tradeoff between standard softmax attention’s strong token-level retrieval and linear attention’s compressed, constant-memory state, then argues that the delta rule offers smarter overwrite and recall behavior than simple additive memory updates. The discussion highlights why earlier DeltaNet variants were bottlenecked by sequential recurrence and poor GPU utilization, and why solving that systems problem matters for scaling to 1.3B-parameter models trained on 100B tokens. Listeners would find it interesting for its clear breakdown of how hardware constraints, associative memory, and long-context language modeling intersect—and why this approach aims to outperform strong linear-time baselines and even some transformer setups.Sources:1. Parallelizing Linear Transformers with the Delta Rule over Sequence Length — Songlin Yang, Bailin Wang, Yu Zhang, Yikang Shen, Yoon Kim, 2024http://arxiv.org/abs/2406.064842. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality — Tri Dao, Albert Gu, 2024https://scholar.google.com/scholar?q=Transformers+are+SSMs%3A+Generalized+Models+and+Efficient+Algorithms+Through+Structured+State+Space+Duality3. Gated Linear Attention Transformers with Hardware-Efficient Training — the GLA / Flash Linear Attention authors cited as [124], 2024https://scholar.google.com/scholar?q=Gated+Linear+Attention+Transformers+with+Hardware-Efficient+Training4. Were RNNs All We Needed? — the authors cited as [92], 2024https://scholar.google.com/scholar?q=Were+RNNs+All+We+Needed%3F5. Mamba: Linear-Time Sequence Modeling with Selective State Spaces — Albert Gu, Tri Dao, 2024https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces6. DeltaNet: A Neural Sequence Model with Fast Weight Programmers — the authors cited as [101] including Imanol Schlag and collaborators, 2024https://scholar.google.com/scholar?q=DeltaNet%3A+A+Neural+Sequence+Model+with+Fast+Weight+Programmers7. The Compact WY Representation for Products of Householder Matrices — the authors cited as [11], 1989https://scholar.google.com/scholar?q=The+Compact+WY+Representation+for+Products+of+Householder+Matrices8. FlashAttention — Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré, 2022https://scholar.google.com/scholar?q=FlashAttention9. The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry — the authors cited as [6], 2024https://scholar.google.com/scholar?q=The+Hedgehog+%26+the+Porcupine%3A+Expressive+Linear+Attentions+with+Softmax+Mimicry10. RazorAttention: Efficient KV Cache Compression Through Retrieval Heads — approx. recent LLM systems/attention-efficiency authors, 2024/2025https://scholar.google.com/scholar?q=RazorAttention%3A+Efficient+KV+Cache+Compression+Through+Retrieval+Heads11. StreamKV: Streaming Video Question-Answering with Segment-Based KV Cache Retrieval and Compression — approx. recent multimodal/streaming inference authors, 2024/2025https://scholar.google.com/scholar?q=StreamKV%3A+Streaming+Video+Question-Answering+with+Segment-Based+KV+Cache+Retrieval+and+Compression12. Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning — approx. recent efficient-inference authors, 2024/2025https://scholar.google.com/scholar?q=Not+All+Heads+Matter%3A+A+Head-Level+KV+Cache+Compression+Method+with+Integrated+Retrieval+and+Reasoning13. Augmenting Language Models with Long-Term Memory — approx. recent long-context/memory-augmented LM authors, 2024/2025https://scholar.google.com/scholar?q=Augmenting+Language+Models+with+Long-Term+Memory14. Retrieval Meets Long Context Large Language Models — approx. recent retrieval/long-context evaluation authors, 2024/2025https://scholar.google.com/scholar?q=Retrieval+Meets+Long+Context+Large+Language+Models15. Gated Delta Networks: Improving Mamba2 with Delta Rule — approx. recent linear-recurrent/model-architecture authors, 2025https://scholar.google.com/scholar?q=Gated+Delta+Networks%3A+Improving+Mamba2+with+Delta+Rule16. Path Attention: Position Encoding via Accumulating Householder Transformations — approx. recent sequence-modeling authors, 2024/2025https://scholar.google.com/scholar?q=Path+Attention%3A+Position+Encoding+via+Accumulating+Householder+Transformations17. DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products — approx. recent linear-RNN authors, 2024/2025https://scholar.google.com/scholar?q=DeltaProduct%3A+Improving+State-Tracking+in+Linear+RNNs+via+Householder+Products18. AI Post Transformers: Mamba-3 for Efficient Sequence Modeling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-16-mamba-3-for-efficient-sequence-modeling-97a22a.mp319. AI Post Transformers: Kimi Linear: Efficient Expressive Attention Architecture — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/kimi-linear-efficient-expressive-attention-architecture/20. AI Post Transformers: RetNet: Retentive Networks: Transformer Successor for Large Language Models — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/retnet-retentive-networks-transformer-successor-for-large-language-models/21. AI Post Transformers: Ring-linear: Efficient Hybrid Architecture for Long-Context Reasoning — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/ring-linear-efficient-hybrid-architecture-for-long-context-reasoning/22. AI Post Transformers: Native Sparse Attention: Efficient Long-Context LLMs — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/native-sparse-attention-efficient-long-context-llms/23. AI Post Transformers: Memory Sparse Attention for 100M-Token Scaling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-07-memory-sparse-attention-for-100m-token-s-377cff.mp324. AI Post Transformers: ALiBi: Attention with Linear Biases Enables Length Extrapolation — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/alibi-attention-with-linear-biases-enables-length-extrapolation/25. AI Post Transformers: Jet-Nemotron and PostNAS for Faster Long Context — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-24-jet-nemotron-and-postnas-for-faster-long-436381.mp326. AI Post Transformers: DRAM-Free In-Flash Computing for LLM Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-09-dram-free-in-flash-computing-for-llm-inf-4ac216.mp3Interactive Visualization: Parallelizing DeltaNet Linear Transformers over Sequence Length20 min
- KVSwap for Disk-Aware Long-Context On-Device InferenceThis episode explores KVSwap, a system for running long-context language models on memory-constrained devices by offloading the growing KV cache to storage such as NVMe, UFS, or eMMC instead of relying on scarce shared RAM. It explains why standard server-style GPU-to-CPU offloading breaks down on phones and edge devices with unified memory, and why disk offloading is only viable if it is carefully designed around storage bottlenecks like low bandwidth, latency, and read amplification. The discussion highlights KVSwap’s core strategy: keep the full KV cache on disk, use a compact in-memory key-side representation to predict needed entries, prefetch them ahead of computation, overlap I/O with decoding, and smooth access patterns with buffering to make reads more sequential. Listeners interested in local AI will find it compelling because it reframes long-context inference as a systems problem at the intersection of transformers, operating systems, and storage architecture.Sources:1. KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference — Huawei Zhang, Chunwei Xia, Zheng Wang, 2025http://arxiv.org/abs/2511.119072. FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU — Tianqi Chen et al., 2023https://scholar.google.com/scholar?q=FlexGen%3A+High-Throughput+Generative+Inference+of+Large+Language+Models+with+a+Single+GPU3. vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention — Woosuk Kwon et al., 2023https://scholar.google.com/scholar?q=vLLM%3A+Easy%2C+Fast%2C+and+Cheap+LLM+Serving+with+PagedAttention4. RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval — Sheng Shen et al., 2024https://scholar.google.com/scholar?q=RetrievalAttention%3A+Accelerating+Long-Context+LLM+Inference+via+Vector+Retrieval5. InfiniGen — Not clearly specified in the excerpt, 2024https://scholar.google.com/scholar?q=InfiniGen6. Mooncake — Not clearly specified in the excerpt, 2024https://scholar.google.com/scholar?q=Mooncake7. SnapKV — Li et al., 2024https://scholar.google.com/scholar?q=SnapKV8. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhang et al., 2023https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models9. StreamingLLM — Xiao et al., 2024https://scholar.google.com/scholar?q=StreamingLLM10. PyramidInfer: Pyramid KV Cache Compression for High-Throughput LLM Inference — approx. recent LLM systems authors, 2024/2025https://scholar.google.com/scholar?q=PyramidInfer%3A+Pyramid+KV+Cache+Compression+for+High-Throughput+LLM+Inference11. Inference-Time Hyper-Scaling with KV Cache Compression — approx. recent LLM inference authors, 2024/2025https://scholar.google.com/scholar?q=Inference-Time+Hyper-Scaling+with+KV+Cache+Compression12. MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference — approx. recent multimodal inference authors, 2024/2025https://scholar.google.com/scholar?q=MadaKV%3A+Adaptive+Modality-Perception+KV+Cache+Eviction+for+Efficient+Multimodal+Long-Context+Inference13. Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks — approx. recent long-context LLM authors, 2024/2025https://scholar.google.com/scholar?q=Model+Tells+You+Where+to+Merge%3A+Adaptive+KV+Cache+Merging+for+LLMs+on+Long-Context+Tasks14. KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments — approx. recent efficient inference authors, 2024/2025https://scholar.google.com/scholar?q=KeyDiff%3A+Key+Similarity-Based+KV+Cache+Eviction+for+Long-Context+LLM+Inference+in+Resource-Constrained+Environments15. CHESS: Context-Aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference — approx. recent long-context inference authors, 2024/2025https://scholar.google.com/scholar?q=CHESS%3A+Context-Aware+Hierarchical+Efficient+Semantic+Selection+for+Long-Context+LLM+Inference16. Compressing Context to Enhance Inference Efficiency of Large Language Models — approx. recent LLM efficiency authors, 2024/2025https://scholar.google.com/scholar?q=Compressing+Context+to+Enhance+Inference+Efficiency+of+Large+Language+Models17. HyperAttention: Long-Context Attention in Near-Linear Time — approx. recent attention-mechanism authors, 2024/2025https://scholar.google.com/scholar?q=HyperAttention%3A+Long-Context+Attention+in+Near-Linear+Time18. Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning — approx. recent hybrid-attention authors, 2024/2025https://scholar.google.com/scholar?q=Every+Attention+Matters%3A+An+Efficient+Hybrid+Architecture+for+Long-Context+Reasoning19. Leave No Context Behind: Efficient Infinite Context Transformers with Infini-Attention — approx. recent long-context architecture authors, 2024https://scholar.google.com/scholar?q=Leave+No+Context+Behind%3A+Efficient+Infinite+Context+Transformers+with+Infini-Attention20. AI Post Transformers: SolidAttention: Co-Designing Sparse Attention and SSD I/O — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-18-solidattention-co-designing-sparse-atten-5a8622.mp321. AI Post Transformers: FlexGen: High-Throughput LLM Inference on a Single GPU — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/flexgen-high-throughput-llm-inference-on-a-single-gpu/22. AI Post Transformers: Lookahead Q-Cache for Consistent KV Eviction — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-25-lookahead-q-cache-for-consistent-kv-evic-d97b09.mp323. AI Post Transformers: Splitwise: Phase-Split LLM Inference — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-26-splitwise-phase-split-llm-inference-e8945b.mp324. AI Post Transformers: Speculative Decoding in Real vLLM Serving — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-speculative-decoding-in-real-vllm-servin-6f4e2b.mp325. AI Post Transformers: TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-25-turboquant-online-vector-quantiz-1967b7.mp326. AI Post Transformers: Accelerating LLM Cold Starts with Programmable Page Cache — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-17-accelerating-llm-cold-starts-with-progra-0912d1.mp3Interactive Visualization: KVSwap for Disk-Aware Long-Context On-Device Inference19 min
- Mamba-3 for Efficient Sequence ModelingThis episode explores Mamba-3, a new state space sequence model that argues architecture should be judged not just by perplexity, but by deployment realities like decode latency, throughput, and hardware efficiency. It explains how Mamba-3 revisits earlier Mamba-style models with three main changes—a new exponential-trapezoidal discretization, complex-valued state dynamics, and a MIMO input-output structure—aimed at improving the quality-efficiency tradeoff for long-sequence inference. The discussion also situates the work against transformers, whose KV-cache costs grow with context, and against competing linear-recurrence approaches like DeltaNet and emerging hybrid industry systems. Listeners would find it interesting because it highlights a broader shift in machine learning: whether the future of sequence models will be decided less by benchmark curves alone and more by how well they actually run in production.Sources:1. Mamba-3: Improved Sequence Modeling using State Space Principles — Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J. Zico Kolter, Tri Dao, Albert Gu, 2026http://arxiv.org/abs/2603.155692. Efficiently Modeling Long Sequences with Structured State Spaces — Albert Gu, Karan Goel, Christopher Re, 2021https://scholar.google.com/scholar?q=Efficiently+Modeling+Long+Sequences+with+Structured+State+Spaces3. On the Parameterization and Initialization of Diagonal State Space Models — Albert Gu, Ankit Gupta, Jonathan Berant, Tri Dao, Christopher Re, 2022https://scholar.google.com/scholar?q=On+the+Parameterization+and+Initialization+of+Diagonal+State+Space+Models4. Mamba: Linear-Time Sequence Modeling with Selective State Spaces — Albert Gu, Tri Dao, 2024https://scholar.google.com/scholar?q=Mamba%3A+Linear-Time+Sequence+Modeling+with+Selective+State+Spaces5. Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality — Tri Dao, Albert Gu, 2024https://scholar.google.com/scholar?q=Transformers+are+SSMs%3A+Generalized+Models+and+Efficient+Algorithms+Through+Structured+State+Space+Duality6. Unitary Evolution Recurrent Neural Networks — Martin Arjovsky, Amar Shah, Yoshua Bengio, 2016https://scholar.google.com/scholar?q=Unitary+Evolution+Recurrent+Neural+Networks7. HiPPO: Recurrent Memory with Optimal Polynomial Projections — Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, Christopher Re, 2020https://scholar.google.com/scholar?q=HiPPO%3A+Recurrent+Memory+with+Optimal+Polynomial+Projections8. The DeltaNet Family: Efficient Sequence Modeling via State Tracking — Michael Schlag, Kazuki Irie, and Jürgen Schmidhuber; later Gated DeltaNet variants by Shang Yang, Boyuan Wang, Yuhang Zhang, et al., 2021 / 2025https://scholar.google.com/scholar?q=The+DeltaNet+Family%3A+Efficient+Sequence+Modeling+via+State+Tracking9. Rotary Position Embedding — Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu, 2023https://scholar.google.com/scholar?q=Rotary+Position+Embedding10. On the Computational Limits of State Space Models and Their Ability to Track State — Ruggero Grazzi, Julien Siems, Arlind Zela, et al., 2025https://scholar.google.com/scholar?q=On+the+Computational+Limits+of+State+Space+Models+and+Their+Ability+to+Track+State11. State Space Models Fail at Simple State Tracking Tasks — Aviad Sarrof, Tom Veitsman, and Michael Hahn, 2024https://scholar.google.com/scholar?q=State+Space+Models+Fail+at+Simple+State+Tracking+Tasks12. Hungry Hungry Hippos: Towards Language Modeling with State Space Models — Atri Rudra? (No—better to omit uncertain authorship) / H3 team, 2023https://scholar.google.com/scholar?q=Hungry+Hungry+Hippos%3A+Towards+Language+Modeling+with+State+Space+Models13. Kimi Linear — Kimi Team, 2025https://scholar.google.com/scholar?q=Kimi+Linear14. Kvzip: Query-agnostic KV Cache Compression with Context Reconstruction — approx. recent LLM systems authors, 2024/2025https://scholar.google.com/scholar?q=Kvzip%3A+Query-agnostic+KV+Cache+Compression+with+Context+Reconstruction15. KVLINK: Accelerating Large Language Models via Efficient KV Cache Reuse — approx. recent LLM systems authors, 2024/2025https://scholar.google.com/scholar?q=KVLINK%3A+Accelerating+Large+Language+Models+via+Efficient+KV+Cache+Reuse16. KV-CAR: KV Cache Compression using Autoencoders and KV Reuse in Large Language Models — approx. recent LLM systems authors, 2024/2025https://scholar.google.com/scholar?q=KV-CAR%3A+KV+Cache+Compression+using+Autoencoders+and+KV+Reuse+in+Large+Language+Models17. Repeat After Me: Transformers Are Better Than State Space Models at Copying — approx. recent sequence-model authors, 2024/2025https://scholar.google.com/scholar?q=Repeat+After+Me%3A+Transformers+Are+Better+Than+State+Space+Models+at+Copying18. Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling — approx. recent hybrid SSM authors, 2024/2025https://scholar.google.com/scholar?q=Samba%3A+Simple+Hybrid+State+Space+Models+for+Efficient+Unlimited+Context+Language+Modeling19. Maximally-Informative Retrieval for State Space Model Generation — approx. recent retrieval/SSM authors, 2024/2025https://scholar.google.com/scholar?q=Maximally-Informative+Retrieval+for+State+Space+Model+Generation20. AI Post Transformers: Kimi Linear: Efficient Expressive Attention Architecture — Hal Turing & Dr. Ada Shannon, 2025https://podcast.do-not-panic.com/episodes/kimi-linear-efficient-expressive-attention-architecture/21. AI Post Transformers: Jet-Nemotron and PostNAS for Faster Long Context — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-24-jet-nemotron-and-postnas-for-faster-long-436381.mp322. AI Post Transformers: Batch-Aware Expert Routing for Faster MoE Decoding — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-batch-aware-expert-routing-for-faster-mo-683ab6.mp323. AI Post Transformers: FengHuang for Rack-Scale LLM Inference Memory — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-12-fenghuang-for-rack-scale-llm-inference-m-62708e.mp324. AI Post Transformers: Memory Sparse Attention for 100M-Token Scaling — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-07-memory-sparse-attention-for-100m-token-s-377cff.mp325. AI Post Transformers: MEMSEARCHER: Reinforcement Learning for LLM Memory Management — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-memsearcher-reinforcement-learning-for-l-e9ad84.mp3Interactive Visualization: Mamba-3 for Efficient Sequence Modeling22 min
- Linear Classifier Probes for Intermediate LayersThis episode explores a 2016 paper on linear classifier probes, a simple method for testing what information is linearly recoverable from a neural network’s intermediate layers by attaching small classifiers to frozen hidden states. It explains the paper’s central finding—that class information often becomes increasingly linearly separable with depth—and why that suggested deep networks develop more organized, task-relevant representations even without being explicitly trained to make every layer separable. The discussion also emphasizes a crucial caveat: probes measure what information is accessible, not which layer causally performs a computation, making them tools for analysis rather than proof of mechanism. Listeners would find it interesting for its clear connection to modern interpretability, transfer learning, and evaluation practices, as well as its argument that this now-standard probing approach was an early step toward opening up the neural network “black box.”Sources:1. Understanding intermediate layers using linear classifier probes — Guillaume Alain, Yoshua Bengio, 2016http://arxiv.org/abs/1610.016442. Probing Classifiers: Promises, Shortcomings, and Advances — Yonatan Belinkov, 2021http://arxiv.org/abs/2102.124523. Towards Best Practices of Activation Patching in Language Models: Metrics and Methods — Fred Zhang, Neel Nanda, 2023http://arxiv.org/abs/2309.160424. https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC8https://docs.google.com/document/d/1WwsnJQstPq91_Yh-Ch2XRL8H_EpsnjrC1dwZXR37PC85. Understanding intermediate layers using linear classifier probes — Guillaume Alain, Yoshua Bengio, 2016https://scholar.google.com/scholar?q=Understanding+intermediate+layers+using+linear+classifier+probes6. On the Transferability of Features in Deep Neural Networks — Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson, 2014https://scholar.google.com/scholar?q=On+the+Transferability+of+Features+in+Deep+Neural+Networks7. Do Better ImageNet Models Transfer Better? — Simon Kornblith, Jonathon Shlens, Quoc V. Le, 2019https://scholar.google.com/scholar?q=Do+Better+ImageNet+Models+Transfer+Better%3F8. A Survey on Probing Methods for Linguistic Information in Neural Language Models — Najoung Kim, Roma Patel, Adam Poliak, Patrick Xia, Alex Wang, Samuel R. Bowman, Yoon Kim, Katharina Kann, 2022https://scholar.google.com/scholar?q=A+Survey+on+Probing+Methods+for+Linguistic+Information+in+Neural+Language+Models9. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps — Karen Simonyan, Andrea Vedaldi, Andrew Zisserman, 2013https://scholar.google.com/scholar?q=Deep+Inside+Convolutional+Networks%3A+Visualising+Image+Classification+Models+and+Saliency+Maps10. Understanding Neural Networks Through Deep Visualization — Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, Hod Lipson, 2015https://scholar.google.com/scholar?q=Understanding+Neural+Networks+Through+Deep+Visualization11. How Transferable Are Features in Deep Neural Networks? — Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson, 2014https://scholar.google.com/scholar?q=How+Transferable+Are+Features+in+Deep+Neural+Networks%3F12. Decaf: A Deep Convolutional Activation Feature for Generic Visual Recognition — Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell, 2014https://scholar.google.com/scholar?q=Decaf%3A+A+Deep+Convolutional+Activation+Feature+for+Generic+Visual+Recognition13. Learning Deep Features for Discriminative Localization — Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba, 2016https://scholar.google.com/scholar?q=Learning+Deep+Features+for+Discriminative+Localization14. The Information Bottleneck Theory of Deep Learning — Naftali Tishby, Noga Zaslavsky, 2015https://scholar.google.com/scholar?q=The+Information+Bottleneck+Theory+of+Deep+Learning15. Visualizing and Understanding Convolutional Networks — Matthew D. Zeiler, Rob Fergus, 2014https://scholar.google.com/scholar?q=Visualizing+and+Understanding+Convolutional+Networks16. Using Linear Classifier Probes — Yonatan Belinkov, 2022https://scholar.google.com/scholar?q=Using+Linear+Classifier+Probes17. What do you learn from context? Probing for sentence structure in contextualized word representations — Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Samuel R. Bowman, Eunsol Choi, 2019https://scholar.google.com/scholar?q=What+do+you+learn+from+context%3F+Probing+for+sentence+structure+in+contextualized+word+representations18. Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models — Ethan Dyer, Guy Gur-Ari, Ishaan Gulrajani, et al., 2024https://scholar.google.com/scholar?q=Beyond+the+Imitation+Game%3A+Quantifying+and+extrapolating+the+capabilities+of+language+models19. Does representation matter? exploring intermediate layers in large language models — unknown from snippet, likely 2024 or 2025https://scholar.google.com/scholar?q=Does+representation+matter%3F+exploring+intermediate+layers+in+large+language+models20. A separability-based approach to quantifying generalization: which layer is best? — unknown from snippet, likely 2023-2025https://scholar.google.com/scholar?q=A+separability-based+approach+to+quantifying+generalization%3A+which+layer+is+best%3F21. The topology and geometry of neural representations — unknown from snippet, likely 2023-2025https://scholar.google.com/scholar?q=The+topology+and+geometry+of+neural+representations22. Context Matters: Analyzing the Generalizability of Linear Probing and Steering Across Diverse Scenarios — unknown from snippet, likely 2024 or 2025https://scholar.google.com/scholar?q=Context+Matters%3A+Analyzing+the+Generalizability+of+Linear+Probing+and+Steering+Across+Diverse+Scenarios23. AI Post Transformers: Xavier Initialization: Deep Feedforward Networks: Training Difficulties and Solutions — Hal Turing & Dr. Ada Shannon, Fri,https://podcast.do-not-panic.com/episodes/xavier-initialization-deep-feedforward-networks-training-difficulties-and-soluti/24. AI Post Transformers: Language Models are Injective and Hence Invertible — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-03-21-language-models-are-injective-an-7545e0.mp3Interactive Visualization: Linear Classifier Probes for Intermediate Layers0 min
- SkillsBench for Evaluating Agent SkillsThis episode explores SkillsBench, a new benchmark for testing whether reusable “skills” — structured procedural packages like runbooks, templates, and verification steps — actually improve LLM agents on real multi-step tasks. It breaks down how the benchmark isolates the value of skills from the underlying model by evaluating 86 tasks across 11 domains under three conditions: no skills, curated skills, and self-generated skills, all with deterministic pass/fail verification. The discussion also examines a key debate over whether skills are genuinely distinct from retrieval-augmented context, arguing that skills encode procedural know-how about when and how to act, not just facts to read. Listeners would find it interesting because it tackles a practical industry problem: how to tell whether accumulated prompt libraries and agent playbooks are useful engineering assets or just extra text that creates the illusion of progress.Sources:1. SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks — Xiangyi Li, Wenbo Chen, Yimin Liu, Shenghan Zheng, Xiaokun Chen, Yifeng He, Yubo Li, Bingran You, Haotian Shen, Jiankai Sun, Shuyi Wang, Binxu Li, Qunhong Zeng, Di Wang, Xuandong Zhao, Yuanli Wang, Roey Ben Chaim, Zonglin Di, Yipeng Gao, Junwei He, Yizhuo He, Liqiang Jing, Luyang Kong, Xin Lan, Jiachen Li, Songlin Li, Yijiang Li, Yueqian Lin, Xinyi Liu, Xuanqing Liu, Haoran Lyu, Ze Ma, Bowei Wang, Runhui Wang, Tianyu Wang, Wengao Ye, Yue Zhang, Hanwen Xing, Yiqi Xue, Steven Dillmann, Han-chung Lee, 2026http://arxiv.org/abs/2602.126702. Terminal-Bench — Merrill et al., 2026https://scholar.google.com/scholar?q=Terminal-Bench3. Harbor Framework — Harbor Framework Team, 2026https://scholar.google.com/scholar?q=Harbor+Framework4. Anthropic Skills documentation / product specification — Anthropic, 2025https://scholar.google.com/scholar?q=Anthropic+Skills+documentation+%2F+product+specification5. Language Agents with Cognitive Architectures — Sumers et al., 2023https://scholar.google.com/scholar?q=Language+Agents+with+Cognitive+Architectures6. Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning — Sutton, Precup, and Singh, 1999https://scholar.google.com/scholar?q=Between+MDPs+and+Semi-MDPs%3A+A+Framework+for+Temporal+Abstraction+in+Reinforcement+Learning7. ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022https://scholar.google.com/scholar?q=ReAct%3A+Synergizing+Reasoning+and+Acting+in+Language+Models8. SWE-bench — Jimenez et al., 2024https://scholar.google.com/scholar?q=SWE-bench9. WebArena — Zhou et al., 2024https://scholar.google.com/scholar?q=WebArena10. Tool Learning / API-Bank style benchmarks — Liu et al., 2023https://scholar.google.com/scholar?q=Tool+Learning+%2F+API-Bank+style+benchmarks11. Group-Evolving Agents: Open-Ended Self-Improvement via Experience Sharing — approx. 2025/2026 multi-author agent-systems paper, 2025/2026https://scholar.google.com/scholar?q=Group-Evolving+Agents%3A+Open-Ended+Self-Improvement+via+Experience+Sharing12. ToolReflection: Improving Large Language Models for Real-World API Calls with Self-Generated Data — approx. 2025 multi-author paper, 2025https://scholar.google.com/scholar?q=ToolReflection%3A+Improving+Large+Language+Models+for+Real-World+API+Calls+with+Self-Generated+Data13. OS-Copilot: Towards Generalist Computer Agents with Self-Improvement — approx. 2024/2025 multi-author paper, 2024/2025https://scholar.google.com/scholar?q=OS-Copilot%3A+Towards+Generalist+Computer+Agents+with+Self-Improvement14. Agent Skills from the Perspective of Procedural Memory: A Survey — approx. 2025 survey paper, 2025https://scholar.google.com/scholar?q=Agent+Skills+from+the+Perspective+of+Procedural+Memory%3A+A+Survey15. Agent skills for large language models: Architecture, acquisition, security, and the path forward — approx. 2025 survey/framework paper, 2025https://scholar.google.com/scholar?q=Agent+skills+for+large+language+models%3A+Architecture%2C+acquisition%2C+security%2C+and+the+path+forward16. DocAgent: An Agentic Framework for Multi-Modal Long-Context Document Understanding — approx. 2025 multi-author paper, 2025https://scholar.google.com/scholar?q=DocAgent%3A+An+Agentic+Framework+for+Multi-Modal+Long-Context+Document+Understanding17. MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understanding — approx. 2025 multi-author paper, 2025https://scholar.google.com/scholar?q=MDocAgent%3A+A+Multi-Modal+Multi-Agent+Framework+for+Document+Understanding18. Multi-agent Verification: Scaling Test-Time Compute with Multiple Verifiers — approx. 2025/2026 multi-author paper, 2025/2026https://scholar.google.com/scholar?q=Multi-agent+Verification%3A+Scaling+Test-Time+Compute+with+Multiple+Verifiers19. Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification — approx. 2025/2026 multi-author paper, 2025/2026https://scholar.google.com/scholar?q=Inference-Time+Scaling+of+Verification%3A+Self-Evolving+Deep+Research+Agents+via+Test-Time+Rubric-Guided+Verification20. AI Post Transformers: AI Agent Traps and Prompt Injection — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-02-ai-agent-traps-and-prompt-injection-7ce4ba.mp321. AI Post Transformers: Real Context Size and Context Rot — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-07-real-context-size-and-context-rot-56cbb4.mp322. AI Post Transformers: MEMSEARCHER: Reinforcement Learning for LLM Memory Management — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-memsearcher-reinforcement-learning-for-l-e9ad84.mp323. AI Post Transformers: Experiential Reinforcement Learning: Internalizing Reflection for Better Policy Training — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/experiential-reinforcement-learning-internalizing-reflection-for-better-policy-t/24. AI Post Transformers: ASI-Evolve for Data, Architectures, and RL — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-05-asi-evolve-for-data-architectures-and-rl-197b2b.mp325. AI Post Transformers: IMO-Bench for Robust Mathematical Reasoning — Hal Turing & Dr. Ada Shannon, 2026https://podcast.do-not-panic.com/episodes/2026-04-04-imo-bench-for-robust-mathematical-reason-143489.mp3Interactive Visualization: SkillsBench for Evaluating Agent Skills0 min
About AI Post Transformers
From the publisher's feed
AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis,…
proper citations, and nerdy humor.