AI Post Transformers

This episode explores how boosted decision trees can be compiled directly into FPGA firmware for ultra-low-latency particle-physics triggers at the Large Hadron Collider. It explains why this setting favors shallow, quantized tree ensembles over larger neural networks: trigger decisions must happen within a tiny hardware budget, with strict limits on latency, power, and on-chip resources. The discussion focuses on a concrete benchmark where a 100-tree, depth-4 gradient-boosted model for five-class jet tagging is mapped to a Xilinx VU9P FPGA and compared against a similarly deployed multilayer perceptron. Listeners would find it interesting because it shows how model choice changes when every nanosecond matters, and how familiar ML methods can become hardwired decision circuits rather than conventional software inference.

Sources:

1. Fast inference of Boosted Decision Trees in FPGAs for particle physics — Sioni Summers, Giuseppe Di Guglielmo, Javier Duarte, Philip Harris, Duc Hoang, Sergo Jindariani, Edward Kreinar, Vladimir Loncar, Jennifer Ngadiuba, Maurizio Pierini, Dylan Rankin, Nhan Tran, Zhenbin Wu, 2020

http://arxiv.org/abs/2002.02534

2. Greedy Function Approximation: A Gradient Boosting Machine — Jerome H. Friedman, 2001

https://scholar.google.com/scholar?q=Greedy+Function+Approximation%3A+A+Gradient+Boosting+Machine

3. XGBoost: A Scalable Tree Boosting System — Tianqi Chen, Carlos Guestrin, 2016

https://scholar.google.com/scholar?q=XGBoost%3A+A+Scalable+Tree+Boosting+System

4. LightGBM: A Highly Efficient Gradient Boosting Decision Tree — Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, Tie-Yan Liu, 2017

https://scholar.google.com/scholar?q=LightGBM%3A+A+Highly+Efficient+Gradient+Boosting+Decision+Tree

5. CatBoost: unbiased boosting with categorical features — Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin, 2018

https://scholar.google.com/scholar?q=CatBoost%3A+unbiased+boosting+with+categorical+features

6. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference — Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, Dmitry Kalenichenko, 2018

https://scholar.google.com/scholar?q=Quantization+and+Training+of+Neural+Networks+for+Efficient+Integer-Arithmetic-Only+Inference

7. Quantizing deep convolutional networks for efficient inference: A whitepaper — Raghuraman Krishnamoorthi, 2018

https://scholar.google.com/scholar?q=Quantizing+deep+convolutional+networks+for+efficient+inference%3A+A+whitepaper

8. Post-training 4-bit quantization of convolution networks for rapid-deployment — Ron Banner, Yury Nahshan, Elad Hoffer, Daniel Soudry, 2019

https://scholar.google.com/scholar?q=Post-training+4-bit+quantization+of+convolution+networks+for+rapid-deployment

9. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — Elias Frantar, Saleh Ashkboos, Torsten Hoefler, Dan Alistarh, 2022

https://scholar.google.com/scholar?q=GPTQ%3A+Accurate+Post-Training+Quantization+for+Generative+Pre-trained+Transformers

10. AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration — Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han, 2023

https://scholar.google.com/scholar?q=AWQ%3A+Activation-aware+Weight+Quantization+for+LLM+Compression+and+Acceleration

11. Fast inference of deep neural networks in FPGAs for particle physics — Javier Duarte, Song Han, Philip Harris, Sergo Jindariani, Edward Kreinar, Benjamin Kreis, Jennifer Ngadiuba, Maurizio Pierini, Ryan Rivera, Nhan Tran, Zhenbin Wu, 2018

https://scholar.google.com/scholar?q=Fast+inference+of+deep+neural+networks+in+FPGAs+for+particle+physics

12. Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree — V. V. Gligorov, M. Williams, 2013

https://scholar.google.com/scholar?q=Efficient%2C+reliable+and+fast+high-level+triggering+using+a+bonsai+boosted+decision+tree

13. Boosted Decision Trees in the Level-1 Muon Endcap Trigger at CMS — CMS Collaboration, 2018

https://scholar.google.com/scholar?q=Boosted+Decision+Trees+in+the+Level-1+Muon+Endcap+Trigger+at+CMS

14. Scalable inference of decision tree ensembles: Flexible design for CPU-FPGA platforms — Muhsen Owaida, Hantian Zhang, Ce Zhang, Gustavo Alonso, 2017

https://scholar.google.com/scholar?q=Scalable+inference+of+decision+tree+ensembles%3A+Flexible+design+for+CPU-FPGA+platforms

15. Machine learning at the energy and intensity frontiers of particle physics — A. Radovic et al., 2018

https://scholar.google.com/scholar?q=Machine+learning+at+the+energy+and+intensity+frontiers+of+particle+physics

16. Low latency transformer inference on FPGAs for physics applications with hls4ml — Zhixing Jiang et al., 2025

https://scholar.google.com/scholar?q=Low+latency+transformer+inference+on+FPGAs+for+physics+applications+with+hls4ml

17. Ultrafast jet classification at the HL-LHC — Patrick Odagiu et al., 2024

https://scholar.google.com/scholar?q=Ultrafast+jet+classification+at+the+HL-LHC

18. AI Post Transformers: FlatAttention for Tile-Based Accelerator Inference — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-04-flatattention-for-tile-based-accelerator-56e6ca.mp3

19. AI Post Transformers: Computation-Bandwidth-Memory Trade-offs for AI Infrastructure — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-09-computation-bandwidth-memory-trade-offs-a83f2b.mp3

20. AI Post Transformers: TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-03-25-turboquant-online-vector-quantiz-1967b7.mp3

Interactive Visualization: Fast FPGA BDT Inference for LHC Triggers

This episode explores why LightGBM became a dominant tool for tabular machine learning by unpacking the algorithmic and systems ideas behind its speed. It explains how gradient boosting decision trees work, why split search becomes expensive on massive sparse datasets, and how LightGBM differs from neural-network-style training despite using gradient information. The discussion focuses on two core contributions: Gradient-based One-Side Sampling, which keeps high-gradient examples while subsampling easier ones without badly distorting split-gain estimates, and Exclusive Feature Bundling, which compresses sparse features by grouping columns that rarely activate together. Listeners would find it interesting for its clear account of how classical ideas like histograms, greedy tree growth, and graph coloring were combined into a highly practical system that reshaped real-world applications such as ranking, fraud detection, credit scoring, and forecasting.

Sources:

1. Why LightGBM Made Boosted Trees Fast

https://proceedings.neurips.cc/paper_files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf

2. Greedy Function Approximation: A Gradient Boosting Machine — Jerome H. Friedman, 2001

https://scholar.google.com/scholar?q=Greedy+Function+Approximation%3A+A+Gradient+Boosting+Machine

3. XGBoost: A Scalable Tree Boosting System — Tianqi Chen and Carlos Guestrin, 2016

https://scholar.google.com/scholar?q=XGBoost%3A+A+Scalable+Tree+Boosting+System

4. LightGBM: A Highly Efficient Gradient Boosting Decision Tree — Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, Tie-Yan Liu, 2017

https://scholar.google.com/scholar?q=LightGBM%3A+A+Highly+Efficient+Gradient+Boosting+Decision+Tree

5. CatBoost: Unbiased Boosting with Categorical Features — Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin, 2018

https://scholar.google.com/scholar?q=CatBoost%3A+Unbiased+Boosting+with+Categorical+Features

6. Feature Hashing for Large Scale Multitask Learning — Kilian Weinberger, Anirban Dasgupta, Josh Attenberg, John Langford, Alex Smola, 2009

https://scholar.google.com/scholar?q=Feature+Hashing+for+Large+Scale+Multitask+Learning

7. An Upper Bound for the Chromatic Number of a Graph and Its Application to Timetabling Problems — D. J. A. Welsh and M. B. Powell, 1967

https://scholar.google.com/scholar?q=An+Upper+Bound+for+the+Chromatic+Number+of+a+Graph+and+Its+Application+to+Timetabling+Problems

8. New Methods to Color the Vertices of a Graph — Daniel Brélaz, 1979

https://scholar.google.com/scholar?q=New+Methods+to+Color+the+Vertices+of+a+Graph

9. Worst Case Behavior of Graph Coloring Algorithms — David S. Johnson, 1974

https://scholar.google.com/scholar?q=Worst+Case+Behavior+of+Graph+Coloring+Algorithms

10. A Communication-Efficient Parallel Algorithm for Decision Tree — Qi Meng, Guolin Ke, Taifeng Wang, Wei Chen, Qiwei Ye, Zhi-Ming Ma, Tie-Yan Liu, 2016

https://scholar.google.com/scholar?q=A+Communication-Efficient+Parallel+Algorithm+for+Decision+Tree

11. Stochastic Gradient Boosting — Jerome H. Friedman, 2002

https://scholar.google.com/scholar?q=Stochastic+Gradient+Boosting

12. Parallel Boosted Regression Trees for Web Search Ranking — Stephen Tyree, Kilian Q. Weinberger, Kunal Agrawal, and Jennifer Paykin, 2011

https://scholar.google.com/scholar?q=Parallel+Boosted+Regression+Trees+for+Web+Search+Ranking

13. Best-First Decision Tree Learning — Haijian Shi, 2007

https://scholar.google.com/scholar?q=Best-First+Decision+Tree+Learning

14. GPU-Acceleration for Large-Scale Tree Boosting — Huan Zhang, Si Si, and Cho-Jui Hsieh, 2017

https://scholar.google.com/scholar?q=GPU-Acceleration+for+Large-Scale+Tree+Boosting

15. Implementing machine learning methods with complex survey data: Lessons learned on the impacts of accounting sampling weights in gradient boosting — authors not identified in the provided snippet, recent (2020s)

https://scholar.google.com/scholar?q=Implementing+machine+learning+methods+with+complex+survey+data%3A+Lessons+learned+on+the+impacts+of+accounting+sampling+weights+in+gradient+boosting

16. Explainable boosting algorithms: sparse-group and interaction-aware variable selection in complex data — authors not identified in the provided snippet, recent (2020s)

https://scholar.google.com/scholar?q=Explainable+boosting+algorithms%3A+sparse-group+and+interaction-aware+variable+selection+in+complex+data

17. Multi-objective optimization of performance and interpretability of tabular supervised machine learning models — authors not identified in the provided snippet, recent (2020s)

https://scholar.google.com/scholar?q=Multi-objective+optimization+of+performance+and+interpretability+of+tabular+supervised+machine+learning+models

18. AI Post Transformers: Breiman's Two Cultures of Statistical Modeling — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-24-breimans-two-cultures-of-statistical-mod-71e49f.mp3

Interactive Visualization: Why LightGBM Made Boosted Trees Fast

This episode explores how fpgaConvNet turns CNN inference into a Synchronous Dataflow problem so embedded FPGA accelerators can be designed with analyzable schedules, buffers, and resource tradeoffs instead of ad hoc hardware tuning. It explains why CNN deployment on robots, drones, and cars is constrained as much by data movement, latency, and power as by raw arithmetic, and why FPGAs can outperform embedded GPUs when the hardware is tailored carefully to a model’s structure. The discussion highlights the paper’s central claim that formalizing the mapping problem enables automated design-space exploration across very different CNN topologies, rather than just optimizing a single benchmark. It also examines where that approach is strong and where listeners should be skeptical, including whether the reported GPU speedups are fair and how well the clean SDF abstraction survives real hardware implementation.

Sources:

1. fpgaConvNet: A Toolflow for Mapping Diverse Convolutional Neural Networks on Embedded FPGAs — Stylianos I. Venieris, Christos-Savvas Bouganis, 2017

http://arxiv.org/abs/1711.08740

2. Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing — Edward A. Lee, David G. Messerschmitt, 1987

https://scholar.google.com/scholar?q=Static+Scheduling+of+Synchronous+Data+Flow+Programs+for+Digital+Signal+Processing

3. Synchronous Data Flow — Edward A. Lee, David G. Messerschmitt, 1987

https://scholar.google.com/scholar?q=Synchronous+Data+Flow

4. Scenario-aware dataflow: modeling, analysis and implementation of dynamic applications — Sander Stuijk, Marc C. W. Geilen, Bart D. Theelen, Twan Basten, 2011

https://scholar.google.com/scholar?q=Scenario-aware+dataflow%3A+modeling%2C+analysis+and+implementation+of+dynamic+applications

5. fpgaConvNet: A Toolflow for Mapping Diverse Convolutional Neural Networks on Embedded FPGAs — Stylianos I. Venieris, Christos-Savvas Bouganis, 2017

https://scholar.google.com/scholar?q=fpgaConvNet%3A+A+Toolflow+for+Mapping+Diverse+Convolutional+Neural+Networks+on+Embedded+FPGAs

6. DNNWeaver: From High-Level Deep Network Models to FPGA Acceleration — Hyoukjun Sharma, Jongse Park, Emmanuel Amaro, Bradley Thwaites, Praneeth Kotha, Anmol Gupta, Joon Kyung Kim, Asit Mishra, and Hsien-Hsin S. Lee, 2016

https://scholar.google.com/scholar?q=DNNWeaver%3A+From+High-Level+Deep+Network+Models+to+FPGA+Acceleration

7. Caffeine: Towards Uniformed Representation and Acceleration for Deep Convolutional Neural Networks — Chen Zhang, Peng Li, Guangyu Sun, Yijin Guan, Bingjun Xiao, and Jason Cong, 2016

https://scholar.google.com/scholar?q=Caffeine%3A+Towards+Uniformed+Representation+and+Acceleration+for+Deep+Convolutional+Neural+Networks

8. FINN: A Framework for Fast, Scalable Binarized Neural Network Inference — Yaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, and Kees Vissers, 2017

https://scholar.google.com/scholar?q=FINN%3A+A+Framework+for+Fast%2C+Scalable+Binarized+Neural+Network+Inference

9. Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks — Yu Wang, Jiajun Xu, Yanzhi Wang, and Huazhong Yang, 2015

https://scholar.google.com/scholar?q=Optimizing+FPGA-based+Accelerator+Design+for+Deep+Convolutional+Neural+Networks

10. Efficient Processing of Deep Neural Networks: A Tutorial and Survey — Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S. Emer, 2017

https://scholar.google.com/scholar?q=Efficient+Processing+of+Deep+Neural+Networks%3A+A+Tutorial+and+Survey

11. ViTA: A Vision Transformer Inference Accelerator for Edge Applications — Shashank Nag, Gourav Datta, Souvik Kundu, Nitin Chandrachoodan, Peter A. Beerel, 2023

https://scholar.google.com/scholar?q=ViTA%3A+A+Vision+Transformer+Inference+Accelerator+for+Edge+Applications

12. ME-ViT: A Single-Load Memory-Efficient FPGA Accelerator for Vision Transformers — Kyle Marino, Pengmiao Zhang, Viktor K. Prasanna, 2024

https://scholar.google.com/scholar?q=ME-ViT%3A+A+Single-Load+Memory-Efficient+FPGA+Accelerator+for+Vision+Transformers

13. DRViT: A Dynamic Redundancy-Aware Vision Transformer Accelerator via Algorithm and Architecture Co-Design on FPGA — Xiangfeng Sun, Yuanting Zhang, Qinyu Wang, Xiaofeng Zou, et al., 2025

https://scholar.google.com/scholar?q=DRViT%3A+A+Dynamic+Redundancy-Aware+Vision+Transformer+Accelerator+via+Algorithm+and+Architecture+Co-Design+on+FPGA

14. Realisation of Early-Exit Dynamic Neural Networks on Reconfigurable Hardware — Anastasios Dimitriou, Lei Xun, Jonathon Hare, Geoff V. Merrett, 2024

https://scholar.google.com/scholar?q=Realisation+of+Early-Exit+Dynamic+Neural+Networks+on+Reconfigurable+Hardware

15. Compute-In-Memory on FPGAs for Deep Learning: A Review — Aman Arora, 2025

https://scholar.google.com/scholar?q=Compute-In-Memory+on+FPGAs+for+Deep+Learning%3A+A+Review

16. A Heterogeneous System With Computing in Memory Processing Elements to Accelerate CNN Inference — Jinkai Wang, Youxiang Chen, Zekun Wang, Zhengkun Gu, et al., 2025

https://scholar.google.com/scholar?q=A+Heterogeneous+System+With+Computing+in+Memory+Processing+Elements+to+Accelerate+CNN+Inference

17. AI Post Transformers: FPGA Neural Network Accelerators for Space — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-26-fpga-neural-network-accelerators-for-spa-3087ae.mp3

Interactive Visualization: Automating CNN Mapping on Embedded FPGAs

This episode explores a 2017 EPFL paper on turning a soft-input soft-output decoder kernel into a reusable RFNoC FPGA block using Vivado HLS, with a focus on software-defined radio and forward error correction. It explains why SISO processing sits at the core of turbo decoding, walking through the BCJR/MAP decoding logic, trellis-based forward and backward recursions, and the hardware challenges created by state metrics, memory traffic, and iterative probabilistic updates. The discussion argues that the paper is more convincing as a case study in HLS-based FPGA block construction and RFNoC integration than as proof of a complete, production-ready decoder system. Listeners would find it interesting for its clear look at the tradeoff between modular FPGA design convenience and the stubborn algorithmic complexity that still demands careful hardware thinking.

Sources:

1. RFNoC SISO Processor via High-Level Synthesis

https://www.epfl.ch/labs/lap/wp-content/uploads/2018/11/GuerrieriSep17_DesigningAnRfnocBlockImplementingASisoProcessorUsingHighLevelSynthesis_GNURC17.pdf

2. RFNoC: RF Network-on-Chip — Martin Braun, Jonathan Pendlum, Matt Ettus, 2016

https://scholar.google.com/scholar?q=RFNoC%3A+RF+Network-on-Chip

3. Designing a RFNoC Block implementing a SISO Processor using High-Level Synthesis — Andrea Guerrieri, 2017

https://scholar.google.com/scholar?q=Designing+a+RFNoC+Block+implementing+a+SISO+Processor+using+High-Level+Synthesis

4. Measured Latency Introduced by RF Network-on-Chip (RFNoC) Architecture — Joshua Sunderlin, 2017

https://scholar.google.com/scholar?q=Measured+Latency+Introduced+by+RF+Network-on-Chip+%28RFNoC%29+Architecture

5. Adventures in RFNoC: Lessons Learned From Developing a Real-Time Spectrum Sensing Block — Rylee G. Mattingly, Justin G. Metcalf, 2022

https://scholar.google.com/scholar?q=Adventures+in+RFNoC%3A+Lessons+Learned+From+Developing+a+Real-Time+Spectrum+Sensing+Block

6. The Software Radio Architecture — Joseph Mitola III, 1995

https://scholar.google.com/scholar?q=The+Software+Radio+Architecture

7. State of the art baseband DSP platforms for Software Defined Radio: A survey — Omer Anjum, Tapani Ahonen, Fabio Garzia, Jari Nurmi, Claudio Brunelli, Heikki Berg, 2011

https://scholar.google.com/scholar?q=State+of+the+art+baseband+DSP+platforms+for+Software+Defined+Radio%3A+A+survey

8. Sora: High-Performance Software Radio Using General-Purpose Multi-Core Processors — Kun Tan, He Liu, Jiansong Zhang, Yongguang Zhang, Ji Fang, Geoffrey M. Voelker, 2011

https://scholar.google.com/scholar?q=Sora%3A+High-Performance+Software+Radio+Using+General-Purpose+Multi-Core+Processors

9. OpenRadio: A Programmable Wireless Dataplane — Manu Bansal, Jeffrey Mehlman, Sachin Katti, Philip Levis, 2012

https://scholar.google.com/scholar?q=OpenRadio%3A+A+Programmable+Wireless+Dataplane

10. A Mathematical Theory of Communication — Claude E. Shannon, 1948

https://scholar.google.com/scholar?q=A+Mathematical+Theory+of+Communication

11. Low-Density Parity-Check Codes — Robert G. Gallager, 1962

https://scholar.google.com/scholar?q=Low-Density+Parity-Check+Codes

12. Design of Capacity-Approaching Irregular Low-Density Parity-Check Codes — Thomas J. Richardson, M. Amin Shokrollahi, Rudiger L. Urbanke, 2001

https://scholar.google.com/scholar?q=Design+of+Capacity-Approaching+Irregular+Low-Density+Parity-Check+Codes

13. Channel Coding: The Road to Channel Capacity — G. David Forney Jr., Daniel J. Costello Jr., 2007

https://scholar.google.com/scholar?q=Channel+Coding%3A+The+Road+to+Channel+Capacity

14. Near Shannon Limit Error-Correcting Coding and Decoding: Turbo-Codes — Claude Berrou, Alain Glavieux, Punya Thitimajshima, 1993

https://scholar.google.com/scholar?q=Near+Shannon+Limit+Error-Correcting+Coding+and+Decoding%3A+Turbo-Codes

15. A Comparison of Optimal and Sub-Optimal MAP Decoding Algorithms Operating in the Log Domain — Patrick Robertson, Emmanuelle Villebrun, Peter Hoher, 1995

https://scholar.google.com/scholar?q=A+Comparison+of+Optimal+and+Sub-Optimal+MAP+Decoding+Algorithms+Operating+in+the+Log+Domain

16. Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm — Robert J. McEliece, David J. C. MacKay, Jung-Fu Cheng, 1998

https://scholar.google.com/scholar?q=Turbo+Decoding+as+an+Instance+of+Pearl%27s+%22Belief+Propagation%22+Algorithm

17. A Survey of Three-Dimensional Turbo Codes and Recent Performance Enhancements — Dhouha Kbaier Ben Ismail, Catherine Douillard, Sylvie Kerouedan, 2013

https://scholar.google.com/scholar?q=A+Survey+of+Three-Dimensional+Turbo+Codes+and+Recent+Performance+Enhancements

18. Optimal Decoding of Linear Codes for Minimizing Symbol Error Rate — Lalit R. Bahl, John Cocke, Frederick Jelinek, Josef Raviv, 1974

https://scholar.google.com/scholar?q=Optimal+Decoding+of+Linear+Codes+for+Minimizing+Symbol+Error+Rate

19. From Low-Architectural Expertise up to High-Throughput Non-Binary LDPC Decoders: Optimization Guidelines Using High-Level Synthesis — Nithin George, Kimon Karras, David Novo, Vitor Silva, Paolo Ienne, Gabriel Falcao, 2015

https://scholar.google.com/scholar?q=From+Low-Architectural+Expertise+up+to+High-Throughput+Non-Binary+LDPC+Decoders%3A+Optimization+Guidelines+Using+High-Level+Synthesis

20. Turbo Decoder Architecture for Beyond-4G Applications — Cheng-Chi Wong, Hsie-Chia Chang, 2014

https://scholar.google.com/scholar?q=Turbo+Decoder+Architecture+for+Beyond-4G+Applications

21. A DSP Shared Is a DSP Earned: HLS Task-Level Multi-Pumping for High-Performance Low-Resource Designs — approx. recent FPGA/HLS architecture authors, 2023-2026

https://scholar.google.com/scholar?q=A+DSP+Shared+Is+a+DSP+Earned%3A+HLS+Task-Level+Multi-Pumping+for+High-Performance+Low-Resource+Designs

22. Making Acceleration More Amenable with Novel High-Level Synthesis Techniques for FPGAs — approx. recent FPGA/HLS systems authors, 2023-2026

https://scholar.google.com/scholar?q=Making+Acceleration+More+Amenable+with+Novel+High-Level+Synthesis+Techniques+for+FPGAs

23. High-Level Synthesis for FPGAs: A Hardware Engineer's Perspective — approx. recent survey/review authors, 2023-2026

https://scholar.google.com/scholar?q=High-Level+Synthesis+for+FPGAs%3A+A+Hardware+Engineer%27s+Perspective

24. Implementation of Cognitive Radio Based on SDR and FPGA: Methods, Architectures and Practical Applicability — approx. SDR/FPGA survey authors, recent

https://scholar.google.com/scholar?q=Implementation+of+Cognitive+Radio+Based+on+SDR+and+FPGA%3A+Methods%2C+Architectures+and+Practical+Applicability

25. StreamPU: A DSEL for High Throughput and Low Latency Software-Defined Radio on Multicore CPUs — approx. Cassagne and collaborators, recent

https://scholar.google.com/scholar?q=StreamPU%3A+A+DSEL+for+High+Throughput+and+Low+Latency+Software-Defined+Radio+on+Multicore+CPUs

26. Run-Time Reconfigurable Systems and Algorithms for RFSoC-Based Software Defined Radio Applications — approx. recent RFSoC/SDR authors, recent

https://scholar.google.com/scholar?q=Run-Time+Reconfigurable+Systems+and+Algorithms+for+RFSoC-Based+Software+Defined+Radio+Applications

27. DecodeX: Exploring and Benchmarking of LDPC Decoding Across CPU, GPU, and ASIC Platforms — approx. recent benchmarking authors, recent

https://scholar.google.com/scholar?q=DecodeX%3A+Exploring+and+Benchmarking+of+LDPC+Decoding+Across+CPU%2C+GPU%2C+and+ASIC+Platforms

28. High-Throughput Software-Defined LDPC Encoder and Decoder with x86-Based Data-Level Parallelism — approx. recent communications authors, recent

https://scholar.google.com/scholar?q=High-Throughput+Software-Defined+LDPC+Encoder+and+Decoder+with+x86-Based+Data-Level+Parallelism

Interactive Visualization: RFNoC SISO Processor via High-Level Synthesis

This episode explores a 2025 survey of reinforcement learning as a statement about how the field now organizes itself, covering value-based, policy-based, model-based, multi-agent, offline, and LLM-related RL. It explains core concepts like Markov decision processes, policies, value functions, delayed credit assignment, and the contrast between direct policy optimization and methods that estimate action values before deriving behavior. The discussion highlights why actor-critic methods became so central, how model-based RL uses world models to plan ahead, and why offline RL is difficult when agents must improve from fixed logged data rather than fresh interaction. Listeners would find it interesting because it turns a broad survey into a clear map of where reinforcement learning stands in 2025, including the tensions between elegant theory, unstable training, and the practical compromises that shaped modern RL.

Sources:

1. Reinforcement Learning: An Overview — Kevin Murphy, 2024

http://arxiv.org/abs/2412.05265

2. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems — Sergey Levine, Aviral Kumar, George Tucker, Justin Fu, 2020

https://scholar.google.com/scholar?q=Offline+Reinforcement+Learning%3A+Tutorial%2C+Review%2C+and+Perspectives+on+Open+Problems

3. D4RL: Datasets for Deep Data-Driven Reinforcement Learning — Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine, 2020

https://scholar.google.com/scholar?q=D4RL%3A+Datasets+for+Deep+Data-Driven+Reinforcement+Learning

4. Conservative Q-Learning for Offline Reinforcement Learning — Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine, 2020

https://scholar.google.com/scholar?q=Conservative+Q-Learning+for+Offline+Reinforcement+Learning

5. Decision Transformer: Reinforcement Learning via Sequence Modeling — Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch, 2021

https://scholar.google.com/scholar?q=Decision+Transformer%3A+Reinforcement+Learning+via+Sequence+Modeling

6. Reinforcement Learning: An Introduction — Richard S. Sutton and Andrew G. Barto, 2018

https://scholar.google.com/scholar?q=Reinforcement+Learning%3A+An+Introduction

7. Algorithms for Reinforcement Learning — Csaba Szepesvari, 2010

https://scholar.google.com/scholar?q=Algorithms+for+Reinforcement+Learning

8. Human-level control through deep reinforcement learning — Volodymyr Mnih, Koray Kavukcuoglu, David Silver and others, 2015

https://scholar.google.com/scholar?q=Human-level+control+through+deep+reinforcement+learning

9. Trust Region Policy Optimization — John Schulman, Sergey Levine, Philipp Moritz, Michael Jordan and Pieter Abbeel, 2015

https://scholar.google.com/scholar?q=Trust+Region+Policy+Optimization

10. Proximal Policy Optimization Algorithms — John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford and Oleg Klimov, 2017

https://scholar.google.com/scholar?q=Proximal+Policy+Optimization+Algorithms

11. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model — Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert and others, 2020

https://scholar.google.com/scholar?q=Mastering+Atari%2C+Go%2C+Chess+and+Shogi+by+Planning+with+a+Learned+Model

12. Fine-Tuning Language Models from Human Preferences — Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu and others, 2019

https://scholar.google.com/scholar?q=Fine-Tuning+Language+Models+from+Human+Preferences

13. Direct Preference Optimization: Your Language Model is Secretly a Reward Model — Rafael Rafailov, Archit Sharma, Eric Mitchell and others, 2023

https://scholar.google.com/scholar?q=Direct+Preference+Optimization%3A+Your+Language+Model+is+Secretly+a+Reward+Model

14. On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimization — Yong Lin et al., 2024

https://scholar.google.com/scholar?q=On+the+Limited+Generalization+Capability+of+the+Implicit+Reward+Model+Induced+by+Direct+Preference+Optimization

15. Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL — Taku Yamagata, Ahmed Khalil, Raul Santos-Rodriguez, 2023

https://scholar.google.com/scholar?q=Q-learning+Decision+Transformer%3A+Leveraging+Dynamic+Programming+for+Conditional+Sequence+Modelling+in+Offline+RL

16. Reinformer: Max-Return Sequence Modeling for Offline RL — Zifeng Zhuang et al., 2024

https://scholar.google.com/scholar?q=Reinformer%3A+Max-Return+Sequence+Modeling+for+Offline+RL

17. Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning — Jialong Wu, Haoyu Ma, Chaoyi Deng, Mingsheng Long, 2023

https://scholar.google.com/scholar?q=Pre-training+Contextualized+World+Models+with+In-the-wild+Videos+for+Reinforcement+Learning

18. PreLAR: World Model Pre-training with Learnable Action Representation — Lixuan Zhang, Meina Kan, Shiguang Shan, Xilin Chen, 2024

https://scholar.google.com/scholar?q=PreLAR%3A+World+Model+Pre-training+with+Learnable+Action+Representation

19. Ctrl-World: A Controllable Generative World Model for Robot Manipulation — Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, Chelsea Finn, 2025

https://scholar.google.com/scholar?q=Ctrl-World%3A+A+Controllable+Generative+World+Model+for+Robot+Manipulation

20. Skill Transfer and Discovery for Sim-to-Real Learning: A Representation-Based Viewpoint — Haitong Ma, Zhaolin Ren, Bo Dai, Na Li, 2024

https://scholar.google.com/scholar?q=Skill+Transfer+and+Discovery+for+Sim-to-Real+Learning%3A+A+Representation-Based+Viewpoint

21. AI Post Transformers: DreamerV3 World Models Across 150 Tasks — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-20-dreamerv3-world-models-across-150-tasks-af5edb.mp3

22. AI Post Transformers: Experience-Based Learning Beyond Human Data — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-19-experience-based-learning-beyond-human-d-b0caa4.mp3

23. AI Post Transformers: Learning to Reason with 13 Parameters — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-14-learning-to-reason-with-13-parameters-54c87f.mp3

24. AI Post Transformers: ASI-Evolve for Data, Architectures, and RL — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-05-asi-evolve-for-data-architectures-and-rl-197b2b.mp3

Interactive Visualization: Reinforcement Learning in 2025: An Overview

This episode explores a paper arguing that language models can reason more effectively at test time if they are trained to use divide-and-conquer strategies instead of defaulting to a single linear chain of thought. It explains the core distinction between ordinary step-by-step reasoning and structured decomposition into subproblems, then situates that idea alongside prior work such as Tree of Thoughts, Least-to-Most prompting, self-consistency, and recent reasoning-focused post-training. The discussion highlights the paper’s main claim that current post-training regimes bias models toward linear reasoning habits, which can make naive divide-and-conquer prompting underperform unless the decomposition behavior itself is explicitly trained. A listener would find it interesting because it gets at a central question in modern AI: whether better inference-time scaling comes from simply generating longer reasoning traces, or from teaching models to search, branch, and recombine intermediate results in a more algorithmic way.

Sources:

1. Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability — Xiao Liang, Zhong-Zhi Li, Zhenghao Lin, Eric Hancheng Jiang, Hengyuan Zhang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Yeyun Gong, Weizhu Chen, 2026

http://arxiv.org/abs/2602.02477

2. Tree of Thoughts: Deliberate Problem Solving with Large Language Models — Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, Karthik Narasimhan, 2023

https://scholar.google.com/scholar?q=Tree+of+Thoughts%3A+Deliberate+Problem+Solving+with+Large+Language+Models

3. Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions — Eric Zelikman, Qian Huang, Gabriel Poesia, Noah Goodman, Nick Haber, 2023

https://scholar.google.com/scholar?q=Parsel%3A+Algorithmic+Reasoning+with+Language+Models+by+Composing+Decompositions

4. Training LLMs for Divide-and-Conquer Reasoning Elevates Test-Time Scalability — Xiao Liang, Zhong-Zhi Li, Zhenghao Lin, Eric Hancheng Jiang, Hengyuan Zhang, Yelong Shen, Kai-Wei Chang, Ying Nian Wu, Yeyun Gong, Weizhu Chen, 2026

https://scholar.google.com/scholar?q=Training+LLMs+for+Divide-and-Conquer+Reasoning+Elevates+Test-Time+Scalability

5. Self-Consistency Improves Chain of Thought Reasoning in Language Models — Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou, 2022

https://scholar.google.com/scholar?q=Self-Consistency+Improves+Chain+of+Thought+Reasoning+in+Language+Models

6. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar, 2024

https://scholar.google.com/scholar?q=Scaling+LLM+Test-Time+Compute+Optimally+can+be+More+Effective+than+Scaling+Model+Parameters

7. Learning to Reason with LLMs — OpenAI, 2024

https://scholar.google.com/scholar?q=Learning+to+Reason+with+LLMs

8. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — Daya Guo, Dejian Yang and the DeepSeek-AI team, 2025

https://scholar.google.com/scholar?q=DeepSeek-R1%3A+Incentivizing+Reasoning+Capability+in+LLMs+via+Reinforcement+Learning

9. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models — Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, Ed Chi, 2022

https://scholar.google.com/scholar?q=Least-to-Most+Prompting+Enables+Complex+Reasoning+in+Large+Language+Models

10. Decomposed Prompting: A Modular Approach for Solving Complex Tasks — Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, Ashish Sabharwal, 2022

https://scholar.google.com/scholar?q=Decomposed+Prompting%3A+A+Modular+Approach+for+Solving+Complex+Tasks

11. DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition — Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin and the DeepSeek team, 2025

https://scholar.google.com/scholar?q=DeepSeek-Prover-V2%3A+Advancing+Formal+Mathematical+Reasoning+via+Reinforcement+Learning+for+Subgoal+Decomposition

12. Decompose, Analyze and Rethink: Solving Intricate Problems with Human-like Reasoning Cycle — Shangzi Xue, Zhenya Huang, Jiayu Liu, Xin Lin, Yuting Ning, Binbin Jin, Xin Li, Qi Liu, 2024

https://scholar.google.com/scholar?q=Decompose%2C+Analyze+and+Rethink%3A+Solving+Intricate+Problems+with+Human-like+Reasoning+Cycle

13. Seed-Prover: Deep and Broad Reasoning for Automated Theorem Proving — Luoxin Chen, Jinming Gu, Liankai Huang, Wenhao Huang, Zhicheng Jiang, Allan Jie, Xiaoran Jin, Xing Jin, Chenggang Li, Kaijing Ma, Cheng Ren, Jiawei Shen, Wenlei Shi, Tong Sun, He Sun, Jiahui Wang, Siran Wang, Zhihong Wang, Chenrui Wei, Shufa Wei, Yonghui Wu, Yuchen Wu, et al., 2025

https://scholar.google.com/scholar?q=Seed-Prover%3A+Deep+and+Broad+Reasoning+for+Automated+Theorem+Proving

14. DAPO: An Open-Source LLM Reinforcement Learning System at Scale — Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, Mingxuan Wang, 2025

https://scholar.google.com/scholar?q=DAPO%3A+An+Open-Source+LLM+Reinforcement+Learning+System+at+Scale

15. Continuous Chain of Thought Enables Parallel Exploration and Reasoning — Halil Alperen Gozeten et al., 2025

https://scholar.google.com/scholar?q=Continuous+Chain+of+Thought+Enables+Parallel+Exploration+and+Reasoning

16. How to Think Step-by-Step: A Mechanistic Understanding of Chain-of-Thought Reasoning — Subhabrata Dutta et al., 2024

https://scholar.google.com/scholar?q=How+to+Think+Step-by-Step%3A+A+Mechanistic+Understanding+of+Chain-of-Thought+Reasoning

17. Decompose-ToM: Enhancing Theory of Mind Reasoning in Large Language Models through Simulation and Task Decomposition — Sneheel Sarangi et al., 2025

https://scholar.google.com/scholar?q=Decompose-ToM%3A+Enhancing+Theory+of+Mind+Reasoning+in+Large+Language+Models+through+Simulation+and+Task+Decomposition

18. Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models — Shuodi Liu et al., 2025

https://scholar.google.com/scholar?q=Select-Then-Decompose%3A+From+Empirical+Analysis+to+Adaptive+Selection+Strategy+for+Task+Decomposition+in+Large+Language+Models

19. LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models — Weibin Liao et al., 2025

https://scholar.google.com/scholar?q=LearNAT%3A+Learning+NL2SQL+with+AST-guided+Task+Decomposition+for+Large+Language+Models

20. Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning — Chang Tian et al., 2025

https://scholar.google.com/scholar?q=Large+Language+Models+Reasoning+Abilities+Under+Non-Ideal+Conditions+After+RL-Fine-Tuning

21. AI Post Transformers: Learning to Reason with 13 Parameters — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-14-learning-to-reason-with-13-parameters-54c87f.mp3

22. AI Post Transformers: IMO-Bench for Robust Mathematical Reasoning — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-04-imo-bench-for-robust-mathematical-reason-143489.mp3

23. AI Post Transformers: Test-time Scaling for Multi-Agent Collaborative Reasoning — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-22-test-time-scaling-for-multi-agent-collab-082570.mp3

24. AI Post Transformers: Chain-of-Thought Reasoning: A Brittle Mirage? — Hal Turing & Dr. Ada Shannon, 2025

https://podcast.do-not-panic.com/episodes/chain-of-thought-reasoning-a-brittle-mirage/

25. AI Post Transformers: NeurIPS 2025: Reinforcement Learning for Reasoning in Large Language Models with One Training Example — Hal Turing & Dr. Ada Shannon, 2025

https://podcast.do-not-panic.com/episodes/neurips-2025-reinforcement-learning-for-reasoning-in-large-language-models-with/

26. AI Post Transformers: TraceRL: Reinforcement Learning for Diffusion Language Models — Hal Turing & Dr. Ada Shannon, 2025

https://podcast.do-not-panic.com/episodes/tracerl-reinforcement-learning-for-diffusion-language-models/

Interactive Visualization: Training LLMs for Divide-and-Conquer Reasoning

This episode explores PackKV, a method for shrinking the transformer KV cache during long-context inference by combining low-bit quantization with GPU-friendly repacking and lossy compression. It explains why KV cache growth can dominate memory use in large models, using examples where cache size exceeds model weights, and frames the problem as a systems bottleneck driven more by memory traffic than raw computation. The discussion compares PackKV to prior approaches such as KV quantization, token pruning, and offloading to CPU memory, highlighting the paper’s argument that compression is only useful if decompression is tightly integrated into the inference pipeline. A listener would find it interesting because it turns a seemingly low-level optimization into a broader claim about how future long-context LLM performance may depend as much on memory layout and kernel design as on model architecture.

Sources:

1. PackKV: Reducing KV Cache Memory Footprint through LLM-Aware Lossy Compression — Bo Jiang, Taolue Yang, Youyuan Liu, Xubin He, Sheng Di, Sian Jin, 2025

http://arxiv.org/abs/2512.24449

2. Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time — Zichang Liu, Aditya Desai, Fangshuo Liao, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, Anshumali Shrivastava, et al., 2023

https://scholar.google.com/scholar?q=Scissorhands%3A+Exploiting+the+Persistence+of+Importance+Hypothesis+for+LLM+KV+Cache+Compression+at+Test+Time

3. H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models — Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Zhao Song, Yuandong Tian, Clark Barrett, Zhangyang Wang, Beidi Chen, et al., 2023

https://scholar.google.com/scholar?q=H2O%3A+Heavy-Hitter+Oracle+for+Efficient+Generative+Inference+of+Large+Language+Models

4. KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache — Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, Xia Hu, 2024

https://scholar.google.com/scholar?q=KIVI%3A+A+Tuning-Free+Asymmetric+2bit+Quantization+for+KV+Cache

5. SnapKV: LLM Knows What You are Looking for Before Generation — Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, Deming Chen, 2024

https://scholar.google.com/scholar?q=SnapKV%3A+LLM+Knows+What+You+are+Looking+for+Before+Generation

6. CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving — Yuyang Liu, Haotian Li, Yao Cheng, Siddhant Ray, Yizhou Huang, Qizhen Zhang, Kaixiang Du, Jinyang Yao, Shan Lu, Ganesh Ananthanarayanan et al., 2024

https://scholar.google.com/scholar?q=CacheGen%3A+KV+Cache+Compression+and+Streaming+for+Fast+Large+Language+Model+Serving

7. Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache — Zhuodong Zhang, Shang Liu, Ruobing Chen, Bhavya Kailkhura, Ben Chen, An Wang, 2024

https://scholar.google.com/scholar?q=Q-Hitter%3A+A+Better+Token+Oracle+for+Efficient+LLM+Inference+via+Sparse-Quantized+KV+Cache

8. PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling — Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, Wen Xiao, 2025

https://scholar.google.com/scholar?q=PyramidKV%3A+Dynamic+KV+Cache+Compression+based+on+Pyramidal+Information+Funneling

9. Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution — Alessio Devoto, Maximilian Jeblick, Simon Jegou, 2025

https://scholar.google.com/scholar?q=Expected+Attention%3A+KV+Cache+Compression+by+Estimating+Attention+from+Future+Queries+Distribution

10. TurboQuant: Online Vector Quantization with Near-Optimal Distortion — Amir Zandieh, Majid Daliri, Majid Hadian, Vahab Mirrokni, 2025

https://scholar.google.com/scholar?q=TurboQuant%3A+Online+Vector+Quantization+with+Near-Optimal+Distortion

11. KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse — Jingbo Yang et al., 2025

https://scholar.google.com/scholar?q=KVLink%3A+Accelerating+Large+Language+Models+via+Efficient+KV+Cache+Reuse

12. HyperRAG: Enhancing Quality-Efficiency Tradeoffs in Retrieval-Augmented Generation with Reranker KV-Cache Reuse — Yuwei An et al., 2025

https://scholar.google.com/scholar?q=HyperRAG%3A+Enhancing+Quality-Efficiency+Tradeoffs+in+Retrieval-Augmented+Generation+with+Reranker+KV-Cache+Reuse

13. ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation — Shihao Wang et al., 2026

https://scholar.google.com/scholar?q=ProphetKV%3A+User-Query-Driven+Selective+Recomputation+for+Efficient+KV+Cache+Reuse+in+Retrieval-Augmented+Generation

14. ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identification — Yefei He et al., 2024

https://scholar.google.com/scholar?q=ZipCache%3A+Accurate+and+Efficient+KV+Cache+Quantization+with+Salient+Token+Identification

15. KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs — Zunhai Su and Kehong Yuan, 2025

https://scholar.google.com/scholar?q=KVSink%3A+Understanding+and+Enhancing+the+Preservation+of+Attention+Sinks+in+KV+Cache+Quantization+for+LLMs

16. ThinK: Thinner Key Cache by Query-Driven Pruning — Yuhui Xu et al., 2024

https://scholar.google.com/scholar?q=ThinK%3A+Thinner+Key+Cache+by+Query-Driven+Pruning

17. KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head — Isaac Rehg, 2024

https://scholar.google.com/scholar?q=KV-Compress%3A+Paged+KV-Cache+Compression+with+Variable+Compression+Rates+per+Attention+Head

18. Paged Attention Meets FlexAttention: Unlocking Long-Context Efficiency in Deployed Inference — Thomas Joshi et al., 2025

https://scholar.google.com/scholar?q=Paged+Attention+Meets+FlexAttention%3A+Unlocking+Long-Context+Efficiency+in+Deployed+Inference

19. AI Post Transformers: TokenDance for Multi-Agent KV Cache Sharing — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-22-tokendance-for-multi-agent-kv-cache-shar-aa9b99.mp3

20. AI Post Transformers: KVSwap for Disk-Aware Long-Context On-Device Inference — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-16-kvswap-for-disk-aware-long-context-on-de-f3c15e.mp3

21. AI Post Transformers: CacheFlow and 3D-Parallel KV Cache Restoration — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-01-cacheflow-and-3d-parallel-kv-cache-resto-8db883.mp3

22. AI Post Transformers: TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-03-25-turboquant-online-vector-quantiz-1967b7.mp3

23. AI Post Transformers: FengHuang for Rack-Scale LLM Inference Memory — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-12-fenghuang-for-rack-scale-llm-inference-m-62708e.mp3

24. AI Post Transformers: Computation-Bandwidth-Memory Trade-offs for AI Infrastructure — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-09-computation-bandwidth-memory-trade-offs-a83f2b.mp3

Interactive Visualization: PackKV Lossy Compression for KV Caches

This episode explores how the OOMB training system tries to break the memory bottleneck that makes million-token language model training impractical, focusing on why training long contexts is much harder than simply extending inference-time context windows. It explains the paper’s core ideas in plain language, including chunk-recurrent training that recomputes activations during backpropagation, O(1)-style activation memory, and the harder remaining problem of storing and moving the KV cache across extremely long sequences. The discussion also weighs the paper’s central claim with healthy skepticism, asking whether fitting multi-million-token training steps on a single GPU proves genuinely useful long-range learning or mainly demonstrates a strong systems optimization. Listeners would find it interesting because it connects deep learning mechanics, hardware limits, and competing long-context strategies like Ring Attention into a clear debate about what real progress in long-context LLMs should look like.

Sources:

1. Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts — Wenhao Li, Daohai Yu, Gen Luo, Yuxin Zhang, Fei Chao, Rongrong Ji, Yifan Wu, Jiaxin Liu, Ziyang Gong, Zimu Liao, 2026

http://arxiv.org/abs/2602.02108

2. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context — Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov, 2019

https://scholar.google.com/scholar?q=Transformer-XL%3A+Attentive+Language+Models+Beyond+a+Fixed-Length+Context

3. Recurrent Memory Transformer — Aydar Bulatov, Yuri Kuratov, Mikhail S. Burtsev, 2022

https://scholar.google.com/scholar?q=Recurrent+Memory+Transformer

4. Ring Attention with Blockwise Transformers for Near-Infinite Context — Hao Liu, Matei Zaharia, Pieter Abbeel, 2023

https://scholar.google.com/scholar?q=Ring+Attention+with+Blockwise+Transformers+for+Near-Infinite+Context

5. Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention — Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth Gopal, 2024

https://scholar.google.com/scholar?q=Leave+No+Context+Behind%3A+Efficient+Infinite+Context+Transformers+with+Infini-attention

6. LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models — Yingfeng Chen et al., 2024

https://scholar.google.com/scholar?q=LongLoRA%3A+Efficient+Fine-tuning+of+Long-Context+Large+Language+Models

7. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon et al., 2023

https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention

8. Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism — Mohammad Shoeybi et al., 2020

https://scholar.google.com/scholar?q=Megatron-LM%3A+Training+Multi-Billion+Parameter+Language+Models+Using+Model+Parallelism

9. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints — Joshua Ainslie et al., 2023

https://scholar.google.com/scholar?q=GQA%3A+Training+Generalized+Multi-Query+Transformer+Models+from+Multi-Head+Checkpoints

10. ZeRO-Offload: Democratizing Billion-Scale Model Training — Samyam Rajbhandari et al., 2020

https://scholar.google.com/scholar?q=ZeRO-Offload%3A+Democratizing+Billion-Scale+Model+Training

11. Long Context Compression with Activation Beacon — approx. Liu et al., 2024

https://scholar.google.com/scholar?q=Long+Context+Compression+with+Activation+Beacon

12. Boosting Long-Context Information Seeking via Query-Guided Activation Refilling — approx. unknown from snippet, 2024 or 2025

https://scholar.google.com/scholar?q=Boosting+Long-Context+Information+Seeking+via+Query-Guided+Activation+Refilling

13. Kvlink: Accelerating Large Language Models via Efficient KV Cache Reuse — approx. unknown from snippet, 2024 or 2025

https://scholar.google.com/scholar?q=Kvlink%3A+Accelerating+Large+Language+Models+via+Efficient+KV+Cache+Reuse

14. SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips — approx. unknown from snippet, 2024 or 2025

https://scholar.google.com/scholar?q=SuperOffload%3A+Unleashing+the+Power+of+Large-Scale+LLM+Training+on+Superchips

15. SPPO: Efficient Long-Sequence LLM Training via Adaptive Sequence Pipeline Parallel Offloading — approx. unknown from snippet, 2024 or 2025

https://scholar.google.com/scholar?q=SPPO%3A+Efficient+Long-Sequence+LLM+Training+via+Adaptive+Sequence+Pipeline+Parallel+Offloading

16. Keep the Cost Down: A Review on Methods to Optimize LLM's KV-Cache Consumption — approx. unknown from snippet, 2024 or 2025

https://scholar.google.com/scholar?q=Keep+the+Cost+Down%3A+A+Review+on+Methods+to+Optimize+LLM%27s+KV-Cache+Consumption

17. AI Post Transformers: DeepSeek-V4 and Practical Million-Token Context — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-25-deepseek-v4-and-practical-million-token-6f4de1.mp3

18. AI Post Transformers: KVSwap for Disk-Aware Long-Context On-Device Inference — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-16-kvswap-for-disk-aware-long-context-on-de-f3c15e.mp3

19. AI Post Transformers: RetrievalAttention for Long-Context LLM Inference — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-17-retrievalattention-for-long-context-llm-ddf566.mp3

20. AI Post Transformers: CacheFlow and 3D-Parallel KV Cache Restoration — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-01-cacheflow-and-3d-parallel-kv-cache-resto-8db883.mp3

21. AI Post Transformers: Gated Linear Attention for Efficient Long Sequences — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-18-gated-linear-attention-for-efficient-lon-c858ab.mp3

22. AI Post Transformers: Parallelizing DeltaNet Linear Transformers over Sequence Length — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-18-parallelizing-deltanet-linear-transforme-2d0377.mp3

Interactive Visualization: Training Million-Token LLMs Beyond the Memory Barrier

This episode explores how transformers split prediction between knowledge stored in their weights and information inferred from the current prompt, using the paper’s synthetic “bigram world” to make those mechanisms visible. It explains the distinction between global statistical knowledge and true in-context knowledge, then walks through induction heads as a concrete circuit for recalling earlier patterns and continuing them later. The discussion highlights the paper’s main finding that models learn easy dataset-wide averages first, while context-sensitive induction behavior emerges later and requires the right architecture, with two-layer transformers succeeding where one-layer models fail. Listeners would find it interesting because it turns a vague claim about in-context learning into a causal, mechanistic story about how temporary memory may actually form during training.

Sources:

1. Birth of a Transformer: A Memory Viewpoint — Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, Leon Bottou, 2023

http://arxiv.org/abs/2306.00802

2. A Mathematical Framework for Transformer Circuits — Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, Chris Olah, 2021

https://scholar.google.com/scholar?q=A+Mathematical+Framework+for+Transformer+Circuits

3. In-context Learning and Induction Heads — Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, Chris Olah, 2022

https://scholar.google.com/scholar?q=In-context+Learning+and+Induction+Heads

4. Birth of a Transformer: A Memory Viewpoint — Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, Leon Bottou, 2023

https://scholar.google.com/scholar?q=Birth+of+a+Transformer%3A+A+Memory+Viewpoint

5. What Learning Algorithm Is In-Context Learning? Investigations with Linear Models — Ekin Akyurek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou, 2023

https://scholar.google.com/scholar?q=What+Learning+Algorithm+Is+In-Context+Learning%3F+Investigations+with+Linear+Models

6. Data Distributional Properties Drive Emergent In-Context Learning in Transformers — Stephanie C. Y. Chan, Adam Santoro, Andrew Lampinen, Jane Wang, Aaditya Singh, Pierre Richemond, Jay McClelland, Felix Hill, 2022

https://scholar.google.com/scholar?q=Data+Distributional+Properties+Drive+Emergent+In-Context+Learning+in+Transformers

7. Transformer Feed-Forward Layers Are Key-Value Memories — Mor Geva, Roei Schuster, Jonathan Berant, Omer Levy, 2021

https://scholar.google.com/scholar?q=Transformer+Feed-Forward+Layers+Are+Key-Value+Memories

8. Dissecting Recall of Factual Associations in Auto-Regressive Language Models — Mor Geva, Jasmijn Bastings, Katja Filippova, Amir Globerson, 2023

https://scholar.google.com/scholar?q=Dissecting+Recall+of+Factual+Associations+in+Auto-Regressive+Language+Models

9. What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation — Aaditya K. Singh, Ted Moskovitz, Felix Hill, Stephanie C. Y. Chan, Andrew M. Saxe, 2024

https://scholar.google.com/scholar?q=What+needs+to+go+right+for+an+induction+head%3F+A+mechanistic+study+of+in-context+learning+circuits+and+their+formation

10. Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks — Tianyu He, Darshil Doshi, Aritra Das, Andrey Gromov, 2024

https://scholar.google.com/scholar?q=Learning+to+grok%3A+Emergence+of+in-context+learning+and+skill+composition+in+modular+arithmetic+tasks

11. Selective Induction Heads: How Transformers Select Causal Structures In Context — Francesco D'Angelo, Francesco Croce, Nicolas Flammarion, 2025

https://scholar.google.com/scholar?q=Selective+Induction+Heads%3A+How+Transformers+Select+Causal+Structures+In+Context

12. Induction Head Toxicity Mechanistically Explains Repetition Curse in Large Language Models — Shuxun Wang, Qingyu Yin, Chak Tou Leong, Qiang Zhang, Linyi Yang, 2025

https://scholar.google.com/scholar?q=Induction+Head+Toxicity+Mechanistically+Explains+Repetition+Curse+in+Large+Language+Models

13. AI Post Transformers: In-Place Test-Time Training for Transformers — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-09-in-place-test-time-training-for-transfor-d0b976.mp3

14. AI Post Transformers: Linear Classifier Probes for Intermediate Layers — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-16-linear-classifier-probes-for-intermediat-927ae3.mp3

15. AI Post Transformers: Parallelizing DeltaNet Linear Transformers over Sequence Length — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-18-parallelizing-deltanet-linear-transforme-2d0377.mp3

16. AI Post Transformers: Gated Delta Networks for Long-Context Retrieval — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-04-17-gated-delta-networks-for-long-context-re-706d85.mp3

Interactive Visualization: How Induction Heads Emerge in Transformers

This episode explores how DeepWalk helped launch modern graph representation learning by turning random walks over a social network into “sentences” and then applying the Skip-Gram ideas behind word2vec to learn dense node embeddings. It explains why that mattered in 2014: instead of relying on heavy spectral or matrix-factorization methods, DeepWalk offered an online, scalable way to learn reusable graph features that worked especially well when labeled data was scarce. The discussion digs into the paper’s main empirical claim that, on social-network benchmarks like BlogCatalog, Flickr, and YouTube, the method substantially improved node classification under sparse-label settings. It is interesting because the conversation goes beyond the headline result and asks what really drove the gains: the language-modeling objective, the community-biased random-walk sampler, or simply a better optimization setup for homophilous graphs.

Sources:

1. DeepWalk and the Rise of Graph Embeddings

https://arxiv.org/pdf/1403.6652

2. Distributed Representations of Words and Phrases and their Compositionality — Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean, 2013

https://scholar.google.com/scholar?q=Distributed+Representations+of+Words+and+Phrases+and+their+Compositionality

3. Efficient Estimation of Word Representations in Vector Space — Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, 2013

https://scholar.google.com/scholar?q=Efficient+Estimation+of+Word+Representations+in+Vector+Space

4. Learning Latent Social Dimensions for Link Prediction — Lei Tang, Huan Liu, 2011

https://scholar.google.com/scholar?q=Learning+Latent+Social+Dimensions+for+Link+Prediction

5. LINE: Large-scale Information Network Embedding — Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, Qiaozhu Mei, 2015

https://scholar.google.com/scholar?q=LINE%3A+Large-scale+Information+Network+Embedding

6. node2vec: Scalable Feature Learning for Networks — Aditya Grover, Jure Leskovec, 2016

https://scholar.google.com/scholar?q=node2vec%3A+Scalable+Feature+Learning+for+Networks

7. Planetoid: Inductive Representation Learning on Large Graphs — Zhilin Yang, William W. Cohen, Ruslan Salakhutdinov, 2016

https://scholar.google.com/scholar?q=Planetoid%3A+Inductive+Representation+Learning+on+Large+Graphs

8. Node Embedding for Homophilous Graphs with ARGEW: Augmentation of Random walks by Graph Edge Weights — authors not shown in snippet, recent; exact year not shown

https://scholar.google.com/scholar?q=Node+Embedding+for+Homophilous+Graphs+with+ARGEW%3A+Augmentation+of+Random+walks+by+Graph+Edge+Weights

9. Graph neural networks for graphs with heterophily: A survey — authors not shown in snippet, recent; exact year not shown

https://scholar.google.com/scholar?q=Graph+neural+networks+for+graphs+with+heterophily%3A+A+survey

10. Learning attribute and homophily measures through random walks — authors not shown in snippet, recent; exact year not shown

https://scholar.google.com/scholar?q=Learning+attribute+and+homophily+measures+through+random+walks

11. Graph Node Embedding by Neighborhood Prediction Based on Multiview Contrastive Learning — authors not shown in snippet, recent; exact year not shown

https://scholar.google.com/scholar?q=Graph+Node+Embedding+by+Neighborhood+Prediction+Based+on+Multiview+Contrastive+Learning

12. Dynamic graph representation learning with neural networks: A survey — authors not shown in snippet, recent; exact year not shown

https://scholar.google.com/scholar?q=Dynamic+graph+representation+learning+with+neural+networks%3A+A+survey

13. AI Post Transformers: GraphSAGE: Inductive Representation Learning on Large Graphs — Hal Turing & Dr. Ada Shannon, Mon,

https://podcast.do-not-panic.com/episodes/graphsage-inductive-representation-learning-on-large-graphs/

Interactive Visualization: DeepWalk and the Rise of Graph Embeddings