Sign up to save your podcasts
Or
Share AI Post Transformers
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
AI Post Transformers
By mcgrofTechnology
AI Post Transformers episodes
- Formal Lean Proof of PAC Learnability for Decision StumpsThis episode explores a 2021 formal proof, in the Lean theorem prover, that decision stumps (one-threshold classifiers) are PAC learnable. It starts from a simple learner that places its threshold at the largest positively labeled training point, so it can only err by calling true positives negative. The discussion explains why formalizing this on the real line is hard. Probability over uncountable spaces needs sigma-algebras and measurability, which textbooks skip, and the Giry monad lets the formal statement stay close to the pencil-and-paper one. It also compares this work with earlier formal ML efforts, including Certigrad's proof of unbiased stochastic backpropagation gradients in Lean, Coq-verified Hoeffding bounds for finite hypothesis classes, and VC-dimension theory, which the paper deliberately leaves out. The episode also sets up the claim that a step in the standard textbook proofs doesn't hold as stated. That claim should interest anyone curious about what machine-checked proofs reveal about familiar learning-theory results.Sources:1. A Formal Proof of PAC Learnability for Decision Stumps — Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, Jean-Baptiste Tristan, 2019http://arxiv.org/abs/1911.003852. A Formal Proof of PAC Learnability for Decision Stumps — Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, Jean-Baptiste Tristan, 2021 (CPP '21; arXiv 1911.00385)https://scholar.google.com/scholar?q=A+Formal+Proof+of+PAC+Learnability+for+Decision+Stumps3. Developing Bug-Free Machine Learning Systems With Formal Mathematics — Daniel Selsam, Percy Liang, David L. Dill, 2017 (ICML)https://scholar.google.com/scholar?q=Developing+Bug-Free+Machine+Learning+Systems+With+Formal+Mathematics4. Formal Verification of Neural Network Robustness and Related Surveys (e.g., Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks) — Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer, 2017 (CAV)https://scholar.google.com/scholar?q=Formal+Verification+of+Neural+Network+Robustness+and+Related+Surveys+%28e.g.%2C+Reluplex%3A+An+Efficient+SMT+Solver+for+Verifying+Deep+Neural+Networks%295. Formal Verification of a Stochastic Gradient Descent Convergence Proof / Formal Verification of Stochastic Gradient Descent (Lean and Coq lines of work) — Various, including Koundinya Vajjha, Avraham Shinnar, Barry Trager, Vincent Pestun, Nathan Fulton (Coq), 2021-2022https://scholar.google.com/scholar?q=Formal+Verification+of+a+Stochastic+Gradient+Descent+Convergence+Proof+%2F+Formal+Verification+of+Stochastic+Gradient+Descent+%28Lean+and+Coq+lines+of+work%296. A Theory of the Learnable — Leslie G. Valiant, 1984https://scholar.google.com/scholar?q=A+Theory+of+the+Learnable7. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting — Yoav Freund, Robert E. Schapire, 1997https://scholar.google.com/scholar?q=A+Decision-Theoretic+Generalization+of+On-Line+Learning+and+an+Application+to+Boosting8. Greedy Function Approximation: A Gradient Boosting Machine — Jerome H. Friedman, 2001https://scholar.google.com/scholar?q=Greedy+Function+Approximation%3A+A+Gradient+Boosting+Machine9. Understanding Machine Learning: From Theory to Algorithms — Shai Shalev-Shwartz, Shai Ben-David, 2014https://scholar.google.com/scholar?q=Understanding+Machine+Learning%3A+From+Theory+to+Algorithms10. Interactive Theorem Proving and Program Development: Coq'Art / Formalization of measure theory and Lebesgue integration in Isabelle/HOL — Johannes Hölzl and Armin Heller, 2011https://scholar.google.com/scholar?q=Interactive+Theorem+Proving+and+Program+Development%3A+Coq%27Art+%2F+Formalization+of+measure+theory+and+Lebesgue+integration+in+Isabelle%2FHOL11. Formalization of Lebesgue Integration and Probability in Coq / Verified probabilistic reasoning (e.g., Coq-based measure theory by Affeldt, Cohen and collaborators) — Reynald Affeldt, Cyril Cohen, and collaborators, 2019-2023https://scholar.google.com/scholar?q=Formalization+of+Lebesgue+Integration+and+Probability+in+Coq+%2F+Verified+probabilistic+reasoning+%28e.g.%2C+Coq-based+measure+theory+by+Affeldt%2C+Cohen+and+collaborators%2912. Formalizing Probability Theory in Lean's mathlib — Mathlib community, including Kexing Ying, Rémy Degenne and others, 2020-2024https://scholar.google.com/scholar?q=Formalizing+Probability+Theory+in+Lean%27s+mathlib13. Machine Learning and Measure Theory textbook treatment: Foundations of Machine Learning / Foundations of Modern Probability — Mohri, Rostamizadeh, Talwalkar; Olav Kallenberg, 2018 / 2002https://scholar.google.com/scholar?q=Machine+Learning+and+Measure+Theory+textbook+treatment%3A+Foundations+of+Machine+Learning+%2F+Foundations+of+Modern+Probability14. The Lean Theorem Prover (System Description) — Leonardo de Moura, Soonho Kong, Jeremy Avigad, Floris van Doorn, Jakob von Raumer, 2015 (CADE)https://scholar.google.com/scholar?q=The+Lean+Theorem+Prover+%28System+Description%2915. The Lean 4 Theorem Prover and Programming Language — Leonardo de Moura and Sebastian Ullrich, 2021 (CADE)https://scholar.google.com/scholar?q=The+Lean+4+Theorem+Prover+and+Programming+Language16. Formal Verification of a Realistic Compiler (CompCert) — Xavier Leroy, 2009https://scholar.google.com/scholar?q=Formal+Verification+of+a+Realistic+Compiler+%28CompCert%2917. seL4: Formal Verification of an OS Kernel — Gerwin Klein et al., 2009https://scholar.google.com/scholar?q=seL4%3A+Formal+Verification+of+an+OS+Kernel18. Certified Programming with Dependent Types / Developing Bug-Free Machine Learning Systems With Formal Mathematics (Certigrad) — Daniel Selsam, Percy Liang, David L. Dill, 2017https://scholar.google.com/scholar?q=Certified+Programming+with+Dependent+Types+%2F+Developing+Bug-Free+Machine+Learning+Systems+With+Formal+Mathematics+%28Certigrad%2919. Certifying the True Error: Machine Learning in Coq with Verified Generalization Guarantees — Alexander Bagnall, Gordon Stewart, 2019https://scholar.google.com/scholar?q=Certifying+the+True+Error%3A+Machine+Learning+in+Coq+with+Verified+Generalization+Guarantees20. Learnability and the Vapnik-Chervonenkis Dimension — Anselm Blumer, Andrzej Ehrenfeucht, David Haussler, Manfred K. Warmuth, 1989https://scholar.google.com/scholar?q=Learnability+and+the+Vapnik-Chervonenkis+Dimension21. Verified Analysis of Random Trees — Manuel Eberl, Max W. Haslbeck, Tobias Nipkow, 2018https://scholar.google.com/scholar?q=Verified+Analysis+of+Random+Trees22. Formal-ML: formal machine learning library for Lean — Martin Zinkevich, 2020https://scholar.google.com/scholar?q=Formal-ML%3A+formal+machine+learning+library+for+Lean23. A Formal Proof of the Expressiveness of Deep Learning — Alexander Bentkamp, Jasmin Christian Blanchette, Dietrich Klakow, 2019https://scholar.google.com/scholar?q=A+Formal+Proof+of+the+Expressiveness+of+Deep+Learning24. A Formally Verified Proof of the Central Limit Theorem — Jeremy Avigad, Johannes Hölzl, Luke Serafin, 2017https://scholar.google.com/scholar?q=A+Formally+Verified+Proof+of+the+Central+Limit+Theorem25. Verified Tail Bounds for Randomized Programs — Joseph Tassarotti, Robert Harper, 2018https://scholar.google.com/scholar?q=Verified+Tail+Bounds+for+Randomized+Programs26. Formal verification of generalization / PAC-Bayes and Rademacher complexity bounds in Lean 4 / mathlib (2023-2025 follow-up work) — Various (Lean/mathlib community; Zinkevich-style learning-theory libraries; Lean 4 'LeanMachineLearning' efforts), 2023-2025https://scholar.google.com/scholar?q=Formal+verification+of+generalization+%2F+PAC-Bayes+and+Rademacher+complexity+bounds+in+Lean+4+%2F+mathlib+%282023-2025+follow-up+work%29Interactive Visualization: Formal Lean Proof of PAC Learnability for Decision Stumps17 min
- Backprop as Functor: Composing Learners Through Category TheoryThis episode examines "Backprop as Functor," which argues that gradient descent on a whole network equals the composition of the gradient-descent learners of its parts. It defines a learner as four pieces: a parameter space, an implementation, an update, and a "request" function. The request tells an upstream module what input it should have received, and this supplies the missing intermediate training target. The discussion explains the categorical machinery in plain terms (categories, functors, monoidal structure, and the Para and Learn categories). It presents the main theorem: a faithful, strong symmetric monoidal functor sends parametrised functions to gradient-descent learners, with the request given by backprop. The episode compares this to difference target propagation, which only approximates backprop, and to lenses and open games, where the request plays the role of the backward "put". It notes that the paper has no experiments and asks whether the formalism does more than restate the fact that the chain rule is functorial. It then starts a worked quadratic-error example showing why the composite update matches plain gradient descent. Listeners get a clear view of what a compositional treatment of backprop does and does not buy.Sources:1. Backprop as Functor: A compositional perspective on supervised learning — Brendan Fong, David I. Spivak, Rémy Tuyéras, 2017http://arxiv.org/abs/1711.104552. Backprop as Functor: A compositional perspective on supervised learning — Brendan Fong, David Spivak, Rémy Tuyéras, 2017 (arXiv); LICS 2019https://scholar.google.com/scholar?q=Backprop+as+Functor%3A+A+compositional+perspective+on+supervised+learning3. Categorical Foundations of Gradient-Based Learning — G.S.H. Cruttwell, Bruno Gavranović, Neil Ghani, Paul Wilson, Fabio Zanasi, 2021 (arXiv); ESOP 2022https://scholar.google.com/scholar?q=Categorical+Foundations+of+Gradient-Based+Learning4. The Simple Essence of Automatic Differentiation — Conal Elliott, 2018https://scholar.google.com/scholar?q=The+Simple+Essence+of+Automatic+Differentiation5. Position: Categorical Deep Learning is an Algebraic Theory of All Architectures — Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković, 2024https://scholar.google.com/scholar?q=Position%3A+Categorical+Deep+Learning+is+an+Algebraic+Theory+of+All+Architectures6. Difference Target Propagation — Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, Yoshua Bengio, 2015https://scholar.google.com/scholar?q=Difference+Target+Propagation7. Compositional Game Theory — Neil Ghani, Jules Hedges, Viktor Winschel, Philipp Zahn, 2016 (arXiv); LICS 2018https://scholar.google.com/scholar?q=Compositional+Game+Theory8. Combinators for Bidirectional Tree Transformations: A Linguistic Approach to the View-Update Problem — J. Nathan Foster, Michael B. Greenwald, Jonathan T. Moore, Benjamin C. Pierce, Alan Schmitt, 2007https://scholar.google.com/scholar?q=Combinators+for+Bidirectional+Tree+Transformations%3A+A+Linguistic+Approach+to+the+View-Update+Problem9. Lenses and Learners — Brendan Fong, Michael Johnson, 2019https://scholar.google.com/scholar?q=Lenses+and+Learners10. From Open Learners to Open Games — Jules Hedges, 2019https://scholar.google.com/scholar?q=From+Open+Learners+to+Open+Games11. Categorical Deep Learning: An Algebraic Theory of Architectures — Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković, 2024https://scholar.google.com/scholar?q=Categorical+Deep+Learning%3A+An+Algebraic+Theory+of+Architectures12. Reverse Derivative Categories — R. Cockett, G. Cruttwell, J. Gallagher, J.-S. P. Lemay, B. MacAdam, G. Plotkin, D. Pronk, 2020https://scholar.google.com/scholar?q=Reverse+Derivative+Categories13. Backpropagation in the Simply Typed Lambda-calculus with Linear Negation — Alois Brunel, Damiano Mazza, Michele Pagani, 2020https://scholar.google.com/scholar?q=Backpropagation+in+the+Simply+Typed+Lambda-calculus+with+Linear+Negation14. Understanding Deep Image Representations by Inverting Them — A. Mahendran, A. Vedaldi, 2015https://scholar.google.com/scholar?q=Understanding+Deep+Image+Representations+by+Inverting+Them15. Automatic Differentiation in Machine Learning: a Survey — A. G. Baydin, B. A. Pearlmutter, A. A. Radul, J. M. Siskind, 2018https://scholar.google.com/scholar?q=Automatic+Differentiation+in+Machine+Learning%3A+a+SurveyInteractive Visualization: Backprop as Functor: Composing Learners Through Category Theory17 min
- Compact Proofs of Model Performance via Mechanistic InterpretabilityThis episode explores a 2024 paper that tries to turn mechanistic interpretability into formal guarantees. The idea is to reverse-engineer a small transformer into an algorithm, then use that algorithm to write a much shorter proof of a lower bound on the model's accuracy. The test case is a one-layer, attention-only model trained to output the maximum of four integers. The discussion contrasts this approach with local-robustness verifiers and PAC-Bayes bounds. It also sets out the two trivial extremes: brute-force enumeration of about sixteen million inputs, and a vacuous accuracy bound of zero. The interpretation finds that the model attends to larger tokens, copies what it attends to, and barely depends on position. Treating everything the interpretation leaves unexplained as worst-case yields a "cubic" proof that cuts the compute roughly four-million-fold, at the cost of certifying about 95% accuracy instead of the true 99.9%. Listeners get a clear look at how a better mechanistic understanding could make guarantees cheaper, along with the caveat that the evidence comes from a single tiny model.Sources:1. Compact Proofs of Model Performance via Mechanistic Interpretability — Jason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong, Chun Hei Yip, Alex Gibson, Soufiane Noubir, Lawrence Chan, 2024http://arxiv.org/abs/2406.117792. Compact Proofs of Model Performance via Mechanistic Interpretability — Jason Gross, Rajashree Agrawal, Thomas Kwa, Euan Ong, Chun Hei Yip, Alex Gibson, Soufiane Noubir, Lawrence Chan, 2024 (arXiv 2406.11779; later versions through Dec 2024)https://scholar.google.com/scholar?q=Compact+Proofs+of+Model+Performance+via+Mechanistic+Interpretability3. Progress measures for grokking via mechanistic interpretability — Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt, 2023 (ICLR 2023)https://scholar.google.com/scholar?q=Progress+measures+for+grokking+via+mechanistic+interpretability4. A Mathematical Framework for Transformer Circuits — Nelson Elhage, Neel Nanda, Catherine Olsson, Chris Olah and others (Anthropic), 2021https://scholar.google.com/scholar?q=A+Mathematical+Framework+for+Transformer+Circuits5. Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems — David 'davidad' Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, Joshua Tenenbaum, 2024https://scholar.google.com/scholar?q=Towards+Guaranteed+Safe+AI%3A+A+Framework+for+Ensuring+Robust+and+Reliable+AI+Systems6. Formal Algorithms/Verification baselines: Towards Fast Computation of Certified Robustness for ReLU Networks (Fast-Lin) and Efficient Formal Safety Analysis of Neural Networks (Neurify); also Reluplex — Reluplex: Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer (2017); Fast-Lin: Tsui-Wei Weng et al. (2018); Neurify: Shiqi Wang et al. (2018), 2017-2018https://scholar.google.com/scholar?q=Formal+Algorithms%2FVerification+baselines%3A+Towards+Fast+Computation+of+Certified+Robustness+for+ReLU+Networks+%28Fast-Lin%29+and+Efficient+Formal+Safety+Analysis+of+Neural+Networks+%28Neurify%29%3B+also+Reluplex7. Formalizing the Presumption of Independence — Paul Christiano, Eric Neyman, Mark Xu, 2022https://scholar.google.com/scholar?q=Formalizing+the+Presumption+of+Independence8. Causal Scrubbing: a method for rigorously testing interpretability hypotheses — Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, Nate Thomas, 2022https://scholar.google.com/scholar?q=Causal+Scrubbing%3A+a+method+for+rigorously+testing+interpretability+hypotheses9. Mathematical Models of Computation in Superposition — Kaarel Hänni, Jake Mendel, Dmitry Vaintrob, Lawrence Chan, 2024https://scholar.google.com/scholar?q=Mathematical+Models+of+Computation+in+Superposition10. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks — Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer, 2017https://scholar.google.com/scholar?q=Reluplex%3A+An+Efficient+SMT+Solver+for+Verifying+Deep+Neural+Networks11. Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data — Gintare Karolina Dziugaite, Daniel M. Roy, 2017https://scholar.google.com/scholar?q=Computing+Nonvacuous+Generalization+Bounds+for+Deep+%28Stochastic%29+Neural+Networks+with+Many+More+Parameters+than+Training+Data12. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation — Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, Martin Vechev, 2018https://scholar.google.com/scholar?q=AI2%3A+Safety+and+Robustness+Certification+of+Neural+Networks+with+Abstract+Interpretation13. How Transformers Learn Causal Structure with Gradient Descent — Eshaan Nichani, Alex Damian, Jason D. Lee, 2024https://scholar.google.com/scholar?q=How+Transformers+Learn+Causal+Structure+with+Gradient+Descent14. Opening the AI Black Box: Program Synthesis via Mechanistic Interpretability — Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, et al., 2024https://scholar.google.com/scholar?q=Opening+the+AI+Black+Box%3A+Program+Synthesis+via+Mechanistic+InterpretabilityInteractive Visualization: Compact Proofs of Model Performance via Mechanistic Interpretability15 min
- Exact Learning Dynamics in Deep Linear Neural NetworksThis episode explores the Saxe, McClelland, and Ganguli paper on exact learning dynamics in deep linear neural networks. It explains why these networks are worth studying even though they collapse to a single linear map: their gradient-descent dynamics are still coupled and non-convex, and they show the long plateaus followed by sudden error drops seen in real deep nets. The discussion covers the Baldi and Hornik result that every critical point is a saddle except the best low-rank solution. It then covers the "decoupled invariant manifold," where each input-output mode evolves independently. The mode-strength equation, whose growth rate is proportional to its own value, explains why small initial weights produce long plateaus. The episode argues that depth need not slow learning if the initial conditions are chosen well, and ties this to unsupervised pretraining and to random initialization. Listeners get a clear, step-by-step derivation of why training time depends on the starting point.Sources:1. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks — Andrew M. Saxe, James L. McClelland, Surya Ganguli, 2013http://arxiv.org/abs/1312.61202. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks — Andrew M. Saxe, James L. McClelland, Surya Ganguli, 2013 (arXiv:1312.6120; ICLR 2014)https://scholar.google.com/scholar?q=Exact+solutions+to+the+nonlinear+dynamics+of+learning+in+deep+linear+neural+networks3. Deep Learning without Poor Local Minima — Kenji Kawaguchi, 2016https://scholar.google.com/scholar?q=Deep+Learning+without+Poor+Local+Minima4. On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization — Sanjeev Arora, Nadav Cohen, Elad Hazan, 2018https://scholar.google.com/scholar?q=On+the+Optimization+of+Deep+Networks%3A+Implicit+Acceleration+by+Overparameterization5. Implicit Regularization in Deep Matrix Factorization — Sanjeev Arora, Nadav Cohen, Wei Hu, Yuping Luo, 2019https://scholar.google.com/scholar?q=Implicit+Regularization+in+Deep+Matrix+Factorization6. A Fast Learning Algorithm for Deep Belief Nets — Geoffrey E. Hinton, Simon Osindero, Yee-Whye Teh, 2006https://scholar.google.com/scholar?q=A+Fast+Learning+Algorithm+for+Deep+Belief+Nets7. Greedy Layer-Wise Training of Deep Networks — Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle, 2006 (NIPS 2007)https://scholar.google.com/scholar?q=Greedy+Layer-Wise+Training+of+Deep+Networks8. Why Does Unsupervised Pre-training Help Deep Learning? — Dumitru Erhan, Yoshua Bengio, Aaron Courville, Pierre-Antoine Manzagol, Pascal Vincent, Samy Bengio, 2010 (JMLR)https://scholar.google.com/scholar?q=Why+Does+Unsupervised+Pre-training+Help+Deep+Learning%3F9. Understanding the difficulty of training deep feedforward neural networks — Xavier Glorot, Yoshua Bengio, 2010https://scholar.google.com/scholar?q=Understanding+the+difficulty+of+training+deep+feedforward+neural+networks10. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification — Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, 2015https://scholar.google.com/scholar?q=Delving+Deep+into+Rectifiers%3A+Surpassing+Human-Level+Performance+on+ImageNet+Classification11. Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks — Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington, 2018https://scholar.google.com/scholar?q=Dynamical+Isometry+and+a+Mean+Field+Theory+of+CNNs%3A+How+to+Train+10%2C000-Layer+Vanilla+Convolutional+Neural+Networks12. Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice — Jeffrey Pennington, Samuel S. Schoenholz, Surya Ganguli, 2017https://scholar.google.com/scholar?q=Resurrecting+the+sigmoid+in+deep+learning+through+dynamical+isometry%3A+theory+and+practice13. Exponential expressivity in deep neural networks through transient chaos — Ben Poole, Subhaneil Lahiri, Maithra Raghu, Jascha Sohl-Dickstein, Surya Ganguli, 2016https://scholar.google.com/scholar?q=Exponential+expressivity+in+deep+neural+networks+through+transient+chaos14. Deep Information Propagation — Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein, 2017https://scholar.google.com/scholar?q=Deep+Information+Propagation15. Neural Tangents: Fast and Easy Infinite Neural Networks in Python — Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz, 2019https://scholar.google.com/scholar?q=Neural+Tangents%3A+Fast+and+Easy+Infinite+Neural+Networks+in+Python16. Learning hierarchical category structure in deep neural networks — A. M. Saxe, J. L. McClelland, S. Ganguli, 2013https://scholar.google.com/scholar?q=Learning+hierarchical+category+structure+in+deep+neural+networks17. Neural networks and principal component analysis: Learning from examples without local minima — P. Baldi, K. Hornik, 1989https://scholar.google.com/scholar?q=Neural+networks+and+principal+component+analysis%3A+Learning+from+examples+without+local+minima18. Effect of Batch Learning In Multilayer Neural Networks — K. Fukumizu, 1998https://scholar.google.com/scholar?q=Effect+of+Batch+Learning+In+Multilayer+Neural+Networks19. On the importance of initialization and momentum in deep learning — I. Sutskever, J. Martens, G. Dahl, G. Hinton, 2013https://scholar.google.com/scholar?q=On+the+importance+of+initialization+and+momentum+in+deep+learning20. Exact learning dynamics of deep linear networks with prior knowledge — L. Braun, C. Dominé, J. Fitzgerald, A. Saxe, 2022https://scholar.google.com/scholar?q=Exact+learning+dynamics+of+deep+linear+networks+with+prior+knowledge21. Deep learning without poor local minima (and related: Kawaguchi 2016) — K. Kawaguchi, 2016https://scholar.google.com/scholar?q=Deep+learning+without+poor+local+minima+%28and+related%3A+Kawaguchi+2016%2922. A mathematical theory of semantic development in deep neural networks — A. Saxe, J. McClelland, S. Ganguli, 2019https://scholar.google.com/scholar?q=A+mathematical+theory+of+semantic+development+in+deep+neural+networks23. Implicit regularization in deep matrix factorization / Arora, Cohen, Hu, Luo — S. Arora, N. Cohen, W. Hu, Y. Luo, 2019https://scholar.google.com/scholar?q=Implicit+regularization+in+deep+matrix+factorization+%2F+Arora%2C+Cohen%2C+Hu%2C+Luo24. On the Convergence of Gradient Descent for Wide/Deep Linear Networks (Arora, Cohen, Golowich, Hu) — S. Arora, N. Cohen, N. Golowich, W. Hu, 2019https://scholar.google.com/scholar?q=On+the+Convergence+of+Gradient+Descent+for+Wide%2FDeep+Linear+Networks+%28Arora%2C+Cohen%2C+Golowich%2C+Hu%2925. Progress measures for grokking via mechanistic interpretability / Saddle-to-saddle dynamics literature (e.g., Jacot et al., 'Saddle-to-saddle dynamics in deep linear networks') — A. Jacot, F. Ged, B. Şimşek, C. Hongler, F. Gabriel, 2021https://scholar.google.com/scholar?q=Progress+measures+for+grokking+via+mechanistic+interpretability+%2F+Saddle-to-saddle+dynamics+literature+%28e.g.%2C+Jacot+et+al.%2C+%27Saddle-to-saddle+dynamics+in+deep+linear+networks%27%2926. Deep Residual Learning for Image Recognition — K. He, X. Zhang, S. Ren, J. Sun, 2016https://scholar.google.com/scholar?q=Deep+Residual+Learning+for+Image+Recognition27. Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer (muP) — G. Yang, E. Hu et al., 2022https://scholar.google.com/scholar?q=Tensor+Programs+V%3A+Tuning+Large+Neural+Networks+via+Zero-Shot+Hyperparameter+Transfer+%28muP%29Interactive Visualization: Exact Learning Dynamics in Deep Linear Neural Networks16 min
- DeLM: Decentralized Coding Agents Sharing Context Without a LeaderThis episode examines DeLM, a Stanford system that runs several coding agents in parallel with no main agent, and asks whether four agents deliver four agents' worth of progress or just a bigger bill. It sorts existing designs by the "bubble" each one creates. Independent sampling wastes effort on redundant work. Peer debate stalls at round barriers and grows quadratically in messages. Centralized orchestrators like Claude Code and Codex leave the lead agent idle and pass lossy summaries between agents. DeLM replaces the lead with a self-service task queue, where the first claim wins, and an append-only shared log of FACT and FAIL entries that agents publish as they learn things. The discussion ties this to the 1980 Hearsay-II blackboard system, which had a scheduler where DeLM has none, and notes the new risk that stale or wrong natural-language findings can mislead peers. The hosts also say how they will judge the claimed accuracy and speed gains: whether the benchmarks are representative, whether the gaps exceed run-to-run noise, and how much of the speedup comes from the design rather than from idle baselines.Sources:1. Decentralized Multi-Agent Systems with Shared Context — Yuzhen Mao, Jerry Gu, Aadi Chauhan, Qizheng Zhang, Hangoo Kang, Azalia Mirhoseini, 2026http://arxiv.org/abs/2606.106622. The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty — Lee D. Erman, Frederick Hayes-Roth, Victor R. Lesser, D. Raj Reddy, 1980https://scholar.google.com/scholar?q=The+Hearsay-II+Speech-Understanding+System%3A+Integrating+Knowledge+to+Resolve+Uncertainty3. Generative Communication in Linda — David Gelernter, 1985https://scholar.google.com/scholar?q=Generative+Communication+in+Linda4. Improving Factuality and Reasoning in Language Models through Multiagent Debate — Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch, 2023https://scholar.google.com/scholar?q=Improving+Factuality+and+Reasoning+in+Language+Models+through+Multiagent+Debate5. Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture — Bochen Han, Songmao Zhang, 2025https://scholar.google.com/scholar?q=Exploring+Advanced+LLM+Multi-Agent+Systems+Based+on+Blackboard+Architecture6. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan, 2024 (ICLR; arXiv 2023)https://scholar.google.com/scholar?q=SWE-bench%3A+Can+Language+Models+Resolve+Real-World+GitHub+Issues%3F7. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press, 2024https://scholar.google.com/scholar?q=SWE-agent%3A+Agent-Computer+Interfaces+Enable+Automated+Software+Engineering8. Introducing SWE-bench Verified — OpenAI (Preparedness team) with human annotators, in collaboration with the SWE-bench authors, 2024https://scholar.google.com/scholar?q=Introducing+SWE-bench+Verified9. Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces — Mike A. Merrill, Alexander G. Shaw, and the Terminal-Bench team (Stanford and Laude Institute); the DeLM paper cites Merrill et al., 2026, 2025-2026https://scholar.google.com/scholar?q=Terminal-Bench%3A+Benchmarking+Agents+on+Hard%2C+Realistic+Tasks+in+Command+Line+Interfaces10. Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, Azalia Mirhoseini, 2024https://scholar.google.com/scholar?q=Large+Language+Monkeys%3A+Scaling+Inference+Compute+with+Repeated+Sampling11. Scheduling Multithreaded Computations by Work Stealing — Robert D. Blumofe, Charles E. Leiserson, 1999https://scholar.google.com/scholar?q=Scheduling+Multithreaded+Computations+by+Work+Stealing12. GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism — Yanping Huang et al., 2019https://scholar.google.com/scholar?q=GPipe%3A+Efficient+Training+of+Giant+Neural+Networks+using+Pipeline+Parallelism13. AOrchestra — Ruan et al., 2026https://scholar.google.com/scholar?q=AOrchestra14. SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models — Emil Biju, Shayan Talaei, Zhemin Huang, Mohammadreza Pourreza, Azalia Mirhoseini, Amin Saberi, 2026https://scholar.google.com/scholar?q=SPRINT%3A+Enabling+Interleaved+Planning+and+Parallelized+Execution+in+Reasoning+Models15. ToolOrchestra — Su et al., 2025https://scholar.google.com/scholar?q=ToolOrchestra16. Recursive Language Models — Zhang et al., 2025https://scholar.google.com/scholar?q=Recursive+Language+Models0 min
- MEMORILLA: Latent Semantic Memory Beats Text Retrieval for Frozen LLMsThis episode examines MEMORILLA, a Stanford-led preprint in which a small trained module reads a whole store of pre-embedded documents and hands a frozen language model just 16 soft-token vectors, instead of pasting retrieved text into the prompt. It motivates the design with a personalization scenario. An old, unrelated chat reveals a preference for blunt feedback, so similarity search may miss it, and the signal is spread across many chats that top-k retrieval can't integrate. The discussion contrasts this approach with full-context prompting, agent memory systems (MemGPT, Mem0, MemoRAG, A-MEM), question-blind compression methods (ICAE, xRAG), and per-corpus cache training (Cartridges). It also traces the design's lineage to Perceiver, BLIP-2's Q-Former and prompt tuning. Listeners get a skeptical framing of the headline claim, including whether gains come from reading the store or from training on the benchmarks, and whether comparing against baselines that weren't trained on them is fair. The evaluation covers personalization, long-document QA, knowledge-intensive QA and a TextWorld agent.Sources:1. Memorilla: Latent Semantic Memory for LLMs — Adibvafa Fallahpour, Parsa Idehpour, Vignesh Kothapalli, Nikita Mounier, Shirley Wu, Shayan Pardis, Jure Leskovec, 2026http://arxiv.org/abs/2610.213752. MemGPT: Towards LLMs as Operating Systems — Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez, 2023https://scholar.google.com/scholar?q=MemGPT%3A+Towards+LLMs+as+Operating+Systems3. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory — Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, Deshraj Yadav, 2025https://scholar.google.com/scholar?q=Mem0%3A+Building+Production-Ready+AI+Agents+with+Scalable+Long-Term+Memory4. In-context Autoencoder for Context Compression in a Large Language Model (ICAE) — Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei, 2024 (ICLR; arXiv 2023)https://scholar.google.com/scholar?q=In-context+Autoencoder+for+Context+Compression+in+a+Large+Language+Model+%28ICAE%295. xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token — Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, Dongyan Zhao, 2024https://scholar.google.com/scholar?q=xRAG%3A+Extreme+Context+Compression+for+Retrieval-augmented+Generation+with+One+Token6. Cartridges: Lightweight and General-Purpose Long Context Representations via Self-Study — Sabri Eyuboglu, Ryan Ehrlich, Simran Arora, Neel Guha, Dylan Zinsley, Emily Liu, William Tennien, Atri Rudra, James Zou, Azalia Mirhoseini, Christopher Ré, 2025 (cited in the Memorilla paper as 2026)https://scholar.google.com/scholar?q=Cartridges%3A+Lightweight+and+General-Purpose+Long+Context+Representations+via+Self-Study7. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models — Junnan Li, Dongxu Li, Silvio Savarese, Steven Hoi, 2023https://scholar.google.com/scholar?q=BLIP-2%3A+Bootstrapping+Language-Image+Pre-training+with+Frozen+Image+Encoders+and+Large+Language+Models8. Perceiver: General Perception with Iterative Attention — Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira, 2021https://scholar.google.com/scholar?q=Perceiver%3A+General+Perception+with+Iterative+Attention9. The Power of Scale for Parameter-Efficient Prompt Tuning — Brian Lester, Rami Al-Rfou, Noah Constant, 2021https://scholar.google.com/scholar?q=The+Power+of+Scale+for+Parameter-Efficient+Prompt+Tuning10. PISCO: Pretty Simple Compression for Retrieval-Augmented Generation — Maxime Louis, Hervé Déjean, Stéphane Clinchant, 2025https://scholar.google.com/scholar?q=PISCO%3A+Pretty+Simple+Compression+for+Retrieval-Augmented+Generation11. CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning — Jie He, Richard He Bai, Sinead Williamson, Jeff Z Pan, Navdeep Jaitly, Yizhe Zhang, 2025https://scholar.google.com/scholar?q=CLaRa%3A+Bridging+Retrieval+and+Generation+with+Continuous+Latent+Reasoning12. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2024https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts13. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval — Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A Huerta, Hao Peng, 2025https://scholar.google.com/scholar?q=Context+Length+Alone+Hurts+LLM+Performance+Despite+Perfect+Retrieval14. Titans: Learning to Memorize at Test Time — Ali Behrouz, Peilin Zhong, Vahab Mirrokni, 2025https://scholar.google.com/scholar?q=Titans%3A+Learning+to+Memorize+at+Test+Time15. Learning to Compress Prompts with Gist Tokens — Jesse Mu, Xiang Li, Noah Goodman, 2023https://scholar.google.com/scholar?q=Learning+to+Compress+Prompts+with+Gist+Tokens16. Memorizing Transformers — Yuhuai Wu, Markus N. Rabe, DeLesley Hutchins, Christian Szegedy, 2022https://scholar.google.com/scholar?q=Memorizing+Transformers0 min
- Multi-Agent Transactive Memory: Sharing Agent Trajectories Across a PopulationThis episode explores Multi-Agent Transactive Memory, a paper asking whether many agents can dump their raw task trajectories into one shared, searchable repository and let other frozen models retrieve from it. That would improve success rates and cut step counts without any joint training or coordination. It builds up from retrieval-augmented generation and single-agent experience memory (Reflexion, Synapse, ExpeL) to the shared-pool idea, borrowing the name from Wegner's group-psychology notion of transactive memory. The discussion stresses that the analogy is loose: the system is closer to a Stack Overflow for agents than to a team, and the trajectories were never written to be read. It also covers the retrieval design, where the key is the task description plus the last five steps embedded with E5-Base and the stored value is the next five steps from a past run. The pool is seeded with public trajectories from models like GPT-4-Turbo, Claude 3.5 Sonnet, Qwen3-32B and GPT-OSS 20B, then grows as agents work. Listeners get a skeptical look at the ALFWorld and WebArena testbeds, including how small and templated they are, and at the missing private-versus-shared memory control that the paper's central claim needs.Sources:1. Multi-Agent Transactive Memory — To Eun Kim, Xuhong He, Dishank Jain, Ambuj Agrawal, Negar Arabzadeh, Fernando Diaz, 2026http://arxiv.org/abs/2606.199112. The Transactive Memory System in Relationships / Cognitive Interdependence in Close Relationships (Transactive Memory: A Contemporary Analysis of the Group Mind) — Daniel M. Wegner, 1985/1987https://scholar.google.com/scholar?q=The+Transactive+Memory+System+in+Relationships+%2F+Cognitive+Interdependence+in+Close+Relationships+%28Transactive+Memory%3A+A+Contemporary+Analysis+of+the+Group+Mind%293. Generative Agents: Interactive Simulacra of Human Behavior — Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein, 2023https://scholar.google.com/scholar?q=Generative+Agents%3A+Interactive+Simulacra+of+Human+Behavior4. Voyager: An Open-Ended Embodied Agent with Large Language Models — Guanzhi Wang, Yuqi Xie, Yunfan Jiang, et al. (Anandkumar, Fan), 2023https://scholar.google.com/scholar?q=Voyager%3A+An+Open-Ended+Embodied+Agent+with+Large+Language+Models5. Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control — Longtao Zheng, Rundong Wang, Xinrun Wang, Bo An, 2024 (ICLR)https://scholar.google.com/scholar?q=Synapse%3A+Trajectory-as-Exemplar+Prompting+with+Memory+for+Computer+Control6. Reflexion: Language Agents with Verbal Reinforcement Learning — Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao, 2023https://scholar.google.com/scholar?q=Reflexion%3A+Language+Agents+with+Verbal+Reinforcement+Learning7. Agent Workflow Memory — Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig, 2024https://scholar.google.com/scholar?q=Agent+Workflow+Memory8. ExpeL: LLM Agents Are Experiential Learners — Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, Gao Huang, 2024https://scholar.google.com/scholar?q=ExpeL%3A+LLM+Agents+Are+Experiential+Learners9. Retrieval-Enhanced Machine Learning — Hamed Zamani, Fernando Diaz, Mostafa Dehghani, Donald Metzler, Michael Bendersky, 2022https://scholar.google.com/scholar?q=Retrieval-Enhanced+Machine+Learning10. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Douwe Kiela, et al., 2020https://scholar.google.com/scholar?q=Retrieval-Augmented+Generation+for+Knowledge-Intensive+NLP+Tasks11. Agent KB: Leveraging Cross-Domain Experience for Agentic Problem Solving — Xiangru Tang et al., 2025https://scholar.google.com/scholar?q=Agent+KB%3A+Leveraging+Cross-Domain+Experience+for+Agentic+Problem+Solving12. Memp: Exploring Agent Procedural Memory — Runnan Fang et al., 2025https://scholar.google.com/scholar?q=Memp%3A+Exploring+Agent+Procedural+Memory13. AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases — Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, Bo Li, 2024https://scholar.google.com/scholar?q=AgentPoison%3A+Red-teaming+LLM+Agents+via+Poisoning+Memory+or+Knowledge+Bases14. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models — Wei Zou, Runpeng Geng, Binghui Wang, Jinyuan Jia, 2025https://scholar.google.com/scholar?q=PoisonedRAG%3A+Knowledge+Corruption+Attacks+to+Retrieval-Augmented+Generation+of+Large+Language+Models15. Evaluating Retrieval Quality in Retrieval-Augmented Generation (eRAG) — Alireza Salemi, Hamed Zamani, 2024https://scholar.google.com/scholar?q=Evaluating+Retrieval+Quality+in+Retrieval-Augmented+Generation+%28eRAG%2916. Retrieval-Augmented Reinforcement Learning — Anirudh Goyal et al., 2022https://scholar.google.com/scholar?q=Retrieval-Augmented+Reinforcement+Learning17. Many-Shot In-Context Learning — Rishabh Agarwal et al., 2024https://scholar.google.com/scholar?q=Many-Shot+In-Context+Learning18. Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory — Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, James Zou, 2025https://scholar.google.com/scholar?q=Dynamic+Cheatsheet%3A+Test-Time+Learning+with+Adaptive+Memory0 min
- PASTA: Steering LLM Attention Toward Highlighted Prompt TextThis episode examines PASTA, a method that lets users highlight part of a prompt and has the model attend to it more strongly, without changing any weights. It covers why markup like asterisks may not work as emphasis for LLMs. It then places PASTA against instruction tuning, activation steering (ActAdd, CAA, Inference-Time Intervention) and knowledge editing (ROME, MEMIT, REMEDI). The key difference is that PASTA changes where selected attention heads read, rather than what they output. The discussion explains the mechanism: scores outside the highlighted span are scaled by a small alpha (0.01) and renormalized, which is equivalent to a roughly hundredfold relative boost on the span. The hosts also raise questions about head selection and evaluation, including the absence of comparisons against the neighboring steering methods. Listeners interested in controllability, long-context failures and conflicts between a prompt and a model's memorized knowledge will find it useful.Sources:1. Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs — Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao, 2023http://arxiv.org/abs/2311.022622. Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs — Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, Tuo Zhao, 2023 (arXiv 2311.02262); ICLR 2024https://scholar.google.com/scholar?q=Tell+Your+Model+Where+to+Attend%3A+Post-hoc+Attention+Steering+for+LLMs3. Prompt-to-Prompt Image Editing with Cross-Attention Control — Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, Daniel Cohen-Or, 2022https://scholar.google.com/scholar?q=Prompt-to-Prompt+Image+Editing+with+Cross-Attention+Control4. Efficient Streaming Language Models with Attention Sinks — Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis, 2023 (ICLR 2024)https://scholar.google.com/scholar?q=Efficient+Streaming+Language+Models+with+Attention+Sinks5. Lost in the Middle: How Language Models Use Long Contexts — Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, 2023 (TACL 2024)https://scholar.google.com/scholar?q=Lost+in+the+Middle%3A+How+Language+Models+Use+Long+Contexts6. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model — Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg, 2023 (NeurIPS 2023)https://scholar.google.com/scholar?q=Inference-Time+Intervention%3A+Eliciting+Truthful+Answers+from+a+Language+Model7. Steering Llama 2 via Contrastive Activation Addition — Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, Alexander Turner, 2023https://scholar.google.com/scholar?q=Steering+Llama+2+via+Contrastive+Activation+Addition8. Activation Addition: Steering Language Models Without Optimization — Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, Monte MacDiarmid, 2023https://scholar.google.com/scholar?q=Activation+Addition%3A+Steering+Language+Models+Without+Optimization9. Trusting Your Evidence: Hallucinate Less with Context-aware Decoding — Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, Scott Wen-tau Yih, 2023https://scholar.google.com/scholar?q=Trusting+Your+Evidence%3A+Hallucinate+Less+with+Context-aware+Decoding10. Retrieval Head Mechanistically Explains Long-Context Factuality — Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, Yao Fu, 2024https://scholar.google.com/scholar?q=Retrieval+Head+Mechanistically+Explains+Long-Context+Factuality11. Model Tells You Where to Merge / AutoPASTA: Model Tells You What to Discard (Adaptive KV cache compression) — Various (follow-up and adjacent work, see AutoPASTA by Zhuang et al. 2024), 2024https://scholar.google.com/scholar?q=Model+Tells+You+Where+to+Merge+%2F+AutoPASTA%3A+Model+Tells+You+What+to+Discard+%28Adaptive+KV+cache+compression%2912. Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering (SPA) — Praveen Venkateswaran, Danish Contractor, 2025https://scholar.google.com/scholar?q=Spotlight+Your+Instructions%3A+Instruction-following+with+Dynamic+Attention+Steering+%28SPA%2913. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning — Tri Dao, 2023https://scholar.google.com/scholar?q=FlashAttention-2%3A+Faster+Attention+with+Better+Parallelism+and+Work+Partitioning14. Attention is not Explanation / Attention is not not Explanation — Sarthak Jain and Byron Wallace; Sarah Wiegreffe and Yuval Pinter, 2019https://scholar.google.com/scholar?q=Attention+is+not+Explanation+%2F+Attention+is+not+not+Explanation15. Are Sixteen Heads Really Better than One? / Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting — Paul Michel, Omer Levy, Graham Neubig; Elena Voita et al., 2019https://scholar.google.com/scholar?q=Are+Sixteen+Heads+Really+Better+than+One%3F+%2F+Analyzing+Multi-Head+Self-Attention%3A+Specialized+Heads+Do+the+Heavy+LiftingInteractive Visualization: PASTA: Steering LLM Attention Toward Highlighted Prompt Text0 min
- Claude 3.5 Haiku Counts Characters on Curved Feature ManifoldsThis episode explores how Claude 3.5 Haiku decides when a hard-wrapped line of text is full, even though it sees only token IDs and is never told the line width. It covers the paper's finding that the model tracks the character count on a curved, roughly six-dimensional manifold. That manifold is tiled by about ten sparse features with widening tuning curves, a bit like place cells. The discussion connects this to earlier work on circular and helical feature geometry and to the rotary position encoding used in transformers, and it explains attribution graphs and sparse dictionary features along the way. It also covers the experimental setup: newlines reinserted at known widths from 15 to 150 characters. Zero-ablating the character-count subspace raises loss mainly on newline tokens, which gives causal evidence that the model uses this subspace and doesn't just encode the count somewhere a probe can find it. Listeners interested in mechanistic interpretability will find a concrete case of a model implementing a continuous quantity as geometry, not as one direction per concept.Sources:1. When Models Manipulate Manifolds: The Geometry of a Counting Task — Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar, Julius Tarng, Adam Pearce, Chris Olah, Joshua Batson, 2026http://arxiv.org/abs/2601.044802. Toy Models of Superposition — Nelson Elhage, Tristan Hume, Catherine Olsson, et al. (Anthropic), 2022https://scholar.google.com/scholar?q=Toy+Models+of+Superposition3. Not All Language Model Features Are Linear — Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, Max Tegmark, 2024https://scholar.google.com/scholar?q=Not+All+Language+Model+Features+Are+Linear4. Towards Monosemanticity: Decomposing Language Models With Dictionary Learning — Trenton Bricken, Adly Templeton, Joshua Batson, et al. (Anthropic), 2023https://scholar.google.com/scholar?q=Towards+Monosemanticity%3A+Decomposing+Language+Models+With+Dictionary+Learning5. Language Models Use Trigonometry to Do Addition (and related work on helical number representations) — Subhash Kantamneni, Max Tegmark, 2025https://scholar.google.com/scholar?q=Language+Models+Use+Trigonometry+to+Do+Addition+%28and+related+work+on+helical+number+representations%296. Circuit Tracing: Revealing Computational Graphs in Language Models — Emmanuel Ameisen, Jack Lindsey, Adam Pearce, Wes Gurnee, Nicholas Turner, Brian Chen, Craig Citro, et al. (Anthropic), 2025https://scholar.google.com/scholar?q=Circuit+Tracing%3A+Revealing+Computational+Graphs+in+Language+Models7. On the Biology of a Large Language Model — Jack Lindsey, Wes Gurnee, Emmanuel Ameisen, Brian Chen, Adam Pearce, et al. (Anthropic), 2025https://scholar.google.com/scholar?q=On+the+Biology+of+a+Large+Language+Model8. Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small — Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, Jacob Steinhardt, 2022https://scholar.google.com/scholar?q=Interpretability+in+the+Wild%3A+a+Circuit+for+Indirect+Object+Identification+in+GPT-2+small9. A Mathematical Framework for Transformer Circuits — Nelson Elhage, Neel Nanda, Catherine Olsson, et al. (Anthropic), 2021https://scholar.google.com/scholar?q=A+Mathematical+Framework+for+Transformer+Circuits10. RoFormer: Enhanced Transformer with Rotary Position Embedding — Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, Yunfeng Liu, 2021https://scholar.google.com/scholar?q=RoFormer%3A+Enhanced+Transformer+with+Rotary+Position+Embedding11. Quanta of model skills: The Quantization Model of Neural Scaling — Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark, 2023https://scholar.google.com/scholar?q=Quanta+of+model+skills%3A+The+Quantization+Model+of+Neural+Scaling12. Space and Time Representations in Language Models (Language Models Represent Space and Time) — Wes Gurnee, Max Tegmark, 2023https://scholar.google.com/scholar?q=Space+and+Time+Representations+in+Language+Models+%28Language+Models+Represent+Space+and+Time%2913. Grid cells / place cells literature: The Hippocampus as a Cognitive Map and Microstructure of a spatial map in the entorhinal cortex — John O'Keefe and Lynn Nadel (1978); Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, Edvard Moser (2005), 1978 / 2005https://scholar.google.com/scholar?q=Grid+cells+%2F+place+cells+literature%3A+The+Hippocampus+as+a+Cognitive+Map+and+Microstructure+of+a+spatial+map+in+the+entorhinal+cortex14. Not All Language Model Features Are One-Dimensionally Linear — Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, Max Tegmark, 2025https://scholar.google.com/scholar?q=Not+All+Language+Model+Features+Are+One-Dimensionally+Linear15. The Origins of Representation Manifolds in Large Language Models — Alexander Modell, Patrick Rubin-Delanchy, Nick Whiteley, 2025https://scholar.google.com/scholar?q=The+Origins+of+Representation+Manifolds+in+Large+Language+Models16. Language Models Use Trigonometry to Do Addition — Subhash Kantamneni, Max Tegmark, 2025https://scholar.google.com/scholar?q=Language+Models+Use+Trigonometry+to+Do+Addition17. Progress Measures for Grokking via Mechanistic Interpretability — Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, Jacob Steinhardt, 2023https://scholar.google.com/scholar?q=Progress+Measures+for+Grokking+via+Mechanistic+Interpretability18. Understanding Sparse Autoencoder Scaling in the Presence of Feature Manifolds — Eric J. Michaud, Liv Gorton, Tom McGrath, 2025https://scholar.google.com/scholar?q=Understanding+Sparse+Autoencoder+Scaling+in+the+Presence+of+Feature+Manifolds19. Curve Detectors / Curve Detector Manifolds in InceptionV1 — Nick Cammarata et al.; Liv Gorton, 2020 / 2024https://scholar.google.com/scholar?q=Curve+Detectors+%2F+Curve+Detector+Manifolds+in+InceptionV120. Tracing Attention Computation Through Feature Interactions (QK attributions) — Harish Kamath, Emmanuel Ameisen, Isaac Kauvar, et al., 2025https://scholar.google.com/scholar?q=Tracing+Attention+Computation+Through+Feature+Interactions+%28QK+attributions%2921. Language Models Need Inductive Biases to Count Inductively — Yingshan Chang, Yonatan Bisk, 2024https://scholar.google.com/scholar?q=Language+Models+Need+Inductive+Biases+to+Count+Inductively22. Length Representations in Large Language Models — Sangjun Moon et al., 2025https://scholar.google.com/scholar?q=Length+Representations+in+Large+Language+Models23. Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics — Yaniv Nikankin, Anja Reusch, Aaron Mueller, Yonatan Belinkov, 2024https://scholar.google.com/scholar?q=Arithmetic+Without+Algorithms%3A+Language+Models+Solve+Math+with+a+Bag+of+Heuristics24. Understanding In-Context Learning of Addition via Activation Subspaces — Xinyan Hu, Kayo Yin, Michael I. Jordan, Jacob Steinhardt, Lijie Chen, 2025https://scholar.google.com/scholar?q=Understanding+In-Context+Learning+of+Addition+via+Activation+Subspaces25. Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry — Sai Sumedh R. Hindupur, Ekdeep Singh Lubana, Thomas Fel, Demba Ba, 2025https://scholar.google.com/scholar?q=Projecting+Assumptions%3A+The+Duality+Between+Sparse+Autoencoders+and+Concept+Geometry26. Language Models Represent Space and Time — Wes Gurnee, Max Tegmark, 2024https://scholar.google.com/scholar?q=Language+Models+Represent+Space+and+Time27. The Quantization Model of Neural Scaling — Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark, 2023https://scholar.google.com/scholar?q=The+Quantization+Model+of+Neural+ScalingInteractive Visualization: Claude 3.5 Haiku Counts Characters on Curved Feature Manifolds17 min
- Harness Learning: Training Small Models to Rewrite Code Around Frozen SolversThis episode explores a paper on harness learning, in which a small proposer model is trained with reinforcement learning to rewrite the code wrapped around a frozen solver. The aim is for the system to adapt to unseen tasks with no weight updates. It defines a harness as the program that controls what the model sees, which tools it uses, and how outputs are validated and retried. It contrasts the approach with search-based optimizers such as DSPy, where the optimizer is just a prompted model. The discussion frames the method as loose meta-learning, with code edits standing in for gradient steps in the style of MAML and RL-squared. It also covers GRPO training on verifiable rewards and a Reasoning Gym testbed with held-out puzzle families. Listeners get a clear account of the paper's three questions: whether learned revisions generalize, whether repeated revision helps, and whether training on whole revision sequences beats single edits. The episode also sets up two critiques, including "Harness Updating Is Not Harness Benefit," as lenses for judging the evidence.Sources:1. Harness Learning Enables Generalizable Test-Time Adaptation — Alvin Zhang, Xuecheng Liu, Zixuan Wang, Fahim Tajwar, Daman Arora, Ruslan Salakhutdinov, Daniel Khashabi, Yuda Song, Andrea Zanette, 2026http://arxiv.org/abs/2609.357382. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines — Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, Christopher Potts, 2023 (ICLR 2024)https://scholar.google.com/scholar?q=DSPy%3A+Compiling+Declarative+Language+Model+Calls+into+Self-Improving+Pipelines3. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press, 2024https://scholar.google.com/scholar?q=SWE-agent%3A+Agent-Computer+Interfaces+Enable+Automated+Software+Engineering4. Automated Design of Agentic Systems (ADAS) — Shengran Hu, Cong Lu, Jeff Clune, 2024https://scholar.google.com/scholar?q=Automated+Design+of+Agentic+Systems+%28ADAS%295. Reflexion: Language Agents with Verbal Reinforcement Learning — Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao, 2023https://scholar.google.com/scholar?q=Reflexion%3A+Language+Agents+with+Verbal+Reinforcement+Learning6. Reasoning Gym: Reasoning Environments for Reinforcement Learning with Verifiable Rewards — Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, Andreas Köpf, 2025https://scholar.google.com/scholar?q=Reasoning+Gym%3A+Reasoning+Environments+for+Reinforcement+Learning+with+Verifiable+Rewards7. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks — Chelsea Finn, Pieter Abbeel, Sergey Levine, 2017https://scholar.google.com/scholar?q=Model-Agnostic+Meta-Learning+for+Fast+Adaptation+of+Deep+Networks8. RL²: Fast Reinforcement Learning via Slow Reinforcement Learning — Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, Pieter Abbeel, 2016https://scholar.google.com/scholar?q=RL%C2%B2%3A+Fast+Reinforcement+Learning+via+Slow+Reinforcement+Learning9. Learning to Learn by Gradient Descent by Gradient Descent — Marcin Andrychowicz, Misha Denil, Sergio Gómez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, Matthew Botvinick, Nando de Freitas, 2016https://scholar.google.com/scholar?q=Learning+to+Learn+by+Gradient+Descent+by+Gradient+Descent10. Mathematical Discoveries from Program Search with Large Language Models (FunSearch) — Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, Alhussein Fawzi, 2023 (Nature 2023/2024)https://scholar.google.com/scholar?q=Mathematical+Discoveries+from+Program+Search+with+Large+Language+Models+%28FunSearch%2911. Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories — Shao et al., 2026https://scholar.google.com/scholar?q=Harness-R1%3A+Learning+to+Edit+Executable+Runtime+Harnesses+from+Agent+Failure+Trajectories12. Rethinking the Evaluation of Harness Evolution for Agents — Wang et al., 2026https://scholar.google.com/scholar?q=Rethinking+the+Evaluation+of+Harness+Evolution+for+Agents13. Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents — Lin et al., 2026https://scholar.google.com/scholar?q=Harness+Updating+Is+Not+Harness+Benefit%3A+Disentangling+Evolution+Capabilities+in+Self-Evolving+LLM+Agents14. Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0 — Wang, Kattakinda, Feizi, 2026https://scholar.google.com/scholar?q=Do+Agent+Optimizers+Compound%3F+A+Continual-Learning+Evaluation+on+Terminal-Bench+2.015. Evo-Bench: Can Language Models Improve Agent Harness? — Huang et al., 2026https://scholar.google.com/scholar?q=Evo-Bench%3A+Can+Language+Models+Improve+Agent+Harness%3F16. Meta-Harness: End-to-End Optimization of Model Harnesses — Lee, Nair, Zhang, Lee, Khattab, Finn, 2026https://scholar.google.com/scholar?q=Meta-Harness%3A+End-to-End+Optimization+of+Model+Harnesses17. AutoHarness: improving LLM agents by automatically synthesizing a code harness — Lou, Lázaro-Gredilla, Dedieu, Wendelken, Lehrach, Murphy, 2026https://scholar.google.com/scholar?q=AutoHarness%3A+improving+LLM+agents+by+automatically+synthesizing+a+code+harness18. WHALE: A simple recipe for joint harness-weight optimization — Kim, Lee, Lee, Finn, Lee, 2026https://scholar.google.com/scholar?q=WHALE%3A+A+simple+recipe+for+joint+harness-weight+optimization19. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents — Zhang, Hu, Lu, Lange, Clune, 2025https://scholar.google.com/scholar?q=Darwin+G%C3%B6del+Machine%3A+Open-Ended+Evolution+of+Self-Improving+Agents20. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation — Zelikman, Lorch, Mackey, Kalai, 2024https://scholar.google.com/scholar?q=Self-Taught+Optimizer+%28STOP%29%3A+Recursively+Self-Improving+Code+Generation21. Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback — Jiang, Zhang, Wang, Andrews, Khashabi, 2025https://scholar.google.com/scholar?q=Feedback+Friction%3A+LLMs+Struggle+to+Fully+Incorporate+External+Feedback22. AdaptFlow: Adaptive Workflow Optimization via Meta-Learning — Zhu et al., 2025https://scholar.google.com/scholar?q=AdaptFlow%3A+Adaptive+Workflow+Optimization+via+Meta-Learning23. Training a Generally Curious Agent — Tajwar, Jiang, Thankaraj, Rahman, Kolter, Schneider, Salakhutdinov, 2025https://scholar.google.com/scholar?q=Training+a+Generally+Curious+Agent24. TTHE: Test-time harness evolution — Nie et al., 2026https://scholar.google.com/scholar?q=TTHE%3A+Test-time+harness+evolutionInteractive Visualization: Harness Learning: Training Small Models to Rewrite Code Around Frozen Solvers18 min
About AI Post Transformers
From the publisher's feed
AI-generated podcast where hosts Hal Turing and Dr. Ada Shannon discuss the latest research papers and reports in machine learning, AI systems, and optimization. Featuring honest critical analysis,…
proper citations, and nerdy humor.