This episode explores Program-as-Weights, a paper that asks whether a natural-language description of a fuzzy software task can be compiled once into a small reusable neural artifact instead of sending every request to a larger remote model. It explains the paper’s architecture in concrete terms: a 4B pseudo-compiler rewrites the task and generates example I/O pairs, a trained 4B compiler plus LoRA mapper turns that specification into adapter weights, and a frozen Qwen3-0.6B interpreter runs the task locally on new inputs. The discussion focuses on why this matters for real problems like log triage, malformed JSON repair, and intent-based reranking, highlighting the promised gains in cost, latency, privacy, offline use, and reproducibility. It also digs into the paper’s broader claim, debating whether this is truly a new programming paradigm or a sharp repackaging of PEFT and LoRA-based adaptation, which makes the episode interesting for listeners thinking about practical deployment rather than just model benchmarks.

Sources:

1. Program-as-Weights: A Programming Paradigm for Fuzzy Functions — Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng, 2026

http://arxiv.org/abs/2607.02512

2. HyperNetworks — David Ha, Andrew Dai, Quoc V. Le, 2016

https://scholar.google.com/scholar?q=HyperNetworks

3. LoRA: Low-Rank Adaptation of Large Language Models — Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, et al., 2021

https://scholar.google.com/scholar?q=LoRA%3A+Low-Rank+Adaptation+of+Large+Language+Models

4. Learning to Compile Programs to Neural Networks — Logan Weber, Jesse Michel, Alex Renda, Michael Carbin, 2024

https://scholar.google.com/scholar?q=Learning+to+Compile+Programs+to+Neural+Networks

5. Text-to-LoRA: Instant Transformer Adaption — Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange, 2025

https://scholar.google.com/scholar?q=Text-to-LoRA%3A+Instant+Transformer+Adaption

6. SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass — Y. Liu, X. Wang, Y. Mao, Y. Gelberg, H. Maron, and M. Zhang, 2026

https://scholar.google.com/scholar?q=SHINE%3A+A+Scalable+In-Context+Hypernetwork+for+Mapping+Context+to+LoRA+in+a+Single+Pass

7. Doc-to-LoRA: Learning to Instantly Internalize Contexts — R. Charakorn, E. Cetin, S. Uesaka, and R. T. Lange, 2026

https://scholar.google.com/scholar?q=Doc-to-LoRA%3A+Learning+to+Instantly+Internalize+Contexts

8. Latent Context Compilation: Distilling Long Context into Compact Portable Memory — Z. Li, Y. Zhou, and Q. Xu, 2026

https://scholar.google.com/scholar?q=Latent+Context+Compilation%3A+Distilling+Long+Context+into+Compact+Portable+Memory

9. The Alchemist: Automated Labeling 500x Cheaper than LLM Data Annotators — T. Huang, C. Cao, V. Bhargava, and F. Sala, 2024

https://scholar.google.com/scholar?q=The+Alchemist%3A+Automated+Labeling+500x+Cheaper+than+LLM+Data+Annotators

10. Learning to Route for Dynamic Adapter Composition in Continual Learning with Language Models — Vladimir Araujo, Marie-Francine Moens, Tinne Tuytelaars, 2024

https://arxiv.org/abs/2408.09053

11. X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design — Eric L. Buehler, Markus J. Buehler, 2024

https://arxiv.org/abs/2402.07148

12. CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning — Yu Feng, Zhen Tian, Yifan Zhu, Zongfu Han, Haoran Luo, Guangwei Zhang, Meina Song, 2024

https://arxiv.org/abs/2407.21043

13. Gradient Projection For Continual Parameter-Efficient Tuning — Jingyang Qiao, Zhizhong Zhang, Xin Tan, Yanyun Qu, Wensheng Zhang, Zhi Han, Yuan Xie, 2024

https://arxiv.org/abs/2405.13383

14. Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading — Minrui Xu, Dusit Niyato, Christopher G. Brinton, 2025

https://arxiv.org/abs/2501.14205

15. AI Post Transformers: SGLang for Faster Structured LLM Programs — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-05-06-sglang-for-faster-structured-llm-program-c59f1c.mp3

16. AI Post Transformers: OpenSkill for Open-World Self-Evolution in LLM Agents — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-08-openskill-for-open-world-self-evolution-19762a.mp3

17. AI Post Transformers: Learning Facts at Scale with Active Reading — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-25-learning-facts-at-scale-with-active-read-161bea.mp3

18. AI Post Transformers: Fine-Tuning LLMs for Human Behavior Prediction — Hal Turing & Dr. Ada Shannon, 2026

https://podcast.do-not-panic.com/episodes/2026-06-21-fine-tuning-llms-for-human-behavior-pred-c79163.mp3