Sign up to save your podcasts
Or
Share Agentic Discovery for Test-Time Scaling
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Agentic Discovery for Test-Time Scaling
This episode explores a paper on test-time scaling that asks whether an LLM agent can automatically discover better inference-time control policies than the hand-built heuristics researchers usually rely on. It explains the core search framework in concrete terms: a controller decides when to branch, continue, probe, prune, or stop, balancing reasoning depth, breadth, and compute budget rather than simply generating more tokens. The discussion highlights the paper’s main technical argument that offline replay over logged reasoning traces, combined with a compact controller parameterization and detailed execution feedback, makes policy discovery cheap enough to be practical. Listeners would find it interesting because it connects abstract ideas about reasoning agents to a very specific claim: smarter inference may come not from larger models, but from better learned strategies for spending compute.
Sources:
1. LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling — Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, Heng Huang, 2026
http://arxiv.org/abs/2605.08083
2. Supervisory Control of a Class of Discrete Event Systems — Peter J. Ramadge, Walter M. Wonham, 1987
https://scholar.google.com/scholar?q=Supervisory+Control+of+a+Class+of+Discrete+Event+Systems
3. On the Synthesis of a Reactive Module — Amir Pnueli, Roni Rosner, 1989
https://scholar.google.com/scholar?q=On+the+Synthesis+of+a+Reactive+Module
4. Formal Methods for Control Synthesis: An Optimization Perspective — Calin Belta, Sadra Sadraddini, 2019
https://scholar.google.com/scholar?q=Formal+Methods+for+Control+Synthesis%3A+An+Optimization+Perspective
5. Formal Synthesis of Controllers for Safety-Critical Autonomous Systems: Developments and Challenges — Xiang Yin, Bingzhao Gao, Xiao Yu, 2024
https://scholar.google.com/scholar?q=Formal+Synthesis+of+Controllers+for+Safety-Critical+Autonomous+Systems%3A+Developments+and+Challenges
6. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems — Sergey Levine, Aviral Kumar, George Tucker, Justin Fu, 2020
https://scholar.google.com/scholar?q=Offline+Reinforcement+Learning%3A+Tutorial%2C+Review%2C+and+Perspectives+on+Open+Problems
7. Off-Policy Deep Reinforcement Learning without Exploration — Scott Fujimoto, David Meger, Doina Precup, 2019
https://scholar.google.com/scholar?q=Off-Policy+Deep+Reinforcement+Learning+without+Exploration
8. Conservative Q-Learning for Offline Reinforcement Learning — Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine, 2020
https://scholar.google.com/scholar?q=Conservative+Q-Learning+for+Offline+Reinforcement+Learning
9. D4RL: Datasets for Deep Data-Driven Reinforcement Learning — Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine, 2020
https://scholar.google.com/scholar?q=D4RL%3A+Datasets+for+Deep+Data-Driven+Reinforcement+Learning
10. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar, 2024
https://scholar.google.com/scholar?q=Scaling+LLM+Test-Time+Compute+Optimally+can+be+More+Effective+than+Scaling+Model+Parameters
11. Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing — Tong Zheng, Chengsong Huang, Runpeng Dai, Yun He, Rui Liu, Xin Ni, Huiwen Bao, Kaishen Wang, Hongtu Zhu, Jiaxin Huang, Furong Huang, Heng Huang, 2026
https://scholar.google.com/scholar?q=Parallel-Probe%3A+Towards+Efficient+Parallel+Thinking+via+2D+Probing
12. Scaling Test-time Compute for LLM Agents — King Zhu, Hanhao Li, Siwei Wu, Tianshun Xing, Dehua Ma, Xiangru Tang, Minghao Liu, Jian Yang, Jiaheng Liu, Yuchen Eleanor Jiang, Changwang Zhang, Chenghua Lin, Jun Wang, Ge Zhang, Wangchunshu Zhou, 2025
https://scholar.google.com/scholar?q=Scaling+Test-time+Compute+for+LLM+Agents
13. Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic — Yichuan Ma, Linyang Li, Yongkang Chen, Peiji Li, Xiaozhe Li, Qipeng Guo, Dahua Lin, Kai Chen, 2026
https://scholar.google.com/scholar?q=Timely+Machine%3A+Awareness+of+Time+Makes+Test-Time+Scaling+Agentic
14. Predicting and improving test-time scaling laws via reward tail-guided search — Muheng Li, Jian Qian, Wenlong Mou, 2026
https://scholar.google.com/scholar?q=Predicting+and+improving+test-time+scaling+laws+via+reward+tail-guided+search
15. TTRL: Test-Time Reinforcement Learning — Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, et al., 2025
https://scholar.google.com/scholar?q=TTRL%3A+Test-Time+Reinforcement+Learning
16. CTRLS: Chain-of-Thought Reasoning via Latent State-Transition — Junda Wu, Yuxin Xiong, Xintong Li, Zhengmian Hu, Tong Yu, Rui Wang, Xiang Chen, Jingbo Shang, Julian McAuley, 2025
https://scholar.google.com/scholar?q=CTRLS%3A+Chain-of-Thought+Reasoning+via+Latent+State-Transition
17. Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification — Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He, 2025
https://scholar.google.com/scholar?q=Reasoning+Models+Know+When+They%27re+Right%3A+Probing+Hidden+States+for+Self-Verification
18. Calibrated Reasoning: An Explanatory Verifier for Dynamic and Efficient Problem-Solving — Anisha Garg, Engin Tekin, Yash More, David Bick, Nishit Neema, Ganesh Venkatesh, 2025
https://scholar.google.com/scholar?q=Calibrated+Reasoning%3A+An+Explanatory+Verifier+for+Dynamic+and+Efficient+Problem-Solving
19. Contextual Drag: How Errors in the Context Affect LLM Reasoning — Yun Cheng, Xingyu Zhu, Haoyu Zhao, Sanjeev Arora, 2026
https://scholar.google.com/scholar?q=Contextual+Drag%3A+How+Errors+in+the+Context+Affect+LLM+Reasoning
20. Reinforcement Learning Teachers of Test Time Scaling — Edoardo Cetin, Tianyu Zhao, Yujin Tang, 2025
https://scholar.google.com/scholar?q=Reinforcement+Learning+Teachers+of+Test+Time+Scaling
21. AI Post Transformers: Test-time Scaling for Multi-Agent Collaborative Reasoning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-22-test-time-scaling-for-multi-agent-collab-082570.mp3
22. AI Post Transformers: ASI-Evolve for Data, Architectures, and RL — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-asi-evolve-for-data-architectures-and-rl-197b2b.mp3
23. AI Post Transformers: Generalist Reward Modeling with Inference-Time Scaling — Hal Turing & Dr. Ada Shannon, Tue,
https://podcast.do-not-panic.com/episodes/generalist-reward-modeling-with-inference-time-scaling/
24. AI Post Transformers: Reasoning Theater and Unfaithful Chain-of-Thought — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-05-reasoning-theater-and-unfaithful-chain-o-a4507e.mp3
25. AI Post Transformers: SGLang for Faster Structured LLM Programs — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-06-sglang-for-faster-structured-llm-program-c59f1c.mp3
Interactive Visualization: Agentic Discovery for Test-Time Scaling
View all episodes
By mcgrofThis episode explores a paper on test-time scaling that asks whether an LLM agent can automatically discover better inference-time control policies than the hand-built heuristics researchers usually rely on. It explains the core search framework in concrete terms: a controller decides when to branch, continue, probe, prune, or stop, balancing reasoning depth, breadth, and compute budget rather than simply generating more tokens. The discussion highlights the paper’s main technical argument that offline replay over logged reasoning traces, combined with a compact controller parameterization and detailed execution feedback, makes policy discovery cheap enough to be practical. Listeners would find it interesting because it connects abstract ideas about reasoning agents to a very specific claim: smarter inference may come not from larger models, but from better learned strategies for spending compute.
Sources:
1. LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling — Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, Heng Huang, 2026
http://arxiv.org/abs/2605.08083
2. Supervisory Control of a Class of Discrete Event Systems — Peter J. Ramadge, Walter M. Wonham, 1987
https://scholar.google.com/scholar?q=Supervisory+Control+of+a+Class+of+Discrete+Event+Systems
3. On the Synthesis of a Reactive Module — Amir Pnueli, Roni Rosner, 1989
https://scholar.google.com/scholar?q=On+the+Synthesis+of+a+Reactive+Module
4. Formal Methods for Control Synthesis: An Optimization Perspective — Calin Belta, Sadra Sadraddini, 2019
https://scholar.google.com/scholar?q=Formal+Methods+for+Control+Synthesis%3A+An+Optimization+Perspective
5. Formal Synthesis of Controllers for Safety-Critical Autonomous Systems: Developments and Challenges — Xiang Yin, Bingzhao Gao, Xiao Yu, 2024
https://scholar.google.com/scholar?q=Formal+Synthesis+of+Controllers+for+Safety-Critical+Autonomous+Systems%3A+Developments+and+Challenges
6. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems — Sergey Levine, Aviral Kumar, George Tucker, Justin Fu, 2020
https://scholar.google.com/scholar?q=Offline+Reinforcement+Learning%3A+Tutorial%2C+Review%2C+and+Perspectives+on+Open+Problems
7. Off-Policy Deep Reinforcement Learning without Exploration — Scott Fujimoto, David Meger, Doina Precup, 2019
https://scholar.google.com/scholar?q=Off-Policy+Deep+Reinforcement+Learning+without+Exploration
8. Conservative Q-Learning for Offline Reinforcement Learning — Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine, 2020
https://scholar.google.com/scholar?q=Conservative+Q-Learning+for+Offline+Reinforcement+Learning
9. D4RL: Datasets for Deep Data-Driven Reinforcement Learning — Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine, 2020
https://scholar.google.com/scholar?q=D4RL%3A+Datasets+for+Deep+Data-Driven+Reinforcement+Learning
10. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar, 2024
https://scholar.google.com/scholar?q=Scaling+LLM+Test-Time+Compute+Optimally+can+be+More+Effective+than+Scaling+Model+Parameters
11. Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probing — Tong Zheng, Chengsong Huang, Runpeng Dai, Yun He, Rui Liu, Xin Ni, Huiwen Bao, Kaishen Wang, Hongtu Zhu, Jiaxin Huang, Furong Huang, Heng Huang, 2026
https://scholar.google.com/scholar?q=Parallel-Probe%3A+Towards+Efficient+Parallel+Thinking+via+2D+Probing
12. Scaling Test-time Compute for LLM Agents — King Zhu, Hanhao Li, Siwei Wu, Tianshun Xing, Dehua Ma, Xiangru Tang, Minghao Liu, Jian Yang, Jiaheng Liu, Yuchen Eleanor Jiang, Changwang Zhang, Chenghua Lin, Jun Wang, Ge Zhang, Wangchunshu Zhou, 2025
https://scholar.google.com/scholar?q=Scaling+Test-time+Compute+for+LLM+Agents
13. Timely Machine: Awareness of Time Makes Test-Time Scaling Agentic — Yichuan Ma, Linyang Li, Yongkang Chen, Peiji Li, Xiaozhe Li, Qipeng Guo, Dahua Lin, Kai Chen, 2026
https://scholar.google.com/scholar?q=Timely+Machine%3A+Awareness+of+Time+Makes+Test-Time+Scaling+Agentic
14. Predicting and improving test-time scaling laws via reward tail-guided search — Muheng Li, Jian Qian, Wenlong Mou, 2026
https://scholar.google.com/scholar?q=Predicting+and+improving+test-time+scaling+laws+via+reward+tail-guided+search
15. TTRL: Test-Time Reinforcement Learning — Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, et al., 2025
https://scholar.google.com/scholar?q=TTRL%3A+Test-Time+Reinforcement+Learning
16. CTRLS: Chain-of-Thought Reasoning via Latent State-Transition — Junda Wu, Yuxin Xiong, Xintong Li, Zhengmian Hu, Tong Yu, Rui Wang, Xiang Chen, Jingbo Shang, Julian McAuley, 2025
https://scholar.google.com/scholar?q=CTRLS%3A+Chain-of-Thought+Reasoning+via+Latent+State-Transition
17. Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification — Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, He He, 2025
https://scholar.google.com/scholar?q=Reasoning+Models+Know+When+They%27re+Right%3A+Probing+Hidden+States+for+Self-Verification
18. Calibrated Reasoning: An Explanatory Verifier for Dynamic and Efficient Problem-Solving — Anisha Garg, Engin Tekin, Yash More, David Bick, Nishit Neema, Ganesh Venkatesh, 2025
https://scholar.google.com/scholar?q=Calibrated+Reasoning%3A+An+Explanatory+Verifier+for+Dynamic+and+Efficient+Problem-Solving
19. Contextual Drag: How Errors in the Context Affect LLM Reasoning — Yun Cheng, Xingyu Zhu, Haoyu Zhao, Sanjeev Arora, 2026
https://scholar.google.com/scholar?q=Contextual+Drag%3A+How+Errors+in+the+Context+Affect+LLM+Reasoning
20. Reinforcement Learning Teachers of Test Time Scaling — Edoardo Cetin, Tianyu Zhao, Yujin Tang, 2025
https://scholar.google.com/scholar?q=Reinforcement+Learning+Teachers+of+Test+Time+Scaling
21. AI Post Transformers: Test-time Scaling for Multi-Agent Collaborative Reasoning — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-22-test-time-scaling-for-multi-agent-collab-082570.mp3
22. AI Post Transformers: ASI-Evolve for Data, Architectures, and RL — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-asi-evolve-for-data-architectures-and-rl-197b2b.mp3
23. AI Post Transformers: Generalist Reward Modeling with Inference-Time Scaling — Hal Turing & Dr. Ada Shannon, Tue,
https://podcast.do-not-panic.com/episodes/generalist-reward-modeling-with-inference-time-scaling/
24. AI Post Transformers: Reasoning Theater and Unfaithful Chain-of-Thought — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-05-reasoning-theater-and-unfaithful-chain-o-a4507e.mp3
25. AI Post Transformers: SGLang for Faster Structured LLM Programs — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-06-sglang-for-faster-structured-llm-program-c59f1c.mp3
Interactive Visualization: Agentic Discovery for Test-Time Scaling