Sign up to save your podcasts
Or
Share Opaque Serial Depth and Chain-of-Thought Limits
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Opaque Serial Depth and Chain-of-Thought Limits
This episode explores a 2026 paper on whether language models can keep doing long, step-by-step reasoning internally or eventually need to expose some of that reasoning in visible chain-of-thought tokens. It explains the paper’s core idea of opaque serial depth, or a model’s hidden reasoning horizon, and argues that this is a better safety-relevant measure than raw model size, parameter count, or informal layer counting. The discussion connects that metric to circuit complexity, fixed-precision computation, and transformer internals, showing why models can perform huge amounts of parallel work in one pass yet still face structural limits on long private sequential reasoning. Listeners would find it interesting because it sharpens a major AI safety question: whether monitoring visible reasoning can meaningfully constrain powerful models, and where that hope may break down.
Sources:
1. Opaque Serial Depth and Chain-of-Thought Limits
https://arxiv.org/pdf/2603.09786
2. Quantifying the Necessity of Chain of Thought through Opaque Serial Depth — Jonah Brown-Cohen, David Lindner, Rohin Shah, 2026
https://scholar.google.com/scholar?q=Quantifying+the+Necessity+of+Chain+of+Thought+through+Opaque+Serial+Depth
3. Chain of Thought Empowers Transformers to Solve Inherently Serial Problems — Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma, 2024
https://scholar.google.com/scholar?q=Chain+of+Thought+Empowers+Transformers+to+Solve+Inherently+Serial+Problems
4. Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety — Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Mark Chen, Rohin Shah, et al., 2025
https://scholar.google.com/scholar?q=Chain+of+Thought+Monitorability%3A+A+New+and+Fragile+Opportunity+for+AI+Safety
5. When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors — Scott Emmons, Erik Jenner, David K. Elson, Rif A. Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, Rohin Shah, 2025
https://scholar.google.com/scholar?q=When+Chain+of+Thought+is+Necessary%2C+Language+Models+Struggle+to+Evade+Monitors
6. Saturated Transformers are Constant-Depth Threshold Circuits — William Merrill, Ashish Sabharwal, Noah A. Smith, 2022
https://scholar.google.com/scholar?q=Saturated+Transformers+are+Constant-Depth+Threshold+Circuits
7. The Parallelism Tradeoff: Limitations of Log-Precision Transformers — William Merrill, Ashish Sabharwal, 2023
https://scholar.google.com/scholar?q=The+Parallelism+Tradeoff%3A+Limitations+of+Log-Precision+Transformers
8. The Expressive Power of Transformers with Chain of Thought — William Merrill, Ashish Sabharwal, 2024
https://scholar.google.com/scholar?q=The+Expressive+Power+of+Transformers+with+Chain+of+Thought
9. Theoretical Limitations of Self-Attention in Neural Sequence Models — Michael Hahn, 2020
https://scholar.google.com/scholar?q=Theoretical+Limitations+of+Self-Attention+in+Neural+Sequence+Models
10. Continuous Chain of Thought Enables Parallel Exploration and Reasoning — Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang, Hrayr Harutyunyan, Ankit Singh Rawat, Samet Oymak, 2025
https://scholar.google.com/scholar?q=Continuous+Chain+of+Thought+Enables+Parallel+Exploration+and+Reasoning
11. Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought — Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow, Atticus Geiger, Owen Lewis, Jack Merullo, 2026
https://scholar.google.com/scholar?q=Reasoning+Theater%3A+Disentangling+Model+Beliefs+from+Chain-of-Thought
12. Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps — Martin Tutek et al., 2025
https://scholar.google.com/scholar?q=Measuring+Chain+of+Thought+Faithfulness+by+Unlearning+Reasoning+Steps
13. Counterfactual Simulation Training for Chain-of-Thought Faithfulness — Peter Hase, Christopher Potts, 2026
https://scholar.google.com/scholar?q=Counterfactual+Simulation+Training+for+Chain-of-Thought+Faithfulness
14. Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models — Richard J. Young, 2026
https://scholar.google.com/scholar?q=Why+Models+Know+But+Don%27t+Say%3A+Chain-of-Thought+Faithfulness+Divergence+Between+Thinking+Tokens+and+Answers+in+Open-Weight+Reasoning+Models
15. Reasoning with Latent Thoughts: On the Power of Looped Transformers — Nikunj Saunshi et al., 2025
https://scholar.google.com/scholar?q=Reasoning+with+Latent+Thoughts%3A+On+the+Power+of+Looped+Transformers
16. Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding — Haolin Chen et al., 2024
https://scholar.google.com/scholar?q=Language+Models+are+Hidden+Reasoners%3A+Unlocking+Latent+Reasoning+Capabilities+via+Self-Rewarding
17. Efficient Post-Training Refinement of Latent Reasoning in Large Language Models — Xinyuan Wang et al., 2025
https://scholar.google.com/scholar?q=Efficient+Post-Training+Refinement+of+Latent+Reasoning+in+Large+Language+Models
18. AI Post Transformers: Reasoning Theater and Unfaithful Chain-of-Thought — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-05-reasoning-theater-and-unfaithful-chain-o-a4507e.mp3
19. AI Post Transformers: Generative Recursive Reasoning in Latent Space — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-21-generative-recursive-reasoning-in-latent-a9371d.mp3
20. AI Post Transformers: How Models Detect Hidden Activation Steering — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-08-how-models-detect-hidden-activation-stee-577f73.mp3
21. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp3
22. AI Post Transformers: Neural Computers as Learned Latent Runtimes — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-11-neural-computers-as-learned-latent-runti-9fa282.mp3
Interactive Visualization: Opaque Serial Depth and Chain-of-Thought Limits
View all episodes
By mcgrofThis episode explores a 2026 paper on whether language models can keep doing long, step-by-step reasoning internally or eventually need to expose some of that reasoning in visible chain-of-thought tokens. It explains the paper’s core idea of opaque serial depth, or a model’s hidden reasoning horizon, and argues that this is a better safety-relevant measure than raw model size, parameter count, or informal layer counting. The discussion connects that metric to circuit complexity, fixed-precision computation, and transformer internals, showing why models can perform huge amounts of parallel work in one pass yet still face structural limits on long private sequential reasoning. Listeners would find it interesting because it sharpens a major AI safety question: whether monitoring visible reasoning can meaningfully constrain powerful models, and where that hope may break down.
Sources:
1. Opaque Serial Depth and Chain-of-Thought Limits
https://arxiv.org/pdf/2603.09786
2. Quantifying the Necessity of Chain of Thought through Opaque Serial Depth — Jonah Brown-Cohen, David Lindner, Rohin Shah, 2026
https://scholar.google.com/scholar?q=Quantifying+the+Necessity+of+Chain+of+Thought+through+Opaque+Serial+Depth
3. Chain of Thought Empowers Transformers to Solve Inherently Serial Problems — Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma, 2024
https://scholar.google.com/scholar?q=Chain+of+Thought+Empowers+Transformers+to+Solve+Inherently+Serial+Problems
4. Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety — Tomek Korbak, Mikita Balesni, Elizabeth Barnes, Yoshua Bengio, Mark Chen, Rohin Shah, et al., 2025
https://scholar.google.com/scholar?q=Chain+of+Thought+Monitorability%3A+A+New+and+Fragile+Opportunity+for+AI+Safety
5. When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors — Scott Emmons, Erik Jenner, David K. Elson, Rif A. Saurous, Senthooran Rajamanoharan, Heng Chen, Irhum Shafkat, Rohin Shah, 2025
https://scholar.google.com/scholar?q=When+Chain+of+Thought+is+Necessary%2C+Language+Models+Struggle+to+Evade+Monitors
6. Saturated Transformers are Constant-Depth Threshold Circuits — William Merrill, Ashish Sabharwal, Noah A. Smith, 2022
https://scholar.google.com/scholar?q=Saturated+Transformers+are+Constant-Depth+Threshold+Circuits
7. The Parallelism Tradeoff: Limitations of Log-Precision Transformers — William Merrill, Ashish Sabharwal, 2023
https://scholar.google.com/scholar?q=The+Parallelism+Tradeoff%3A+Limitations+of+Log-Precision+Transformers
8. The Expressive Power of Transformers with Chain of Thought — William Merrill, Ashish Sabharwal, 2024
https://scholar.google.com/scholar?q=The+Expressive+Power+of+Transformers+with+Chain+of+Thought
9. Theoretical Limitations of Self-Attention in Neural Sequence Models — Michael Hahn, 2020
https://scholar.google.com/scholar?q=Theoretical+Limitations+of+Self-Attention+in+Neural+Sequence+Models
10. Continuous Chain of Thought Enables Parallel Exploration and Reasoning — Halil Alperen Gozeten, M. Emrullah Ildiz, Xuechen Zhang, Hrayr Harutyunyan, Ankit Singh Rawat, Samet Oymak, 2025
https://scholar.google.com/scholar?q=Continuous+Chain+of+Thought+Enables+Parallel+Exploration+and+Reasoning
11. Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought — Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow, Atticus Geiger, Owen Lewis, Jack Merullo, 2026
https://scholar.google.com/scholar?q=Reasoning+Theater%3A+Disentangling+Model+Beliefs+from+Chain-of-Thought
12. Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps — Martin Tutek et al., 2025
https://scholar.google.com/scholar?q=Measuring+Chain+of+Thought+Faithfulness+by+Unlearning+Reasoning+Steps
13. Counterfactual Simulation Training for Chain-of-Thought Faithfulness — Peter Hase, Christopher Potts, 2026
https://scholar.google.com/scholar?q=Counterfactual+Simulation+Training+for+Chain-of-Thought+Faithfulness
14. Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models — Richard J. Young, 2026
https://scholar.google.com/scholar?q=Why+Models+Know+But+Don%27t+Say%3A+Chain-of-Thought+Faithfulness+Divergence+Between+Thinking+Tokens+and+Answers+in+Open-Weight+Reasoning+Models
15. Reasoning with Latent Thoughts: On the Power of Looped Transformers — Nikunj Saunshi et al., 2025
https://scholar.google.com/scholar?q=Reasoning+with+Latent+Thoughts%3A+On+the+Power+of+Looped+Transformers
16. Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding — Haolin Chen et al., 2024
https://scholar.google.com/scholar?q=Language+Models+are+Hidden+Reasoners%3A+Unlocking+Latent+Reasoning+Capabilities+via+Self-Rewarding
17. Efficient Post-Training Refinement of Latent Reasoning in Large Language Models — Xinyuan Wang et al., 2025
https://scholar.google.com/scholar?q=Efficient+Post-Training+Refinement+of+Latent+Reasoning+in+Large+Language+Models
18. AI Post Transformers: Reasoning Theater and Unfaithful Chain-of-Thought — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-05-reasoning-theater-and-unfaithful-chain-o-a4507e.mp3
19. AI Post Transformers: Generative Recursive Reasoning in Latent Space — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-21-generative-recursive-reasoning-in-latent-a9371d.mp3
20. AI Post Transformers: How Models Detect Hidden Activation Steering — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-05-08-how-models-detect-hidden-activation-stee-577f73.mp3
21. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp3
22. AI Post Transformers: Neural Computers as Learned Latent Runtimes — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-11-neural-computers-as-learned-latent-runti-9fa282.mp3
Interactive Visualization: Opaque Serial Depth and Chain-of-Thought Limits