Sign up to save your podcasts
Or
Share Teaching Language Models to Verbalize Uncertainty
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Teaching Language Models to Verbalize Uncertainty
This episode explores whether language models can express uncertainty in natural language in a way that is actually calibrated and useful, rather than merely sounding cautious or confident. It focuses on the paper’s central distinction between token uncertainty and epistemic uncertainty, arguing that next-token probabilities are a poor proxy for whether a model truly knows an answer. The discussion situates this idea alongside earlier work on Bayesian approximations, dataset shift, and newer hallucination-detection methods such as semantic entropy, all pointing to the same challenge: uncertainty should attach to claims, not just strings. A listener would find it interesting because it connects a seemingly simple design choice, having models state confidence in words, to the much larger problem of building AI systems that can warn users when they are likely guessing.
Sources:
1. Teaching Models to Express Their Uncertainty in Words — Stephanie Lin, Jacob Hilton, Owain Evans, 2022
http://arxiv.org/abs/2205.14334
2. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods — Eyke Hullermeier, Willem Waegeman, 2021
https://scholar.google.com/scholar?q=Aleatoric+and+Epistemic+Uncertainty+in+Machine+Learning%3A+An+Introduction+to+Concepts+and+Methods
3. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning — Yarin Gal, Zoubin Ghahramani, 2016
https://scholar.google.com/scholar?q=Dropout+as+a+Bayesian+Approximation%3A+Representing+Model+Uncertainty+in+Deep+Learning
4. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift — Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, Jasper Snoek, 2019
https://scholar.google.com/scholar?q=Can+You+Trust+Your+Model%27s+Uncertainty%3F+Evaluating+Predictive+Uncertainty+Under+Dataset+Shift
5. Detecting Hallucinations in Large Language Models Using Semantic Entropy — Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and colleagues including Yarin Gal, 2024
https://scholar.google.com/scholar?q=Detecting+Hallucinations+in+Large+Language+Models+Using+Semantic+Entropy
6. Language Models (Mostly) Know What They Know — Katherine Kadavath, Roberta Raileanu, Akul Arora, et al., 2022
https://scholar.google.com/scholar?q=Language+Models+%28Mostly%29+Know+What+They+Know
7. On Calibration of Modern Neural Networks — Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, 2017
https://scholar.google.com/scholar?q=On+Calibration+of+Modern+Neural+Networks
8. Can Language Models Be Too Honest? The Curious Case of Adaptive Honesty — Owain Evans, Jacob Hilton, Lukas Heim, et al., 2021
https://scholar.google.com/scholar?q=Can+Language+Models+Be+Too+Honest%3F+The+Curious+Case+of+Adaptive+Honesty
9. TruthfulQA: Measuring How Models Mimic Human Falsehoods — Stephanie Lin, Jacob Hilton, Owain Evans, 2021
https://scholar.google.com/scholar?q=TruthfulQA%3A+Measuring+How+Models+Mimic+Human+Falsehoods
10. Measuring Calibration in Deep Learning — Mahdi Pakdaman Naeini, Gregory Cooper, Milos Hauskrecht, 2015
https://scholar.google.com/scholar?q=Measuring+Calibration+in+Deep+Learning
11. Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation — Sebastian Kuhn, Yarin Gal, et al., 2023
https://scholar.google.com/scholar?q=Semantic+Uncertainty%3A+Linguistic+Invariances+for+Uncertainty+Estimation+in+Natural+Language+Generation
12. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models — Potsawee Manakul, Adian Liusie, Mark Gales, 2023
https://scholar.google.com/scholar?q=SelfCheckGPT%3A+Zero-Resource+Black-Box+Hallucination+Detection+for+Generative+Large+Language+Models
13. Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence? — Shiyu Ni, Keping Bi, Lulu Yu, Jiafeng Guo, 2024
https://scholar.google.com/scholar?q=Are+Large+Language+Models+More+Honest+in+Their+Probabilistic+or+Verbalized+Confidence%3F
14. Calibrating Verbalized Probabilities for Large Language Models — Cheng Wang, Gyuri Szarvas, Georges Balazs, Pavel Danchenko, Patrick Ernst, 2024
https://scholar.google.com/scholar?q=Calibrating+Verbalized+Probabilities+for+Large+Language+Models
15. Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning — Jiaqi Li, Yixuan Tang, Yi Yang, 2025
https://scholar.google.com/scholar?q=Know+the+Unknown%3A+An+Uncertainty-Sensitive+Method+for+LLM+Instruction+Tuning
16. From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered — Siddartha Devic, Tejas Srinivasan, Jesse Thomason, Willie Neiswanger, Vatsal Sharan, 2025
https://scholar.google.com/scholar?q=From+Calibration+to+Collaboration%3A+LLM+Uncertainty+Quantification+Should+Be+More+Human-Centered
17. Closing the Confidence-Faithfulness Gap in Large Language Models — Miranda Muqing Miao, Lyle Ungar, 2026
https://scholar.google.com/scholar?q=Closing+the+Confidence-Faithfulness+Gap+in+Large+Language+Models
18. AI Post Transformers: CLUE: Hidden-State Clustering for Non-parametric Verification — Hal Turing & Dr. Ada Shannon, 2025
https://podcast.do-not-panic.com/episodes/clue-hidden-state-clustering-for-non-parametric-verification/
19. AI Post Transformers: Neural Chameleons and Evading Activation Monitors — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-14-neural-chameleons-and-evading-activation-bc470e.mp3
20. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp3
Interactive Visualization: Teaching Language Models to Verbalize Uncertainty
View all episodes
By mcgrofThis episode explores whether language models can express uncertainty in natural language in a way that is actually calibrated and useful, rather than merely sounding cautious or confident. It focuses on the paper’s central distinction between token uncertainty and epistemic uncertainty, arguing that next-token probabilities are a poor proxy for whether a model truly knows an answer. The discussion situates this idea alongside earlier work on Bayesian approximations, dataset shift, and newer hallucination-detection methods such as semantic entropy, all pointing to the same challenge: uncertainty should attach to claims, not just strings. A listener would find it interesting because it connects a seemingly simple design choice, having models state confidence in words, to the much larger problem of building AI systems that can warn users when they are likely guessing.
Sources:
1. Teaching Models to Express Their Uncertainty in Words — Stephanie Lin, Jacob Hilton, Owain Evans, 2022
http://arxiv.org/abs/2205.14334
2. Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods — Eyke Hullermeier, Willem Waegeman, 2021
https://scholar.google.com/scholar?q=Aleatoric+and+Epistemic+Uncertainty+in+Machine+Learning%3A+An+Introduction+to+Concepts+and+Methods
3. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning — Yarin Gal, Zoubin Ghahramani, 2016
https://scholar.google.com/scholar?q=Dropout+as+a+Bayesian+Approximation%3A+Representing+Model+Uncertainty+in+Deep+Learning
4. Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift — Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, Jasper Snoek, 2019
https://scholar.google.com/scholar?q=Can+You+Trust+Your+Model%27s+Uncertainty%3F+Evaluating+Predictive+Uncertainty+Under+Dataset+Shift
5. Detecting Hallucinations in Large Language Models Using Semantic Entropy — Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and colleagues including Yarin Gal, 2024
https://scholar.google.com/scholar?q=Detecting+Hallucinations+in+Large+Language+Models+Using+Semantic+Entropy
6. Language Models (Mostly) Know What They Know — Katherine Kadavath, Roberta Raileanu, Akul Arora, et al., 2022
https://scholar.google.com/scholar?q=Language+Models+%28Mostly%29+Know+What+They+Know
7. On Calibration of Modern Neural Networks — Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, 2017
https://scholar.google.com/scholar?q=On+Calibration+of+Modern+Neural+Networks
8. Can Language Models Be Too Honest? The Curious Case of Adaptive Honesty — Owain Evans, Jacob Hilton, Lukas Heim, et al., 2021
https://scholar.google.com/scholar?q=Can+Language+Models+Be+Too+Honest%3F+The+Curious+Case+of+Adaptive+Honesty
9. TruthfulQA: Measuring How Models Mimic Human Falsehoods — Stephanie Lin, Jacob Hilton, Owain Evans, 2021
https://scholar.google.com/scholar?q=TruthfulQA%3A+Measuring+How+Models+Mimic+Human+Falsehoods
10. Measuring Calibration in Deep Learning — Mahdi Pakdaman Naeini, Gregory Cooper, Milos Hauskrecht, 2015
https://scholar.google.com/scholar?q=Measuring+Calibration+in+Deep+Learning
11. Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation — Sebastian Kuhn, Yarin Gal, et al., 2023
https://scholar.google.com/scholar?q=Semantic+Uncertainty%3A+Linguistic+Invariances+for+Uncertainty+Estimation+in+Natural+Language+Generation
12. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models — Potsawee Manakul, Adian Liusie, Mark Gales, 2023
https://scholar.google.com/scholar?q=SelfCheckGPT%3A+Zero-Resource+Black-Box+Hallucination+Detection+for+Generative+Large+Language+Models
13. Are Large Language Models More Honest in Their Probabilistic or Verbalized Confidence? — Shiyu Ni, Keping Bi, Lulu Yu, Jiafeng Guo, 2024
https://scholar.google.com/scholar?q=Are+Large+Language+Models+More+Honest+in+Their+Probabilistic+or+Verbalized+Confidence%3F
14. Calibrating Verbalized Probabilities for Large Language Models — Cheng Wang, Gyuri Szarvas, Georges Balazs, Pavel Danchenko, Patrick Ernst, 2024
https://scholar.google.com/scholar?q=Calibrating+Verbalized+Probabilities+for+Large+Language+Models
15. Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning — Jiaqi Li, Yixuan Tang, Yi Yang, 2025
https://scholar.google.com/scholar?q=Know+the+Unknown%3A+An+Uncertainty-Sensitive+Method+for+LLM+Instruction+Tuning
16. From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered — Siddartha Devic, Tejas Srinivasan, Jesse Thomason, Willie Neiswanger, Vatsal Sharan, 2025
https://scholar.google.com/scholar?q=From+Calibration+to+Collaboration%3A+LLM+Uncertainty+Quantification+Should+Be+More+Human-Centered
17. Closing the Confidence-Faithfulness Gap in Large Language Models — Miranda Muqing Miao, Lyle Ungar, 2026
https://scholar.google.com/scholar?q=Closing+the+Confidence-Faithfulness+Gap+in+Large+Language+Models
18. AI Post Transformers: CLUE: Hidden-State Clustering for Non-parametric Verification — Hal Turing & Dr. Ada Shannon, 2025
https://podcast.do-not-panic.com/episodes/clue-hidden-state-clustering-for-non-parametric-verification/
19. AI Post Transformers: Neural Chameleons and Evading Activation Monitors — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-14-neural-chameleons-and-evading-activation-bc470e.mp3
20. AI Post Transformers: Latent Space as a New Computational Paradigm — Hal Turing & Dr. Ada Shannon, 2026
https://podcast.do-not-panic.com/episodes/2026-04-05-latent-space-as-a-new-computational-para-810f39.mp3
Interactive Visualization: Teaching Language Models to Verbalize Uncertainty