REVIEW 2 major objections 7 minor 1 cited by
Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
T0 review · 2 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper argues that the aleatoric/epistemic uncertainty dichotomy breaks down in interactive LLM agent settings and should be replaced by underspecification detection, interactive learning, and verbalized output uncertainties.
desk verdict A defensible, useful position piece: the aleatoric/epistemic dichotomy is indeed a poor fit for interactive LLM agents, but the authors' strongest claim hinges on a modeler-relative semantics they don't fully justify. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the conversation itself viewed as a moving input point in a fixed input space. In a traditional setup the input $x$ is fixed and the uncertainty about the output $y$ is split into aleatoric and epistemic parts; in a chatbot, the input is the concatenated conversation string $x_t$, which changes with each turn, so the same question $y$ is predicted from different information at different times. Because the agent can choose to ask follow-up questions, any current uncertainty can be classified as reducible (epistemic) if the agent will keep asking, or irreducible (aleatoric) if it stops — yielding the paper's central claim that the labels flip and therefore lose objective meaning. The proposed alternative is a three-part structure: task- and context-underspecification uncertainties, interactive learning as a mechanism for reducing them, and output uncertainties as the communicative layer that replaces scalar scores with sets of possibilities, explanations, and verbal or prosodic expressions.
What would settle it
Build a controlled chat where the user's question is ambiguous but all further information is unavailable (the user answers "I don't know" to any follow-up). If an uncertainty estimator labels the state epistemic while follow-up questions are permitted and aleatoric as soon as the agent is forced to stop, while the available information is identical, the paper's agent-relative labeling is demonstrated. If the label instead stays stable, or tracks only the information actually supplied, the central breakdown claim is refuted.
Extended reading notes
Core claim
The paper's central claim is that the aleatoric/epistemic dichotomy, as commonly used in machine learning, is inapplicable to modern LLM agent interactions. To support this, it collects schools of thought that define epistemic uncertainty as the number of plausible models, as disagreement among models, or as distance from training data, and shows these give opposite answers to a simple two-Dirac belief example. It collects analogous conflicts for aleatoric uncertainty, where one school counts all residual risk of the Bayes-optimal model inside a fixed model class as aleatoric while another counts only the variance intrinsic to the data. It then argues that in a conversation the same uncertainty changes label depending on whether the agent continues to ask follow-up questions, so reducibility is a subjective modeling choice rather than a property of the world. The constructive part of the position is that research should develop three complementary notions — underspecification uncertainty, interactive learning, and output uncertainties — and that numerical probabilities remain useful mainly for machine-to-machine interfaces.
Load-bearing premise
The argument that epistemic uncertainty becomes aleatoric the moment an agent stops asking questions assumes that reducibility is judged relative to the agent's chosen stopping point rather than relative to the world; if more information exists that would settle the answer, the claimed label flip dissolves.
Editorial extensions
If this is right
- Uncertainty quantification for LLM agents should be re-oriented from decomposing total uncertainty into aleatoric and epistemic parts toward detecting when a user's request is underspecified.
- Agents should be built and evaluated on their ability to ask helpful follow-up questions, with an explicit trade-off between informativeness and user patience.
- Uncertainty communication research should develop benchmarks and metrics for textual and speech-based expressions of uncertainty, including the coverage of listed possibilities.
- Numerical uncertainty scores remain appropriate for automated consumers of agent output, such as abstention thresholds in pipelines, but are insufficient for human users.
- Existing evidence that ambiguity appears in a large fraction of real user questions (for example, 56% of one open-domain QA benchmark) becomes a first target for the proposed underspecification direction.
Reading between the lines
- One extension the paper leaves implicit: if reducibility labels are agent-relative, then the right formal object is not a label but a decision — the agent's stopping rule — and uncertainty could be expressed as the expected value of asking one more question, connecting to value-of-information theory.
- A testable extension is to measure whether existing aleatoric and epistemic estimators, which the paper cites as almost perfectly correlated in static settings, predict the usefulness of follow-up questions in dialogue; if they do, the proposed research directions can reuse existing tools rather than requiring new ones.
- The same reasoning applies beyond text chat to any interactive agent that can acquire information at a cost, such as robots or diagnostic systems, so the three proposed uncertainty types could serve as a general template for embodied uncertainty quantification.
- A concrete experiment to sharpen the position would show users the same uncertainty expressed as a calibrated number versus as a short list of possibilities with reasons, then measure decision quality and trust; the paper predicts the verbal form will support human decision making better, but it does not itself run this study.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This position paper argues that the traditional aleatoric/epistemic uncertainty dichotomy is insufficient for LLM agents in open, interactive, multi-turn settings. It reviews conflicting definitions of the two uncertainty types (Sections 2.1–2.2), presents empirical and theoretical evidence that their estimators are entangled (Section 2.3), and argues that in chatbot conversations the epistemic/aleatoric label can flip depending on whether the agent continues to ask follow-up questions (Section 2.4). Based on these arguments, the paper proposes three research directions: underspecification uncertainties, interactive learning, and output uncertainties (Section 3), while acknowledging in Section 4 that traditional uncertainties and numerical scores remain useful in some contexts, such as automated downstream systems.
Significance. If the position is accepted, it would shift UQ research priorities for LLM agents away from scalar aleatoric/epistemic decomposition toward interactive clarification, underspecification detection, and verbalized output uncertainty. The paper provides a valuable synthesis of a fragmented literature, including a useful table (Table 1) of conflicting schools of thought and an explicit discussion of counter-positions. The three proposed research directions are concrete and lead to falsifiable benchmarks (e.g., ambiguity detection accuracy, helpfulness of follow-up questions). However, the paper's strongest claim—that the dichotomy is 'inapplicable' to LLM agents—rests substantially on the Section 2.4 argument, which is not fully justified.
major comments (2)
- [2.4] The epistemic-to-aleatoric flip argument relies on an agent-relative notion of reducibility, following Der Kiureghian and Ditlevsen (2009), but the paper does not independently justify why this semantics is appropriate for LLM-agent uncertainty. Under the standard feature-relative view (aleatoric uncertainty is H[Y | X=x_t], and reducibility is evaluated by whether some additional observation Z exists with H[Y | X=x_t, Z] < H[Y | X=x_t]), the agent's decision to stop asking questions merely changes the conditioning point; it does not change the label. Without such a justification, the claim in Section 1 that the dichotomy is 'inapplicable' is not established by Section 2.4. The paper should either defend the agent-relative semantics explicitly or weaken the claim to 'ambiguous relative to the modeler's chosen stopping point.'
- [2.1–2.3] The paper moves from 'different schools of thought define these terms differently' and 'estimators are correlated' to the conclusion that aleatoric and epistemic uncertainty 'cannot be split exactly.' The two-Diracs example in Section 2.1 shows that different axiomatic systems yield different numerical values, which is a lack of consensus rather than a logical contradiction. The empirical correlation results (Mucsányi et al. 2024; de Jong et al. 2024) show that current estimates are entangled, but they do not establish that the underlying quantities are inherently inseparable; the paper acknowledges the approximation-error objection in Section 2.3 but does not fully rebut it. The theoretical argument from Gruber et al. (2023) concerns a specific linear-model prediction interval and does not generalize to all settings. These considerations support the need for careful definitions and enrichment, but not the strong claim that the dichotomy is inapplicable.
minor comments (7)
- [Page 1] The 'Apple Confidential–Internal Use Only' header appears to be a version-control artifact and should be removed.
- [3.2] Typo: 'resemblence' should be 'resemblance'.
- [3.3] Typo: 'possiblities' should be 'possibilities'.
- [2.4] The notation 'P(Y)' is used without defining Y as the random variable for the answer token or string; please introduce it explicitly.
- [3.1] Equation (2) writes an integral over T, which appears to assume a continuous task space; if T is intended to be discrete (or a mixture), the equation should be written accordingly.
- [References] There are two entries for Belém et al. (2024) with slightly different spellings (Belem vs. Belém) and identical content; these should be consolidated.
- [Figure 2] The vertical axis label 'Q(θ)' is not explained as a probability density; the caption should state that Q is a second-order distribution.
Circularity Check
No circularity found: the position is an externally grounded literature synthesis, not a derivation that reduces to its own inputs.
full rationale
The paper advances a position rather than a fitted derivation, and none of its load-bearing steps reduce to its own inputs by construction. The definitional-conflict examples in Sections 2.1-2.3 are presented as observed disagreements between named external schools of thought (e.g., Houlsby et al. 2011, Wimmer et al. 2023, Mukhoti et al. 2023) and are not derived from the paper's conclusion. The empirical disentanglement claim in Section 2.3 does cite the authors' own Mucsányi et al. (2024) benchmark, but that benchmark is an externally published, reproduced result and is corroborated by independent citations (de Jong et al. 2024; Bouvier et al. 2022; Gruber et al. 2023; Hüllermeier et al. 2022; Valdenegro-Toro & Mori 2022), so the self-citation is not load-bearing. The other self-citations (Kirchhof et al. 2023, 2025; Kasneci et al. 2023) appear only as examples or pointers and do not carry the argument. Section 2.4's epistemic-to-aleatoric flip is explicitly framed as 'one can argue... however, one can equally argue' and is attributed to the external modeler-relative convention of Der Kiureghian & Ditlevsen (2009); whether that convention is the right semantics for LLM agents is a substantive justification question, not a self-referential reduction. Equations (1) and (2) are standard decompositions and marginalizations cited from the literature rather than new predictions. The 'Apple Confidential' header is a metadata artifact and does not affect the argument chain.
Assumptions & free parameters
assumptions (5)
- standard math Second-order (Bayesian) distributions formalize the epistemic state of a learner.
- ad hoc to paper Reducibility of uncertainty is relative to the agent's decision to stop gathering information.
- ad hoc to paper The input space of a chatbot is the set of all finite strings and remains fixed across conversation turns.
- domain assumption Calibrated language models must hallucinate, as shown by Kalai and Vempala.
- domain assumption Human-interaction transparency is a primary goal for LLM-agent uncertainty quantification.
Cite this review
Pith. "Pith review of Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents." pith.science (2026). https://pith.science/paper/QIS5JY5O
@misc{pith2026250522655,
author = {Pith},
title = {Pith review of: Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents},
year = {2026},
howpublished = {\url{https://pith.science/paper/QIS5JY5O}},
note = {Machine review of arXiv:2505.22655}
}
read the original abstract
Large-language models (LLMs) and chatbot agents are known to provide wrong outputs at times, and it was recently found that this can never be fully prevented. Hence, uncertainty quantification plays a crucial role, aiming to quantify the level of ambiguity in either one overall number or two numbers for aleatoric and epistemic uncertainty. This position paper argues that this traditional dichotomy of uncertainties is too limited for the open and interactive setup that LLM agents operate in when communicating with a user, and that we need to research avenues that enrich uncertainties in this novel scenario. We review the literature and find that popular definitions of aleatoric and epistemic uncertainties directly contradict each other and lose their meaning in interactive LLM agent settings. Hence, we propose three novel research directions that focus on uncertainties in such human-computer interactions: Underspecification uncertainties, for when users do not provide all information or define the exact task at the first go, interactive learning, to ask follow-up questions and reduce the uncertainty about the current context, and output uncertainties, to utilize the rich language and speech space to express uncertainties as more than mere numbers. We expect that these new ways of dealing with and communicating uncertainties will lead to LLM agent interactions that are more transparent, trustworthy, and intuitive.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
Uncertainty-Driven Reliability: Selective Prediction and Trustworthy Deployment in Modern Machine Learning
A training-dynamics abstention method matches deep ensembles at a fraction of the training cost, and a five-term error budget explains why selective classifiers still fall short of the oracle.
Reference graph
Works this paper leans on
-
[3]
Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. A multitask, mul- tilingual, multimodal evaluation of chatgpt on reason- ing, hallucination, and interactivity.arXiv preprint arXiv:2302.04023,
-
[7]
How disentangled are your classifi- cation uncertainties?arXiv preprint arXiv:2408.12175,
Ivo Pascal de Jong, Andreea Ioana Sburlea, and Matias Valdenegro-Toro. How disentangled are your classifi- cation uncertainties?arXiv preprint arXiv:2408.12175,
-
[9]
Ensembling over Classifiers: a Bias-Variance Perspective
10 Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents Neha Gupta, Jamie Smith, Ben Adlam, and Zelda Mariet. Ensembling over classifiers: a bias-variance perspective. arXiv preprint arXiv:2206.10566,
-
[11]
Large language models must be taught to know what they don’t know.arXiv preprint arXiv:2406.08391,
Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. Large language models must be taught to know what they don’t know.arXiv preprint arXiv:2406.08391,
-
[13]
URL https://twitter.com/BlackHC/ status/1817556167687569605. Accessed on 04.08.2024. Zahra Kolagar and Alessandra Zarcone. Aligning uncer- tainty: Leveraging llms to analyze uncertainty transfer in text summarization. InProceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024), pp. 41– 61,
-
[14]
Nikita Kotelevskii and Maxim Panov. Predictive uncertainty quantification via risk decompositions for strictly proper scoring rules.arXiv preprint arXiv:2402.10727,
-
[17]
AmbigQA: Answering ambiguous open- domain questions
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. AmbigQA: Answering ambiguous open- domain questions. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP),
work page 2020
-
[20]
On uncertainty in natural language process- ing.arXiv preprint arXiv:2410.03446,
Dennis Ulmer. On uncertainty in natural language process- ing.arXiv preprint arXiv:2410.03446,
Show all 24 references
-
[21]
Sayself: Teach- ing llms to express confidence with self-reflective ra- tionales, 2024a
Tianyang Xu, Shujin Wu, Shizhe Diao, Xiaoze Liu, Xingyao Wang, Yangyi Chen, and Jing Gao. Sayself: Teach- ing llms to express confidence with self-reflective ra- tionales, 2024a. URL https://arxiv.org/abs/ 2405.20974. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucina- ti...
-
[22]
Gal Yona, Roee Aharoni, and Mor Geva
URLhttps://arxiv.org/abs/2505.16922. Gal Yona, Roee Aharoni, and Mor Geva. Can large language models faithfully express their intrinsic uncertainty in words? In Yaser Al-Onaizan, Mohit Bansal, and Yun- Nung Chen (eds.),Proceedings of the 2024 Conference on Empirical Methods in...
2024
-
[23]
Boxuan Zhang and Ruqi Zhang
URL https://arxiv.org/abs/ 2505.14489. Boxuan Zhang and Ruqi Zhang. CoT-UQ: Improving response-wise uncertainty quantification in llms with chain-of-thought,
-
[24]
Michael J.Q
URL https://arxiv.org/ abs/2502.17214. Michael J.Q. Zhang and Eunsol Choi. Clarify when neces- sary: Resolving ambiguity through interaction with lms,
-
[1990]
Uncertainty in natural language generation: From theory to applications.arXiv preprint arXiv:2307.15703,
Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ulmer, Haau- Sing Li, Raquel Fern´andez, Barbara Plank, Rico Sennrich, Chrysoula Zerva, and Wilker Aziz. Uncertainty in natural language generation: From theory to applications.arXiv preprint arXiv:2307.15703,
-
[1997]
Bayesian active learning for classification and preference learning.arXiv preprint arXiv:1112.5745,
Neil Houlsby, Ferenc Husz ´ar, Zoubin Ghahramani, and M´at´e Lengyel. Bayesian active learning for classification and preference learning.arXiv preprint arXiv:1112.5745,
-
[2004]
Towards clear expectations for uncertainty estimation.arXiv preprint arXiv:2207.13341,
Victor Bouvier, Simona Maggio, Alexandre Abraham, and L´eo Dreyfus-Schmidt. Towards clear expectations for uncertainty estimation.arXiv preprint arXiv:2207.13341,
-
[2013]
Know what you don’t know: Unanswerable questions for squad
Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:1806.03822,
-
[2017]
Sources of un- certainty in machine learning–a statisticians’ view.arXiv preprint arXiv:2305.16703,
Cornelia Gruber, Patrick Oliver Schenk, Malte Schierholz, Frauke Kreuter, and G¨oran Kauermann. Sources of un- certainty in machine learning–a statisticians’ view.arXiv preprint arXiv:2305.16703,
-
[2018]
On information-theoretic measures of predictive uncertainty.arXiv preprint arXiv:2410.10786,
Kajetan Schweighofer, Lukas Aichberger, Mykyta Ielan- skyi, and Sepp Hochreiter. On information-theoretic measures of predictive uncertainty.arXiv preprint arXiv:2410.10786,
-
[2019]
Perceptions of linguistic un- certainty by language models and humans.arXiv preprint arXiv:2407.15814,
Catarina G Belem, Markelle Kelly, Mark Steyvers, Sameer Singh, and Padhraic Smyth. Perceptions of linguistic un- certainty by language models and humans.arXiv preprint arXiv:2407.15814,
-
[2020]
Teach- ing models to express their uncertainty in words.arXiv preprint arXiv:2205.14334,
Stephanie Lin, Jacob Hilton, and Owain Evans. Teach- ing models to express their uncertainty in words.arXiv preprint arXiv:2205.14334,
-
[2022]
Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Sla...
-
[2023]
Phiseg: Capturing uncertainty in medical image segmentation
Christian F Baumgartner, Kerem C Tezcan, Krishna Chai- tanya, Andreas M H¨otker, Urs J Muehlematter, Khoschy Schawkat, Anton S Becker, Olivio Donati, and Ender Konukoglu. Phiseg: Capturing uncertainty in medical image segmentation. InMedical Image Computing and Computer Assist...
2019
-
[2024]
Llms will always hallucinate, and we need to live with this
Sourav Banerjee, Ayushi Agarwal, and Saloni Singla. Llms will always hallucinate, and we need to live with this. arXiv preprint arXiv:2409.05746,
-
[2025]
Andreas Kirsch
URL https: //arxiv.org/abs/2505.20295. Andreas Kirsch. (implicit) ensembles of ensembles: Epis- temic uncertainty collapse in large models.arXiv preprint arXiv:2409.02628,
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.