REVIEW 6 cited by
Relying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
As natural language becomes the default interface for human-AI interaction, there is a need for LMs to appropriately communicate uncertainties in downstream applications. In this work, we investigate how LMs incorporate confidence in responses via natural language and how downstream users behave in response to LM-articulated uncertainties. We examine publicly deployed models and find that LMs are reluctant to express uncertainties when answering questions even when they produce incorrect responses. LMs can be explicitly prompted to express confidences, but tend to be overconfident, resulting in high error rates (an average of 47%) among confident responses. We test the risks of LM overconfidence by conducting human experiments and show that users rely heavily on LM generations, whether or not they are marked by certainty. Lastly, we investigate the preference-annotated datasets used in post training alignment and find that humans are biased against texts with uncertainty. Our work highlights new safety harms facing human-LM interactions and proposes design recommendations and mitigating strategies moving forward.
Forward citations
Cited by 6 Pith papers
-
Humans overrely on overconfident language models, across languages
LLMs produce overconfident-sounding answers in all five tested languages, and bilingual users show the highest overreliance risk in Japanese despite its frequent hedges.
-
Rethinking Prospect Theory for LLMs: Revealing the Instability of Decision-Making under Epistemic Uncertainty
Prospect Theory parameters estimated for LLMs become unstable when epistemic markers are injected into prompts, indicating the model is not robust for decision-making under linguistic uncertainty.
-
Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies
Explanations increase user reliance on both correct and incorrect LLM answers, while sources and inconsistent explanations reduce overreliance on incorrect answers in a controlled experiment.
-
"All that Glitters": Approaches to Evaluations with Unreliable Model and Human Annotations
Encoder models trained on noisy classroom ratings look super-human under standard concordance metrics, but generalizability, disattenuation, and hierarchical rater analyses show the apparent advantage is partly spurio...
-
Evaluating LLMs Without Oracle Feedback: Agentic Annotation Evaluation Through Unsupervised Consistency Signals
The ratio of agreement to disagreement between a small student model and an LLM correlates with the LLM's annotation accuracy across ten datasets and can heuristically select better models.
-
Bridging Expertise Gaps: The Role of LLMs in Human-AI Collaboration for Cybersecurity
In n=58 non-expert participants, human-AI collaboration improved phishing precision and intrusion recall, with confident LLM responses strongly influencing user decisions.
Discussion (0). Continue with ORCID to comment.