pith. sign in

arxiv: 2606.22633 · v1 · pith:ZAV3G5N6new · submitted 2026-06-21 · 💻 cs.AI

Confident but Conflicted: Internal Uncertainty and Cognitive Dissonance Resolution in LLMs

Weihong Qi , Kristina Lerman This is my paper

Pith reviewed 2026-06-26 10:28 UTC · model grok-4.3

classification 💻 cs.AI
keywords large language modelscognitive dissonancetrust elasticityinternal uncertaintypersuasionconfidence miscalibrationhealth claims
0
0 comments X

The pith

LLMs vary substantially in how readily they update their outputs when presented with conflicting evidence, and this variation tracks measurable internal uncertainty signals in at least two models.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how large language models handle inputs that contradict their earlier statements, framing the process as cognitive dissonance resolution. It introduces Trust Elasticity as a quantitative measure of how easily a model shifts toward new evidence when source authority and evidence quality are varied across health-science claims. Four models show clear differences in this measure, with clearly false claims producing almost no shift in any of them. In two open-weight models the behavioral differences align with distinct internal uncertainty signals: one tied to confidence miscalibration and the other to changes in internal uncertainty estimates. The work therefore connects observable persuasion outcomes to properties inside the models themselves.

Core claim

Trust Elasticity, defined as the readiness of an LLM to revise its stance toward conflicting evidence, differs substantially across four tested models while remaining near zero for clearly false claims in all of them. On Qwen this variation correlates with Confidence Miscalibration; on Llama it correlates with Internal Uncertainty Change. These associations are obtained by applying controlled persuasion attempts that differ in source authority and evidence quality to twelve health-science claims of varying epistemic status.

What carries the argument

Trust Elasticity (TE), an econometrics-inspired metric that quantifies how readily a model revises its output in response to conflicting evidence of controlled authority and quality.

If this is right

  • Models can be grouped by their typical TE values, with false claims treated as a special case that elicits almost no change.
  • Targeting internal uncertainty signals offers a possible route to altering how readily a given model revises its outputs.
  • Behavioral outcomes (persuasion, backfire, or immunity) can be predicted from internal signals in at least two open-weight architectures.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the internal-uncertainty link holds, fine-tuning procedures that deliberately adjust confidence calibration could produce models with more consistent persuasion resistance.
  • The near-zero TE on false claims suggests a built-in immunity mechanism that future work could test by injecting fabricated evidence of varying quality.
  • Extending the protocol beyond health claims would show whether the same uncertainty indicators govern persuasion on other domains such as historical or technical facts.

Load-bearing premise

Differences in how models respond to conflicting evidence can be attributed to the measured internal uncertainty indicators rather than to unmeasured differences in training data or architecture.

What would settle it

Run the same persuasion protocol on additional models and find that the observed TE variation shows no systematic correlation with either Confidence Miscalibration or Internal Uncertainty Change.

Figures

Figures reproduced from arXiv: 2606.22633 by Kristina Lerman, Weihong Qi.

Figure 1
Figure 1. Figure 1: Experimental framework and key findings. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Cross-model behavioral comparison. Mean Trust Elasticity (TE) per topic across four models, with error [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Internal uncertainty indicators associated with [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-topic Trust Elasticity (TE) matrix for Qwen3.5-9B. Each [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-topic Trust Elasticity (TE) matrix for Grok-3. See Figure [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per-topic Trust Elasticity (TE) matrix for Llama-3.3-70B. See Figure [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per-topic Trust Elasticity (TE) matrix for GPT-4o. See Figure [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Mean stance shift (∆ stance) as a function of authority level, separated by epistemic category. Each line represents one model, with error bars indicating standard deviation across topics and evidence levels within each authority bin. ∆ stance is computed as post-intervention stance minus baseline stance on the 7-point scale. 0.3 0.4 0.5 0.6 0.7 Normalized Entropy GMO 5G Vaccines MSG Sweeteners Fasting Vit… view at source ↗
Figure 9
Figure 9. Figure 9: Internal entropy shifts from baseline to intervention for Qwen3.5-9B [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
read the original abstract

Large language models (LLMs) frequently encounter inputs that disagree with their prior outputs, through user pushback, retrieved documents, or web search results. While the way they resolve such conflicts -- a process we frame as cognitive dissonance resolution -- has been characterized behaviorally, its connection to internal model uncertainty is not well understood. To study this systematically, we vary persuasion attempts along two dimensions, source authority and evidence quality, across 12 health-science claims of stratified epistemic status. Dissonance can be resolved through persuasion, backfire, or immunity. We introduce Trust Elasticity (TE), an econometrics-inspired measure of how readily a model is persuaded toward conflicting evidence. Across four LLMs, TE varies substantially, while clearly false claims elicit near-zero TE across all models. On two open-weight models, we further find that this variation is associated with two complementary internal uncertainty indicators, Confidence Miscalibration in Qwen and Internal Uncertainty Change in Llama. These results link cross-model behavioral variation to a measurable internal property and point to interventions targeting internal uncertainty as future work.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper introduces Trust Elasticity (TE), an econometrics-inspired measure of how readily LLMs update outputs when presented with conflicting evidence that varies in source authority and evidence quality. Across 12 health-science claims of varying epistemic status, it reports that TE varies substantially across four LLMs but is near-zero for clearly false claims. On two open-weight models, TE variation is associated with two model-specific internal uncertainty indicators (Confidence Miscalibration for Qwen; Internal Uncertainty Change for Llama), linking behavioral dissonance resolution to measurable internal properties.

Significance. If the reported associations prove robust to controls for model differences, the work would usefully connect observable persuasion outcomes to internal uncertainty metrics, supporting future interventions that target uncertainty calibration. The TE metric itself provides a quantifiable behavioral lens on cognitive dissonance resolution that could generalize beyond the health-science domain tested.

major comments (1)
  1. [Results section on open-weight models] Results section reporting associations for open-weight models: The link between TE variation and the two internal uncertainty indicators is presented without ablations, controls, or regressions that isolate the indicators from model identity (e.g., no multi-model analysis holding architecture or training regime constant). Because the indicators are defined and measured separately per model and the two models differ in both architecture and training data, the association could be driven by those unmeasured factors rather than the indicators themselves.
minor comments (2)
  1. [Abstract] Abstract and methods: The exact computational definitions and formulas for 'Confidence Miscalibration' and 'Internal Uncertainty Change' are not summarized, which would help readers assess independence from the TE outcome measure.
  2. The manuscript could clarify how the 12 claims were selected and stratified for epistemic status to allow replication.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on the associations reported for open-weight models. We address the major comment point by point below.

read point-by-point responses

  1. Referee: The link between TE variation and the two internal uncertainty indicators is presented without ablations, controls, or regressions that isolate the indicators from model identity (e.g., no multi-model analysis holding architecture or training regime constant). Because the indicators are defined and measured separately per model and the two models differ in both architecture and training data, the association could be driven by those unmeasured factors rather than the indicators themselves.

    Authors: We agree that the reported associations lack ablations, controls, or regressions isolating the uncertainty indicators from model identity. The indicators (Confidence Miscalibration for Qwen; Internal Uncertainty Change for Llama) were selected and operationalized separately because each model exposes different internal states, precluding direct cross-model comparison or a unified regression. With only two open-weight models available, multi-model analyses holding architecture or training regime constant are not feasible. The manuscript presents these as model-specific exploratory associations rather than a general claim. We will revise the Results and Limitations sections to explicitly note this constraint and the possibility that unmeasured model differences contribute to the observed patterns. revision: yes

Circularity Check

0 steps flagged

No circularity: TE and uncertainty associations are independently defined empirical measures

full rationale

The paper introduces Trust Elasticity (TE) as a new econometrics-inspired measure of persuasion susceptibility and reports observed associations with separately measured internal uncertainty indicators (Confidence Miscalibration, Internal Uncertainty Change). No equations, definitions, or self-citations are shown that reduce TE, the associations, or any claimed result to fitted parameters or prior inputs by construction. The central content consists of empirical variation and correlations across models, which remain independent of the target claims.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Information is limited to the abstract; full details on parameters, assumptions, and any invented constructs are unavailable.

axioms (1)
  • domain assumption Conflict resolution in LLMs can be meaningfully framed as cognitive dissonance resolution
    Central framing of the study stated in the abstract without further justification.
invented entities (1)
  • Trust Elasticity (TE) no independent evidence
    purpose: Econometrics-inspired measure of how readily a model is persuaded toward conflicting evidence
    Newly introduced quantity whose exact formula is not supplied in the abstract.

pith-pipeline@v0.9.1-grok · 5717 in / 1136 out tokens · 33536 ms · 2026-06-26T10:28:06.064264+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

18 extracted references · 3 canonical work pages · 1 internal anchor

  1. [1]

    1957 , address =

    Festinger, Leon , title =. 1957 , address =

    1957

  2. [2]

    Bowman and Newton Cheng and Esin Durmus and Zac Hatfield-Dodds and Scott R

    Mrinank Sharma and Meg Tong and Tomasz Korbak and David Duvenaud and Amanda Askell and Samuel R. Bowman and Newton Cheng and Esin Durmus and Zac Hatfield-Dodds and Scott R. Johnston and Shauna Kravec and Timothy Maxwell and Sam McCandlish and Kamal Ndousse and Oliver Rausch and Nicholas Schiefer and Da Yan and Miranda Zhang and Ethan Perez , title =. Proc...

  3. [3]

    Bowman and Amanda Askell and Roger Grosse and Danny Hernandez and Deep Ganguli and Evan Hubinger and Nicholas Schiefer and Jared Kaplan , title =

    Ethan Perez and Sam Ringer and Kamile Lukosiute and Karina Nguyen and Edwin Chen and Scott Heiner and Craig Pettit and Catherine Olsson and Sandipan Kundu and Saurav Kadavath and Andy Jones and Anna Chen and Benjamin Mann and Brian Israel and Bryan Seethor and Cameron McKinnon and Christopher Olah and Da Yan and Daniela Amodei and Dario Amodei and Dawn Dr...

    2023

  4. [4]

    Manning , title =

    Katherine Tian and Eric Mitchell and Allan Zhou and Archit Sharma and Rafael Rafailov and Huaxiu Yao and Chelsea Finn and Christopher D. Manning , title =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2023 , address =

    2023

  5. [5]

    Language Models (Mostly) Know What They Know

    Saurav Kadavath and Tom Conerly and Amanda Askell and Tom Henighan and Dawn Drain and Ethan Perez and Nicholas Schiefer and Zac Hatfield-Dodds and Nova DasSarma and Eli Tran-Johnson and Scott Johnston and Sheer El-Showk and Andy Jones and Nelson Elhage and Tristan Hume and Anna Chen and Yuntao Bai and Sam Bowman and Stanislav Fort and Deep Ganguli and Dan...

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    Kevin Liu and Stephen Casper and Dylan Hadfield-Menell and Jacob Andreas , title =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2023 , address =

    2023

  7. [7]

    1890 , address =

    Marshall, Alfred , title =. 1890 , address =

  8. [8]

    Gregory , title =

    Mankiw, N. Gregory , title =. 2020 , address =

    2020

  9. [9]

    Le , title =

    Jerry Wei and Da Huang and Yifeng Lu and Denny Zhou and Quoc V. Le , title =. 2023 , eprint =

    2023

  10. [10]

    Lin and Shujian Yang and Tianqi Zhang and Weiyan Shi and Tianwei Zhang and Zhixuan Fang and Wei Xu and Han Qiu , title =

    Rongwu Xu and Brian S. Lin and Shujian Yang and Tianqi Zhang and Weiyan Shi and Tianwei Zhang and Zhixuan Fang and Wei Xu and Han Qiu , title =. Proceedings of ACL 2024 , year =

    2024

  11. [11]

    Chen and Roy Ka-Wei Lee , title =

    Bryan Chen Zhengyu Tan and Daniel Wai Kit Chin and Zhengyuan Liu and Nancy F. Chen and Roy Ka-Wei Lee , title =. arXiv preprint arXiv:2508.17450 , year =

    work page arXiv

  12. [12]

    arXiv preprint arXiv:2601.13590 , year =

    Fan Huang and Haewoon Kwak and Jisun An , title =. arXiv preprint arXiv:2601.13590 , year =

    work page arXiv

  13. [13]

    Nature , volume =

    Sebastian Farquhar and Jannik Kossen and Lorenz Kuhn and Yarin Gal , title =. Nature , volume =. 2024 , doi =

    2024

  14. [14]

    Weinberger , title =

    Chuan Guo and Geoff Pleiss and Yu Sun and Kilian Q. Weinberger , title =. Proceedings of the 34th International Conference on Machine Learning (ICML) , pages =. 2017 , url =

    2017

  15. [15]

    2026 , howpublished=

    2026

  16. [16]

    Grattafiori, Aaron and Dubey, Abhimanyu and Jauhri, Abhinav and others , journal=. The

  17. [17]

    and others , journal=

    Hurst, Aaron and Lerer, Adam and Goucher, Adam P. and others , journal=

  18. [18]

    2025 , howpublished=

    Grok 3. 2025 , howpublished=

    2025