Pith. sign in

REVIEW 2 major objections 4 minor 49 references

Correctness probes, tested on OC–SJ conflicts, follow the model's own judgement rather than objective correctness in every cross-domain condition.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Transferable LLM 'truth' probes preserve a self-judgement-associated polarity rather than an objective-correctness polarity when the two conflict.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection Conflict-based factorial design is a real contribution and the transfer asymmetry is robust enough to change validation practice; the component-level 'SJ polarity' claim leans on the linear model assumption, but the broader cautionary conclusion survives it. the 2 major comments →

arxiv 2607.16799 v1 pith:WKEE3SQD submitted 2026-07-18 cs.CL cs.LG

Diagnosing Correctness Probes under Self-Judgement Confounding

Yi-Long Lu This is my paper
classification cs.CL cs.LG
keywords correctness probingself-judgementobjective correctnesshidden-state readoutsconflict settransferabilitylinear representationslarge language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what a correctness probe actually reads out of a language model's hidden states when 'is the answer right?' and 'does the model think it's right?' disagree. It constructs conflict cases — objectively correct answers the model rejects versus wrong answers it endorses — and finds that conventional correctness-labelled readouts rank the wrong-but-endorsed answers higher, following self-judgement rather than objective correctness. After factorially separating the two variables, the self-judgement-associated direction transfers across math and factual recall domains for every tested model, while the objective-correctness-associated direction does not. The authors conclude that transferability alone does not establish that a probe encodes objective correctness; the transferable component largely preserves the model's own evaluation.

Core claim

The paper's central claim is that when objective correctness (OC) and self-judgement (SJ) are separated, the component of a correctness probe that transfers across domains preserves SJ-associated polarity, not OC-associated polarity. On conflict cases, the conventional mixed contrast Wmix ranks wrong/self-endorsed responses above correct/self-rejected responses, and the OC-only mass-mean control does the same in all eight cross-domain conditions. The factorial contrast Wmeta (SJ-associated) predicts held-out SJ above chance within domain and across domains in all eight model-by-direction evaluations, whereas Wtruth (OC-associated) is at or below chance for OC ordering. This pattern develops

What carries the argument

The load-bearing object is a factorial decomposition of the conventional 'correct minus incorrect' contrast. With cell means mu_A (correct, endorsed), mu_B (correct, rejected), mu_C (wrong, endorsed), mu_D (wrong, rejected), the paper defines Wmix = mu_A - mu_D, Wmeta = [(mu_A - mu_B)+(mu_C - mu_D)]/2, and Wtruth = [(mu_A - mu_C)+(mu_B - mu_D)]/2, and interprets them under an additive linear model of the answer-token residual stream in which OC and SJ each contribute a direction and the interaction cancels. The conflict set B/C provides the test: an OC readout should rank B above C, an SJ readout should rank C above B. These directions are fitted on one domain and scored on held-out or cross

Load-bearing premise

The interpretation rests on the assumption that the answer-token residual stream is a linear additive mixture of separate OC and SJ directions, with the interaction cancelling in factorial contrasts; if the true encoding is nonlinear or confounded with other variables like answer style, the 'OC-associated' and 'SJ-associated' labels are fitted contrasts rather than true latent directions.

What would settle it

Find a model or domain where, on the same B/C conflict set, the OC-associated direction Wtruth transfers above chance while the SJ-associated direction Wmeta does not, with confidence intervals excluding the paper's asymmetry. Alternatively, show that controlling for a specific confound (e.g., response verbosity or topic) reverses the B/C ordering of Wmix, or that a causal intervention that changes self-judgement while leaving objective correctness fixed fails to move the probe's ranking.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If a correctness probe transfers across tasks, the transfer may reflect the model's confidence or endorsement rather than objective truth; transferability alone does not validate a truth readout.
  • Conflict cases (correct-but-rejected vs wrong-but-endorsed) provide a practical diagnostic for deciding what a probe tracks.
  • The SJ-associated direction is the robust transferable component across four models and two source domains, including zero-target-fitting transfer to MMLU and TruthfulQA.
  • The distinction between OC and SJ is operationalized by a separate Yes/No judgement; the probe's answer-token signal predictably aligns with that later judgement.
  • The observed asymmetry develops in middle-to-late transformer layers and persists under controls for answer likelihood, sequence length, token count, and null directions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to test the same factorial conflict diagnostic on larger or non-instruction-tuned models; the paper only covers up to 14B instruction-tuned models, so it is an open question whether the asymmetry holds elsewhere.
  • If the SJ signal is largely endorsement or commitment, then interventions that shift a model's self-judgement (e.g., prompt-induced confidence) might move the probe's ranking of B versus C without changing objective correctness — a testable prediction that would sharpen the causal reading.
  • The results suggest re-examining published 'truth direction' findings: without conflict-based validation, those directions may have been tracking confidence all along.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a diagnostic for correctness probes that separates objective correctness (OC) from the model's self-judgement (SJ) by constructing conflict cases in which the two variables predict opposite orderings. It extracts final-answer-token residual-stream activations, defines factorial contrasts Wmeta and Wtruth intended to isolate SJ- and OC-associated directions, and evaluates their within-domain and cross-domain transfer across four instruction-tuned models and two free-response domains, plus zero-target-fitting OOD evaluations on MMLU and binary TruthfulQA. The central empirical finding is that the SJ-associated direction transfers above chance in essentially every condition, while the OC-associated direction has below-chance or near-chance AUCs for the expected OC ordering, and the conventional mixed contrast often ranks wrong/self-endorsed responses above correct/self-rejected responses. The paper concludes that transferability alone does not establish objective-correctness semantics.

Significance. If the result holds, it is an important and timely caution for the interpretability literature: it provides a concrete, multi-model demonstration that a transferable correctness readout can preserve the polarity of the model's own judgement rather than external correctness. The paper's strengths are substantial: four models, two free-response domains and two OOD benchmarks, strict confidence-filtered conflict sets, cluster-bootstrap inference that resamples source and target questions, source-label-shuffle and random-direction nulls, token-count and answer-likelihood residualization, and a second strict-pair draw. The OC-only mass-mean control is particularly valuable because it shows that even a direction fitted without any SJ labels follows SJ on conflicts. The paper is also unusually candid about its scope and the conditional nature of its estimates.

major comments (2)
  1. [Abstract; §5.1/Eq. S1; Scope (S12)] The component-level labels Wmeta and Wtruth are derived from the additive linear model x = α o vOC + β s vSJ + η o s vINT + ε. If the true residual-stream encoding is nonlinear, or if OC and SJ directions are not separable in the way this model assumes, then these vectors are fitted contrasts rather than semantic directions. The paper acknowledges this in Scope, but the abstract and Discussion state the component-level conclusion ('SJ-associated polarity', 'the SJ-associated direction') without the caveat. Please either qualify the abstract explicitly (e.g., 'under a linear factorial decomposition of activation means') or provide a nonlinear-probe check to show the labels are not artifacts of the linear estimator. The broader cautionary conclusion is supported independently by the OC-only mass-mean control, but the component-level interpretation rests on Eq. S1.
  2. [Method (SJ elicitation); Exp2B; S12] Source and target SJ labels are generated by the identical yes/no self-judgement prompt, whereas OC labels come from dataset-specific parsers or answer keys. Cross-domain transfer of Wmeta could therefore partly reflect shared surface alignment of the elicitation procedure rather than a stable semantic self-judgement direction. The answer-likelihood residualization, the OC-only control, and the counterbalanced X/Y control reduce this concern materially, but the Discussion should state the asymmetry explicitly: the SJ transfer evidence is symmetric in task format in a way that the OC evidence is not. S12 mentions that SJ may contain variance from confidence, familiarity, or response policy, but does not connect this to the source-target sharing of the judgement prompt.
minor comments (4)
  1. [Eq. S1 and surrounding text] The terms 'latent directions vOC and vSJ' may suggest independent latent axes. Since Wmeta and Wtruth are defined as factorial contrasts and are not shown to be orthogonal or unique, please clarify in the main text that these are operational contrast directions, not necessarily independent semantic axes in activation space.
  2. [Sample flow and Scope] The attrition from the original pools to the strict paired sample is severe (e.g., OLMo-3-7B Math retains 362 strict pairs from 5,549 questions). The main text states the conditional nature of the estimand, but it would help readers if the approximate retention rates were stated in the main experimental protocol rather than only in the supplement.
  3. [Figures 2–4] The figures are information-dense and the colors (teal/amber, blue/orange) may be hard to distinguish in grayscale. Adding direct text labels at the curve level or using line styles would improve accessibility.
  4. [Results, OOD section] The all-layer counts such as 458/560 (81.8%) are described as descriptive in S6, but the main text could explicitly remind readers that adjacent layers are not independent observations so these proportions are summaries, not primary inference.

Circularity Check

0 steps flagged

No circularity: transfer results are held-out/OOD with zero target fitting; factorial labels are explicit constructions, not fitted-then-renamed predictions.

full rationale

The central empirical chain is not circular. Wmeta and Wtruth are constructed as factorial contrasts from source-domain cell means (S5.1, Eqs. S6–S7), explicitly labelled as 'operational labels for the SJ-associated and OC-associated factorial contrasts.' The transfer claims (Exp2B, Table 1) freeze these source directions and evaluate them on B/C conflicts in a different domain, including MMLU and TruthfulQA with no target-domain direction fitting (S10). This is a genuine out-of-sample generalization test, not a re-fit on the target labels. Exp1's Wmix = µA − µD is a conventional correctness-labelled contrast; the OC-only mass-mean control (S8.2) is fitted with OC labels only, yet ranks C above B on target conflicts, so the Exp1 reversal is not forced by using SJ labels in fitting. Held-out grouped folds (Exp2A), source-label-shuffle and random-direction nulls (S8.3), token-count matching, answer-likelihood residualization, and counterbalanced X/Y judgement labels further support the empirical asymmetry. The linear associative model in Eq. S1 is an interpretative assumption; the paper explicitly scope-limits the factorial contrasts to 'linear activation-mean directions' and notes they 'do not assess nonlinear or task-adapted correctness predictors' (Scope, S12). An assumption that is stated and testable is a correctness risk, not circularity. The only self-citation (Lu, Song, and Wang 2025) is contextual and not load-bearing for the derivation.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 1 invented entities

Central claims rest on three non-empirical premises: the linear separable activation model, the one-token Yes/No judgement as a valid SJ measure, and the assumption that the high-confidence strict-pair subset supports inference about OC-SJ relations. The confidence threshold, fixed layer window, and pilot filtering thresholds are analyst-chosen but are not fitted to the transfer result; sensitivity analyses are reported for tau and the window is not peak-selected. No new physical entities are introduced beyond latent activation directions vOC/vSJ, which are operationalized by contrasts and tested by transfer.

free parameters (3)
  • SJ confidence threshold tau = 0.7
    Symmetric high-confidence cut for defining binary SJ labels; sensitivity analyses over 0.5-0.8 show the same pattern (S9).
  • Fixed layer window = normalized depth [0.40, 0.80]
    Pre-specified middle-to-late window used to summarize transfer and avoid selecting a model-specific peak layer (Method, S6).
  • Pilot filtering thresholds = ncorrect < 8; mean pilot output length < 380 tokens
    Filters the Math pool by removing questions solved on all pilot attempts and the longest ~10% of responses; shapes the diagnostic subset (S2.1).
axioms (3)
  • domain assumption Final-answer-token residual streams are well approximated by x = alpha*o*vOC + beta*s*vSJ + eta*o*s*vINT + epsilon with additive, separable OC and SJ signals
    Underlies the interpretation of Wmeta and Wtruth as SJ- and OC-associated directions (Eq. S1, S5.1). If false, the factorial contrasts do not cleanly separate OC from SJ.
  • domain assumption A one-token Yes/No self-judgement after the answer is a valid operational measure of self-judgement
    SJ is defined by this elicitation; X/Y counterbalancing is checked for target-side OOD labels, but source directions still rely on Yes/No (S3.1, S10.2).
  • domain assumption The high-confidence strict-pair selection and question-level filtering do not systematically bias the OC-SJ conflict set toward the observed asymmetry
    Analyses are conditional on questions with both correct and incorrect usable answers and confident judgements; the paper states this scope in Discussion and S4, but the inference to general probe semantics assumes no systematic selection-induced polarity.
invented entities (1)
  • Latent task directions vOC and vSJ independent evidence
    purpose: Conceptual carriers of objective-correctness and self-judgement information; Wtruth and Wmeta are claimed to estimate 2*alpha*vOC and 2*beta*vSJ
    Not directly observed; they are inferred from factorial contrasts and tested via cross-domain transfer and OOD benchmark AUCs, providing a falsifiable handle.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Diagnosing Correctness Probes under Self-Judgement Confounding." pith.science (2026). https://pith.science/paper/WKEE3SQD

@misc{pith2026260716799,
  author       = {Pith},
  title        = {Pith review of: Diagnosing Correctness Probes under Self-Judgement Confounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WKEE3SQD}},
  note         = {Machine review of arXiv:2607.16799}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Hidden-state readouts can predict whether language-model outputs are correct, but objective correctness (OC) usually agrees with the model's own self-judgement (SJ), leaving the decoded signal semantically ambiguous. We construct conflict cases in which OC and SJ predict opposite readout orderings. On high-confidence disagreements, conventional correctness-labelled contrasts often rank incorrect/self-endorsed responses above correct/self-rejected responses, following SJ rather than OC. We estimate factorial SJ- and OC-associated directions and evaluate their polarity across mathematical reasoning and factual recall. Across four instruction-tuned models up to 14B parameters, the SJ-associated direction transfers above chance in both cross-domain directions for every model, whereas the OC-associated direction has a below-chance point estimate for the expected OC ordering in every corresponding condition. This transfer asymmetry develops across middle-to-late layers, persists under answer-likelihood, sequence-length, and null-direction controls, and extends to MMLU and binary TruthfulQA without target-domain direction fitting. Across the studied models and diagnostic subsets, the most reliably transferable component preserves SJ-associated polarity. Transferability alone therefore does not establish objective-correctness semantics.

Figures

Figures reproduced from arXiv: 2607.16799 by Yi-Long Lu.

Figure 1
Figure 1. Figure 1: Conflict-based semantic validation of correctness probes. (a) Objective correctness (OC) and self-judgement (SJ) define [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Traditional mixed direction on OC–SJ conflicts. Curves show the AUC for ranking C (wrong/self-endorsed) above B [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cross-domain component transfer across layers. Directions fitted in one domain are evaluated on B/C conflicts in the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: OOD transfer without target-domain direction fitting. Rows show (a) Math [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 3 canonical work pages

  1. [1]

    T ruthful QA : Measuring How Models Mimic Human Falsehoods

    Lin, Stephanie and Hilton, Jacob and Evans, Owain. T ruthful QA : Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2022. doi:10.18653/v1/2022.acl-long.229

    doi:10.18653/v1/2022.acl-long.229 2022

  2. [2]

    Lin, Stephanie and Hilton, Jacob and Evans, Owain , year =

  3. [3]

    International Conference on Learning Representations , year =

    Measuring Massive Multitask Language Understanding , author =. International Conference on Learning Representations , year =

  4. [4]

    arXiv preprint arXiv:2110.14168 , year =

    Training Verifiers to Solve Math Word Problems , author =. arXiv preprint arXiv:2110.14168 , year =

    Pith/arXiv arXiv

  5. [5]

    A Diverse Corpus for Evaluating and Developing E nglish Math Word Problem Solvers

    Miao, Shen-yun and Liang, Chao-Chun and Su, Keh-Yih. A Diverse Corpus for Evaluating and Developing E nglish Math Word Problem Solvers. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.92

    doi:10.18653/v1/2020.acl-main.92 2020

  6. [6]

    Are NLP Models really able to Solve Simple Math Word Problems?

    Patel, Arkil and Bhattamishra, Satwik and Goyal, Navin. Are NLP Models really able to Solve Simple Math Word Problems?. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021. doi:10.18653/v1/2021.naacl-main.168

    doi:10.18653/v1/2021.naacl-main.168 2021

  7. [7]

    Measuring Mathematical Problem Solving With the MATH Dataset , url =

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , booktitle =. Measuring Mathematical Problem Solving With the MATH Dataset , url =

  8. [8]

    Solving General Arithmetic Word Problems

    Roy, Subhro and Roth, Dan. Solving General Arithmetic Word Problems. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015. doi:10.18653/v1/D15-1202

    doi:10.18653/v1/d15-1202 2015

  9. [9]

    Lightman, Hunter and Kosaraju, Vineet and Burda, Yuri and Edwards, Harrison and Baker, Bowen and Lee, Teddy and Leike, Jan and Schulman, John and Sutskever, Ilya and Cobbe, Karl , booktitle =. Let. 2024 , url =

    2024

  10. [10]

    International Conference on Learning Representations , year =

    Discovering Latent Knowledge in Language Models Without Supervision , author =. International Conference on Learning Representations , year =

  11. [11]

    The Internal State of an LLM Knows When It ' s Lying

    Azaria, Amos and Mitchell, Tom. The Internal State of an LLM Knows When It ' s Lying. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023. doi:10.18653/v1/2023.findings-emnlp.68

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  12. [12]

    Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , url =

    Li, Kenneth and Patel, Oam and Vi\'. Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , url =. Advances in Neural Information Processing Systems , editor =

  13. [13]

    First Conference on Language Modeling , year=

    The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. First Conference on Language Modeling , year=

  14. [14]

    and Wang, Zifan and Mallen, Alex and Basart, Steven and Koyejo, Sanmi and Song, Dawn and Fredrikson, Matt and Kolter, J

    Zou, Andy and Phan, Long and Chen, Sarah and Campbell, James and Guo, Phillip and Ren, Richard and Pan, Alexander and Yin, Xuwang and Mazeika, Mantas and Dombrowski, Ann-Kathrin and Goel, Shashwat and Li, Nathaniel and Byun, Michael J. and Wang, Zifan and Mallen, Alex and Basart, Steven and Koyejo, Sanmi and Song, Dawn and Fredrikson, Matt and Kolter, J. ...

    Pith/arXiv arXiv

  15. [15]

    2022 , eprint=

    Language Models (Mostly) Know What They Know , author=. 2022 , eprint=

    2022

  16. [16]

    Transactions on Machine Learning Research , issn=

    Teaching Models to Express Their Uncertainty in Words , author=. Transactions on Machine Learning Research , issn=. 2022 , url=

    2022

  17. [17]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =

    Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =. 2023 , address =. doi:10.18653/v1/2023.emnlp-main.330 , url =

    doi:10.18653/v1/2023.emnlp-main.330 2023

  18. [18]

    Xiong, Miao and Hu, Zhiyuan and Lu, Xinyang and Li, Yifei and Fu, Jie and He, Junxian and Hooi, Bryan , booktitle =. Can. 2024 , url =

    2024

  19. [19]

    Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages =

    Designing and Interpreting Probes with Control Tasks , author =. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing , pages =. 2019 , publisher =. doi:10.18653/v1/D19-1275 , url =

    doi:10.18653/v1/d19-1275 2019

  20. [20]

    Computational Linguistics , volume =

    Probing Classifiers: Promises, Shortcomings, and Advances , author =. Computational Linguistics , volume =. 2022 , doi =

    2022

  21. [21]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  22. [22]

    arXiv preprint arXiv:2407.21783 , year =

    The Llama 3 Herd of Models , author =. arXiv preprint arXiv:2407.21783 , year =

    Pith/arXiv arXiv

  23. [23]

    ACM Transactions on Information Systems , volume =

    A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions , author =. ACM Transactions on Information Systems , volume =. 2025 , publisher =. doi:10.1145/3703155 , url =

    doi:10.1145/3703155 2025

  24. [24]

    2025 , eprint =

    A Unified Representation Underlying the Judgment of Large Language Models , author =. 2025 , eprint =. doi:10.48550/arXiv.2510.27328 , url =

    doi:10.48550/arxiv.2510.27328 2025

  25. [25]

    Decoupling Metacognition from Cognition: A Framework for Quantifying Metacognitive Ability in

    Wang, Guoqing and Wu, Wen and Ye, Guangze and Cheng, Zhenxiao and Chen, Xi and Zheng, Hong , booktitle =. Decoupling Metacognition from Cognition: A Framework for Quantifying Metacognitive Ability in. 2025 , doi =

    2025

  26. [26]

    Do Large Language Models Know What They Don

    Yin, Zhangyue and Sun, Qiushi and Guo, Qipeng and Wu, Jiawen and Qiu, Xipeng and Huang, Xuanjing , booktitle =. Do Large Language Models Know What They Don. 2023 , address =. doi:10.18653/v1/2023.findings-acl.551 , url =

    doi:10.18653/v1/2023.findings-acl.551 2023

  27. [27]

    Kumaran, Dharshan and Patraucean, Viorica and Osindero, Simon and Veli. How. 2026 , eprint =. doi:10.48550/arXiv.2604.22271 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.22271 2026

  28. [28]

    2026 , eprint=

    How do LLMs Compute Verbal Confidence , author=. 2026 , eprint=

    2026

  29. [29]

    , year =

    Haller, Patrick and Ibrahim, Mark and Kirichenko, Polina and Sagun, Levent and Bell, Samuel J. , year =. doi:10.48550/arXiv.2510.11905 , url =. 2510.11905 , archivePrefix =

    doi:10.48550/arxiv.2510.11905

  30. [30]

    2025 , url =

    Orgad, Hadas and Toker, Michael and Gekhman, Zorik and Reichart, Roi and Szpektor, Idan and Kotek, Hadas and Belinkov, Yonatan , booktitle =. 2025 , url =

    2025

  31. [31]

    , booktitle =

    Li, Ji-An and Xiong, Huadong and Wilson, Robert and Mattar, Marcelo G and Benna, Marcus K. , booktitle =. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations , url =

  32. [32]

    Cheang, Chi Seng and Chan, Hou Pong and Zhang, Wenxuan and Deng, Yang , editor =. Do. Findings of the Association for Computational Linguistics: ACL 2026 , month = jul, year =. doi:10.18653/v1/2026.findings-acl.34 , pages =

    work page doi:10.18653/v1/2026.findings-acl.34 2026

  33. [33]

    Current Directions in Psychological Science , volume =

    Metacognition and Uncertainty Communication in Humans and Large Language Models , author =. Current Directions in Psychological Science , volume =. 2026 , doi =

    2026

  34. [34]

    2025 , url =

    The Geometries of Truth Are Orthogonal Across Tasks , author =. 2025 , url =

    2025

  35. [35]

    2026 , eprint=

    Olmo 3 , author=. 2026 , eprint=

    2026

  36. [36]

    and Chen, Jia , year =

    Patel, Het and Chen, Tiejin and Wei, Hua and Papalexakis, Evangelos E. and Chen, Jia , year =. Are. doi:10.48550/arXiv.2604.19974 , url =. 2604.19974 , archivePrefix =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.19974

  37. [37]

    2026 , eprint =

    The Truthfulness Spectrum Hypothesis , author =. 2026 , eprint =. doi:10.48550/arXiv.2602.20273 , url =

    doi:10.48550/arxiv.2602.20273 2026

  38. [38]

    Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in

    Bao, Yuntai and Zhang, Xuhong and Du, Tianyu and Zhao, Xinkui and Feng, Zhengwen and Peng, Hao and Yin, Jianwei , booktitle =. Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in. 2025 , publisher =. doi:10.18653/v1/2025.findings-acl.38 , url =

    doi:10.18653/v1/2025.findings-acl.38 2025

  39. [39]

    Testing the Limits of Truth Directions in

    Poulis, Angelos and Crovella, Mark and Terzi, Evimaria , year =. Testing the Limits of Truth Directions in. doi:10.48550/arXiv.2604.03754 , url =. 2604.03754 , archivePrefix =

    Pith/arXiv arXiv doi:10.48550/arxiv.2604.03754

  40. [40]

    Proceedings of the 2nd Conference on Language Modeling , year =

    Language Models Fail to Introspect About Their Knowledge of Language , author =. Proceedings of the 2nd Conference on Language Modeling , year =

  41. [41]

    Singh, Shashwat and Linzen, Tal and Ravfogel, Shauli , year =. Can. doi:10.48550/arXiv.2605.26242 , url =. 2605.26242 , archivePrefix =

    Pith/arXiv arXiv doi:10.48550/arxiv.2605.26242

  42. [42]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =

    When Truthful Representations Flip Under Deceptive Instructions? , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =. 2025 , publisher =. doi:10.18653/v1/2025.emnlp-main.826 , url =

    work page doi:10.18653/v1/2025.emnlp-main.826 2025

  43. [43]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  44. [44]

    Enhancing Uncertainty Estimation in

    Xiao, Zeguan and Dou, Diyang and Xiong, Boya and Chen, Yun and Chen, Guanhua , journal =. Enhancing Uncertainty Estimation in. 2026 , doi =

    2026

  45. [45]

    and Wang, Rui , booktitle =

    Wang, Yiming and Zhang, Pei and Yang, Baosong and Wong, Derek F. and Wang, Rui , booktitle =. Latent Space Chain-of-Embedding Enables Output-Free. 2025 , url =

    2025

  46. [46]

    Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in

    Servedio, Giovanni and De Bellis, Alessandro and Di Palma, Dario and Anelli, Vito Walter and Di Noia, Tommaso , booktitle =. Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in. 2025 , publisher =. doi:10.18653/v1/2025.acl-long.304 , url =

    doi:10.18653/v1/2025.acl-long.304 2025

  47. [47]

    Schouten and Peter Bloem and Ilia Markov and Piek Vossen , booktitle=

    Stefan F. Schouten and Peter Bloem and Ilia Markov and Piek Vossen , booktitle=. Truth-value judgment in language models:. 2025 , url=

    2025

  48. [48]

    Masked by Consensus: Disentangling Privileged Knowledge in

    Ashuach, Tomer and Gretz, Shai and Katz, Yoav and Belinkov, Yonatan and Ein-Dor, Liat , booktitle =. Masked by Consensus: Disentangling Privileged Knowledge in. 2026 , address =. doi:10.18653/v1/2026.acl-long.483 , pages =

    work page doi:10.18653/v1/2026.acl-long.483 2026

  49. [49]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =

    Cognitive Dissonance: Why Do Language Model Outputs Disagree with Internal Representations of Truthfulness? , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =. doi:10.18653/v1/2023.emnlp-main.291 , pages =

    doi:10.18653/v1/2023.emnlp-main.291 2023

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.