Pith. sign in

REVIEW 3 major objections 4 minor 48 references

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Knowledge-probing methods for LLMs contradict themselves and each other: shuffling answer options drops intra-method agreement to about 40%, and different probes agree on as little as 7% of accept/reject decisions.

desk verdict Valuable measurement study with a clean cross-method result; the intra-method claim is confounded by model-output variability and threshold re-tuning. read the letter →

arxiv 2505.21701 v2 pith:IGBE4MDU submitted 2025-05-27 cs.CL

Raoyuan Zhao , Abdullatif Köksal , Ali Modarressi , Michael A. Hedderich , Hinrich Schütze This is my paper
classification cs.CL
keywords knowledgeprobinggapsLLMhallucinationpromptsensitivityconsistencymetricscalibrationabstentionmultiple-choicequestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that methods designed to reveal what large language models do not know are themselves unreliable, and it demonstrates that unreliability with a systematic consistency test. The test applies minimal, meaning-preserving prompt changes (extra spaces, shuffled answer options, typos, and a one-shot example) and measures whether each probing method labels the same questions as known or unknown. The authors report that shuffling answer options lowers agreement with the original prompt to roughly 40%, and that different methods applied to the same model, dataset, and prompt agree on accept/reject decisions as little as 7% of the time. Their conclusion is that current knowledge-gap probes, and any refusal or abstention system built on them, cannot be trusted at the level of individual questions.

What carries the argument

The argument is carried by a consistency-evaluation procedure built from prompt variants and decision-set metrics. For each prompt variant (space insertion, option shuffling, typo, and an added one-shot example), the probe produces an accept set and a reject set; the metrics $IoU_{acc}$ and $IoU_{rej}$ measure intersection-over-union of those sets between two setups, $IoU_{cons}$ is their harmonic mean, and $DecCons$ measures the fraction of all questions with a matching accept/reject decision. These set comparisons turn the abstract question of probe reliability into concrete numbers that can be compared across methods, models, datasets, and perturbations.

What would settle it

Run a calibration-based probe such as ASKCAL with a single fixed threshold chosen once on the original prompt, apply that same threshold to all shuffled-option variants, and compare IoUcons with the paper's per-variant re-calibration results; a large jump in consistency would show that the reported probe instability is threshold drift rather than intrinsic to the probe.

Watch

Extended reading notes

Core claim

The central discovery is that knowledge probing outcomes are highly unstable in two ways: the same probe gives different verdicts under semantically irrelevant prompt changes, and different probes give conflicting verdicts under identical conditions. To quantify this, the paper defines set-based metrics over the questions each probe accepts (model knows the answer) and rejects (knowledge gap), with IoUcons as the harmonic mean of acceptance and rejection intersection-over-union and DecCons as the fraction of questions on which two setups make the same decision. Across six probing methods, two multiple-choice datasets, and models from 1B to 70B parameters, the authors observe intra-method IoUcons values as low as 0.27 under option shuffling and cross-method DecCons values as low as 0.07. They also show that the established aggregate metric Abstain F1 stays roughly flat across prompt variants even while the underlying per-question decisions change substantially, which they argue makes that metric insufficient for evaluating probe reliability.

Load-bearing premise

The load-bearing assumption is that the observed inconsistency lives in the probing methods rather than in the way their accept/reject cutoffs are re-tuned for each prompt: the paper re-calibrates thresholds on the development set for every variant, reports values ranging from 0.98 to 0.01, and only applies a safeguard to one of the methods.

Editorial extensions

If this is right

  • If shuffling answer options changes which questions a probe rejects, then any refusal or abstention system built on that probe will reject different questions depending on option order.
  • Aggregate metrics such as Abstain F1 cannot certify a probe's reliability, because they can stay stable while the specific questions marked as knowledge gaps change substantially.
  • Cross-method disagreements mean that published probing results on the same model, dataset, and prompt may be answering materially different questions about what the model knows.
  • Scaling to larger models does not by itself make probing consistent; some methods such as NOTA are less consistent at 70B parameters than at smaller sizes.
  • One-shot prompting, often expected to stabilize behavior, can shift probe decisions even more than typos or extra spaces, with MOREINFO's $IoU_{cons}$ dropping to values near 0.04 in some settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is consensus probing: aggregating verdicts across several prompt variants or several methods might recover accuracy, since the data imply no single-query probe can be trusted.
  • The low cross-method agreement may partly reflect that 'knows the answer' is not a single property: token probability, verbalized confidence, hidden-state training, and self-reflection measure different signals, so the 7% floor may indicate a measurement-construct mismatch as much as method failure.
  • The stability of Abstain F1 alongside unstable per-question decisions suggests that future probe evaluations should routinely report a per-question consistency statistic such as $IoU_{cons}$ under option shuffling, since aggregate scores can hide label flips.
  • Because option shuffling is the most destructive zero-shot variant and is trivial to apply, it could serve as a minimal stress test for any newly proposed knowledge-gap probe.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a consistency-evaluation framework for knowledge-gap probing methods in LLMs. It defines four metrics (IoUcons, IoUacc, IoUrej, DecCons, plus Agreement) and applies them to six probing methods (TOKPROB, ASKCAL, EMBEDDING, NOTA, MOREINFO, SELFREF) under four prompt variants (inserted spaces, shuffled options, typos, and a one-shot exemplar) across seven models (Mistral-7B, LLaMA-3.2-1B/3B, LLaMA-3.1-8B/70B, OLMo-2-7B) on MMLU and HellaSwag. The main findings are that intra-method accept/reject decisions change substantially under surface-level perturbations (e.g., shuffled options reduce IoUcons to about 0.4), that cross-method decision consistency can be as low as 7%, that scaling does not consistently improve probing consistency, and that aggregate metrics like Abstain F1 hide this instability. The paper also reports an analysis showing that ASKCAL's poor intra-method consistency is partly caused by unstable threshold selection, and proposes a threshold-correction safeguard.

Significance. If the central claims survive scrutiny, the paper makes a useful contribution: it demonstrates that aggregate abstention metrics can be stable while the specific items rejected change dramatically, and it documents large disagreement among probes on identical inputs. The experimental scale is a strength — six probes, multiple model families and sizes, two datasets, multiple seeds, and publicly released code — and the reported variance across seeds is near zero. The proposed metrics are simple and likely to be adopted. However, the headline paradox ('the tools themselves suffer from alarming inconsistencies') is currently supported by measurements that do not cleanly separate probe behavior from model-output instability or from threshold-recalibration artifacts. The cross-method inconsistency is the least confounded result; the intra-method results need additional conditioning to support the paper's interpretation.

major comments (3)
  1. [§5.1, Table 1 and §3] The intra-method consistency metrics compare probe decisions across prompt variants while allowing the model's output to vary, so they conflate probe instability with the model's own answer instability. Table 1's Agr. column shows that under shuffled options the model's answer changes on a large fraction of commonly accepted questions (e.g., TOKPROB Agr.=0.66 and NOTA Agr.=0.57 on Mistral/MMLU). Because each probe is a function of these outputs, a low IoUcons may simply reflect the model's sensitivity rather than an internal defect of the probe. The paper acknowledges this in the 'Source of Inconsistency' paragraph of §5.1, but does not act on it; the conclusion in §5.3 that the inconsistency 'should be attributed to the knowledge probing methods themselves' is therefore not supported by the reported measurements. To support the central claim, the authors should report the consistency metrics restricted to items where the model's answer (and, ideally, its confidence) is unchanged across variants, or otherwise decompose the observed variance.
  2. [§5.5, Table 3 vs. Table 8] The threshold-recalibration confound is demonstrated inside the paper but not resolved in the main results. Section 5.5 describes a threshold correction for ASKCAL that raises HellaSwag IoUcons values from 0.24/0.05 to 0.53/0.48 (Table 3), yet the main intra-method results for ASKCAL in Table 8 still report the uncorrected values (0.243 and 0.049 for Mistral). No equivalent threshold-stability analysis is provided for TOKPROB, which also selects a threshold per prompt variant, and the correction is not applied uniformly to all calibration-based methods. Because a poorly chosen threshold alone can make a probe appear arbitrarily inconsistent across variants, the intra-method numbers for calibration-based probes cannot be interpreted as measuring intrinsic probe reliability unless threshold selection is controlled (e.g., by fixing one threshold or by reporting rejection sets under several thresholds). The same concern applies to the cross-method heatmaps in Figure 3, where ASKCAL's decisions are based on a per-variant threshold whose stability is not checked.
  3. [§5.3, Figure 2] The scaling-consistency analysis inherits the same confounds and adds a format confound. Figure 2 averages IoUcons over variants that include one-shot prompts, which are not a minimal surface-level perturbation: they change the task format and, as §5.1 reports, can collapse a method's abstention rate (e.g., MOREINFO IoUcons 0.04 on Mistral/MMLU). The non-monotonic trend (e.g., NOTA peaking at 3B and declining at 70B) is therefore not a clean statement about probing consistency under semantically equivalent perturbations. The authors should either restrict the scaling analysis to the three zero-shot variants or show that the pattern is unchanged when the one-shot variant is removed.
minor comments (4)
  1. [Abstract and §3] The one-shot variant is described as a 'minimal non-semantic perturbation,' but it substantially changes the prompt structure and the instructions. The abstract and framing should be qualified so that the one-shot results are not grouped with the zero-shot perturbation results.
  2. [§5.2, Figure 3] When interpreting a DecCons value as low as 0.07, the main text should explicitly reference the base abstention rates of the two methods (available in Appendix B), since large differences in abstention rates mechanically depress DecCons even when the underlying uncertainty signals are correlated.
  3. [§5.5, Table 3] The threshold-correction safeguard sets the threshold to 0.5 without a stated justification; the paper should report sensitivity to this choice (e.g., 0.3 and 0.7) and should clarify whether the main results in Tables 1, 6, and 8 use corrected or uncorrected ASKCAL thresholds.
  4. [§5.1] The heading 'Inconsistency is Consistent' is confusing; a clearer heading such as 'Variance Across Runs Is Low' would better convey the finding that the reported mean values have near-zero standard deviation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the consistency metrics are directly computed from model outputs and public benchmarks, with no fitted parameter disguised as a prediction.

full rationale

The paper's central claim is an empirical measurement, not a derivation from fitted inputs. Intra-method and cross-method consistency are defined directly from probe accept/reject decisions (IoUacc, IoUrej, IoUcons, DecCons) computed on held-out MMLU and HellaSwag test sets, with thresholds calibrated on a separate development set (Section 4). The reported low consistency values follow from the raw experimental outputs and do not presuppose the conclusion. The ASKCAL threshold correction in Section 5.5 is a transparent post-hoc safeguard applied to one method, reported both with and without correction, rather than a hidden parameter that forces the headline result; if anything, correcting the threshold makes the method look more consistent, which weakens rather than manufactures the paper's conclusion. The paper's argument does not depend on any self-citation chain or imported uniqueness theorem: the probes are standard published methods (TOKPROB, ASKCAL, EMBEDDING, NOTA, MOREINFO, SELFREF) whose behavior is re-measured here, and the claimed paradox is an observed property of these measurements. Concerns about whether probe inconsistency is inherited from model-output variability are legitimate external-validity caveats, but they are not circularity: the paper explicitly states it evaluates probing methods, and comparing decisions while allowing model outputs to vary is a definitional choice, not an equation that reduces to its own input. No step in the paper fits the enumerated circularity patterns, so the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; the new consistency metrics are measurements, not entities. The free parameters are the thresholds used by the evaluated methods and the ad hoc ASKCAL correction.

free parameters (2)
  • Threshold per probe and prompt variant = e.g., 0.98, 0.01
    Calibration-based probes set an accept/reject threshold on a development set for each prompt variant; these thresholds shift across variants and influence consistency measurements.
  • ASKCAL threshold correction = 0.5
    Ad hoc safeguard applied when the calibrated threshold is deemed unreasonable; it is not part of the original method and is only applied to ASKCAL.
assumptions (4)
  • domain assumption Surface-level perturbations do not change the model's ground-truth knowledge gaps
    Section 1: 'the underlying knowledge gaps should remain unchanged, and probing methods should be robust to such slight perturbations'.
  • domain assumption The one-shot prompting examples introduce no new knowledge
    Section 3: 'They are simple questions that do not introduce new knowledge'.
  • domain assumption MMLU and HellaSwag are representative datasets for knowledge probing evaluation
    Used as benchmarks for knowledge and reasoning; their representativeness is assumed without further justification.
  • standard math Development set is sufficient for threshold calibration
    Required for reliably setting thresholds; the paper does not verify sufficiency across all variants.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing." pith.science (2026). https://pith.science/paper/IGBE4MDU

@misc{pith2026250521701,
  author       = {Pith},
  title        = {Pith review of: Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IGBE4MDU}},
  note         = {Machine review of arXiv:2505.21701}
}
read the original abstract

The reliability of large language models (LLMs) is greatly compromised by their tendency to hallucinate, underscoring the need for precise identification of knowledge gaps within LLMs. Various methods for probing such gaps exist, ranging from calibration-based to prompting-based methods. To evaluate these probing methods, in this paper, we propose a new process based on using input variations and quantitative metrics. Through this, we expose two dimensions of inconsistency in knowledge gap probing. (1) Intra-method inconsistency: Minimal non-semantic perturbations in prompts lead to considerable variance in detected knowledge gaps within the same probing method; e.g., the simple variation of shuffling answer options can decrease agreement to around 40%. (2) Cross-method inconsistency: Probing methods contradict each other on whether a model knows the answer. Methods are highly inconsistent -- with decision consistency across methods being as low as 7% -- even though the model, dataset, and prompt are all the same. These findings challenge existing probing methods and highlight the urgent need for perturbation-robust probing frameworks.

Figures

Figures reproduced from arXiv: 2505.21701 by the authors.

Figure 1
Figure 1. Examples of the two major dimensions of inconsistency in knowledge gap probing that we focus on in this paper. In the intra-method comparison (top), the same probing method gives contradictory assess￾ments (certain vs not certain) for the same LLM and the same question with options shuffled, revealing in￾ternal inconsistency. In the cross-method comparison (bottom), different probing methods applied to the same LLM … view at source ↗
Figure 2
Figure 2. Average IoUcons across different model sizes (LLaMA3) for intra-method consistency of each method. The scaling trend does not consistently hold across all probing methods, see [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Heatmaps of cross-method consistency evaluation results (DecCons) under the original prompt across [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Heatmap of cross-method consistency evaluation results for Hellaswag. The values represent the average [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Heatmap of cross-method consistency evaluation results for MMLU. The values represent the average [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Heatmap of cross-method consistency evaluation results. The values represent the average consistency [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Heatmap of cross-method consistency evaluation results. The values represent the average consistency [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Decision consistency heatmaps for three LLaMA models across three prompt variants. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 12 canonical work pages

  1. [1]

    Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. 2024. https://openreview.net/forum?id=pH3XAQME6c Refusal in language models is mediated by a single direction . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  2. [2]

    Amos Azaria and Tom Mitchell. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.68 The internal state of an LLM knows when it ' s lying . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967--976, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  4. [4]

    Lang Cao. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.212 Learn to refuse: Making large language models more controllable and reliable through knowledge scope limitation and refusal mechanism . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 3628--3646, Miami, Florida, USA. Association for Computation...

    doi:10.18653/v1/2024.emnlp-main.212 2024

  5. [5]

    Anwoy Chatterjee, H S V N S Kowndinya Renduchintala, Sumit Bhatia, and Tanmoy Chakraborty. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.852 POSIX : A prompt sensitivity index for large language models . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14550--14565, Miami, Florida, USA. Association for Computational ...

    doi:10.18653/v1/2024.findings-emnlp.852 2024

  6. [6]

    Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Dayiheng Liu, Dongmei Zhang, Zhixu Li, and Yanghua Xiao. 2023. Hallucination detection: Robustly discerning reliable answers in large language models. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 245--255

    2023

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  8. [8]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

Show all 48 references
  1. [9]

    Federico Errica, Davide Sanvito, Giuseppe Siracusano, and Roberto Bifulco. 2025. https://aclanthology.org/2025.naacl-long.73/ What did I do wrong? quantifying LLM s' sensitivity and consistency to prompt engineering . In Proceedings of the 2025 Conference of the Nations of the...

    2025

  2. [10]

    Shangbin Feng, Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. 2023. Knowledge card: Filling llms' knowledge gaps with plug-in specialized language models. In The Twelfth International Conference on Learning Representations

    2023

  3. [11]

    Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Orevaoghene Ahia, Shuyue Stella Li, Vidhisha Balachandran, Sunayana Sitaram, and Yulia Tsvetkov. 2024 a . https://doi.org/10.18653/v1/2024.emnlp-main.239 Teaching LLM s to abstain across languages via multilingual feedback . ...

    2024 doi

  4. [12]

    Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024 b . https://aclanthology.org/2024.acl-long.786 Don ' t hallucinate, abstain: Identifying LLM knowledge gaps via multi- LLM collaboration . In Proceedings of the 62nd Annual Meet...

    2024

  5. [13]

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. In International conference on machine learning, pages 1321--1330. PMLR

    2017

  6. [14]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. In International Conference on Learning Representations

    2021

  7. [15]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023 a . https://doi.org/10.1145/3571730 Survey of hallucination in natural language generation . ACM Comput. Surv., 55(12)

    2023 doi

  8. [16]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023 b . Survey of hallucination in natural language generation. ACM computing surveys, 55(12):1--38

    2023

  9. [17]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  10. [18]

    Haoming Jiang et al. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics

    2021

  11. [19]

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav For...

    2022 arXiv

  12. [20]

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. https://openreview.net/forum?id=VD-AYtP0dve Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation . In The Eleventh International Conference on Learning Representations

    2023

  13. [21]

    Jing Liu et al. 2023. Knowledge card: Filling llms’ knowledge gaps with plug-in specialized language models. In Proceedings of the 2023 Annual Meeting of the Association for Computational Linguistics (ACL)

    2023

  14. [22]

    Potsawee Manakul, Adian Liusie, and Mark Gales. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.557 S elf C heck GPT : Zero-resource black-box hallucination detection for generative large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural...

    2023 doi

  15. [23]

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. https://doi.org/10.18653/v1/2020.acl-main.173 On faithfulness and factuality in abstractive summarization . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1...

    2020 doi

  16. [24]

    Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning. arXiv preprint arXiv:2308.00436

    2023 arXiv

  17. [25]

    Niels M \"u ndler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2024. https://openreview.net/forum?id=EmQSOi1X2f Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation . In The Twelfth International Conference on Learning Representations

    2024

  18. [26]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Mi...

    2024 arXiv

  19. [27]

    Pouya Pezeshkpour and Estevam Hruschka. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.130 Large language models sensitivity to the order of options in multiple-choice questions . In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2006--2017...

    2024 doi

  20. [28]

    Abel Salinas and Fred Morstatter. 2024. https://doi.org/10.18653/v1/2024.findings-acl.275 The butterfly effect of altering prompts: How small changes and jailbreaks affect large language model performance . In Findings of the Association for Computational Linguistics: ACL 2024...

    2024 doi

  21. [29]

    Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. https://openreview.net/forum?id=RIu5lyNXjT Quantifying language models' sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting . In The Twelfth International...

    2024

  22. [30]

    Chenglei Si, Weijia Shi, Chen Zhao, Luke Zettlemoyer, and Jordan Boyd-Graber. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.552 Getting M o RE out of mixture of language model reasoning experts . In Findings of the Association for Computational Linguistics: EMNLP 2023,...

    2023 doi

  23. [31]

    Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.220 The curious case of hallucinatory (un)answerability: Finding truths in the hidden states of over-confident large language models . In Proceedings ...

    2023 doi

  24. [32]

    Rickard Stureborg, Dimitris Alikaniotis, and Yoshi Suhara. 2024. Large language models are inconsistent and biased evaluators. arXiv preprint arXiv:2405.01724

    2024 arXiv

  25. [33]

    Meiqi Sun, Wilson Yan, Pieter Abbeel, and Igor Mordatch. 2022. Quantifying uncertainty in foundation models via ensembles. In NeurIPS 2022 Workshop on Robustness in Sequence Modeling

    2022

  26. [34]

    Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2023. https://doi.org/10.18653/v1/2023.findings-acl.322 Evaluating the factual consistency of large language models through news summarization . In Findings of the Association for Computat...

    2023 doi

  27. [35]

    Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.330 Just ask for calibration: Strategies for eliciting calibrated confidence scores from language mod...

    2023 doi

  28. [36]

    Boshi Wang, Xiang Yue, and Huan Sun. 2023 a . Can chatgpt defend its belief in truth? evaluating llm reasoning via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11865--11881

    2023

  29. [37]

    Xinpeng Wang, Chengzhi Hu, Paul R \"o ttger, and Barbara Plank. 2024. Surgical, cheap, and flexible: Mitigating false refusal in language models via single vector ablation. arXiv preprint arXiv:2410.03415

    2024 arXiv

  30. [38]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023 b . https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Confer...

    2023

  31. [39]

    Zezhong Wang, Luyao Ye, Hongru Wang, Wai-Chung Kwan, David Ho, and Kam-Fai Wong. 2023 c . https://doi.org/10.18653/v1/2023.findings-emnlp.501 R ead P rompt: A readable prompting method for reliable knowledge probing . In Findings of the Association for Computational Linguistic...

    2023 doi

  32. [40]

    Spencer Whitehead, Suzanne Petryk, Vedaad Shakib, Joseph Gonzalez, Trevor Darrell, Anna Rohrbach, and Marcus Rohrbach. 2022. Reliable visual question answering: Abstain rather than answer incorrectly. In European Conference on Computer Vision, pages 148--166. Springer

    2022

  33. [41]

    Hongshen Xu, Zichen Zhu, Situo Zhang, Da Ma, Shuai Fan, Lu Chen, and Kai Yu. 2024. Rejection improves reliability: Training llms to refuse unknown questions using rl from knowledge feedback. arXiv preprint arXiv:2403.18349

    2024 arXiv

  34. [42]

    Yunzhi Yao, Ningyu Zhang, Zekun Xi, Mengru Wang, Ziwen Xu, Shumin Deng, and Huajun Chen. 2024. https://doi.org/10.48550/ARXIV.2405.17969 Knowledge circuits in pretrained transformers . CoRR, abs/2405.17969

    2024 doi arXiv

  35. [43]

    o rg Schl \

    Paul Youssef, Osman Kora s , Meijie Li, J \"o rg Schl \"o tterer, and Christin Seifert. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.1043 Give me the facts! a survey on factual knowledge probing in pre-trained language models . In Findings of the Association for Compu...

    2023 doi

  36. [44]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. https://doi.org/10.18653/v1/P19-1472 H ella S wag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4...

    2019 doi

  37. [45]

    Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. 2024. https://doi.org/10.18653/v1/2024.naacl-long.394 R -tuning: Instructing large language models to say I don`t know' . In Proceedings of the 2024 Conference of the ...

    2024 doi

  38. [46]

    Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.108 P ro SA : Assessing and understanding the prompt sensitivity of LLM s . In Findings of the Association for Computational Linguistics: EMN...

    2024 doi

  39. [47]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.