Pith. sign in

REVIEW 4 major objections 5 minor 65 references

Pretrained LLMs Learn Multiple Types of Uncertainty

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large language models that were only pretrained encode several distinct, nearly independent linear directions for uncertainty in their hidden states, and these directions can predict whether the model's own next token will be wrong.

desk verdict Broad probing study with a plausible core idea, but the unbalanced-label accuracy reporting makes the headline numbers uninterpretable as published. read the letter →

arxiv 2505.21218 v1 pith:AY6QVFKL submitted 2025-05-27 cs.CL cs.AI

Roi Cohen , Omri Fahn , Gerard de Melo This is my paper
classification cs.CLcs.AI
keywords uncertaintyrepresentationlinearprobinghallucinationdetectioncorrectnesspredictionlargelanguagemodelsinstructiontuningIDKtokenlatentspace
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a pretrained large language model does not hold one unified notion of uncertainty but many task-specific ones, and that these notions are linearly readable from its hidden states. Using frozen models and no weight updates, the authors fit a linear probe at each transformer layer to predict whether the model's own next token will be correct, and the resulting uncertainty direction predicts correctness well above chance on 16 QA benchmarks. The directions found for different datasets are nearly orthogonal, yet directions from related math benchmarks transfer across each other, suggesting uncertainty is organized by knowledge type rather than as a single scalar. The paper also argues that intermediate layers carry the strongest signal, that model scale alone does not improve uncertainty encoding, and that instruction-tuning or [IDK]-token tuning both sharpens the signal and pushes it into earlier layers. If correct, this gives a lightweight, training-free way to detect when a model is about to be wrong, and points to training method rather than size as the lever for making models more reliable.

What carries the argument

The central object is the linear uncertainty direction: a weight vector $u_i(D)$ in the hidden state space at the end of transformer layer $i$, fitted by logistic regression to separate states whose next-token prediction will be correct from states whose prediction will be wrong. Its role is to act as a read-out: projecting the hidden state onto this direction yields a scalar that classifies correctness, and comparing the directions found for different datasets via cosine similarity and cross-testing is what reveals that uncertainty is multiple rather than unified.

What would settle it

Take one of the reported model–dataset pairs, relabel every example by full-sequence exact match (or by executing generated code for HumanEval-X and MBPP), retrain the same linear probes, and check whether above-chance accuracy and cross-dataset near-orthogonality persist; if they collapse, the central claim describes top-token matching rather than uncertainty.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is that uncertainty is a linearly separable concept in LLM hidden space, and it splits into multiple dataset-specific vectors. For each model layer $i$ and dataset $D$, a logistic-regression probe on hidden states $h_i(x)$ yields a vector $u_i(D)$ and bias $b_i$; the classifier $\operatorname{sign}(u_i(D)^\top h_i(x) + b_i)$ predicts whether the model's highest-probability next token matches the gold answer, at accuracies well above 0.5 across Llama, Mistral, and Qwen models. Across layers and datasets these vectors show low cosine similarity, indicating near-linear independence; only a probe trained on a union of all datasets (UNIFIED) and vectors from within the same thematic group (e.g., math) generalize. The paper finds intermediate layers most informative, little gain from scaling, and consistent improvements from instruction-tuning and [IDK]-tuning, with the optimal layer shifting earlier; zero-shot verbal abstention correlates with probe accuracy in the range 0.38 to 0.45.

Load-bearing premise

All probe labels depend on defining a model answer as correct exactly when its single highest-probability next token equals the gold token; for multi-token, exact-match benchmarks this proxy can disagree with genuine correctness.

Editorial extensions

If this is right

  • A frozen model's hidden states can flag its own likely-wrong answers with a linear probe, enabling hallucination detection without fine-tuning or extra inference-time sampling.
  • Uncertainty should not be treated as a single score: probes trained per task or knowledge type capture signal that a unified probe may miss.
  • Probing should target intermediate layers, where the strongest correctness signal lives, rather than the final layer.
  • Scaling model size alone is a weak lever for uncertainty awareness; instruction-tuning and [IDK]-tuning improve correctness prediction more and make the signal appear earlier.
  • Because uncertainty vectors from related math benchmarks transfer, a detector built on one benchmark can serve nearby benchmarks within the same knowledge type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the near-orthogonality of uncertainty vectors reflects fragmented training-data groupings, then a model may hold conflicting internal confidence directions for the same question; unifying them during tuning could reduce self-contradictory hallucinations, a testable consequence the paper only gestures at.
  • The top-token correctness label is a narrow proxy; re-running the same probes with full-answer exact match or open-ended factuality judgments would show whether the multiple-vector structure survives when correct means the whole answer, not just the first token.
  • The layered probes could be converted into a calibration score for selective answering: use the intermediate-layer uncertainty direction's margin as a confidence threshold and measure precision-recall against abstention, giving a practical deployment test of the paper's mechanism.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a linear-probing framework to detect uncertainty in frozen pretrained LLMs. For each transformer layer and dataset, the authors train a logistic-regression probe on hidden states to predict a binary label L(q)=1 iff the model's highest-probability next token exactly matches the gold answer token (Section 2, Eq. 1). They report correctness-prediction accuracy across 9 models and 16 datasets (Table 1), claim that LLMs encode multiple near-orthogonal dataset-specific uncertainty directions, show layer-wise and model-size trends, and argue that instruction-tuning and [IDK]-tuning improve uncertainty encoding. The paper includes transfer matrices, cosine similarities, and a correlation with zero-shot abstention accuracy.

Significance. The central question—whether pretrained LLMs linearly encode something usable as uncertainty—is timely and relevant for hallucination detection and mechanistic interpretability. The paper has genuine strengths: it covers multiple model families, keeps model weights frozen, probes all layers, and presents a broad transfer analysis. However, the empirical support as reported is not yet convincing because Table 1 compares raw accuracy to 0.5 without reporting label base rates or a majority-class baseline, the correctness definition in Section 2.2 is a single next-token match for benchmarks that require multi-token exact-match answers, and no validation procedure is given for the best-layer selection. These issues are load-bearing for the paper's main claims, but they are fixable within the manuscript's scope, so the appropriate disposition is major revision.

major comments (4)
  1. [Section 4.1 / Table 1] The headline evidence compares classifier accuracy against 'the random baseline of 0.5', but the labels are imbalanced: each dataset has a fraction p of examples where the model's top token is correct, and a constant classifier that always predicts the majority class achieves accuracy max(p, 1-p) >= 0.5. The paper never reports p, balanced accuracy, AUC, or a majority-class baseline, so entries such as 0.857 (Llama-3.2-1B on PopQA) and 0.444 (Llama-3.2-1B on GSM8K) cannot be distinguished from a trivial classifier that ignores hidden states entirely. This concern is load-bearing because the existence of linearly separable uncertainty directions, the transfer matrices in Figures 2, 3, 5, and 8, and the layer-wise curves in Figures 6-11 are all interpreted through these raw accuracy numbers. Please report the label base rate for every model-layer-dataset cell, recompute results as balanced accuracy or AUC, and compare against a constant majority-class baseline.
  2. [Section 2.2 / Eq. (1)-(2)] Correctness is defined as the model's single highest-probability next token exactly matching the gold answer token. Most of the 16 benchmarks (e.g., GSM8K, TriviaQA, NaturalQuestions, HumanEval-X, MBPP, QAMPARI, RoMQA) require multi-token exact-match or free-form answers, so this proxy conflates partial token coincidence with genuine answer correctness. If the top token often matches while the full greedy decoding is wrong, or vice versa, then the reported probe accuracy, cross-dataset transfer, and layer-wise trends do not establish what the paper claims about predicting generation correctness. Please either restrict the analysis to single-token-answer benchmarks or re-label correctness using greedy decoding with the benchmarks' own evaluation metrics (e.g., exact match for QA and code) and verify that the qualitative conclusions in Sections 4 and 5 survive that change.
  3. [Section 3 / Section 4.1] Table 1 reports results 'from the best-performing layer only', but the paper does not describe any validation procedure for selecting that layer. If the best layer is chosen by test-set performance, the reported peak accuracy is an optimistically biased estimate, and the comparisons across models and datasets in Table 1 are not honest evaluations of the method. Please define an explicit split: for example, select the best layer on a held-out validation subset derived from the training split, then report accuracy on the untouched test split for that chosen layer. The same clarification is needed for the layer choices underlying Figures 2, 3, 4, and 5.
  4. [Section 3 / NeurIPS Checklist item 7] No error bars, confidence intervals, or significance tests are reported anywhere in the paper. The checklist response says statistical errors are 'likely negligible' because of the large scale, but this is not an adequate substitute for quantification. The claims in Section 5.2 that model size 'doesn't seem to matter' and in Section 5.3 that instruction-tuning consistently improves accuracy are comparisons of small differences in averages; without bootstrap confidence intervals or paired significance tests across the same test instances, these conclusions are not supported. Please add at least bootstrap confidence intervals over test sets or a small number of random train/test split replicates, and report them in Table 1 and Figures 6-11.
minor comments (5)
  1. [Table 1 and Figure 5] The dataset name is written as 'SV AMP' in the Table 1 header and in Figure 5, while the text and Section 3 refer to 'SVAMP'; please make the spelling consistent.
  2. [Section 2.1] The notation D={(q_j, a_j)}_{j=0}^n with m<n is used informally; please clarify the number of training examples and the split proportions, and define m and n explicitly.
  3. [Section 2.2 and Section 3] The term 'Precision' is introduced in Section 2.2 but defined only in Section 3 as the ratio of actually wrong completions among those predicted wrong; this is a class-precision for the 'incorrect' class and should be labeled as such to avoid ambiguity with standard precision on the 'correct' class.
  4. [Figure 4] Figure 4 reports cosine similarities at layer 22, whereas Figures 2 and 3 use layers 26 and 27; no rationale is given for these particular layer choices, which matters given the best-layer selection issue described above.
  5. [Appendix A] The limitations section acknowledges that 'correctness is treated as a proxy for uncertainty' and that 'performance may be influenced by dataset-specific biases', but it does not mention the single-token correctness definition of Eq. (1) or the lack of majority-class baselines; these should be disclosed in the limitations discussion.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline probe accuracies are best-layer test-set maxima rather than independent predictions, and the multiple-types evidence relies on a near-orthogonality artifact without a random baseline.

  1. fitted input called prediction [Section 4.1, Table 1; Section 3 Evaluation]
    "While the uncertainty vector search is conducted independently at each transformer layer for every model–dataset pair, the table reports results from the best-performing layer only (a detailed layer-wise analysis is provided in a subsequent section)."

    The experimental setup defines only a train split 'used to derive our uncertainty vectors' and a test split 'used to evaluate their performance' (Section 3). Each Table 1 cell is thus the maximum over the layer-wise probes of test-set accuracy, so the layer index is a hyperparameter selected on the same test labels that are then reported as the prediction target. The headline claim that 'classification accuracy significantly exceeds the random baseline of 0.5' is made for this selected maximum, not for a fixed pre-chosen layer; any single layer would on average score lower, and the multiple-comparison inflation is never corrected. The reported 'prediction' is therefore partly constructed by the layer-selection step.

  2. renaming known result [Section 4.2, Figure 4]
    "Furthermore, the cosine similarity between ui(D1) and ui(D2) is frequently near-zero, indicating near-linear independence between these vectors. ... Aside from the unified classifier trained on a dataset union (UNIFIED), nearly all vectors are close to orthogonal."

    These vectors live in the model's high-dimensional residual stream (e.g., 4096 dimensions for Llama-3.1-8B). For any two independent random or arbitrary directions in such a space, expected squared cosine is about 1/d, so 'near-zero, near-linear independence' is the default outcome for unrelated fitted weight vectors, regardless of whether they encode shared uncertainty semantics. The paper presents this expected geometric baseline as evidence that 'LLMs encode uncertainty through multiple distinct and largely independent internal representations,' without a random-vector or permutation baseline; the discovery is a renaming of the dimensionality artifact rather than an independent finding about uncertainty types.

full rationale

The probing framework itself is not definitionally circular: correctness labels come from external gold answers, hidden states come from a frozen model, and the train/test split provides some out-of-sample test. The main circularity is the best-layer selection performed on the test split, which makes Table 1's reported accuracies fitted maxima rather than independent predictions; this is the paper's primary evidence that linear uncertainty directions exist. The multiple-types conclusion additionally rests on near-zero cosine similarity between dataset-specific probes, which is the expected null in high-dimensional spaces and is reported without any random or permutation baseline; this is a renaming artifact rather than an independent empirical discovery. The absence of base rates and majority-class baselines (a constant classifier achieves max(p,1-p) >= 0.5) is a separate statistical validity problem, not itself a circularity, but it compounds the difficulty of interpreting the headline numbers. The [IDK]-tuned and instruction-tuned checkpoints are drawn from prior work by the same authors, but they are externally trained model artifacts rather than load-bearing definitions, so those self-citations do not raise the circularity score further. Overall, the central correctness-prediction results are partially circular because the reported peaks are selected on the test set, yielding a score of 6 rather than 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper's evidence rests on the next-token correctness proxy, the choice of probe hyperparameters, and the layer-selection procedure. None of these are externally anchored, so they act as free parameters in interpreting the reported accuracies.

free parameters (3)
  • best_layer_index = varies by model and dataset; e.g., layer 17 for Mistral-7B-v0.1
    The reported accuracy in Table 1 and Figures 2-5 is for the best-performing layer only. No validation split is described for layer selection, so the layer index is effectively a number chosen to maximize test performance.
  • logistic regression regularization and optimizer settings
    The paper does not report any hyperparameters for the probes, so any tuning choices are implied fitted variables rather than specified constants.
  • train/test split proportions
    The paper only says 'a fixed train split' without giving sample counts or split method, so the split itself is an unspecified modeling choice.
assumptions (3)
  • domain assumption The hidden state at the end of a transformer layer is a valid representation for training a linear correctness probe.
    Used throughout Section 2; the probe is trained on hi(x). The paper interprets probe success as evidence the model captures uncertainty, which is an interpretive step.
  • domain assumption Correctness can be defined as the highest-probability next token matching the gold token.
    Section 2.2 defines L(q) this way. All accuracy numbers are based on this reduced definition of correctness, which is likely not equivalent to full answer correctness for multi-token QA answers.
  • domain assumption The train/test splits provide enough statistical power to support three-decimal accuracy differences without error bars.
    The NeurIPS checklist claims 'statistical errors are negligible' based on scale, but no variance, seeds, or tests are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pretrained LLMs Learn Multiple Types of Uncertainty." pith.science (2026). https://pith.science/paper/AY6QVFKL

@misc{pith2026250521218,
  author       = {Pith},
  title        = {Pith review of: Pretrained LLMs Learn Multiple Types of Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AY6QVFKL}},
  note         = {Machine review of arXiv:2505.21218}
}
read the original abstract

Large Language Models are known to capture real-world knowledge, allowing them to excel in many downstream tasks. Despite recent advances, these models are still prone to what are commonly known as hallucinations, causing them to emit unwanted and factually incorrect text. In this work, we study how well LLMs capture uncertainty, without explicitly being trained for that. We show that, if considering uncertainty as a linear concept in the model's latent space, it might indeed be captured, even after only pretraining. We further show that, though unintuitive, LLMs appear to capture several different types of uncertainty, each of which can be useful to predict the correctness for a specific task or benchmark. Furthermore, we provide in-depth results such as demonstrating a correlation between our correction prediction and the model's ability to abstain from misinformation using words, and the lack of impact of model scaling for capturing uncertainty. Finally, we claim that unifying the uncertainty types as a single one using instruction-tuning or [IDK]-token tuning is helpful for the model in terms of correctness prediction.

Figures

Figures reproduced from arXiv: 2505.21218 by the authors.

Figure 1
Figure 1. Illustration of identifying multiple data-specific uncertainty linear vectors when investigating [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Correctness prediction accuracy results of the classifier induced by u27(y − axis − dataset), using Mistral-7B-v0.1, while testing on the test set of the x-axis dataset. Notably, for each of these, we create a fixed train split which will be used to derive our uncertainty vectors, and a test split which will be used to evaluate their performance. Linear Uncertainty Search Details. For every model M, transformer laye… view at source ↗
Figure 5
Figure 5. Correctness prediction accuracy results of the classifier induced by u21(y − axis − dataset), using Qwen2.5-7B, while testing on the test set of the x-axis dataset. The results demonstrate that, for a substantial number of datasets across all models, classification accuracy significantly exceeds the random baseline of 0.5. This provides strong empirical evidence that uncertainty is encoded within LLMs in a manner th… view at source ↗
Figures from the paper (3 more)
Figure 7
Figure 7. Figure 7: Correctness prediction pre￾cision averaged over all datasets of the induced classifier, considering the Llama family: Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B, and Llama-3.1-8B-Instruct. 4.4 Comparing to Zero-Shot Abstaining Skills As an additional evaluation of the un…
Figure 9
Figure 9. Figure 9: Correctness prediction accu￾racy averaged over all datasets of the induced classifier, considering the Qwen family: Qwen2.5-7B, Qwen3-14B, and Qwen3-14B-Instruct [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Correctness prediction accuracy averaged over all datasets of the induced clas￾sifier, comparing Mistral-7B-v0.1 against IDK-tuned-Mistral-7B-v0.1 [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

65 extracted references · 11 canonical work pages

  1. [1]

    Prompting as Probing: Using Language Models for Knowledge Base Construction

    Dimitrios Alivanistos, Selene B \'a ez Santamar \' a, Michael Cochez, Jan-Christoph Kalo, Emile van Krieken, and Thiviyan Thanapalasingam. Prompting as probing: Using language models for knowledge base construction. arXiv preprint arXiv:2208.11057, 2022

    work page Pith review arXiv 2022

  2. [2]

    QAMPARI : A benchmark for open-domain questions with many answers

    Samuel Amouyal, Tomer Wolfson, Ohad Rubin, Ori Yoran, Jonathan Herzig, and Jonathan Berant. QAMPARI : A benchmark for open-domain questions with many answers. In Sebastian Gehrmann, Alex Wang, Jo \ a o Sedoc, Elizabeth Clark, Kaustubh Dhole, Khyathi Raghavi Chandu, Enrico Santus, and Hooman Sedghamiz, editors, Proceedings of the Third Workshop on Natural ...

    work page 2023

  3. [3]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    arXiv 2021

  4. [4]

    The internal state of an LLM knows when it ' s lying

    Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it ' s lying. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 967--976, Singapore, December 2023. Association for Computational Linguistics. doi:10.18653/v1/2023.findings-emnlp.68. URL https://aclantholo...

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  5. [5]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenhang Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, K. Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, ...

    arXiv 2023

  6. [6]

    Attributed question answering: Evaluation and modeling for attributed large language models

    Bernd Bohnet, Vinh Q Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, et al. Attributed question answering: Evaluation and modeling for attributed large language models. arXiv preprint arXiv:2212.08037, 2022

    arXiv 2022

  7. [7]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 0 1877--1901, 2020

    1901

  8. [8]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24 0 (240): 0 1--113, 2023

    2023

Show all 65 references
  1. [9]

    Think you have solved question answering? try ARC , the AI2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC , the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  2. [10]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  3. [11]

    Crawling the internal knowledge-base of language models

    Roi Cohen, Mor Geva, Jonathan Berant, and Amir Globerson. Crawling the internal knowledge-base of language models. In Andreas Vlachos and Isabelle Augenstein, editors, Findings of the Association for Computational Linguistics: EACL 2023, pages 1856--1869, Dubrovnik, Croatia, M...

    2023 doi

  4. [12]

    LM vs LM : Detecting factual errors via cross examination

    Roi Cohen, May Hamri, Mor Geva, and Amir Globerson. LM vs LM : Detecting factual errors via cross examination. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12621--12640, Sin...

    2023 doi

  5. [13]

    Evaluating the ripple effects of knowledge editing in language models

    Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effects of knowledge editing in language models. Transactions of the Association for Computational Linguistics, 12: 0 283--298, 2024 a

    2024

  6. [14]

    I D on't K now: E xplicit modeling of uncertainty with an [IDK] token

    Roi Cohen, Konstantin Dobler, Eden Biran, and Gerard de Melo. I D on't K now: E xplicit modeling of uncertainty with an [IDK] token. Advances in Neural Information Processing Systems, 37: 0 10935--10958, 2024 b

    2024

  7. [15]

    Calibration of pre-trained transformers

    Shrey Desai and Greg Durrett. Calibration of pre-trained transformers. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 295--302, Online, November 2020. Associa...

    2020 doi

  8. [16]

    Evaluating factuality in text simplification

    Ashwin Devaraj, William Sheffield, Byron Wallace, and Junyi Jessy Li. Evaluating factuality in text simplification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7331--7345, Dublin, Ireland, May 2022. ...

    2022 doi

  9. [17]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  10. [18]

    Measuring and improving consistency in pretrained language models

    Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Sch \"u tze, and Yoav Goldberg. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9: 0 1012--1031, 2021. doi...

    2021 doi

  11. [19]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9: 0 346--361, 2021 a . doi:10.1162/...

    2021 doi

  12. [20]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495, Online and Punta Cana, Dominican Republic, November 202...

    2021 doi

  13. [21]

    Dissecting recall of factual associations in auto-regressive language models

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. Dissecting recall of factual associations in auto-regressive language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro...

    2023 doi

  14. [22]

    Weinberger

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 13...

    2017

  15. [23]

    Finding neurons in a haystack: Case studies with sparse probing

    Wes Gurnee, Neel Nanda, Matthew Pauly, Katherine Harvey, Dmitrii Troitskii, and Dimitris Bertsimas. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610, 2023

    2023 arXiv

  16. [24]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv., 55 0 (12), mar 2023. ISSN 0360-0300. doi:10.1145/3571730. URL https://doi.or...

    2023 doi

  17. [26]

    Mistral 7b

    Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, T...

    arXiv

  18. [27]

    How can we know when language models know? on the calibration of language models for question answering

    Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9: 0 962--977, 2021. doi:10.1162/tacl_a_00407. URL ...

    2021 doi

  19. [28]

    TriviaQA : A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. TriviaQA : A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017

    2017 arXiv

  20. [29]

    Brown, Jack Clark, Nicholas Joseph, Benjamin Mann, Sam McCandlish, Christopher Olah, and Jared Kaplan

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zachary Dodds, Nova Dassarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, De...

    2022 arXiv

  21. [30]

    Large language models must be taught to know what they don't know

    Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. Large language models must be taught to know what they don't know. arXiv preprint arXiv:2406.08391, 2024

    2024 arXiv

  22. [31]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664, 2023

    2023 arXiv

  23. [32]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  24. [33]

    Bloom: A 176b-parameter open-access multilingual language model

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \'e , et al. Bloom: A 176b-parameter open-access multilingual language model. 2023

    2023

  25. [34]

    Teaching models to express their uncertainty in words

    Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334, 2022

    2022 arXiv

  26. [35]

    Lin, Jacob Hilton, and Owain Evans

    Stephanie C. Lin, Jacob Hilton, and Owain Evans. TruthfulQA : Measuring how models mimic human falsehoods. In Annual Meeting of the Association for Computational Linguistics, 2021. URL https://api.semanticscholar.org/CorpusID:237532606

    2021

  27. [36]

    On controlling fallback responses for grounded dialogue generation

    Hongyuan Lu, Wai Lam, Hong Cheng, and Helen Meng. On controlling fallback responses for grounded dialogue generation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2591--2601, Dublin, Ireland, May 2022. Association for Computational Linguistics....

    2022 doi

  28. [37]

    When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511, 2022

    2022 arXiv

  29. [38]

    On the challenges and opportunities in Generative AI

    Laura Manduchi, Kushagra Pandey, Robert Bamler, Ryan Cotterell, Sina Däubener, Sophie Fellenz, Asja Fischer, Thomas Gärtner, Matthias Kirchler, Marius Kloft, Yingzhen Li, Christoph Lippert, Gerard de Melo, Eric Nalisnick, Björn Ommer, Rajesh Ranganath, Maja Rudolph, Karen Ullr...

    2024 arXiv

  30. [39]

    On faithfulness and factuality in abstractive summarization

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906--1919, Online, July 2020. Association for Computa...

    1906 doi

  31. [40]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT . In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/fo...

    2022

  32. [41]

    A diverse corpus for evaluating and developing E nglish math word problem solvers

    Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. A diverse corpus for evaluating and developing E nglish math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975--984, Online, July 2020. Association for Com...

    2020 doi

  33. [42]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    2018 arXiv

  34. [43]

    Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation

    Niels M \"u ndler, Jingxuan He, Slobodan Jenko, and Martin Vechev. Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation. arXiv preprint arXiv:2305.15852, 2023

    2023 arXiv

  35. [44]

    In-context learning and induction heads

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al. In-context learning and induction heads. arXiv preprint arXiv:2209.11895, 2022

    2022 arXiv

  36. [45]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 0 2...

    2022

  37. [46]

    Jeff Z. Pan, Simon Razniewski, Jan-Christoph Kalo, Sneha Singhania, Jiaoyan Chen, Stefan Dietze, Hajira Jabeen, Janna Omeliyanenko, Wen Zhang, Matteo Lissandrini, Russa Biswas, Gerard de Melo, Angela Bonifati, Edlira Vakaj, Mauro Dragoni, and Damien Graux. Large Language Model...

    2023 doi

  38. [47]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080--2094...

    2021 doi

  39. [48]

    Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    Fabio Petroni, Tim Rockt \"a schel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases? arXiv preprint arXiv:1909.01066, 2019

    1909 arXiv

  40. [49]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  41. [50]

    Measuring attribution in natural language generation models

    Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring attribution in natural language generation models. Computational Linguistics, 49 0 (4): 0 777--840, 2023

    2023

  42. [51]

    How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910, 2020

    Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? arXiv preprint arXiv:2002.08910, 2020

    2002 arXiv

  43. [52]

    C ommonsense QA : A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. C ommonsense QA : A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human La...

    2019 doi

  44. [53]

    Evaluating the factual consistency of large language models through news summarization

    Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. Evaluating the factual consistency of large language models through news summarization. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Com...

    2023 doi

  45. [54]

    Transformer memory as a differentiable search index

    Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. Transformer memory as a differentiable search index. Advances in Neural Information Processing Systems, 35: 0 21831--21843, 2022

    2022

  46. [55]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  47. [56]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  48. [57]

    Do large language models know what they don't know? arXiv preprint arXiv:2305.18153, 2023

    Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. Do large language models know what they don't know? arXiv preprint arXiv:2305.18153, 2023

    2023 arXiv

  49. [58]

    Narrowing the knowledge evaluation gap: Open-domain question answering with multi-granularity answers

    Gal Yona, Roee Aharoni, and Mor Geva. Narrowing the knowledge evaluation gap: Open-domain question answering with multi-granularity answers. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational L...

    2024 doi

  50. [59]

    Answering questions by meta-reasoning over multiple chains of thought

    Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. Answering questions by meta-reasoning over multiple chains of thought. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023 doi

  51. [60]

    Mechanistic understanding and mitigation of language model non-factual hallucinations

    Lei Yu, Meng Cao, Jackie Chi Kit Cheung, and Yue Dong. Mechanistic understanding and mitigation of language model non-factual hallucinations. arXiv preprint arXiv:2403.18167, 2024

    2024 arXiv

  52. [61]

    Characterizing mechanisms for factual recall in language models

    Qinan Yu, Jack Merullo, and Ellie Pavlick. Characterizing mechanisms for factual recall in language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9924--9959, Singapor...

    2023 doi

  53. [62]

    Automatic evaluation of attribution by large language models

    Xiang Yue, Boshi Wang, Ziru Chen, Kai Zhang, Yu Su, and Huan Sun. Automatic evaluation of attribution by large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4615--4635, Singa...

    2023 doi

  54. [63]

    R -tuning: Instructing large language models to say I don`t know'

    Hanning Zhang, Shizhe Diao, Yong Lin, Yi Fung, Qing Lian, Xingyao Wang, Yangyi Chen, Heng Ji, and Tong Zhang. R -tuning: Instructing large language models to say I don`t know'. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the N...

    2024

  55. [64]

    Instruction tuning for large language models: A survey

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792, 2023

    2023

  56. [65]

    Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Di...

    2023

  57. [66]

    RoMQA : A benchmark for robust, multi-evidence, multi-answer question answering

    Victor Zhong, Weijia Shi, Wen tau Yih, and Luke Zettlemoyer. RoMQA : A benchmark for robust, multi-evidence, multi-answer question answering. In Conference on Empirical Methods in Natural Language Processing, 2022. URL https://api.semanticscholar.org/CorpusID:253116788

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.