Pith. sign in

REVIEW 3 major objections 4 minor 80 references

Attention-Path Fragility as an Uncertainty Signal in Large Language Models

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM uncertainty is visible in whether a confident prediction collapses when attention heads are masked.

desk verdict Careful, honest empirical study of a new perturbation-based uncertainty estimator, but the central interpretation is confounded by per-sample variation in the number of masked heads. read the letter →

arxiv 2608.11138 v1 pith:CH7JY7SP submitted 2026-08-11 cs.CL cs.AI

Minsoo Kim , Sungyoung Ji , Kisung Moon , Ilyong Yoon This is my paper · ORCID
classification cs.CLcs.AI
keywords uncertaintyestimationattention-headmaskingmutualinformationconfident-but-fragilepredictionsgroundedquestionansweringselectivepredictionlargelanguagemodelssemanticagreement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that an LLM's uncertainty about a next token shows up not only in how spread out its output probabilities are, but in whether a confident prediction is fragile: whether it depends on one specific attention pathway among many redundant ones. To measure this, the authors introduce ASMI, a training-free estimator that repeatedly masks random attention heads in one transformer layer and measures how much the resulting predictive distributions disagree, discounting surface-form variation with a semantic-similarity kernel. The central empirical claim is that ASMI is not a proxy for ordinary confidence: out-of-fold residual tests on grounded question answering show it adds error-predictive information beyond confidence and entropy, concentrated in confident-but-fragile predictions, where using it roughly halves the retained error of a confidence filter. The signal is regime-graded by design: it helps when answers are routed through provided context and reverts to or below a zero-cost baseline when answers are recalled from parametric memory, matching the paper's stated prediction.

What carries the argument

The central object is ASMI (Attention-Subnetwork Mutual Information), a training-free token-level score. At a chosen transformer layer, each of $S$ Monte Carlo samples draws a Bernoulli mask over attention heads and recomputes only the suffix of the network; the disagreement among the resulting predictive distributions is measured by the BALD mutual information $\mathrm{MI}_t = H(\bar{p}_t) - \tfrac{1}{S}\sum_s H(p_t^{(s)})$, approximated over a top-$K$ support with a tail bucket. Sem-ASMI multiplies this by $1 - A_t$, where $A_t$ is the pairwise semantic agreement of the top-$K$ distributions computed from the output-projection embedding, and Adapt-ASMI gates that factor by the diversity of $N$ sampled responses. The mechanism being probed is redundancy among attention heads: because many heads are prunable, an answer that survives random masking is genuinely supported, while an answer that collapses was carried by one critical path.

What would settle it

Run the paper's matched-perturbation control on a fresh grounded-QA benchmark: mask the MLP sublayer at the same target layer instead of attention heads with the same budget, and compare rankings; if the non-attention perturbation reproduces ASMI's error-detection AUROC, then attention-path fragility specifically is not the carrier and the central mechanism is falsified.

Watch

Extended reading notes

Core claim

A model's uncertainty about a token is reflected in whether the prediction is fragile under perturbation of its attention pathways, and this fragility carries error information that single-pass confidence and entropy miss. Under random Bernoulli masking of attention-head outputs at a target layer, the mutual information between the masked subnetwork and the next-token distribution, in the BALD form, measures this fragility; a semantic-agreement kernel multiplies the mutual information by one minus agreement to discount surface-form variation. The paper shows, on grounded QA benchmarks across four pretrained transformers, that the residual of this score after regressing out confidence and entropy still detects errors out of fold, and that among predictions the model itself rates as confident, abstaining those the score flags as fragile roughly halves retained error compared with filtering by entropy. On closed-book trivia QA the same score drops to or below the zero-cost maximum-sequence-probability baseline, and the paper interprets this boundary as design: attention-path fragility is informative where correctness depends on routing context through specific heads, and uninformative where uncertainty originates in parametric recall.

Load-bearing premise

The whole signal rests on treating random head masking as a probe of genuine path fragility rather than generic degradation; when a model is too robust for the mask to move its outputs, the score carries no information, as the paper reports on one of its grounded cells.

Editorial extensions

If this is right

  • On retrieval-grounded QA, a confidence filter that abstains on the most ASMI-fragile confident predictions retains roughly half the error of one that abstains on the same budget by predictive entropy.
  • A usable uncertainty score can be read from a single greedy response plus a few dozen masked suffix passes, with near-deterministic rankings, instead of requiring ten stochastic generations.
  • The method's domain is predictable before correctness labels are needed: it should be applied where answers are routed through provided context, and output-distribution baselines should be used for closed-book recall.
  • Because the added information lives in the confident stratum, gains appear only in targeted selective-prediction filters, not in a single global risk-coverage selector.
  • The masking response itself can serve as a label-free screen for over-robust models where the signal is uninformative.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same structural-perturbation logic could be applied to other redundant components, such as MLP sublayers or key/value groups, to map where fragility lives in different tasks.
  • A prospective test suggested by the paper's operating-envelope analysis is to measure mean token-level mutual-information magnitude before running the full estimator; a near-zero response predicts the signal will be uninformative, a rule that could be validated as a general screen.
  • The regime-graded boundary generalizes into a design principle: an uncertainty estimator that probes a specific computation inherits the locus of that computation's errors, so applicability is a design input rather than a discovered property.
  • Because entropy and mutual information are highly rank-correlated, aggregate ranking scores alone would mask the localized effect; future comparisons should report residual-based out-of-fold AUROC together with confident-stratum error reductions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes ASMI (Attention-Subnetwork Mutual Information), a training-free token-level uncertainty estimator that randomly masks attention heads at a chosen layer and measures the BALD mutual information among the resulting subnetworks, with a semantic-agreement kernel to discount surface-form variants. The central claims are that this signal is distinct from single-pass confidence and entropy, that its error-predictive information is concentrated in confident-but-fragile predictions, and that the signal is regime-graded: it helps on context-routed QA (CoQA, SQuAD, BabiQA) and reverts to or below the free MSP baseline on closed-book parametric recall (TriviaQA). The paper reports extensive experiments across four backbones against 17 baselines, including paired cluster bootstraps, a depth sweep, a residual-based distinctness test, a same-dataset open-/closed-book deconfound, a truncation-coverage deconfound, a deployment-payoff analysis, estimator-stability redraws, and a head-level causal analysis. The paper is unusually transparent about its limitations, including a mapped failure cell on Llama-2-7B/BabiQA and a candidate label-free screen that is acknowledged to be retrospective.

Significance. If the central claims hold, ASMI is a meaningful contribution: it occupies a relatively unexplored corner of uncertainty-estimation design (structural perturbation measuring path dependence), it is cheap because it scores a single greedy response with 40 masked passes, and it makes a falsifiable prediction about where the signal should and should not work. The empirical protocol is a clear strength: 17 baselines, cluster-respecting paired bootstrap, out-of-fold residualization, split-half depth-stability checks, and an honest limitations section. The main weakness is that the estimator's operationalization does not cleanly separate the identity of the ablated attention pathway from the random number of heads dropped per sample, which weakens the mechanistic interpretation of the signal. A second concern is that the representative operating depth is selected on the same benchmark data used for the headline comparisons.

major comments (3)
  1. [Methodology, Eq. (3)-(5); Appendix G] The claim near Eq. (4) that the scale reduction is "common to all samples, so it does not itself generate the inter-sample disagreement that MI_t measures" is mathematically incorrect. Each sample draws a fresh Bernoulli mask, so the number of active heads varies as Binomial(H, p); with H=32 and p=0.15, the number of kept heads has mean 27.2 and standard deviation about 2.0. Different samples therefore have different scale reductions even when the set of kept heads is otherwise irrelevant, and this count variation alone produces positive MI_t through the standard H(mean) - mean H decomposition. This is a generic-degradation component orthogonal to attention-path identity. The fidelity bound in Appendix G limits the per-sample number of dropped heads (k_max=8 at the 95th percentile) but does not control the across-sample variance in counts. The central claim of the paper is that disagreement reflects whether the prediction hinges on a specific attention pathway; as written, Eq. (5) conflates that with how many heads happened to be dropped. Please re-run the main experiments with stratified masks that hold the number of dropped heads fixed across samples (e.g., exactly round(H*p) heads per mask), or provide a quantitative demonstration that fixed-count masks yield the same MI_t rankings and the same distinctness results.
  2. [Implementation details; Appendix I; Table 13] The operating depth d=60% is selected by aggregate PRR on the same sixteen cells that are then used for all reported head-to-head comparisons and significance tests. The split-half stability analysis in Appendix I is helpful and shows that 60% is selected in 99.3% of resamples, which mitigates but does not eliminate the selection-on-test-data problem: the final point estimates and the paired-bootstrap intervals in Table 13 are computed at a depth chosen by the full data, so the reported "ties or leads" counts are not fully out-of-fold. Please provide a sensitivity analysis for the headline comparisons across the four depths, or re-state the counts as "at the aggregate-selected depth" with explicit acknowledgment that the selection used the evaluation labels.
  3. [Deployment payoff; Table 4; Appendix E] The abstract and main text state that acting on the confident-but-fragile signal "roughly halves the retained error" of a confidence filter. This is supported on BabiQA and CoQA for Qwen3-4B-base, but Appendix E reports that SQuAD shows no confident-stratum gain on that backbone, and no other backbone is tested in the deployment analysis. The claim should be qualified as holding on two of the three grounded benchmarks for one backbone, or extended to additional backbones and datasets. As written, the headline overstates the generality of the deployment payoff.
minor comments (4)
  1. [Abstract; Conclusions] The phrase "adds error-predictive information beyond single-pass confidence and entropy" should be qualified: Table 6 shows that the residual AUROC on one of the two SQuAD draws for Qwen3-4B sits at chance (0.501, CI [0.466, 0.535]) and that CoQA on Qwen3-8B also fails to exclude chance. The main text does mention these caveats, but the abstract states the claim categorically.
  2. [Figure 4] The panel labels "BabiQA×4b" and "CoQA×4b" use a lower-case 'b' that is inconsistent with the backbone notation "Qwen3-4B" used elsewhere; please standardize the capitalization.
  3. [Appendix K, Table 14] The table header contains the visible artifact "uni00AD" in "Adapt/uni00ADASMI"; this should be corrected to "Adapt-ASMI".
  4. [Throughout] No statement of code or data availability is provided. Given the paper's emphasis on reproducibility and the detailed appendices, please state whether the implementation and evaluation scripts will be released.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: ASMI is not defined in terms of the errors it predicts, and its distinctness claim is tested by out-of-fold residualization rather than by construction.

full rationale

The derivation chain is self-contained. The estimator MIt is defined as BALD mutual information over Bernoulli-masked attention subnetworks (Eqs. 3-5), using only the model's predictive distributions and a fixed semantic kernel; no error label or confidence score enters the estimator's definition. The central distinctness claim is tested by residualizing usem on MSP and single-pass entropy out of fold and measuring whether the residual still predicts errors (Table 6); this is an incremental-information test, not a restatement of the fit. The semantic kernel uses a fixed projection matrix, and the adaptive gate's hyperparameters are fixed across tasks; no parameter is fitted to the target error labels. The only tuned hyperparameter, target layer depth, is selected by aggregate PRR on the same benchmarks (Appendix I), which is a test-set selection concern rather than a circularity: the depth is not part of the estimator's definition, and the reported PRR at d=60% is not equal to the selection criterion by construction. The Bernoulli mask count variation, while a genuine construct-validity threat to the pure 'path fragility' interpretation, is a confound rather than a circular reduction, since the paper's claim that the scale reduction is 'common to all samples' is inaccurate only in expectation, not in per-sample realization. The paper contains no load-bearing self-citations and invokes no uniqueness theorem from the authors' own prior work. It also explicitly flags its own limitations, including the retrospective label-free screen, the over-robust Llama-2-7B/BabiQA cell, and the CoQA residual on Qwen3-8B that 'does not exclude chance'; these admissions support a non-circular reading of the empirical claims.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several empirical assumptions about attention redundancy and the locus of knowledge; these are grounded in cited mechanistic studies, not derived here. No new physical or ontological entities are introduced; ASMI is a statistical estimator.

free parameters (5)
  • Mask rate p = 0.15
    Chosen from a fidelity upper bound with k_max=8 (one quarter of heads), following head-pruning literature; not fitted to errors, but k_max is a heuristic.
  • Monte Carlo sample count S = 40
    Smallest S satisfying the coverage lower bound at p=0.15 for H=32 and delta=0.05; not fitted to errors.
  • Target layer depth d = 60% of layers
    Selected as representative depth by aggregate PRR over the evaluation benchmarks (Appendix I); this is selection on the test data, though split-half stable.
  • Top-K truncation K = 64
    Chosen from coverage analysis on Qwen3-4B (Appendix B); a truncation parameter, not an error fit.
  • Adaptive gate hyperparameters (tau, beta) = (0.3, 10)
    Fixed heuristics across all tasks and backbones; not tuned on errors.
assumptions (4)
  • domain assumption Multi-head attention is redundant: many heads can be pruned with minimal loss.
    Cited from Michel et al. 2019 and Voita et al. 2019; the entire masking probe relies on this.
  • domain assumption Context-routed QA answers are carried by attention paths in middle-to-late-middle layers, while parametric recall lives in MLP knowledge.
    Cited mechanistic literature (Geva et al. 2021, 2023; Meng et al. 2022; Wu et al. 2025); used to predict the regime boundary.
  • ad hoc to paper Output embedding rows of W_lm are a valid semantic similarity space for the agreement kernel.
    The semantic kernel (Eq. 10) assumes cosine similarity in the output projection matrix space approximates semantic equivalence; not independently validated.
  • ad hoc to paper Random Bernoulli masks with no rescaling yield a functional ensemble whose disagreement is a meaningful uncertainty signal.
    The core probe assumption of Eqs. 3-5; known to fail on Llama-2-7B/BabiQA where the model is over-robust.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attention-Path Fragility as an Uncertainty Signal in Large Language Models." pith.science (2026). https://pith.science/paper/CH7JY7SP

@misc{pith2026260811138,
  author       = {Pith},
  title        = {Pith review of: Attention-Path Fragility as an Uncertainty Signal in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CH7JY7SP}},
  note         = {Machine review of arXiv:2608.11138}
}
read the original abstract

We propose that a model's uncertainty about a token is reflected not only in the breadth of its output distribution but also in whether a confident prediction is \emph{fragile} under perturbation of its attention pathways. We instantiate this as ASMI (Attention-Subnetwork Mutual Information), a training-free estimator that masks attention heads and measures the BALD mutual information among the resulting subnetworks, with a semantic-agreement kernel to discount surface-form disagreement. The signal is not a restatement of output confidence: on grounded QA an out-of-fold test shows it adds error-predictive information beyond single-pass confidence and entropy, concentrated in \emph{confident-but-fragile} predictions, where acting on it roughly halves the retained error of a confidence filter. The distinctness is regime-graded, so ASMI predicts its own domain of applicability, strong where answers are routed through provided context and bounded by design where they are recalled from parametric knowledge. Sem-ASMI reads the signal from a single greedy response, without the stochastic generations the strongest baselines require, and ties or beats Semantic Entropy on ten of the twelve grounded benchmark-backbone settings. Across the same twelve settings, the best ASMI variant, typically the adaptive one reusing the ten samples already drawn for the baselines, ties or leads the strongest baseline in eight, significantly in three under a paired test. On parametric QA all variants revert to or below the zero-cost MSP baseline, exactly as predicted, and the estimates are near-deterministic across reruns. A head-level analysis shows that what tracks this boundary is not the presence of head-level fragility but whether that fragility couples to errors.

Figures

Figures reproduced from arXiv: 2608.11138 by the authors.

Figure 1
Figure 1. ASMI at a glance. Masking heads leaves a confident [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of ASMI. The prefix below the target layer [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Token-level mechanism on CoQA. (a) Joint dis￾tribution of MIt and At (Pearson r = −0.57). High MIt with low agreement is flagged, while high MIt with high agreement is benign instability that the kernel discounts. (b) Pairwise Wlm cosine similarity among the top-8 candidates at a representative benign-instability token, the surface-form cluster that drives At up. The semantically-weighted token-level score is u sem … view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The confident-but-fragile signal. (a) BabiQA error rate by MSP confidence and u sem fragility: confident-but￾fragile predictions fail 31% against 10% for confident-robust. (b) The effect is strong on grounded QA and fades to chance under parametric recall. Measured cos…
Figure 6
Figure 6. Figure 6: Coverage lower bound and fidelity upper bound in [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Error separation on BabiQA. Violins show per [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 43 canonical work pages

  1. [1]

    arXiv preprint arXiv:2307.16789 , year=

    Toolllm: Facilitating large language models to master 16000+ real-world apis , author=. arXiv preprint arXiv:2307.16789 , year=

    arXiv

  2. [2]

    arXiv preprint arXiv:2508.16153 , year=

    Memento: Fine-tuning llm agents without fine-tuning llms , author=. arXiv preprint arXiv:2508.16153 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2509.04664 , year=

    Why language models hallucinate , author=. arXiv preprint arXiv:2509.04664 , year=

    arXiv

  4. [4]

    Proceedings of ACL , pages=

    Shifting attention to relevance: Towards the predictive uncertainty quantification of free-form large language models , author=. Proceedings of ACL , pages=

    work page

  5. [5]

    arXiv preprint arXiv:2207.05221 , year=

    Language models (mostly) know what they know , author=. arXiv preprint arXiv:2207.05221 , year=

    arXiv

  6. [6]

    Proceedings of EMNLP , pages=

    Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models , author=. Proceedings of EMNLP , pages=

    work page

  7. [7]

    arXiv preprint arXiv:2306.13063 , year=

    Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms , author=. arXiv preprint arXiv:2306.13063 , year=

    arXiv

  8. [8]

    Proceedings of EMNLP: System Demonstrations , pages=

    LM-polygraph: Uncertainty estimation for language models , author=. Proceedings of EMNLP: System Demonstrations , pages=

    work page

Show all 80 references
  1. [9]

    Proceedings of ACL , pages=

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned , author=. Proceedings of ACL , pages=

  2. [10]

    Advances in Neural Information Processing Systems , volume=

    Are sixteen heads really better than one? , author=. Advances in Neural Information Processing Systems , volume=

  3. [11]

    Advances in Neural Information Processing Systems , volume=

    Attention is all you need , author=. Advances in Neural Information Processing Systems , volume=

  4. [12]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  5. [13]

    arXiv preprint arXiv:2002.07650 , year=

    Uncertainty estimation in autoregressive structured prediction , author=. arXiv preprint arXiv:2002.07650 , year=

    2002 arXiv

  6. [14]

    arXiv preprint arXiv:2302.09664 , year=

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation , author=. arXiv preprint arXiv:2302.09664 , year=

    arXiv

  7. [15]

    Advances in Neural Information Processing Systems , volume=

    Kernel language entropy: Fine-grained uncertainty quantification for llms from semantic similarities , author=. Advances in Neural Information Processing Systems , volume=

  8. [16]

    arXiv preprint arXiv:2402.03744 , year=

    INSIDE: LLMs' internal states retain the power of hallucination detection , author=. arXiv preprint arXiv:2402.03744 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2406.15927 , year=

    Semantic entropy probes: Robust and cheap hallucination detection in llms , author=. arXiv preprint arXiv:2406.15927 , year=

    arXiv

  10. [18]

    Findings of EMNLP , pages=

    The internal state of an LLM knows when it's lying , author=. Findings of EMNLP , pages=

  11. [19]

    Transactions of the Association for Computational Linguistics , volume=

    Benchmarking uncertainty quantification methods for large language models with lm-polygraph , author=. Transactions of the Association for Computational Linguistics , volume=

  12. [20]

    Vashurin, Roman and Goloburda, Maiya and Nakov, Preslav and Shelmanov, Artem and Panov, Maxim , journal=

  13. [21]

    arXiv preprint arXiv:2510.20460 , year=

    Systematic Evaluation of Uncertainty Estimation Methods in Large Language Models , author=. arXiv preprint arXiv:2510.20460 , year=

  14. [22]

    arXiv preprint arXiv:2412.15176 , year=

    Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure , author=. arXiv preprint arXiv:2412.15176 , year=

    arXiv

  15. [23]

    arXiv preprint arXiv:1112.5745 , year=

    Bayesian active learning for classification and preference learning , author=. arXiv preprint arXiv:1112.5745 , year=

    arXiv

  16. [24]

    Findings of ACL , pages=

    Fact-checking the output of large language models via token-level uncertainty quantification , author=. Findings of ACL , pages=

  17. [25]

    arXiv preprint arXiv:2502.16820 , year=

    Uncertainty quantification of large language models through multi-dimensional responses , author=. arXiv preprint arXiv:2502.16820 , year=

    arXiv

  18. [26]

    Proceedings of EMNLP , pages=

    Luq: Long-text uncertainty quantification for llms , author=. Proceedings of EMNLP , pages=

  19. [27]

    Findings of EMNLP , pages=

    GENUINE: Graph Enhanced Multi-level Uncertainty Estimation for Large Language Models , author=. Findings of EMNLP , pages=

  20. [28]

    Nature , volume=

    Detecting hallucinations in large language models using semantic entropy , author=. Nature , volume=

  21. [29]

    arXiv preprint arXiv:2505.11737 , year=

    TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoning , author=. arXiv preprint arXiv:2505.11737 , year=

    arXiv

  22. [30]

    arXiv preprint arXiv:2505.20045 , year=

    Uncertainty-aware attention heads: Efficient unsupervised uncertainty quantification for LLMs , author=. arXiv preprint arXiv:2505.20045 , year=

    arXiv

  23. [31]

    Transactions of the Association for Computational Linguistics , volume=

    Coqa: A conversational question answering challenge , author=. Transactions of the Association for Computational Linguistics , volume=

  24. [32]

    arXiv preprint arXiv:1502.05698 , year=

    Towards ai-complete question answering: A set of prerequisite toy tasks , author=. arXiv preprint arXiv:1502.05698 , year=

    arXiv

  25. [33]

    Proceedings of EMNLP , pages=

    HotpotQA: A dataset for diverse, explainable multi-hop question answering , author=. Proceedings of EMNLP , pages=

  26. [34]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    arXiv

  27. [35]

    Proceedings of ICLR , year=

    Measuring Massive Multitask Language Understanding , author=. Proceedings of ICLR , year=

  28. [36]

    arXiv preprint arXiv:1808.08745 , year=

    Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization , author=. arXiv preprint arXiv:1808.08745 , year=

    arXiv

  29. [37]

    Proceedings of ACL , pages=

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension , author=. Proceedings of ACL , pages=

  30. [38]

    Proceedings of ACL , pages=

    Truthfulqa: Measuring how models mimic human falsehoods , author=. Proceedings of ACL , pages=

  31. [39]

    Proceedings of the 2nd Workshop on New Frontiers in Summarization , pages=

    SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization , author=. Proceedings of the 2nd Workshop on New Frontiers in Summarization , pages=

  32. [40]

    Proceedings of ACL , pages=

    This email could save your life: Introducing the task of email subject line generation , author=. Proceedings of ACL , pages=

  33. [41]

    Proceedings of ACL , pages=

    AlignScore: Evaluating factual consistency with a unified alignment function , author=. Proceedings of ACL , pages=

  34. [42]

    arXiv preprint arXiv:2406.15786 , year=

    What matters in transformers? not all attention is needed , author=. arXiv preprint arXiv:2406.15786 , year=

    arXiv

  35. [43]

    arXiv preprint arXiv:2310.06825 , year=

    Mistral 7B , author=. arXiv preprint arXiv:2310.06825 , year=

    arXiv

  36. [44]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  37. [45]

    Proceedings of EMNLP , pages=

    Transformer feed-forward layers are key-value memories , author=. Proceedings of EMNLP , pages=

  38. [46]

    Proceedings of ACL , pages=

    Knowledge neurons in pretrained transformers , author=. Proceedings of ACL , pages=

  39. [47]

    Advances in Neural Information Processing Systems , volume=

    Locating and editing factual associations in gpt , author=. Advances in Neural Information Processing Systems , volume=

  40. [48]

    Proceedings of EMNLP , pages=

    Dissecting recall of factual associations in auto-regressive language models , author=. Proceedings of EMNLP , pages=

  41. [49]

    Advances in Neural Information Processing Systems , volume=

    Investigating gender bias in language models using causal mediation analysis , author=. Advances in Neural Information Processing Systems , volume=

  42. [50]

    arXiv preprint arXiv:2211.00593 , year=

    Interpretability in the wild: a circuit for indirect object identification in gpt-2 small , author=. arXiv preprint arXiv:2211.00593 , year=

    arXiv

  43. [51]

    Proceedings of ICLR , year=

    Retrieval head mechanistically explains long-context factuality , author=. Proceedings of ICLR , year=

  44. [52]

    arXiv preprint arXiv:2307.15771 , year=

    The hydra effect: Emergent self-repair in language model computations , author=. arXiv preprint arXiv:2307.15771 , year=

    arXiv

  45. [53]

    Proceedings of ICML , pages=

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning , author=. Proceedings of ICML , pages=

  46. [54]

    arXiv preprint arXiv:1606.05250 , year=

    SQuAD: 100,000+ Questions for Machine Comprehension of Text , author=. arXiv preprint arXiv:1606.05250 , year=

    arXiv

  47. [55]

    arXiv preprint arXiv:2502.18581 , year=

    Scalable Best-of-N Selection for Large Language Models via Self-Certainty , author=. arXiv preprint arXiv:2502.18581 , year=

  48. [56]

    Proceedings of the First Workshop on NLP for Conversational AI , pages=

    Relevant and Informative Response Generation using Pointwise Mutual Information , author=. Proceedings of the First Workshop on NLP for Conversational AI , pages=

  49. [57]

    Proceedings of EMNLP , pages=

    Mutual Information Alleviates Hallucinations in Abstractive Summarization , author=. Proceedings of EMNLP , pages=

  50. [58]

    Proceedings of EMNLP , pages=

    Characterizing mechanisms for factual recall in language models , author=. Proceedings of EMNLP , pages=

  51. [59]

    Proceedings of ICLR , year=

    Function vectors in large language models , author=. Proceedings of ICLR , year=

  52. [60]

    Proceedings of ACL , pages=

    On faithfulness and factuality in abstractive summarization , author=. Proceedings of ACL , pages=

  53. [61]

    ACM Transactions on Information Systems , volume=

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions , author=. ACM Transactions on Information Systems , volume=

  54. [62]

    Proceedings of UAI , pages=

    Understanding Measures of Uncertainty for Adversarial Example Detection , author=. Proceedings of UAI , pages=

  55. [63]

    Proceedings of EMNLP , pages=

    Entity-based knowledge conflicts in question answering , author=. Proceedings of EMNLP , pages=

  56. [64]

    Proceedings of ACL , pages=

    When not to trust language models: Investigating effectiveness of parametric and non-parametric memories , author=. Proceedings of ACL , pages=

  57. [65]

    Findings of ACL , pages=

    Cutting off the head ends the conflict: A mechanism for interpreting and mitigating knowledge conflicts in language models , author=. Findings of ACL , pages=

  58. [66]

    Advances in Neural Information Processing Systems , volume=

    Selective classification for deep neural networks , author=. Advances in Neural Information Processing Systems , volume=

  59. [67]

    Proceedings of ACL , pages=

    Selective question answering under domain shift , author=. Proceedings of ACL , pages=

  60. [68]

    Proceedings of UAI , pages=

    Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? , author=. Proceedings of UAI , pages=

  61. [69]

    Proceedings of ICML , pages=

    On calibration of modern neural networks , author=. Proceedings of ICML , pages=

  62. [70]

    Transactions of the Association for Computational Linguistics , volume=

    How can we know when language models know? on the calibration of language models for question answering , author=. Transactions of the Association for Computational Linguistics , volume=

  63. [71]

    Proceedings of ICLR , year=

    On the role of attention heads in large language model safety , author=. Proceedings of ICLR , year=

  64. [72]

    Proceedings of EMNLP , pages=

    Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps , author=. Proceedings of EMNLP , pages=

  65. [73]

    arXiv preprint arXiv:2209.11895 , year=

    In-context learning and induction heads , author=. arXiv preprint arXiv:2209.11895 , year=

    arXiv

  66. [74]

    Proceedings of ACL , pages=

    Competition of mechanisms: Tracing how language models handle facts and counterfactuals , author=. Proceedings of ACL , pages=

  67. [75]

    Proceedings of ICLR , year=

    ReDeEP: Detecting hallucination in retrieval-augmented generation via mechanistic interpretability , author=. Proceedings of ICLR , year=

  68. [76]

    Advances in Neural Information Processing Systems , volume=

    To believe or not to believe your llm: Iterative prompting for estimating epistemic uncertainty , author=. Advances in Neural Information Processing Systems , volume=

  69. [77]

    Proceedings of ICLR , year=

    Attention satisfies: A constraint-satisfaction lens on factual errors of language models , author=. Proceedings of ICLR , year=

  70. [78]

    arXiv preprint arXiv:2410.02707 , year=

    LLMs know more than they show: On the intrinsic representation of LLM hallucinations , author=. arXiv preprint arXiv:2410.02707 , year=

    arXiv

  71. [79]

    Proceedings of EMNLP , pages=

    Knowledge conflicts for LLMs: A survey , author=. Proceedings of EMNLP , pages=

  72. [80]

    Proceedings of NAACL , pages=

    From language modeling to instruction following: Understanding the behavior shift in LLMs after instruction tuning , author=. Proceedings of NAACL , pages=

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.