{"id":"16e5e920-f41a-4e13-aa1c-2e0a50324dff","arxiv_id":"2608.11138","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ASMI, an attention-head masking mutual information score, adds error-predictive power beyond confidence and entropy on grounded QA and degrades to chance on parametric recall, matching its design prediction.","lead":"This paper introduces a training-free way to estimate an LLM's uncertainty by randomly switching off attention heads in one layer and measuring how much the answer changes. The fragility signal catches confident-but-wrong answers on reading-based QA, and it works from a single greedy response, making it cheaper than sampling-based alternatives.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bernoulli mask count variation confounds ASMI's fragility measure; the claim that scale reduction is common to all samples is false and untested.","rationale":"The central claim is that ASMI measures whether a prediction hinges on a specific attention pathway, so MI_t must reflect which heads are ablated rather than merely how many. Eq. 4 uses independent Bernoulli masks, making the number of active heads vary across samples; the text's assertion that the scale reduction is common to all samples is only true in expectation. This injects a generic-degradation component into MI_t that is orthogonal to path fragility, directly threatening the construct validity of the headline signal.\n\nThe concern is load-bearing because the distinctness and confident-stratum results (Table 6, Figure 5) are produced by the confounded estimator. The causal probe in Appendix F does show that ablating the most critical heads flips high-MI tokens, which supports path-specificity, but it does not quantify what fraction of the ASMI score is carried by count variation rather than head identity. Without a matched-count control, the abstract's mechanistic language is not fully supported.\n\nAlternative concerns are weaker. Depth selection is defended by a split-half stability test (60% selected in 99.3% of splits), and the main significance claims are partially protected by a pre-specified Semantic Entropy comparison. The Bernoulli count confound, by contrast, is an internal inconsistency in the method's own justification, and it is cheap to test.\n\nThe proposed exact-k control is decisive: if the results survive, the concern is refuted; if they degrade, the estimator's mechanistic claims and perhaps the regime boundary need revision. Because this is an additional condition rather than a demonstrated fatal flaw, the conditional verdict stands, with the matched-count control added to the list of requirements.","tokens_in":31256,"tokens_out":10060,"duration_ms":110305,"concrete_test":"Recompute all grounded and parametric results with a matched-count masking scheme: for each sample, drop exactly k = round(pH) = 5 heads chosen uniformly without replacement (same S=40, same target layer, no rescaling), so every subnetwork has the same scale reduction. Compare MI_t distributions, Table 1 PRR, Table 6 residual AUROC, and Figure 5 confident-stratum error rates on Qwen3-4B CoQA/BabiQA/TriviaQA. If the matched-count results lie within bootstrap error of the Bernoulli results, the count-variation confound is negligible; if they degrade (e.g., residual AUROC drops toward 0.5 or PRR drops by more than 0.02), the reported fragility signal is partly an artifact of variable ensemble size and the estimator's mechanistic claims require revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The estimator's identity as a measure of attention-path fragility is confounded by ensemble-size variation. In Eq. 4 each sample draws an independent Bernoulli mask, so the number of active heads varies across samples (Binomial(32,0.85): mean 27.2, sd about 2.0). The paper states that the scale reduction is 'common to all samples, so it does not itself generate the inter-sample disagreement that MI_t measures.' This is incorrect: the expected reduction is common, but each sample has a different reduction, so MI_t in Eq. 5 includes disagreement driven by how many heads are dropped, not only by which heads are dropped. This is a generic-degradation component orthogonal to path fragility: even a token supported by fully redundant heads will show nonzero MI solely from random count variation in the attention-output norm. The fidelity bound in Appendix G limits the count to at most about 8 heads dropped at p=0.15, but a 7% coefficient of variation in the number of kept heads remains. The central claim requires that disagreement reflects the identity of the ablated pathway; the current implementation does not separate identity from count. The failure on Llama-2-7B/BabiQA (near-zero MI) shows that when the model is too robust, even this confounded signal vanishes, but it does not establish that the signal is otherwise clean.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ASMI (Attention-Subnetwork Mutual Information), a training-free token-level uncertainty estimator that randomly masks attention heads at a chosen layer and measures the BALD mutual information among the resulting subnetworks, with a semantic-agreement kernel to discount surface-form variants. The central claims are that this signal is distinct from single-pass confidence and entropy, that its error-predictive information is concentrated in confident-but-fragile predictions, and that the signal is regime-graded: it helps on context-routed QA (CoQA, SQuAD, BabiQA) and reverts to or below the free MSP baseline on closed-book parametric recall (TriviaQA). The paper reports extensive experiments across four backbones against 17 baselines, including paired cluster bootstraps, a depth sweep, a residual-based distinctness test, a same-dataset open-/closed-book deconfound, a truncation-coverage deconfound, a deployment-payoff analysis, estimator-stability redraws, and a head-level causal analysis. The paper is unusually transparent about its limitations, including a mapped failure cell on Llama-2-7B/BabiQA and a candidate label-free screen that is acknowledged to be retrospective.","tokens_in":31583,"tokens_out":5859,"duration_ms":68087,"significance":"If the central claims hold, ASMI is a meaningful contribution: it occupies a relatively unexplored corner of uncertainty-estimation design (structural perturbation measuring path dependence), it is cheap because it scores a single greedy response with 40 masked passes, and it makes a falsifiable prediction about where the signal should and should not work. The empirical protocol is a clear strength: 17 baselines, cluster-respecting paired bootstrap, out-of-fold residualization, split-half depth-stability checks, and an honest limitations section. The main weakness is that the estimator's operationalization does not cleanly separate the identity of the ablated attention pathway from the random number of heads dropped per sample, which weakens the mechanistic interpretation of the signal. A second concern is that the representative operating depth is selected on the same benchmark data used for the headline comparisons.","major_comments":[{"comment":"The claim near Eq. (4) that the scale reduction is \"common to all samples, so it does not itself generate the inter-sample disagreement that MI_t measures\" is mathematically incorrect. Each sample draws a fresh Bernoulli mask, so the number of active heads varies as Binomial(H, p); with H=32 and p=0.15, the number of kept heads has mean 27.2 and standard deviation about 2.0. Different samples therefore have different scale reductions even when the set of kept heads is otherwise irrelevant, and this count variation alone produces positive MI_t through the standard H(mean) - mean H decomposition. This is a generic-degradation component orthogonal to attention-path identity. The fidelity bound in Appendix G limits the per-sample number of dropped heads (k_max=8 at the 95th percentile) but does not control the across-sample variance in counts. The central claim of the paper is that disagreement reflects whether the prediction hinges on a specific attention pathway; as written, Eq. (5) conflates that with how many heads happened to be dropped. Please re-run the main experiments with stratified masks that hold the number of dropped heads fixed across samples (e.g., exactly round(H*p) heads per mask), or provide a quantitative demonstration that fixed-count masks yield the same MI_t rankings and the same distinctness results.","section":"Methodology, Eq. (3)-(5); Appendix G"},{"comment":"The operating depth d=60% is selected by aggregate PRR on the same sixteen cells that are then used for all reported head-to-head comparisons and significance tests. The split-half stability analysis in Appendix I is helpful and shows that 60% is selected in 99.3% of resamples, which mitigates but does not eliminate the selection-on-test-data problem: the final point estimates and the paired-bootstrap intervals in Table 13 are computed at a depth chosen by the full data, so the reported \"ties or leads\" counts are not fully out-of-fold. Please provide a sensitivity analysis for the headline comparisons across the four depths, or re-state the counts as \"at the aggregate-selected depth\" with explicit acknowledgment that the selection used the evaluation labels.","section":"Implementation details; Appendix I; Table 13"},{"comment":"The abstract and main text state that acting on the confident-but-fragile signal \"roughly halves the retained error\" of a confidence filter. This is supported on BabiQA and CoQA for Qwen3-4B-base, but Appendix E reports that SQuAD shows no confident-stratum gain on that backbone, and no other backbone is tested in the deployment analysis. The claim should be qualified as holding on two of the three grounded benchmarks for one backbone, or extended to additional backbones and datasets. As written, the headline overstates the generality of the deployment payoff.","section":"Deployment payoff; Table 4; Appendix E"}],"minor_comments":[{"comment":"The phrase \"adds error-predictive information beyond single-pass confidence and entropy\" should be qualified: Table 6 shows that the residual AUROC on one of the two SQuAD draws for Qwen3-4B sits at chance (0.501, CI [0.466, 0.535]) and that CoQA on Qwen3-8B also fails to exclude chance. The main text does mention these caveats, but the abstract states the claim categorically.","section":"Abstract; Conclusions"},{"comment":"The panel labels \"BabiQA×4b\" and \"CoQA×4b\" use a lower-case 'b' that is inconsistent with the backbone notation \"Qwen3-4B\" used elsewhere; please standardize the capitalization.","section":"Figure 4"},{"comment":"The table header contains the visible artifact \"uni00AD\" in \"Adapt/uni00ADASMI\"; this should be corrected to \"Adapt-ASMI\".","section":"Appendix K, Table 14"},{"comment":"No statement of code or data availability is provided. Given the paper's emphasis on reproducibility and the detailed appendices, please state whether the implementation and evaluation scripts will be released.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is empirically careful and unusually honest, but the Bernoulli count-variation confound is a genuine load-bearing issue for the mechanistic interpretation of ASMI. I would be willing to accept a revised version that re-runs the main experiments with fixed-count masks or otherwise demonstrates that the MI signal is not driven by the random number of dropped heads. The depth-selection concern is secondary and could be addressed with a sensitivity analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Three things. First, the ASMI estimator is a genuinely new entry in the perturbation-based uncertainty family, and the empirical work is unusually careful: 17 baselines, cluster-respecting paired bootstrap, ablations, deconfounds, and a limitations section that names real caveats. The consistent grounded-versus-parametric boundary is a good designed contrast, and the deployment claim is scoped to the confident stratum rather than oversold. Credit is due for the mask-batched implementation and the split-half depth-selection stability check.\n\nSecond, the central interpretation does not survive the stress-test. The Bernoulli masks at Eq. 4 leave the number of active heads varying across samples (Binomial(32,0.85), sd about 2 heads). The paper says the scale reduction is \"common to all samples, so it does not itself generate the inter-sample disagreement\" — that is wrong. The expected reduction is common; each draw is not. MI_t in Eq. 5 therefore mixes disagreement about which heads are ablated with disagreement about how many heads are ablated. Even a token supported by fully redundant heads will show nonzero MI from count variation alone, since the logit scale shifts with the number of active heads and no rescaling is applied. This is a generic-degradation component, orthogonal to path fragility. The fidelity bound in Appendix G caps the count at about 8 dropped heads, but a 7% coefficient of variation in kept-head count remains, and the paper never separates identity from count. The Llama-2-7B/BabiQA failure shows the signal vanishes when the model is over-robust, but it does not demonstrate that the signal is otherwise clean.\n\nThird, the other soft spots are real but smaller. The operating depth d=60% is selected on the evaluation benchmarks, even though the split-half analysis suggests the choice is stable. One SQuAD draw on Qwen3-4B is borderline at the estimator's own Monte Carlo resolution. No code or data are released, which matters for a method whose value is empirical. None of these by themselves change the verdict, but they compound the main confound.\n\nWho is this for? Researchers in selective prediction and RAG hallucination detection will find the empirical pattern useful. The paper deserves a serious referee, and I would send it to the main NLP venues. But the authors should be asked to address the mask-count confound, for example by comparing to a fixed-active-count mask ensemble, and to release code and data. The empirical claim about confident-but-fragile errors may survive the fix; the mechanism claim currently does not.","headline":"Careful, honest empirical study of a new perturbation-based uncertainty estimator, but the central interpretation is confounded by per-sample variation in the number of masked heads.","tokens_in":32063,"tokens_out":2000,"would_cite":false,"duration_ms":23493,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLM uncertainty is visible in whether a confident prediction collapses when attention heads are masked.","keywords":["uncertainty estimation","attention-head masking","mutual information","confident-but-fragile predictions","grounded question answering","selective prediction","large language models","semantic agreement"],"falsifier":"Run the paper's matched-perturbation control on a fresh grounded-QA benchmark: mask the MLP sublayer at the same target layer instead of attention heads with the same budget, and compare rankings; if the non-attention perturbation reproduces ASMI's error-detection AUROC, then attention-path fragility specifically is not the carrier and the central mechanism is falsified.","tokens_in":31076,"feed_emoji":"🎯","tokens_out":7645,"duration_ms":68160,"temperature":0.7,"pith_summary":"This paper proposes that an LLM's uncertainty about a next token shows up not only in how spread out its output probabilities are, but in whether a confident prediction is fragile: whether it depends on one specific attention pathway among many redundant ones. To measure this, the authors introduce ASMI, a training-free estimator that repeatedly masks random attention heads in one transformer layer and measures how much the resulting predictive distributions disagree, discounting surface-form variation with a semantic-similarity kernel. The central empirical claim is that ASMI is not a proxy for ordinary confidence: out-of-fold residual tests on grounded question answering show it adds error-predictive information beyond confidence and entropy, concentrated in confident-but-fragile predictions, where using it roughly halves the retained error of a confidence filter. The signal is regime-graded by design: it helps when answers are routed through provided context and reverts to or below a zero-cost baseline when answers are recalled from parametric memory, matching the paper's stated prediction.","feed_headline":"Fragile attention paths flag LLM errors confidence misses","feed_subtitle":"A training-free score reads uncertainty from a single greedy answer and roughly halves retained error in grounded QA.","key_machinery":"The central object is ASMI (Attention-Subnetwork Mutual Information), a training-free token-level score. At a chosen transformer layer, each of $S$ Monte Carlo samples draws a Bernoulli mask over attention heads and recomputes only the suffix of the network; the disagreement among the resulting predictive distributions is measured by the BALD mutual information $\\mathrm{MI}_t = H(\\bar{p}_t) - \\tfrac{1}{S}\\sum_s H(p_t^{(s)})$, approximated over a top-$K$ support with a tail bucket. Sem-ASMI multiplies this by $1 - A_t$, where $A_t$ is the pairwise semantic agreement of the top-$K$ distributions computed from the output-projection embedding, and Adapt-ASMI gates that factor by the diversity of $N$ sampled responses. The mechanism being probed is redundancy among attention heads: because many heads are prunable, an answer that survives random masking is genuinely supported, while an answer that collapses was carried by one critical path.","core_discovery":"A model's uncertainty about a token is reflected in whether the prediction is fragile under perturbation of its attention pathways, and this fragility carries error information that single-pass confidence and entropy miss. Under random Bernoulli masking of attention-head outputs at a target layer, the mutual information between the masked subnetwork and the next-token distribution, in the BALD form, measures this fragility; a semantic-agreement kernel multiplies the mutual information by one minus agreement to discount surface-form variation. The paper shows, on grounded QA benchmarks across four pretrained transformers, that the residual of this score after regressing out confidence and entropy still detects errors out of fold, and that among predictions the model itself rates as confident, abstaining those the score flags as fragile roughly halves retained error compared with filtering by entropy. On closed-book trivia QA the same score drops to or below the zero-cost maximum-sequence-probability baseline, and the paper interprets this boundary as design: attention-path fragility is informative where correctness depends on routing context through specific heads, and uninformative where uncertainty originates in parametric recall.","pith_inferences":["The paper leaves implicit that the same structural-perturbation logic could be applied to other redundant components, such as MLP sublayers or key/value groups, to map where fragility lives in different tasks.","A prospective test suggested by the paper's operating-envelope analysis is to measure mean token-level mutual-information magnitude before running the full estimator; a near-zero response predicts the signal will be uninformative, a rule that could be validated as a general screen.","The regime-graded boundary generalizes into a design principle: an uncertainty estimator that probes a specific computation inherits the locus of that computation's errors, so applicability is a design input rather than a discovered property.","Because entropy and mutual information are highly rank-correlated, aggregate ranking scores alone would mask the localized effect; future comparisons should report residual-based out-of-fold AUROC together with confident-stratum error reductions."],"forward_implications":["On retrieval-grounded QA, a confidence filter that abstains on the most ASMI-fragile confident predictions retains roughly half the error of one that abstains on the same budget by predictive entropy.","A usable uncertainty score can be read from a single greedy response plus a few dozen masked suffix passes, with near-deterministic rankings, instead of requiring ten stochastic generations.","The method's domain is predictable before correctness labels are needed: it should be applied where answers are routed through provided context, and output-distribution baselines should be used for closed-book recall.","Because the added information lives in the confident stratum, gains appear only in targeted selective-prediction filters, not in a single global risk-coverage selector.","The masking response itself can serve as a label-free screen for over-robust models where the signal is uninformative."],"supporting_citations":[{"why":"This study establishes that many attention heads can be pruned with minimal loss, supplying the redundancy premise behind masking as a probe.","marker":"Michel, Levy, and Neubig 2019"},{"why":"This study shows that specialized heads do the heavy lifting while the rest are prunable, grounding the fragile-versus-redundant distinction the estimator reads.","marker":"Voita et al. 2019"},{"why":"This work supplies the BALD mutual-information decomposition used to define the token-level disagreement score.","marker":"Houlsby et al. 2011"},{"why":"This work defines Semantic Entropy, the primary sampling-based baseline that the sampling-free Sem-ASMI is compared against.","marker":"Kuhn, Gal, and Farquhar 2023"},{"why":"This work provides the information-based baselines and the Prediction Rejection Ratio metric used to score all methods.","marker":"Malinin and Gales 2020"},{"why":"This work supplies SAR and the temperature-0.5 ten-sample protocol that the sampling baselines and the adaptive variant reuse.","marker":"Duan et al. 2024"},{"why":"This work provides AlignScore, the entailment-based quality metric that labels correctness of generated answers.","marker":"Zha et al. 2023"},{"why":"This work locates factual associations in feed-forward layers, supporting the paper's explanation for why the signal fades under parametric recall.","marker":"Geva et al. 2021"}],"fun_headline_variants":["Attention fragility spots errors confidence misses","Training-free attention signal halves grounded QA errors","Masked-attention mutual info flags confident LLM mistakes","Fragile paths, not confidence, predict LLM errors","Attention perturbation reveals LLM uncertainty that entropy misses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole signal rests on treating random head masking as a probe of genuine path fragility rather than generic degradation; when a model is too robust for the mask to move its outputs, the score carries no information, as the paper reports on one of its grounded cells.","fun_headline_variants_meta":{"raw":{"variants":["Attention fragility spots errors confidence misses","Training-free attention signal halves grounded QA errors","Masked-attention mutual info flags confident LLM mistakes","Fragile paths, not confidence, predict LLM errors","Attention perturbation reveals LLM uncertainty that entropy misses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1358,"prompt_tokens":1057,"completion_tokens":301,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":229}},"tokens_in":673,"tokens_out":301,"duration_ms":3608,"temperature":1.0,"reasoning_tokens":229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:24:54.192401+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's matched-perturbation control on a fresh grounded-QA benchmark: mask the MLP sublayer at the same target layer instead of attention heads with the same budget, and compare rankings; if the non-attention perturbation reproduces ASMI's error-detection AUROC, then attention-path fragility specifically is not the carrier and the central mechanism is falsified.","supporting_citations":[],"review_version":1}