{"id":"c7440888-cfc9-4100-be73-d03706c5b5bc","arxiv_id":"2506.18628","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AggTruth uses four ways of aggregating attention scores over the retrieved passage to train a logistic regression detector, achieving stable cross-task AUROC across four LLMs.","lead":"AggTruth detects when an LLM hallucinates while answering using a retrieved passage by reading the model's internal attention to that passage. It reports stable cross-task accuracy across four LLMs, which matters for building safer retrieval-augmented systems.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported per-model results are selected on test-set Gap values (Tables 5–6), so the headline 'high and stable' AUROCs may be optimistically biased; without a validation-based selection protocol the central outperformance claim is not securely established.","rationale":"I read the paper as attempting to establish an online, windowed, attention-only hallucination detector that generalizes across QA and summarization tasks and across four LLMs. The core aggregation idea is clearly motivated, and the token-alignment discussion in Section 3.3 is a thoughtful and non-obvious detail. The most load-bearing weakness is not the internal logic of the features but the experimental evaluation: the paper's own reporting convention in Tables 5 and 6 selects configurations by test-set Gap, which makes the headline numbers optimistically biased. The reader flagged this in the rationale but did not place it as the weakest assumption; the reader's weakest_assumption focuses on GPT-4o labels, which is also a real concern. Both are genuine, but selection-on-test is more directly tied to the claimed outperformance and can be checked computationally without new annotation. I therefore keep the verdict at CONDITIONAL and recommend requiring a validation-based selection protocol and artifact release before the claim 'outperforms the current SOTA' is accepted at face value.","tokens_in":11338,"tokens_out":3890,"duration_ms":49141,"concrete_test":"Re-run the pipeline exactly as in Sections 4–5, but choose the aggregation method, selector, and head percentage on 5-fold validation splits (or on a pre-registered rule), then evaluate once on held-out test sets. If the best-on-test configurations from Tables 5–6 are not among the validation-selected configurations, or if their held-out Gap values increase by more than a small margin, the reported 'stable outperformance' is selection bias rather than a robust property of AggTruth.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 7 ('high and stable results across all examined datasets and various LLMs') rests on Tables 5 and 6, but these tables report 'best obtained results w.r.t. Gap value,' where Gap (Eq. 5) is computed on the three test sets. This is selection-on-test: multiple AggTruth variants, selectors (Center, Random, Random+, Lasso, Spearman), and head percentages are explored, and the configuration with the lowest test Gap is then presented as the method's performance. That procedure inflates expected AUROC because the test set has already influenced model/variant choice. Table 5 illustrates the magnitude: for Llama-2 QA→SUM, Random+3,3 lowers Gap from 6.836 to 2.048 relative to no selection; for Gemma-2 SUM→QA, Spearman0.1 lowers it from 8.642 to 5.901. No multiple-comparison correction or validation-based selector selection is reported. A second, compounding issue is that GPT-4o token labels are used as ground truth, with only 75 human samples (Kappa 0.7); if GPT-4o labels are biased, the classifier learns GPT-4o's notion of hallucination. Either issue alone prevents the headline claim from being taken at face value.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AggTruth, an online, windowed hallucination detector for RAG settings that uses attention scores restricted to the provided passage, aggregated via four techniques (Sum, CosSim, Entropy, JS-Div). A Logistic Regression classifier is trained on token-level labels produced by GPT-4o, with features optionally selected by several head-selection methods (Center, Random/Random+, Lasso, Spearman). The method is evaluated on QA and summarization datasets across four LLMs, comparing against Lookback Lens and hidden-state baselines in same-task and cross-task settings. The authors report that AggTruth achieves high and stable AUROC values and outperforms baselines in multiple scenarios, and that head selection can improve performance while reducing the number of heads.","tokens_in":11581,"tokens_out":2723,"duration_ms":27846,"significance":"If the reported results hold, AggTruth offers a practical, interpretable, and computationally light approach to detecting contextual hallucinations during decoding, with the passage-only attention aggregation being a simple and reusable feature. The strengths of the paper include evaluation across four LLMs and four datasets, comparison against a recent SOTA (Lookback Lens) and hidden-state baselines, and a systematic exploration of feature/head selection. However, the central claim of 'high and stable results' rests on two assumptions that need tightening: the labels are generated by GPT-4o with only a small human check, and the reported configurations are selected based on test-set Gap values, which risks optimistic bias. The paper's contribution is valuable, but these issues must be addressed before the headline claim is secure.","major_comments":[{"comment":"The central outperformance claim in Section 7 is based on Tables 5 and 6, which report 'best obtained results w.r.t. Gap value.' Since Gap (Eq. 5) is computed on the three test sets, the selection of the aggregation variant, selector, and head percentage is performed using test-set performance. This is selection on the test set, which inflates expected AUROC. For example, in Table 5, for Llama-2 QA→SUM, Random+3,3 reduces Gap from 6.836 to 2.048 relative to no selection; for Gemma-2 SUM→QA, Spearman0.1 reduces Gap from 8.642 to 5.901. No validation-based selection protocol or multiple-comparison correction is described. The paper should either fix the configuration using only training/validation data before computing test AUROC, or use nested cross-validation, and then report the resulting test numbers as the primary results.","section":"Section 5, Eq. (5), Table 5, Table 6"},{"comment":"All token-level labels are produced by GPT-4o, with human agreement checked on only 75 samples (Cohen's Kappa 0.7). If GPT-4o's judgments are biased or noisy, the classifier learns to predict GPT-4o's notion of hallucination rather than true unfaithfulness, and every reported AUROC inherits this label assumption. The paper should provide evidence that the GPT-4o labels are reliable on a larger human-annotated sample, or use human labels as ground truth for at least one full dataset to confirm the ranking of methods.","section":"Section 4.1"}],"minor_comments":[{"comment":"The description of the Center selector is ambiguous: 'keep the top r/2 fraction of heads with the highest ratio and r/2 fraction with the lowest ratio' should clarify whether r is the total fraction of selected heads and how ties are broken.","section":"Section 3.2"},{"comment":"The column labeled 'truthfulness evaluation results' reports percentages that look like hallucination rates, but the caption does not state this explicitly. Please clarify whether higher percentages mean more hallucinations or more truthful responses.","section":"Table 1"},{"comment":"The caption says 'best obtained hidden states-based method w.r.t. Gap value,' which again implies selection on the test set; this should be reworded to avoid reinforcing the impression of test-set selection in the main results.","section":"Figure 4 caption"},{"comment":"The windowing procedure is described as 'overlapping chunks of size 8, sliding token by token,' but the degree of overlap and the label assignment for a chunk that contains both hallucinated and non-hallucinated tokens are not fully specified; please define this precisely.","section":"Section 4, Experimental Setup"},{"comment":"There are numerous formatting typos (e.g., 'T able 1', 'AggT ruth' in Table 5, 'AUCROC' instead of 'AUROC') that should be corrected in a revision.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the proposed method is simple and plausible, but the test-set-based selection of configurations is a serious methodological concern that undermines the strength of the empirical claims. I would recommend that the editor require a validation-based selection protocol and a more robust human evaluation of the labels before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know two things about this one. First, the core idea is genuinely useful: AggTruth restricts attention maps to the provided passage and aggregates them with four simple functions, and it works surprisingly well as a windowed, online hallucination detector across four LLMs and four datasets. Second, the reported “high and stable” results are probably optimistic, because the authors picked the best feature selector and head percentage per model/task based on the test-set Gap value. That is selection-on-test, and it undermines the central outperformance claim.\n\nWhat’s new: the passage-only attention restriction, the four aggregation methods (Sum, CosSim, Entropy, JS-Div), and the feature selection comparison across four LLMs. The Lookback Lens baseline is honestly cited, and the paper clearly positions itself as a modification. The head-selection analysis is also useful: they show that using all heads is not always best, which contradicts Lookback Lens’s finding. The experiments cover more models than most papers in this area.\n\nWhere it goes soft. Label quality is a concern: GPT-4o produces token-level labels, with only 75 human-checked samples and Kappa 0.7. The classifier learns GPT-4o’s notion of hallucination, not necessarily truth. That alone should make you cautious. But the bigger problem is the selection protocol. Tables 5 and 6 report “best obtained results w.r.t. Gap value,” and Gap is computed on the test sets. They explore multiple selectors (Center, Random, Random+, Lasso, Spearman) and head percentages, then present the minimum-Gap configuration as the method’s performance. Without validation-based selection or multiple-comparison correction, those numbers are expected to be optimistic. The stress-test note gives a concrete example: for Llama-2 QA→SUM, Random+3,3 lowers Gap from 6.836 to 2.048. That is a large effect. There are also no error bars or significance tests anywhere, and the Lookback Lens baseline is only run on Llama-2 because the released code doesn’t generalize.\n\nThe paper is worth a serious referee, because the method is novel and the experiments are extensive. But it needs major revision: release code and data, pre-register selectors or select them on validation folds, report variance across runs, and validate a larger sample of GPT-4o labels. With those fixes, the central claim might hold. As written, take the headline numbers with a grain of salt.\n\nRecommendation: send to peer review, but expect heavy revision.","headline":"Useful passage-only attention aggregation for hallucination detection, but the headline AUROCs are likely optimistic because best configurations were selected on test-set Gap.","tokens_in":12166,"tokens_out":2553,"would_cite":true,"duration_ms":26762,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AggTruth detects contextual hallucinations in LLM outputs online by aggregating attention scores over the source passage, and it transfers across QA and summarization tasks and across four LLMs.","keywords":["hallucination detection","attention map","retrieval-augmented generation","contextual hallucination","large language models","feature selection","online detection","attention aggregation"],"falsifier":"Re-annotate a random sample of, say, 400 responses with token-level human labels, ideally with two or more annotators per token, and recompute the AUROC of the best AggTruth configuration against these human labels instead of GPT-4o labels. If the AUROC drops to near random, the method is detecting the judge's labeling style rather than hallucination; if it stays high, the central claim survives.","tokens_in":11139,"feed_emoji":"🎯","tokens_out":9655,"duration_ms":102509,"temperature":0.7,"pith_summary":"The paper introduces AggTruth, a detector that watches the attention an LLM pays to the provided context passage while generating each token, and turns that attention into a small set of features for classifying tokens as hallucinated or faithful. It claims this passage-only signal is enough to detect contextual hallucinations in real time, during decoding, and that the method stays accurate when moving between question answering and summarization and across four different LLMs. The authors test four aggregation techniques: summing the attention placed on the passage, cosine similarity between heads, entropy of the attention distribution, and Jensen-Shannon distance from the layer's average attention. They report that AggTruth matches or beats the previous attention-map baseline on most settings while being cheaper, and that selecting a subset of attention heads can improve performance. A reader should care because online hallucination detection is a prerequisite for interventions like stopping, correcting, or regenerating untruthful output in retrieval-augmented generation systems.","feed_headline":"Passage-only attention scores detect LLM hallucinations across tasks","feed_subtitle":"AggTruth reads attention to the source passage and flags untruthful tokens as they are generated.","key_machinery":"The mechanism is the passage-only attention aggregate: for each layer $l$, head $h$, and generated token $t$, the attention scores over the context tokens are reduced to one scalar by one of four formulas. Sum simply adds the attention placed on passage tokens; CosSim averages the cosine similarity between a head and all other heads in the same layer; Entropy treats the attention vector as a pseudo-probability distribution; and JS-Div measures the Jensen-Shannon distance between a head's attention and the layer's average head, with an extra appended value so the pseudo-distribution sums to one. These scalars become features for an eight-token sliding window, and the window label is positive if GPT-4o judged any token in it hallucinated; a logistic-regression classifier then makes the final decision. The paper's additional machinery is head selection: it reduces the feature set via Spearman correlation with the target, a simplified Boruta-style 'above random' selector, Lasso, or a center-ratio baseline.","core_discovery":"The central discovery is that you do not need the full attention map, hidden states, or multiple sampled generations to detect contextual hallucinations: the attention scores the model assigns to the provided passage, aggregated per head per generated token, carry enough signal. AggTruth reduces the attention tensor to a single scalar per layer-head for each generated token, by summing the scores over passage tokens, by measuring how similar each head is to its layer-mates, or by treating the attention distribution as a pseudo-probability and measuring its entropy or Jensen-Shannon distance to the layer average. These per-token features are then windowed into overlapping chunks of eight tokens, labeled by GPT-4o as containing a hallucination or not, and fed to a logistic-regression classifier. In same-task and cross-task tests across four LLMs on Natural Questions, HotPotQA, CNN/Daily Mail, and XSum, the authors find that AggTruth variants achieve small gaps between source and target test performance and outperform hidden-state classifiers and, where comparable, the Lookback Lens baseline. The paper also finds that selecting a subset of attention heads, often 10 to 50 percent, can improve or preserve performance compared with using all heads.","pith_inferences":["Beyond the paper: because AggTruth only needs attention onto a reference passage, the same recipe could apply to other grounded-generation tasks such as dialogue grounded in retrieved documents, clinical note generation, or legal citation, but the paper only tests QA and summarization, so this is an open question.","Beyond the paper: all responses were generated with greedy decoding; under sampling or higher temperatures the attention distributions would shift, so thresholds and head subsets would likely need recalibration, a setting the paper does not evaluate.","Beyond the paper: the paper itself notes that AggTruth requires access to attention scores for context tokens, which flash-attention and sparse or window attention implementations do not offer, so deployment is restricted to models that expose raw attention maps, and those regimes are left untested.","Beyond the paper: since GPT-4o supplied the training labels, a fully human-annotated benchmark would clarify whether AggTruth detects unfaithfulness itself or merely predicts the judge's labeling style."],"forward_implications":["A retrieval-augmented generation system using AggTruth could detect hallucinated tokens while the answer is still being generated, making post-hoc correction unnecessary.","Because the features come only from attention on the source passage, the detector transfers between question answering and summarization and across four different LLMs without target-task retraining.","Selecting a subset of attention heads, often between 10 and 50 percent, can match or slightly beat using all heads, so the classifier can be trained and run more cheaply.","The Sum aggregation is the recommended default: it is the fastest, most interpretable, and most stable variant across the tested settings."],"supporting_citations":[{"why":"Supplies the Lookback-Lens attention-map baseline, the eight-token window size, and the GPT-4o-as-judge labeling precedent that AggTruth extends and compares against.","marker":"[2]"},{"why":"Provides the hidden-states baseline method using upper-layer hidden states that AggTruth is compared with across models.","marker":"[1]"},{"why":"Natural Questions is the QA dataset used in the QA-to-summarization training and test setup.","marker":"[12]"},{"why":"HotPotQA is the multi-hop QA dataset used as a QA test and source in the cross-task setups.","marker":"[18]"},{"why":"CNN/Daily Mail is one of the summarization datasets used in the summarization-to-QA setup.","marker":"[17]"},{"why":"XSum is the other summarization dataset, used as source or target in the summarization setups.","marker":"[15]"},{"why":"Gemma 2's sliding-window attention motivates the 4,096-token context limit and contributes the Gemma-2 results to the across-LLM claim.","marker":"[6]"},{"why":"Boruta's feature-selection procedure is simplified into the Random and Random+ head selectors that improve performance with fewer heads.","marker":"[11]"}],"fun_headline_variants":["Passage attention alone detects LLM hallucination in real time","AggTruth: Four attention stats outperform hidden-state hallucination checks","Careful head choice beats using all heads for hallucination detection","Attention to source passage is enough to spot LLM untruths","Lightweight attention aggregation detects hallucinations across tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every reported accuracy figure assumes GPT-4o's token-level hallucination labels are trustworthy; the only human check covered 75 samples and reached moderate agreement (a Cohen's Kappa of 0.7), so if GPT-4o's judgment is biased, the detector is really predicting GPT-4o's notion of hallucination rather than ground truth.","fun_headline_variants_meta":{"raw":{"variants":["Passage attention alone detects LLM hallucination in real time","AggTruth: Four attention stats outperform hidden-state hallucination checks","Careful head choice beats using all heads for hallucination detection","Attention to source passage is enough to spot LLM untruths","Lightweight attention aggregation detects hallucinations across tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001122,"raw_usage":{"total_tokens":4653,"prompt_tokens":918,"completion_tokens":3735,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":3651}},"tokens_in":534,"tokens_out":3735,"duration_ms":27273,"temperature":1.0,"reasoning_tokens":3651,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:15:25.381039+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of, say, 400 responses with token-level human labels, ideally with two or more annotators per token, and recompute the AUROC of the best AggTruth configuration against these human labels instead of GPT-4o labels. If the AUROC drops to near random, the method is detecting the judge's labeling style rather than hallucination; if it stays high, the central claim survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Lookback-Lens attention-map baseline, the eight-token window size, and the GPT-4o-as-judge labeling precedent that AggTruth extends and compares against."},{"cited_title":"In: EMNLP","cited_arxiv_id":null,"evidence_quote":"Provides the hidden-states baseline method using upper-layer hidden states that AggTruth is compared with across models."},{"cited_title":"Transactions of the ACL7, 452–466 (2019)","cited_arxiv_id":null,"evidence_quote":"Natural Questions is the QA dataset used in the QA-to-summarization training and test setup."},{"cited_title":"In: ACL 2017","cited_arxiv_id":null,"evidence_quote":"CNN/Daily Mail is one of the summarization datasets used in the summarization-to-QA setup."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"XSum is the other summarization dataset, used as source or target in the summarization setups."},{"cited_title":"Journal of statistical software36, 1–13 (2010)","cited_arxiv_id":null,"evidence_quote":"Boruta's feature-selection procedure is simplified into the Random and Random+ head selectors that improve performance with fewer heads."}],"review_version":1}