{"id":"447d2750-f2cf-4cf5-b7fd-97817e328e48","arxiv_id":"2504.19414","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"GMAR weights ViT attention heads by the magnitude of their gradients to the predicted class and feeds them into attention rollout, improving four interpretability metrics over standard rollout in one fine-tuned setting.","lead":"A new interpretability method weighs each attention head in a Vision Transformer by the magnitude of its gradient with respect to the predicted class, then applies these weights during attention rollout. On Tiny-ImageNet, it reports better faithfulness scores than standard attention rollout, without statistical error bars and for a single model only.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GMAR's Algorithm 1 recursion conflicts with Eq. (1) and depends on an unspecified residual ratio alpha, leaving the reported gains non-reproducible as written.","rationale":"The reader's weakest_assumption correctly identifies that the recursion in Algorithm 1 differs from Eq. (1) and that alpha is unspecified. My stress-test focuses on this as the single most load-bearing concern because it directly affects reproducibility and the validity of the reported comparisons. The paper claims GMAR outperforms attention rollout across all four metrics, but the differences on Insertion (12.15 vs 11.97) and Deletion (10.62 vs 12.17) are small, and the algorithm that generates these numbers is not well-defined: the residual term alpha could be any value, and the recursion is not derivable from the stated rollout formula. Without a specified alpha and with an inconsistent recursion, the experimental results cannot be independently verified, and the advantage over the baseline may be an artifact of the arbitrary aggregation rule rather than of gradient-based head weighting. The proposed concrete test directly resolves this by recomputing the metrics under the alternative, standard recursion and a range of alpha values. If GMAR remains superior under the corrected recursion and any reasonable alpha, the central claim survives; otherwise the paper needs revision. Therefore, the verdict should remain CONDITIONAL, pending this reproducibility check, matching the reader's assessment.","tokens_in":6465,"tokens_out":3556,"duration_ms":38625,"concrete_test":"Run GMAR on Tiny-ImageNet with ViT-Large-Patch16-224 under four conditions: (1) the algorithm as printed with alpha=1; (2) the algorithm as printed with alpha=0.5; (3) the corrected standard recursion A_rollout = A_rollout * (A_weighted + I), with alpha=1 implicit; and (4) the corrected recursion with alpha=0.5. For each condition, recompute all four metrics in Table 1 on the same subset of images with multiple seeds to get error bars. If the relative ordering of GMAR versus Attention Rollout changes on any metric across these conditions, the paper's central claim is not robust to the recursion/alpha ambiguity. If ordering is stable, the concern is alleviated, but the alpha value still must be reported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that GMAR outperforms attention rollout rests on the algorithm's exact definition, but Algorithm 1 is internally inconsistent with the paper's own Eq. (1). In Eq. (1), attention rollout is defined as the product over layers of (A_l + I), with the identity added to each layer's attention matrix to model the residual connection. Algorithm 1 instead updates A_rollout = A_rollout * A_weighted + alpha * I, adding the identity only after multiplication. This is not equivalent to the standard recursive form A_rollout = A_rollout * (A_weighted + I) for any alpha, except for the first layer. The two recursions diverge from layer 2 onward; with L=24 layers in ViT-Large, the resulting maps differ substantially. Moreover, alpha is introduced as a 'residual ratio' but no value is given anywhere in the paper, and no sensitivity analysis is provided. Because the reported metrics in Table 1 depend on this underspecified and likely incorrect recursion, a reader cannot reproduce GMAR's results, and it is unclear whether the claimed advantage over attention rollout comes from the gradient-based head weighting or from the ad-hoc residual term. This is a load-bearing flaw because fixing the recursion or setting alpha arbitrarily could flip the ordering of methods on Insertion or Deletion, which are currently separated by less than 0.2 percentage points.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Gradient-Driven Multi-Head Attention Rollout (GMAR), a method that computes per-head importance weights from the gradient of the predicted logit with respect to attention weights, normalizes them with L1 or L2 norms, and incorporates them into a weighted attention rollout for Vision Transformers. The method is evaluated on Tiny-ImageNet with a ViT-Large model using four faithfulness metrics (Average Drop, Average Increase, Insertion, Deletion), and the authors claim that GMAR consistently outperforms standard attention rollout. Qualitative overlay and difference-map visualizations are also provided.","tokens_in":6741,"tokens_out":3683,"duration_ms":35013,"significance":"If the reported results hold, the contribution is a moderate, incremental improvement to attention-rollout-based interpretability, with the useful property of producing head-level attribution. The paper uses four standard faithfulness metrics and compares against Grad-CAM and Attention Rollout, which is appropriate for a first validation. However, the experimental evidence is narrow (one model, one dataset, no error bars), and the method's definition contains an underspecified residual parameter and an inconsistency with the paper's own Eq. (1), so the central comparison is not yet reproducible. The idea of combining gradient-based head weighting with rollout is reasonable and worth pursuing, but the current manuscript does not yet establish the claimed advantage.","major_comments":[{"comment":"The definition of attention rollout in Eq. (1) is A_rollout = product over l of (A^(l) + I), which corresponds to the recursion A_rollout = A_rollout * (A^(l) + I). Algorithm 1 and Eq. (3) instead implement A_rollout = A_rollout * A_weighted + alpha * I. These two recursions coincide only for the first layer; with L=24 layers in ViT-Large they produce substantially different maps, and no value of alpha is given anywhere in the manuscript. Since the central claim is that GMAR outperforms attention rollout, the exact form of the recursion is load-bearing: the reported gains could be due to the ad-hoc residual term rather than to gradient-driven head weighting. Please reconcile Eq. (1) with Algorithm 1/Eq. (3), specify alpha (or remove it), and re-run the comparison.","section":"Eq. (1), Algorithm 1, Eq. (3)"},{"comment":"Table 1 reports a single value per method and metric, with no error bars, no multiple runs, and no significance tests. Several differences that support the main claim are small (Insertion: 12.15 vs 11.97; Deletion: 10.62 vs 12.17), so without variance estimates or statistical testing the statement that GMAR 'consistently outperforms' attention rollout is not supported. Please report standard deviations or confidence intervals over multiple seeds or image subsets, and state the number of images used for evaluation.","section":"Section 5.1, Table 1"},{"comment":"The central premise that the gradient norm of the predicted logit with respect to each head's attention weights measures head importance is asserted without justification or supporting citation. Gradients of attention weights are known to be susceptible to saturation and can be noisy; the paper should at least discuss this risk and provide a sensitivity analysis, for example comparing L1/L2 weighting against alternative head-importance measures such as attention variance or removal-based importance.","section":"Section 3.3.1"},{"comment":"The description of gradient extraction is ambiguous: it says 'G <- Extract gradients of attention layers' and 'G_h <- split(G, num_head) by channel', but attention weight tensors in ViT are 4D (batch, heads, keys, queries) and gradients are typically taken with respect to the attention logits before softmax or the normalized attention weights after softmax. The paper does not specify which quantity G represents. This ambiguity, together with the unspecified alpha, prevents reproduction of the reported numbers.","section":"Algorithm 1"}],"minor_comments":[{"comment":"The typesetting of Eq. (2) and the normalization formula is garbled: 'pP G2 hi' should presumably be the square root of the sum of squared gradient elements, and 'w = GRP GR' is missing a division symbol. Please fix the notation and define the index ranges explicitly.","section":"Section 3.3.1, Eq. (2)"},{"comment":"Attention Flow is mentioned in Section 2.1 but never defined or compared against; either add a brief description or remove the reference, since the paper does not use it in the experiments.","section":"Related Work"},{"comment":"The citation for ViT-Large-Patch16-224 is given as [23, 24], but [23] is a paper titled 'Visual transformers' rather than the original ViT reference. Please verify the citations; the original ViT paper [1] appears to be the correct source for the architecture.","section":"Section 4.1"},{"comment":"The qualitative section says 'integrating Grad-CAM with attention rollout confirms the appropriate application of gradients', but no such integration is described in the method section; please clarify what integration was performed.","section":"Section 5.2"},{"comment":"The evaluation section does not state the number of images used for computing the metrics, the random seed, or the fine-tuned model's accuracy on Tiny-ImageNet. Providing these details would help readers assess the reliability of the results.","section":"Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript would be more convincing with a comparison to contemporary ViT interpretability methods (e.g., Chefer et al. 2021, LRP-based methods) and with a public code release. The citation of [23] as the ViT-Large model appears incorrect. The narrow evaluation (one model, one dataset, no error bars) and the underspecified algorithm make the central claim hard to verify as written; however, the core idea is fixable within the scope of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"GMAR is a reasonable incremental idea: compute per-head gradient magnitudes from the target logit, normalize them L1 or L2, and use those as weights in attention rollout. The reported numbers support the narrow claim that this beats plain attention rollout on Tiny-ImageNet with ViT-Large. But the paper, as printed, cannot be reproduced, because the recursion in Algorithm 1 conflicts with Eq. (1) and the residual ratio alpha is never given a value. Eq. (1) defines rollout as the product over layers of (A_l + I). Algorithm 1 updates A_rollout = A_rollout * A_weighted + alpha * I. These are algebraically different from the second layer onward. Since the margins over attention rollout on Insertion and Deletion are roughly 0.2 percentage points, this is a load-bearing ambiguity, not a copyedit issue.\n\nWhat earns credit: the L1/L2 normalization of per-head gradient norms is simple, cheap, and clearly described, and the direct comparison to attention rollout on four standard faithfulness metrics is the right experimental design. The idea of gradient-weighting heads is not new — Chefer et al. already does gradient-weighted attention propagation — but the specific norm choice is a legitimate small extension. The qualitative figures are fine but add little beyond the metrics.\n\nSoft spots, in proportion: no error bars or significance testing; one model and one dataset; only Grad-CAM and attention rollout as baselines, with the most relevant gradient-weighted baseline not benchmarked; no code. The strongest issue remains the undefined alpha and the recursion mismatch. A reader cannot know whether the improvement comes from the gradient weighting or from an ad hoc residual term. The citation pattern looks fair; the paper cites Chefer et al. and related work, even if it does not compare against them numerically.\n\nWho this is for: a practitioner who wants a slightly sharper attention-rollout map might use the idea after the method is pinned down. Right now I would not cite it, and I would desk reject it. A revised version with the recursion fixed, alpha swept, error bars, and the Chefer baseline added would deserve a serious referee.","headline":"Gradient-weighted attention rollout is a reasonable incremental idea, but Algorithm 1 conflicts with Eq. (1), alpha is never specified, and the reported gains rest on a quarter-point margin — not ready as printed.","tokens_in":7233,"tokens_out":3425,"would_cite":false,"duration_ms":34143,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Gradient-driven head weighting makes attention rollout more faithful for Vision Transformers.","keywords":["Attention Rollout","Explainable AI","Multi-head Attention","Vision Transformer","Gradient-based importance","Head weighting","Faithfulness metrics","Interpretability"],"falsifier":"Shuffle the GMAR head weights across heads and rerun the same rollout on the same images; if the shuffled maps match GMAR's Insertion and Deletion scores, then the specific assignment of gradient-based weights carries no information and only the act of weighting matters.","tokens_in":6290,"feed_emoji":"🔍","tokens_out":5796,"duration_ms":54636,"temperature":0.7,"pith_summary":"The paper sets out to fix a blind spot in attention rollout for Vision Transformers: rollout averages all attention heads together, even though heads specialize and some matter more than others. GMAR assigns each head a scalar weight from the gradient of the predicted class logit with respect to that head's attention weights, then multiplies each layer's attention matrix by these weights before the usual rollout recursion. On Tiny-ImageNet with a fine-tuned ViT-Large, the weighted maps come out ahead of plain attention rollout on all four faithfulness metrics: Average Drop, Average Increase, Insertion, and Deletion. The intended payoff is that ViT explanations become class-specific and head-aware, rather than an undifferentiated average of all attention.","feed_headline":"Weighting ViT heads by gradient makes attention rollout more faithful","feed_subtitle":"GMAR beats plain attention rollout on all four fidelity metrics on Tiny-ImageNet with ViT-Large.","key_machinery":"The carrying object is the per-head gradient score $w_h = \\mathrm{GR}_h / \\sum_j \\mathrm{GR}_j$, where $\\mathrm{GR}_h = \\sum |G_h|$ for L1 or $\\sqrt{\\sum G_h^2}$ for L2, and $G_h$ is the gradient of the predicted class logit with respect to the attention weights of head $h$. The weights are reshaped to $(1,H,1,1)$ and multiply each layer's attention map $A_\\ell$ to form $A_{\\text{weighted}}$. The rollout then follows $A_{\\text{rollout}} = A_{\\text{rollout}} \\cdot A_{\\text{weighted}} + \\alpha I$, with $\\alpha$ controlling the strength of the residual identity. This mechanism converts a class-specific, gradient-based importance signal at the head level into the spatial attention map, which is what lets GMAR keep rollout's global aggregation while adding head sensitivity.","core_discovery":"The central claim, stated on the paper's own terms, is that gradient-driven head weighting makes attention rollout more faithful. For a given input, GMAR computes the gradient of the predicted logit with respect to each head's attention weights, aggregates it with an L1 or L2 norm, and normalizes across heads to get weights $w$. Each layer's attention map is scaled by $w$ before the recursive rollout $A_{\\text{rollout}} = A_{\\text{rollout}} \\cdot A_{\\text{weighted}} + \\alpha I$, so heads that respond strongly to the predicted class dominate the final map. In the reported experiments, GMAR outperforms Attention Rollout across all evaluation metrics; with the L2 norm it reduces Average Drop from 25.78 to 22.13 and raises Insertion from 11.97 to 12.16, while also improving Deletion. The authors conclude that attention-weight-only methods are less faithful than gradient-informed ones, and that combining the two signals yields more reliable visual explanations.","pith_inferences":["A natural but untested use of GMAR's weights is head pruning: if gradient magnitude really measures head importance, heads with near-zero weights should be removable with little accuracy loss, but the paper does not run that experiment.","The residual weight $\\alpha$ in Algorithm 1 is never given a value; comparing $\\alpha=1$ against tuned values would separate the benefit of head weighting from the benefit of the residual recursion.","Because the method only requires access to attention maps and gradients, it should transfer to non-vision transformers; verifying that on a text model would test whether the finding is about attention heads in general rather than ViT specifics."],"forward_implications":["GMAR needs only one backward pass per input, so head-weighted explanations are nearly as cheap as standard attention rollout.","Because the weights come from the predicted class logit, GMAR maps vary with the class being explained, unlike unweighted rollout's class-agnostic integration.","On the reported Tiny-ImageNet/ViT-Large setup, GMAR improves all four fidelity metrics over attention rollout; the L2 variant's Average Drop (22.13) is even slightly below Grad-CAM's (22.61), while its Insertion (12.16) beats both baselines.","The L1 and L2 variants behave differently: L1 tracks large abrupt changes, while L2 captures gradual cumulative variation, giving users two complementary explanation views."],"supporting_citations":[{"why":"Defines attention rollout, the baseline method GMAR extends and compares against.","marker":"[3]"},{"why":"Introduces the Vision Transformer architecture whose attention layers GMAR explains.","marker":"[1]"},{"why":"Provides Grad-CAM, the gradient-based baseline GMAR is compared with and contrasted against.","marker":"[6]"},{"why":"Motivates that equal treatment of attention heads obscures meaningful patterns, the problem GMAR addresses.","marker":"[19]"},{"why":"Supplies the pretrained ViT-Large-Patch16-224 backbone used in the experiments.","marker":"[23]"},{"why":"Provides Tiny-ImageNet, the dataset on which the model is fine-tuned and all metrics are reported.","marker":"[25]"},{"why":"Defines Average Drop, Average Increase, Insertion, and Deletion, the four metrics used to judge faithfulness.","marker":"[26]"}],"fun_headline_variants":["Gradient-weighted heads improve ViT attention rollout","GMAR: gradient-informed head weights for ViT explainability","Weighing attention heads by gradient boosts rollout fidelity","Gradient-weighted attention rollout beats plain rollout","ViT interpretability: gradient head weighting outperforms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the gradient magnitude of the predicted class score with respect to a head's attention weights faithfully measures how much that head contributes to the prediction.","fun_headline_variants_meta":{"raw":{"variants":["Gradient-weighted heads improve ViT attention rollout","GMAR: gradient-informed head weights for ViT explainability","Weighing attention heads by gradient boosts rollout fidelity","Gradient-weighted attention rollout beats plain rollout","ViT interpretability: gradient head weighting outperforms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1337,"prompt_tokens":969,"completion_tokens":368,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":293}},"tokens_in":585,"tokens_out":368,"duration_ms":3853,"temperature":1.0,"reasoning_tokens":293,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:52:31.104457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Shuffle the GMAR head weights across heads and rerun the same rollout on the same images; if the shuffled maps match GMAR's Insertion and Deletion scores, then the specific assignment of gradient-based weights carries no information and only the act of weighting matters.","supporting_citations":[{"cited_title":"Overview of Attention Rollout Attention rollout is a technique designed to interpret and visu- alize the internal mechanisms of ViTs","cited_arxiv_id":null,"evidence_quote":"Defines attention rollout, the baseline method GMAR extends and compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Grad-CAM, the gradient-based baseline GMAR is compared with and contrasted against."},{"cited_title":"Swin transformer: Hierarchical vision transformer us- ing shifted windows,","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained ViT-Large-Patch16-224 backbone used in the experiments."},{"cited_title":"Transformer in- terpretability beyond attention visualization,","cited_arxiv_id":null,"evidence_quote":"Provides Tiny-ImageNet, the dataset on which the model is fine-tuned and all metrics are reported."}],"review_version":1}