{"id":"1969f0e1-30d2-4580-8504-fc9362257984","arxiv_id":"2411.16760","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"LibraGrad prunes and scales backward gradient paths in Vision Transformers to make attribution maps more complete and more faithful, improving existing gradient-based explanation methods.","lead":"This paper introduces LibraGrad, a post-hoc technique that changes how error signals travel backwards through vision models to produce sharper, more faithful attribution maps. The authors report that it improves many existing explanation methods on standard benchmarks, with no extra compute.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The theoretical guarantee is tautological: LibraGrad's FG-completeness holds only for its modified backward Jacobians, not the model's true gradients, so it cannot certify faithfulness.","rationale":"The central claim has two pillars: (i) LibraGrad is theoretically grounded because it restores FG-completeness, and (ii) it universally enhances gradient-based attribution methods. For (i) to support the method, FG-completeness would have to be a property of the model that the attribution method faithfully reflects. But the paper's construction enforces the completeness equation by redefining the gradient, so the theorem is not about the model. The only bridge from FG-completeness to the reported faithfulness metrics is empirical, and the abstract's universality is already falsified by the paper's own MURA and SigLIP-L numbers. I do not think this warrants rejection: the method may still be practically useful, the experiments are broad, the analysis is detailed, and the code is promised. However, the manuscript needs to state explicitly that LibraGrad's completeness is relative to modified backward Jacobians, drop or carefully qualify 'universally', and discuss the negative cases. This is consistent with the reader's CONDITIONAL verdict; the reader identified the same general gap, but I emphasize that the property is not merely unproven for faithfulness but is satisfied by construction for the modified gradients.","tokens_in":98305,"tokens_out":8844,"duration_ms":87551,"concrete_test":"On a small ViT (e.g., ViT-T) with 100 ImageNet images, compute Libra FullGrad+ attributions and then evaluate Completeness Error using the true Jacobians of the original forward model (standard autograd, without detaching attention scores or LayerNorm denominators). If CE is substantially above zero, the FG-completeness claimed in Corollary 3 is an artifact of the modified backward pass rather than a property of the model being explained.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Corollary 3 is the paper's theoretical anchor, but it does not say what the abstract needs it to say. Definition 1 defines FG-completeness for a function f using the true Jacobians Jxf and Jbi f. LibraGrad, however, replaces those Jacobians: Libra-Attention detaches softmax(QK^T), so dQ=dK=0 (Section 3.5), and Libra-LayerNorm detaches the denominator (Corollary 2). The resulting backward pass is not the gradient of the forward function. For LayerNorm, Proposition 3 shows the true JxLN·x tends to 0 as epsilon approaches 0, while the Libra backward pass returns LN(x); for attention, the true Jacobian has additional terms through Q and K that LibraGrad drops. Hence the equation f(x)=J^Libra_x f·x+Σ J^Libra_bi f·bi is satisfied by construction, not derived from the model. Table 4's CE=0 is therefore an implementation check of the custom backward pass, not evidence that the attributions reflect the model's computation. The paper concedes the link to faithfulness is empirical ('empirical results demonstrate...', Section 3.5), and the 'Universal Improvement' paragraph is not a theorem. Moreover, the empirical universal claim is contradicted by the paper's own tables: MURA Input×Grad drops from 25.5 to 21.6 (Table 2), and SigLIP-L GradCAM+ drops from 44.3 to 41.7 in Segmentation AP (Table 3). Thus both the theoretical grounding and the 'universally enhances' phrasing are weaker than claimed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LibraGrad, a post-hoc attribution enhancement for vision transformers that modifies backward gradients by zeroing gradients through attention softmax scores and LayerNorm denominators, pruning non-linear gates, and scaling self-gating branches. It claims that these changes restore FullGrad-completeness, that this is crucial for faithful interpretability, and that LibraGrad universally enhances a wide range of gradient-based attribution methods across faithfulness, completeness error, and segmentation metrics. The experiments cover eight architectures, four model sizes, and four datasets, with code released.","tokens_in":98594,"tokens_out":7225,"duration_ms":67510,"significance":"If the universal-improvement claim held, the contribution would be significant: it offers a cheap, drop-in gradient surgery applicable to IG, FullGrad+, GradCAM+, and other methods, and it poses a clear hypothesis about why gradients fail on transformers. The paper's theoretical toolbox (constant operator, SwapBackward, Theorems 4 and 5) is clean and reusable, and the empirical sweep is unusually broad, spanning multiple architectures, model sizes, datasets, and metric families. However, as detailed below, the theoretical guarantee applies to modified backward Jacobians rather than to the model's true Jacobians, and the 'universal' claim is contradicted by the paper's own tables. The method remains a plausible strong heuristic; its significance depends on an honest reframing and on reporting the negative cases.","major_comments":[{"comment":"FG-completeness is stated for the true Jacobians of f, but LibraGrad's definitions replace those Jacobians with modified backward values. For example, Libra-LayerNorm keeps the forward value but zeroes the gradient through the denominator, and Libra-Attention zeroes gradients through softmax(QK^T); these are not the Jacobians of the forward function. Corollary 3 therefore certifies that a modified backward pass satisfies f(x)=J^Libra_x f·x+Σ J^Libra_bi f·bi, which is enforced by construction rather than derived from the model. Table 4's CE=0 (Libra FullGrad) is an implementation sanity check of that custom backward pass, as Section B.1 acknowledges, and it cannot by itself support the abstract's 'theoretically grounded' faithfulness claim. The divergence is concrete: Proposition 3 shows the true JxLN·x tends to zero as epsilon approaches zero, while the Libra backward pass returns LN(x). The paper should either prove a formal connection between modified-Jacobian completeness and the faithfulness metrics or explicitly reframe the contribution as a heuristic gradient reweighting with empirical support.","section":"§3.4, Definition 1, Corollary 3"},{"comment":"The claim of universal enhancement across all metrics is not supported by the paper's own results. In Table 2 (MIF accuracy on ViT-B), Libra Input×Grad drops on MURA from 25.5 to 21.6, and Libra TokenTM is unchanged at 28.0. In Table 3 (Segmentation AP), Libra GradCAM+ drops on SigLIP-L from 44.3±0.4 to 41.7±0.3 and on DeiT3-H from 60.3±0.4 to 46.7±0.4; similar decreases appear in other cells. Section 4.2's statement that 'LibraGrad universally enhances gradient-based attribution methods across all tested models, architectures, and datasets' is therefore false as written. The authors should present a systematic account of cases with no gain or a loss, and the abstract and conclusion should be revised to describe large but not universal gains.","section":"Tables 2 and 3; §4.2"},{"comment":"The choice of scaling versus pruning coefficients is underdetermined by the theory and not justified empirically. Theorem 4 allows any a,b with a+b=1, and the paper fixes a=b=1/2 for self-gating while setting a=0 (pruning) for attention and LayerNorm; Theorem 5 justifies pruning only when the surviving branch is FG-complete. The decision to zero Q and K gradients rather than scale them is presented as a design choice in Section 3.5 with no ablation or sensitivity analysis over a,b or over pruning versus scaling for any component. Since these choices determine the attribution maps, the paper should at least report an ablation over the free coefficients and over scaling alternatives for the attention path.","section":"§3.4–§3.5"},{"comment":"The ablation's interaction pattern complicates the 'LayerNorm is the most significant factor' interpretation. Removing LayerNorm alone drops MIF predicted accuracy from 71.7 to 49.9, but removing both attention and LayerNorm gives 61.2, which is higher than removing LayerNorm alone; similar nonmonotonicity appears in the GT-accuracy column (65.2 vs 49.9 vs 63.6). The text in Section 4.2 asserts the LayerNorm effect is the most significant without discussing this interaction. The authors should explain the non-additivity or soften the causal attribution.","section":"Table 5"}],"minor_comments":[{"comment":"Table 2 reports that standard deviations were bounded by 0.1 and omitted, but it does not state the number of random seeds or images used for each cell; a single sentence on replication would help.","section":"§4.1"},{"comment":"The Completeness Error table excludes attention-based methods 'as their incompleteness is evident'; this should be stated as a scope restriction in the main text so that the comparison is not read as a complete ranking.","section":"Table 4 caption"},{"comment":"The expression 'Libra-Attention(Q, K, V) = [softmax(QK^T)]_cst · V' has ambiguous operator precedence; adding parentheses would clarify that the constant operator applies to the whole softmax argument.","section":"§3.5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript would benefit from an explicit statement of what completeness of a surrogate backward pass can and cannot certify. The editor may also want the authors to include a 'negative results' subsection listing the cells where LibraGrad hurts, since the current abstract overclaims. The fit to the journal is acceptable if the claims are reframed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a worthwhile method paper with an unusually broad evaluation, but the headline claims need reining in. The core idea is simple and sound—Transformer components like softmax attention, LayerNorm division, and gated activations violate the FullGrad conservation identity, and LibraGrad fixes this by detaching or scaling those backward paths. The proofs are elementary but correct, and the sweep is wide: eight architectures, multiple datasets, faithfulness, segmentation, and qualitative CLIP checks. Showing that the same trick helps MLP-Mixer is a nice control that rules out attention-specific explanations. Credit where due: the method is new, easy to implement, and likely to become a standard baseline for ViT attribution.\n\nThe soft spots are two. First, the abstract and text say \"universally enhances,\" but the paper's own tables contradict that. MURA Input×Grad drops from 25.5 to 21.6 (Table 2), SigLIP-L GradCAM+ drops from 44.3 to 41.7 (Table 3), and there are small regressions on some hybrid methods. Not universal. The authors should soften the claim and analyze the exceptions rather than sweeping them under the average.\n\nSecond, the theory is oversold. Corollary 3 says the Libra-modified backward pass is FG-complete, but that is true by construction—softmax QK and LayerNorm denominators are detached, so the backward path is no longer the Jacobian of the forward function. The completeness equation holds for a surrogate gradient, not for the model's actual sensitivity. The paper does acknowledge the link to faithfulness is empirical, which is honest, but the phrase \"theoretically grounded\" in the abstract implies more than that. Table 4's zero completeness error is a sanity check of the custom backward pass, as the authors themselves note, not evidence of faithful attribution.\n\nThese are fixable problems. The method appears to work most of the time, and the empirical evidence for the imbalance hypothesis is reasonably strong. I would not trust the \"universal\" claim, but I would use LibraGrad as a strong baseline and would want the analysis of negative cases in the revision. Statistical significance testing would also strengthen the paper, since the standard errors are tiny but the dataset sizes are modest.\n\nWho is this for: anyone working on ViT interpretability or using gradient attributions in practice. It deserves serious peer review with major revisions aimed at the claims and the theoretical framing.","headline":"A genuinely useful gradient-surgery method for ViT attributions with broad empirical gains, but the 'universally enhances' claim is contradicted by its own tables and the completeness theory is a conservation identity for a surrogate gradient, not a faithfulness guarantee.","tokens_in":99160,"tokens_out":2150,"would_cite":true,"duration_ms":22262,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pruning and rescaling backward gradients restores an exact-output-completeness property in Transformers, improving every gradient attribution method tested.","keywords":["LibraGrad","gradient attribution","Transformer interpretability","FullGrad-completeness","faithfulness metrics","LayerNorm","attention attributions","post-hoc explanation"],"falsifier":"Take a Transformer fine-tuned on ImageNet and compute attributions on the paper's own zebra/elephant COCO images with only the attention-gradient pruning applied and LayerNorm gradients untouched: if the pruned attention scores, rather than LayerNorm, encode the co-occurring-class distinction, the maps will localize the wrong animal. Conversely, the theory predicts that a Transformer whose normalizations are replaced by affine scaling needs no Libra LayerNorm fix, so a faithfulness gain there would disprove FG-completeness as the operative mechanism.","tokens_in":98092,"feed_emoji":"⚖️","tokens_out":5975,"duration_ms":51160,"temperature":0.7,"pith_summary":"Gradient-based attribution methods that work well on CNNs fail on Vision Transformers, and this paper says the reason is not attention itself but unbalanced gradient flow in the backward pass. It formalizes a property, FullGrad-completeness, that makes attributions decompose a model's output exactly, shows that classical CNNs have it naturally while Transformer components break it, and introduces LibraGrad, a set of backward-pass modifications that prune and rescale gradients by treating attention weights and LayerNorm denominators as constants and splitting self-gating gradients. Across eight architectures, four model sizes, and four datasets, the modified gradients improve every gradient-based attribution method tested, including hybrids designed for Transformers, and the general-purpose Libra FullGrad+ becomes the best overall method. The package requires no forward-pass changes and no extra compute, which matters because faithful explanations of Transformer decisions are a prerequisite for deploying these models in high-stakes settings.","feed_headline":"Balanced gradients fix Transformer explanations","feed_subtitle":"LibraGrad prunes and rescales backward paths, making general-purpose attribution methods beat Transformer-specific ones.","key_machinery":"The central object is FullGrad-completeness (FG-completeness), the identity $f(x) = J_x f \\cdot x + \\sum_i J_{b_i} f \\cdot b_i$ that forces attributions to sum exactly to the model output. The argument is carried by two backward-pass manipulation operators defined on top of an unchanged forward pass: the constant operator $[\\cdot]_{\\mathrm{cst}}$, which zeroes the gradient while keeping the value, and SwapBackward, which evaluates one function but propagates the Jacobian of another. Theorems 1–3 show that locally affine functions, compositions, and sums are FG-complete, which explains why CNNs inherit the property, and Propositions 1–3 show which Transformer operations break it; Theorems 4–5 then give the repair rules, scale product branches by coefficients summing to one or prune a non-FG branch. The practical Libra modules instantiate these rules: attention contributes gradients only through the value branch, LayerNorm treats its denominator as constant, gated activations discard the gate's gradient, and SwiGLU splits its backward gradient equally between the two branches.","core_discovery":"LibraGrad claims that the reason gradient-based explanations underperform on Transformers is a set of non-locally-affine operations — attention softmax scores, gated activations such as GELU and SiLU, SwiGLU self-gating, and LayerNorm — that violate FullGrad-completeness, the identity $f(x) = J_x f \\cdot x + \\sum_i J_{b_i} f \\cdot b_i$ that CNNs satisfy automatically. The paper proves that naive element-wise multiplication over-counts attributions by a factor of two, that division makes FullGrad vanish identically, and that LayerNorm's denominator drives FullGrad to zero as $\\varepsilon \\to 0$. It then proves that scaling the two Jacobian branches of a product by coefficients summing to one, or pruning one branch entirely by treating it as constant in the backward pass, restores the identity (Theorems 4 and 5, Corollary 2), and that applying these fixes to attention, gated activations, self-gating, and LayerNorm yields a Transformer that is FG-complete (Corollary 3). Empirically, Libra versions of Input×Grad, GradCAM+, HiResCAM, XGradCAM+, FullGrad+, AttCAT, GenAtt, and TokenTM beat their unmodified counterparts on faithfulness, completeness error, and segmentation alignment on nearly every model and dataset, with Libra FullGrad+ at 0.0 completeness error on all tested models and the top average segmentation AP.","pith_inferences":["The argument suggests a general design rule for future architectures: whenever a new layer introduces multiplicative gating or normalization, the backward pass should be audited against FG-completeness before any attribution method is built, and the Libra recipes could be applied preemptively.","A testable prediction follows: on a Transformer variant whose normalizations are already affine, the gain from the Libra LayerNorm fix should shrink, cleanly separating the normalization effect from the attention effect.","The pruning choice is a modeling decision, and the scaling coefficients of Theorem 4 could be tuned to interpolate between full gradients and pruned gradients, trading exact completeness against retaining some attention-gradient signal; the paper does not explore this.","If FG-completeness is the operative mechanism, faithfulness gains should correlate with measured completeness error across architectures and checkpoints, a correlation the paper reports only indirectly."],"forward_implications":["Once gradients are rebalanced, general-purpose gradient methods such as FullGrad+ outperform attention-based and Transformer-specific attribution methods on faithfulness and segmentation metrics, suggesting that specialized explainers for Transformers are unnecessary.","Libra FullGrad+ reaches 0.0 completeness error on every large model tested and raises average segmentation AP from 43.4 to 67.9, placing it above every attention-based method in the comparison.","LibraGrad imposes no forward-pass modification and no extra computational or memory overhead, so it can be dropped into any existing gradient pipeline.","The method transfers to the attention-free MLP-Mixer architecture, supporting the claim that gradient imbalance, not attention per se, is the root cause of attribution failures.","The ablation study attributes most of the gain to fixing LayerNorm, with attention and self-gating secondary and bias terms negligible, identifying the highest-value repair for practitioners."],"supporting_citations":[{"why":"FullGrad: supplies the completeness-style attribution decomposition that FG-completeness formalizes and extends.","marker":"[75]"},{"why":"SkipPLUS / FullGrad+: the base method that LibraGrad enhances, and the source of the evaluation protocol.","marker":"[49]"},{"why":"Integrated Gradients: the standard completeness baseline that LibraGrad is compared against and whose fixed-step approximation the paper proves unstable.","marker":"[77]"},{"why":"AttnLRP: a Transformer-specific comparison baseline evaluated across all models.","marker":"[2]"},{"why":"TokenTM: a Transformer-specific gradient-attention hybrid baseline that LibraGrad composes with and outperforms.","marker":"[87]"},{"why":"AttCAT: a Transformer-specific gradient-attention hybrid baseline in the comparison.","marker":"[61]"},{"why":"GenAtt: a Transformer-specific gradient-attention hybrid baseline in the comparison.","marker":"[16]"},{"why":"GLU variants: the source of the self-gating (SwiGLU) operation whose gradient splitting the paper targets.","marker":"[69]"},{"why":"DecompX: a comparison baseline used for completeness error and faithfulness metrics.","marker":"[52]"}],"fun_headline_variants":["LibraGrad rebalances gradients to fix Transformer attributions","Gradient balancing makes generic XAI beat Transformer-specific","Prune and scale backward paths to restore attribution completeness","Universal fix for Transformer explanations: balance gradient flow","Why Transformers distort saliency maps and how LibraGrad corrects it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that making the gradient-sensitive parts of a Transformer behave as constants in the backward pass, which guarantees attributions sum to the output, is enough to make attributions faithful, and that the discarded gradient signal through attention weights and LayerNorm denominators carries no essential explanation.","fun_headline_variants_meta":{"raw":{"variants":["LibraGrad rebalances gradients to fix Transformer attributions","Gradient balancing makes generic XAI beat Transformer-specific","Prune and scale backward paths to restore attribution completeness","Universal fix for Transformer explanations: balance gradient flow","Why Transformers distort saliency maps and how LibraGrad corrects it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1634,"prompt_tokens":1100,"completion_tokens":534,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":716,"tokens_out":534,"duration_ms":6340,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:47:56.321496+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a Transformer fine-tuned on ImageNet and compute attributions on the paper's own zebra/elephant COCO images with only the attention-gradient pruning applied and LayerNorm gradients untouched: if the pruned attention scores, rather than LayerNorm, encode the co-occurring-class distinction, the maps will localize the wrong animal. Conversely, the theory predicts that a Transformer whose normalizations are replaced by affine scaling needs no Libra LayerNorm fix, so a faithfulness gain there would disprove FG-completeness as the operative mechanism.","supporting_citations":[{"cited_title":"SkipPLUS: Skip the first few layers to better explain vision transform- ers","cited_arxiv_id":null,"evidence_quote":"SkipPLUS / FullGrad+: the base method that LibraGrad enhances, and the source of the evaluation protocol."},{"cited_title":"Full-gradient represen- tation for neural network visualization","cited_arxiv_id":null,"evidence_quote":"FullGrad: supplies the completeness-style attribution decomposition that FG-completeness formalizes and extends."},{"cited_title":"Axiomatic attribution for deep networks","cited_arxiv_id":null,"evidence_quote":"Integrated Gradients: the standard completeness baseline that LibraGrad is compared against and whose fixed-step approximation the paper proves unstable."},{"cited_title":"Token transformation matters: Towards faithful post-hoc ex- planation for vision transformer","cited_arxiv_id":null,"evidence_quote":"TokenTM: a Transformer-specific gradient-attention hybrid baseline that LibraGrad composes with and outperforms."},{"cited_title":"AttCAT: Explaining transformers via attentive class activation tokens","cited_arxiv_id":null,"evidence_quote":"AttCAT: a Transformer-specific gradient-attention hybrid baseline in the comparison."},{"cited_title":"De- compX: Explaining transformers decisions by propagating token decomposition","cited_arxiv_id":null,"evidence_quote":"DecompX: a comparison baseline used for completeness error and faithfulness metrics."}],"review_version":1}