{"id":"e1405f40-702f-41a3-a7de-90e2e7fffaf6","arxiv_id":"1908.04351","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SGLRP initializes layer-wise relevance propagation with the softmax gradient, yielding class-discriminative pixel attribution maps that outperform LRP and CLRP on ImageNet localization metrics.","lead":"This paper proposes SGLRP, a small change to layer-wise relevance propagation that starts the explanation signal from the gradient of the softmax output instead of the raw pre-softmax score. The result is a pixel-attribution map that suppresses non-target classes more strongly than earlier LRP variants, and the paper reports better class-discriminative localization on ImageNet.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that SGLRP yields more faithful target-class attributions is not yet established: the softmax-gradient initialization is a zero-sum derivative signal, and the quantitative support is an uncalibrated patch-drop proxy without error bars.","rationale":"The softmax-gradient derivative in Eq. (10) is computed correctly, and the qualitative examples show plausible class-discriminative behavior. The concern is not the derivation but the interpretation: LRP rules are designed to decompose an output score, and propagating a zero-sum derivative with the z+ rule produces a map whose total relevance is zero before clipping, so it is unclear what, if anything, the final positive map conserves or decomposes. The patch-masking metric is a reasonable causal-impact proxy for explaining a classifier, but it is not calibrated as an explanation-faithfulness measure, and the absence of error bars or significance tests makes the claimed improvement over CLRP uncertain. An ablation against Input×Gradient is the decisive check, because it separates the contribution of the softmax-gradient signal from the contribution of the LRP propagation rules. If SGLRP is essentially Input×Gradient in disguise, the novelty is smaller than claimed; if the confidence interval against CLRP crosses zero, the quantitative headline is unsupported. These are addressable concerns rather than fatal flaws, so the reader's CONDITIONAL verdict remains appropriate without change.","tokens_in":12668,"tokens_out":8444,"duration_ms":100413,"concrete_test":"On a fixed 5,000-image subset of ILSVRC2012, compute SGLRP maps alongside (i) plain Input×Gradient of ŷ_t, (ii) LRP with the standard one-hot output relevance, (iii) CLRP, and (iv) Guided Grad-CAM. Run the same maximal patch masking and Pointing Game protocols as Figs. 11 and 13, but report paired bootstrap 95% confidence intervals for the per-image differences. If SGLRP does not significantly beat the Input×Gradient baseline, the improvement cannot be attributed to the LRP propagation machinery; if the SGLRP-vs-CLRP interval crosses zero, the headline comparison is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (10) sets the output-layer relevance to the softmax derivative ∂ŷ_t/∂z_n. This is mathematically correct, but it is a derivative, not a score, and its entries sum to zero. Propagating this with the z+ LRP rule preserves the sum, so the input relevance is also zero-sum before the max(·,0) in Eq. (9) discards all negative evidence. The paper gives no argument that the remaining positive map decomposes ŷ_t or any other network output, and the statement that the maps 'relate directly to the probability' is a heuristic. The main quantitative comparison (Sec. 5.3.1, Fig. 11) uses maximal patch masking, a causal-impact proxy that can reward a heatmap pointing at a correlated background patch the classifier relies on rather than the object; the metric is never calibrated against human judgments or known causal structure, and Figs. 11 and 13 are plotted without error bars or significance tests. Because the claimed advantage over CLRP is the central result, the load-bearing assumption is that the softmax-gradient LRP signal is both class-discriminative and faithful—exactly what is untested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Softmax-Gradient Layer-wise Relevance Propagation (SGLRP), an extension of LRP in which the initial relevance at the output layer is set to the gradient of the target softmax probability with respect to the pre-softmax logits (Eq. 10), rather than LRP's one-hot initialization (Eq. 3) or CLRP's uniform penalty (Eq. 7). The relevance is then propagated to the input using the standard LRP rules (Eqs. 1 and 2), and the final map is the positive part (Eq. 9). The authors evaluate SGLRP against LRP, CLRP, Guided Grad-CAM, and random baselines on ImageNet/VGG16 using qualitative heatmaps, maximal patch masking of the ground-truth and second-most-probable classes, and the extended Pointing Game. They report that SGLRP outperforms LRP and CLRP and is competitive with Guided Grad-CAM, with an open-source implementation released.","tokens_in":12880,"tokens_out":4916,"duration_ms":51250,"significance":"If the central claims were fully established, SGLRP would be a simple, parameter-free modification of LRP that improves class-discriminative attribution without retraining, and the authors provide an open-source implementation using iNNvestigate. The proposal is well grounded in the LRP framework, and the qualitative examples do show sharper separation of target and distractor objects. The main limitation is that the quantitative evidence for the central claim is currently not conclusive: the figures lack error bars or significance tests, and the principal evaluation metric is an uncalibrated causal-impact proxy. With additional statistical rigor and a check of the attribution-theoretic interpretation, this could be a useful contribution to the interpretability literature.","major_comments":[{"comment":"The output-layer relevance vector defined in Eq. (10) is a derivative signal, not a score, and it sums to zero over the output nodes. Because the propagation rules in Eqs. (1) and (2) are homogeneous linear operations in the incoming relevance, the input-layer relevance also sums to zero before the max(., 0) operation in Eq. (9); the negative entries are discarded rather than explained. The paper therefore does not show that the final positive map decomposes y_t or any other network output, and the statement in Section 4 that SGLRP propagates values \"that relate directly to the probability\" remains a heuristic. The authors should either provide a derivation of what the clipped positive map represents or reposition SGLRP as an empirical heuristic and support it with stronger quantitative evidence.","section":"Section 4, Eqs. (9) and (10)"},{"comment":"Fig. 11 reports average changes in y_t without error bars, confidence intervals, or significance tests, even though the evaluation is run on 50,000 test images and the differences among methods are small (e.g., for the ground-truth target, SGLRP is close to Guided Grad-CAM for patch sizes p >= 5). Without variance estimates, the claim that SGLRP outperforms CLRP across all patch sizes cannot be assessed reliably. The authors should report standard errors or confidence intervals and a paired significance test (e.g., paired bootstrap or Wilcoxon signed-rank test) over images.","section":"Section 5.3.1, Fig. 11"},{"comment":"The maximal-patch-masking metric is an indirect causal proxy: masking the maximally attributed patch and measuring the drop in y_t rewards any heatmap that points at an input region whose removal lowers the probability, including correlated background patches that the classifier has learned to rely on rather than the target object. The paper never calibrates this proxy against human judgments or known causal structure, so the quantitative advantage of SGLRP may not reflect better attribution quality. I recommend adding a sanity check, such as comparing against a baseline that masks the region from an object detector or reporting agreement with human-annotated object regions, to show that the metric tracks explanation quality rather than merely model sensitivity.","section":"Section 5.3.1, maximal patch masking"},{"comment":"In the Pointing Game, SGLRP is only the best method for energy levels E >= 67%, while Guided Grad-CAM achieves higher Pointing Accuracy for E < 67%; no error bars are shown and the crossing point is not statistically tested. This result is consistent with SGLRP producing more diffuse positive maps, so the claim that SGLRP \"had the highest Pointing Accuracy out of all of the LRP-based methods\" is accurate, but the comparison to Guided Grad-CAM is more nuanced than the text suggests and needs a significance test.","section":"Section 5.3.3, Fig. 13"}],"minor_comments":[{"comment":"The caption says that z_2^(2) contributes to both z_3^(2) and z_3^(2), which appears to be a typo; it should probably refer to two different output nodes.","section":"Section 3.1, Fig. 3 caption"},{"comment":"The sentence \"z_2^(2) is not specifically relevant to neither class 1 nor class 2\" contains a double negative; it should be \"not specifically relevant to either class 1 or class 2.\"","section":"Section 3.1"},{"comment":"The definition of Energy is written as E = #(R_t^(1) >= tau) / #(R_t^(1) > 0); the subscript t on R^(1) is unexplained, since the input-layer relevance map already corresponds to the target class being explained.","section":"Section 5.3.3, Eq. (12)"},{"comment":"The accuracy 69.63% should be identified as top-1 accuracy at the point where it is first introduced, since the later discussion in Section 5.3.2 relies on this distinction.","section":"Section 5.1"},{"comment":"Integrated Gradients and DeepLIFT are mentioned in the related-work section but are not included in the empirical comparisons; a brief sentence explaining why they are excluded would help the reader position SGLRP.","section":"Related Work"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis is a small but legitimate extension of LRP. Instead of CLRP's uniform penalty on non-target classes, SGLRP uses the softmax gradient as output-layer relevance, so each competing class is penalized in proportion to its predicted probability. The formula is correct, the method is clearly specified, and the authors promise code. The qualitative examples on VGG16 and other networks show the intended class-discriminative behavior, and they honestly report failure cases where the target object fills the frame or is tiny.\n\nThe genuinely new element is Eq. (10), and the paper does not oversell it: it is a small delta over CLRP with a plausible motivation. The quantitative evaluation—patch masking of both the ground-truth and second-probable class, plus the extended Pointing Game—consistently puts SGLRP ahead of LRP and CLRP, although Guided Grad-CAM stays competitive on the ground-truth masking test at larger patch sizes.\n\nThe soft spots are real but not fatal. The figures have no error bars or significance tests, and the patch-masking metric is an indirect proxy for attribution quality; the paper never calibrates it against human judgment or known causal structure. The conceptual story is also thin: the softmax gradient sums to zero, the propagated relevance therefore sums to zero before the max(0,·) clipping, and the paper never explains why the surviving positive map should be read as a decomposition of the target probability. That is a genuine gap, though CLRP shares the same zero-sum starting point, so it is not unique to this method.\n\nThe citation pattern is fair, the comparison with CLRP is direct, and there are no fitted parameters or circular evaluations. The paper is a solid incremental contribution to the attribution toolbox, not a game-changer. It deserves a serious referee. I would accept after the authors add error bars, run significance tests, and tighten the justification of the zero-sum initialization.\n\nBest,\n[You]","headline":"Small, honest LRP variant with modest class-discrimination gains; the zero-sum initialization is under-justified and the quantitative evidence lacks error bars.","tokens_in":13426,"tokens_out":3703,"would_cite":true,"duration_ms":35412,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes that a CNN heatmap method that seeds layer-wise relevance propagation with the softmax gradient can isolate the pixels that justify a chosen class, and that it outperforms existing LRP-based attributions.","keywords":["softmax gradient","layer-wise relevance propagation","class-discriminative attribution","CNN interpretability","deep Taylor decomposition","contrastive LRP","VGG16","pixel-wise relevance map"],"falsifier":"A direct test would compare SGLRP's most relevant patch with human-annotated object masks: if the top patch frequently falls outside the target object's bounding box while masking it barely changes the target probability, whereas masking a patch inside the box changes it more, then the relevance ranking is not tracking the true cause. The paper's pointing game partially addresses this, but a causal counterfactual—systematically masking the top-ranked patch against the ground-truth mask—would settle it.","tokens_in":12428,"feed_emoji":"🎯","tokens_out":4110,"duration_ms":41002,"temperature":0.7,"pith_summary":"This paper proposes SGLRP (Softmax-Gradient Layer-wise Relevance Propagation), a way to produce pixel-level heatmaps showing which parts of an image drove a convolutional neural network's decision for a chosen class. It claims that starting layer-wise relevance propagation from the gradient of the softmax output, rather than from a one-hot target as in standard LRP or from a fixed uniform penalty as in contrastive LRP, makes the heatmaps class-discriminative: regions belonging to non-target objects are actively suppressed, with suppression weight proportional to how probable those classes are. On ImageNet with VGG16, the resulting relevance maps localize target objects more tightly and, in the paper's quantitative tests, outperform other LRP-based attributions. A sympathetic reader would care because class-discriminative attribution is what lets a heatmap answer not just 'where is the model looking' but 'which object in a cluttered scene is being classified.'","feed_headline":"Softmax-gradient heatmaps separate target objects from distractors","feed_subtitle":"Starting relevance from the softmax derivative subtracts non-target classes in proportion to their probability.","key_machinery":"The mechanism is the softmax-gradient initialization of the top-layer relevance. In a network with softmax outputs $\\hat{y}_n$ and pre-softmax logits $z_n$, the paper sets $R_n^{(L)} = \\partial \\hat{y}_t / \\partial z_n$, which evaluates to $\\hat{y}_t(1-\\hat{y}_t)$ on the target and $-\\hat{y}_t\\hat{y}_n$ on every other class. This signal is then pushed down through the usual Deep Taylor Decomposition and LRP propagation rules. The key property is that the negative terms are probability-weighted, so an object that the network actually treats as a competing class receives a larger subtractive relevance, whereas classes that are absent or unlikely are barely penalized.","core_discovery":"The central claim is that the output-layer relevance seed should be the derivative of the target softmax probability with respect to each pre-softmax logit: $R_t^{(L)} = \\hat{y}_t(1-\\hat{y}_t)$ for the target node and $R_n^{(L)} = -\\hat{y}_t\\hat{y}_n$ for every other class. This replaces LRP's one-hot choice and CLRP's uniform penalty with a probability-weighted subtraction. The paper argues that this is more natural because it ties the propagated relevance directly to the probability that the image belongs to the target class, and it demonstrates empirically that SGLRP's maps are more class-discriminative and score better on patch-masking and pointing-game evaluations.","pith_inferences":["An implication not stated in the paper is that the same softmax-gradient seed could be combined with other propagation rules or with higher-order attribution schemes, potentially improving class-discriminative heatmaps beyond the LRP family.","The paper's failure cases (small targets, targets that fill the frame) suggest that SGLRP depends on the target being one among distinguishable objects; a natural extension would be to combine it with objectness priors or multi-scale relevance aggregation.","Because the softmax gradient is already computed during standard backpropagation, SGLRP adds very little computational overhead, making it a plausible routine debugging tool for large deployed classifiers."],"forward_implications":["If the central claim holds, SGLRP can be applied to any CNN with a softmax output layer without retraining or architectural changes.","Because the maps are class-discriminative, the method can target a class that is not the predicted class, revealing where a particular object sits in a multi-object scene.","Probability-weighted subtraction should scale more gracefully to datasets with many similar classes, where a uniform penalty over-penalizes all non-target classes equally.","SGLRP preserves the balance property that the target class relevance equals the sum of the subtractive relevance, so the propagated signal remains conservative in the same sense as LRP."],"supporting_citations":[{"why":"Defines the original Layer-wise Relevance Propagation framework that SGLRP extends.","marker":"[3]"},{"why":"Introduces Contrastive LRP and the patch-masking and extended pointing-game evaluation protocols used as baselines.","marker":"[8]"},{"why":"Provides the Deep Taylor Decomposition propagation rules that SGLRP inherits for layers below the output.","marker":"[19]"},{"why":"Defines Guided Grad-CAM, one of the main comparative baselines in the qualitative and quantitative evaluations.","marker":"[26]"},{"why":"Supplies the VGG16 architecture on which all experiments are run.","marker":"[29]"},{"why":"Supplies the ILSVRC2012 dataset and the 50,000-image test set used for evaluation.","marker":"[24]"},{"why":"Introduces the original Pointing Game, later extended by [8], used to measure localization accuracy.","marker":"[42]"}],"fun_headline_variants":["Softmax gradient makes LRP class-discriminative","SGLRP: Softmax-seeded relevance maps separate classes","Probability-weighted relevance outperforms one-hot LRP","Softmax-gradient relevance improves class attribution","Class-discriminative heatmaps via softmax gradient"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative comparison assumes that the drop in the target class probability when the maximally relevant patch is masked is a faithful measure of attribution quality, and the paper does not calibrate that proxy against human judgments or causal interventions.","fun_headline_variants_meta":{"raw":{"variants":["Softmax gradient makes LRP class-discriminative","SGLRP: Softmax-seeded relevance maps separate classes","Probability-weighted relevance outperforms one-hot LRP","Softmax-gradient relevance improves class attribution","Class-discriminative heatmaps via softmax gradient"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2787,"prompt_tokens":866,"completion_tokens":1921,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":1847}},"tokens_in":482,"tokens_out":1921,"duration_ms":15006,"temperature":1.0,"reasoning_tokens":1847,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:52:56.364073+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would compare SGLRP's most relevant patch with human-annotated object masks: if the top patch frequently falls outside the target object's bounding box while masking it barely changes the target probability, whereas masking a patch inside the box changes it more, then the relevance ranking is not tracking the true cause. The paper's pointing game partially addresses this, but a causal counterfactual—systematically masking the top-ranked patch against the ground-truth mask—would settle it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the original Layer-wise Relevance Propagation framework that SGLRP extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Contrastive LRP and the patch-masking and extended pointing-game evaluation protocols used as baselines."},{"cited_title":"Montavon, S","cited_arxiv_id":null,"evidence_quote":"Provides the Deep Taylor Decomposition propagation rules that SGLRP inherits for layers below the output."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Guided Grad-CAM, one of the main comparative baselines in the qualitative and quantitative evaluations."},{"cited_title":"Russakovsky, J","cited_arxiv_id":null,"evidence_quote":"Supplies the ILSVRC2012 dataset and the 50,000-image test set used for evaluation."},{"cited_title":"Zhang, Z","cited_arxiv_id":null,"evidence_quote":"Introduces the original Pointing Game, later extended by [8], used to measure localization accuracy."}],"review_version":1}