{"id":"a4309271-6961-43a3-b9bf-ad16a4c2a7c1","arxiv_id":"2501.14322","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"R-LRP is a hyperparameter-free LRP variant claimed to outperform prior methods, but it reduces to Input*Gradient and relies on an unproven conservation law.","lead":"The paper introduces R-LRP, a hyperparameter-free rule for explaining image classifiers by scoring pixel importance. It claims better focus and accuracy than earlier LRP methods, but the rule is a known Input*Gradient variant and its conservation-law justification is wrong.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"R-LRP violates its own conservation law: for a linear layer the input-relevance sum equals y^2/N, not c*y, invalidating the 'up to a factor' claim and Proposition 14.","rationale":"The strongest claim is the conservation law 'up to a factor' and the absence of divisions by small values. The no-division property holds by construction, but the conservation law fails already in the simplest linear case. This is an internal inconsistency, not a disagreement with consensus: it can be checked by direct substitution into equations (6)-(7). The reader's stated weakest_assumption (top-k masking) is about empirical evaluation, and the paper itself concedes quality metrics are only indicative; but even if masking were a perfect proxy, the theoretical claim remains false. The practical consequence is that Proposition 14's normalization is built on an unproven premise, so residual-network attribution has no theoretical support. The method may still be useful as a heuristic and could be benchmarked as a baseline; the empirical comparison with other LRP methods is not invalidated by the theoretical flaw, but the paper's central novelty claim is. Therefore the REJECT verdict stands, but the reason is the conservation-law error, not the evaluation proxy alone.","tokens_in":17450,"tokens_out":4700,"duration_ms":43591,"concrete_test":"Take a one-neuron linear layer y = sum_i w_i x_i + b with random w and x (N = 10). Compute R-LRP contributions via Eq. (6)-(7): z_i = (w_i x_i y) / N. Sum them and compare with y for several random x. Conservation 'up to a factor' predicts sum = c*y for a constant c; the formula predicts sum = y*(y-b)/N. This analytic check settles the issue without any network training; the same check on a two-layer linear network confirms the failure propagates.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is that R-LRP obeys conservation of relevance up to a multiplicative factor (Section 2.1 and Proposition 14). Combining Eq. (6) and (7) for a single dense layer with one output neuron, y = Wx + b, gives sum_i z_i,k = (1/N) * y * (y - b). With b = 0 this is y^2/N, not c*y for any constant c across different inputs. The factor y/N changes with the input, so 'up to a factor' is not a single global factor; the conservation law fails. Since Proposition 14's residual-block normalization is derived from the exact equality sum_N(l) z_p = sum_N(m) z_p, the residual handling inherits the same error. This is not a matter of evaluation protocol: it is an internal inconsistency in the method's defining property. The method may still behave as a plausible attribution heuristic, but the paper's central claim that R-LRP satisfies the classical conservation law up to a factor is false.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new attribution method, Relative Layer-Wise Relevance Propagation (R-LRP), defined by Eqs. (6)-(7). The rule propagates a selected output backward through dense, convolutional, and residual layers without dividing by pre-activations, and it is claimed to involve no hyperparameters and to satisfy the classical LRP conservation law up to a multiplicative factor. The empirical sections compare R-LRP with LRP0, LRP-epsilon, LRP-gamma, and LRP-alpha-beta on a modified MNIST task, a cat-vs-dog dataset, and a subset of Synthetic ImageNet-1K with VGG16, VGG19, and ResNet50, using a top-k pixel/input masking accuracy protocol and two mask-based qualitative measures.","tokens_in":17547,"tokens_out":4160,"duration_ms":40988,"significance":"The proposal is attractive in spirit: the base R-LRP rule is simple, contains no learned parameters beyond the network weights, and the authors release source code. The experimental study covers several architectures and includes both quantitative and qualitative evaluations. If the central conservation-law claim were correct, R-LRP would be a valuable parameter-free alternative to existing LRP variants. However, the central theoretical claim is not correct: the sum of input relevances is not proportional to the selected output by a constant factor. The empirical superiority claim is also weakened by the chosen masking protocol and limited sampling, so the main contributions of the paper do not currently hold.","major_comments":[{"comment":"The claimed conservation law \"up to a multiplicative factor\" is false. Consider a single fully connected layer with one output neuron y = sum_i w_i x_i and bias zero. With z^{(q+1)} = y, Eq. (6) gives z_i = (1/N^(0)) w_i x_i y and Eq. (7) gives sum_i z_i = (1/N^(0)) y sum_i w_i x_i = y^2 / N^(0). The ratio (sum_i z_i)/y = y/N^(0) depends on the input through y, so there is no constant c satisfying sum_i z_i = c y for all inputs. Thus Definition 8 is not satisfied even up to a global factor, contradicting the abstract and the statement in Section 2.1 that \"the sum of the inputs' contributions is equal to the selected output (up to a factor for our case).\"","section":"Section 2.1, Definition 9, Eqs. (6)-(7)"},{"comment":"The residual-block normalization in Eq. (13) is derived from the equality sum_N(l) z_p = sum_N(m) z_p, which is the conservation law shown above to fail for R-LRP. The rescaling factors in Eq. (13) are therefore data-dependent corrections rather than consequences of a valid conservation property. In addition, Eq. (13) divides by sums of relevances, which can be zero or arbitrarily small; this contradicts the paper's advertised property of avoiding division by small values, and the \"except for Resnet skip connection\" qualification does not cover these normalization denominators. The residual-block handling is therefore not soundly justified.","section":"Section 2.3, Proposition 14, Eq. (13)"},{"comment":"The main empirical evidence for superiority is the top-k masking protocol in which all but the most relevant inputs/pixels are set to zero and the network accuracy is measured. This protocol rewards methods that produce concentrated maps, regardless of whether the selected regions correspond to the actual decision process, and it does not compare against any ground-truth attribution. The ImageNet-1K evaluation in Section 5 uses only two random images per category, and no error bars or per-image variance are reported. These limitations are partially acknowledged only for the qualitative mask-based evaluation in Section 6, but they also affect the quantitative claims of \"better results\" in Section 7. As presented, the empirical evidence does not support the claim that R-LRP is superior to other LRP methods in general.","section":"Sections 3-5, evaluation protocol"}],"minor_comments":[{"comment":"The word \"eXplaination\" in the title should be \"Explanation.\"","section":"Title and Abstract"},{"comment":"The affiliation text \"partial secondment at Buawei\" appears to contain a typo; it should likely read \"Huawei.\"","section":"Authors' affiliations"},{"comment":"The dataset attribution \"Charm Myae Zaw and all\" is unclear; a formal Kaggle citation or a URL should be provided.","section":"Section 4.1"},{"comment":"The factor Card(J)/N^{(l+1)} in Eq. (7) is redundant when J is the full set of neurons in layer l+1, and its meaning for boundary positions in padded convolutions is not defined; the relationship between J and N^{(l+1)} should be clarified.","section":"Definition 9, Eq. (7)"}],"recommendation":"reject","confidential_remarks":"The core theoretical claim is demonstrably false for a single linear layer, and the residual normalization inherits the same error. The empirical protocol is also too weak to establish the claimed superiority. Since the central conceptual contribution of the paper is the conservation-law property, this is not a local fix that can be handled in a minor revision. The authors would need to reposition the work as a heuristic attribution method and substantially rework the empirical validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my take on the R-LRP paper. The core idea is genuinely simple: propagate the output back through the network by multiplying by weights and dividing by the layer width, never by the layer's weighted sum. That avoids the numerical blowups that plague LRP-0 and eliminates the need for epsilon/gamma/alpha-beta tuning. The authors ship code, test on MNIST, cat-vs-dog, and ImageNet-style data with multiple architectures, and report consistent improvements in top-k masking accuracy and pointing-game scores. That empirical work is real and the method is easy to implement, so as a practical baseline it has some value.\n\nBut the central theoretical claim does not survive contact with the math. In Definition 9, for a single linear layer with one output y = w^T x, the relevance sum over inputs is (1/N) * y^2 when the bias is zero, not a constant multiple of y. So Equation (2) fails even \"up to a factor\", because the factor y/N depends on the input. The paper asserts the conservation law holds up to a multiplicative factor and then builds the residual-block normalization (Proposition 14) on that exact equality. Since the premise is false, the residual handling is a self-referential fix, not a derivation. This is an internal inconsistency, not just a missing proof.\n\nThere's a second issue: the recursion is essentially Input*Gradient with a per-layer scaling constant and a specific seed. For linear layers the relative contribution ranking is identical to gradient*input. The paper doesn't cite or benchmark against Integrated Gradients or simple gradient*input, which overstates novelty. The ImageNet evaluation is also thin: two images per category, no error bars, assumed baseline accuracies of 90%/92%.\n\nThe masking protocol itself is a known evaluation style, so I don't fault the authors for using it; it's the interpretation that's fragile. The positive results are likely real in the sense that concentration on the object helps top-k accuracy, but that doesn't establish that the explanations are faithful.\n\nBottom line: the paper is a mixed bag. The method is simple and reproducible, the empirical comparisons are broad, but the load-bearing conservation claim is false and the novelty relative to Input*Gradient is undeclared. I'd send it to peer review, because a referee can pinpoint the algebra and the authors might salvage it as an empirical heuristic. But as it stands, I would not accept it as a novel scientific contribution, and I wouldn't cite it as a principled attribution method.","headline":"R-LRP is a simple, hyperparameter-free attribution rule that empirically looks strong, but the paper's central conservation-law claim is false: for a linear layer the relevance sum is y^2/N, not c*y, so the residual normalization rests on a broken premise.","tokens_in":18194,"tokens_out":2935,"would_cite":false,"duration_ms":22857,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a new relative layer-wise relevance propagation rule, R-LRP, explains dense, convolutional, and residual image classifiers without dividing by near-zero values or tuning hyperparameters, and that it outperforms…","keywords":["Layer-Wise Relevance Propagation","eXplainable Machine Learning","interpretability","neural network attribution","saliency maps","image classification","residual networks","hyperparameter-free"],"falsifier":"A controlled benchmark with known discriminative features (for example, a synthetic dataset where a specific image region determines the class) would settle the claim: if R-LRP's top-k pixels do not localize that region with higher precision than random selection or LRP0, the reported superiority of R-LRP would be an artifact of the masking protocol.","tokens_in":17154,"feed_emoji":"🧠","tokens_out":6008,"duration_ms":49062,"temperature":0.7,"pith_summary":"The paper proposes a new layer-wise relevance propagation rule, R-LRP, that assigns each input pixel a share of a chosen network output without ever dividing by the neuron pre-activations that make classic LRP0 unstable. The authors claim R-LRP needs no hyperparameters and, when the lowest-relevance inputs are masked out, it preserves classification accuracy better than LRP0, LRP-epsilon, LRP-gamma, and LRP-alpha-beta on MNIST, a cat-versus-dog benchmark, and ImageNet-scale data with VGG16, VGG19, and ResNet50. If the claim holds, explainability for deep image classifiers becomes simpler and more reliable because the same rule works across dense, convolutional, and residual architectures. The paper also argues that negative relevance values carry useful signal, so taking absolute values of the contributions improves object localization.","feed_headline":"R-LRP beats standard LRP without tuning or unstable divisions","feed_subtitle":"Attribution maps for deep image classifiers stay stable and object-focused with zero hyperparameter choices.","key_machinery":"The central object is the relative contribution formula $z_{i,j,k}^{(l)} = \\frac{1}{M_j^{(l+1)}} w_{i,j}^{(l+1)} x_i^{(l)} z_{j,k}^{(l+1)}$, followed by a layer-wise averaging step $z_{i,k}^{(l)} = \\frac{\\mathrm{Card}(J)}{N^{(l+1)}} \\sum_J z_{i,j,k}^{(l)}$. It replaces the ratio-based LRP rule (which divides by the pre-activation sum over inputs) with a multiplication-only recurrence, so near-zero denominators never appear. The factor $\\mathrm{Card}(J)/N^{(l+1)}$ rescales the sum of contributions so that the conservation law is preserved up to a global multiplicative constant, and the same machinery is extended to convolutions (via transposed convolutions), pooling, and residual blocks with a normalization of skip-connection and residual-path contributions.","core_discovery":"R-LRP computes the contribution of neuron i in layer l to output k through a backward product of weights, activations, and already-computed upper-layer relevance, normalized by the number of inputs to each neuron and averaged over connected outputs, as in equations (6)–(7). Because no denominator is formed from the network's weighted sums, the rule does not blow up when a neuron's pre-activation is near zero. The authors state that the sum of all input contributions equals the selected output up to a single multiplicative factor, so the classical LRP conservation law holds in relative form. On their masked-image accuracy benchmarks, R-LRP outperforms the standard LRP variants at small keep-fractions, and it produces more object-focused maps in pointing-game and distance-to-mask tests.","pith_inferences":["The top-k masking protocol may reward concentrated maps even when they are not faithful to the network's actual decision; a fairer test would compare against a random pixel baseline and use multiple masking strategies.","Because R-LRP's conservation law holds only up to a global factor, the raw relevance values are not directly comparable across different output classes; a calibration step would be needed for cross-class comparisons.","The normalization step for residual blocks is heuristic, and its validity for very deep networks with many branches could be tested on architectures that are even deeper than ResNet50.","A natural and testable extension is applying R-LRP to transformer-based architectures, which the authors explicitly list as future work; whether the relative-relevance rule survives attention layers is an open empirical question."],"forward_implications":["R-LRP can be dropped into existing dense, convolutional, and residual image classifiers without tuning epsilon, gamma, alpha, or beta, removing a common barrier to adopting LRP.","Explanations from R-LRP should remain stable in very deep networks because no stage of the backward pass divides by a quantity that can approach zero.","The masked-image accuracy protocol used in the paper, if it is accepted, provides a quantitative ranking of attribution methods that does not depend on a human looking at heatmaps.","Cross-network comparisons become feasible: the paper uses R-LRP to show that VGG16 and VGG19 rely on similar pixels while ResNet50 relies on different, more conceptual regions.","Absolute-value relevance can be reported as an additional map that highlights negative contributors, which the paper shows improves object localization."],"supporting_citations":[{"why":"Defines LRP0, the conservation law, and the LRP-alpha-beta rule that serve as the baseline and the theoretical target R-LRP is compared with.","marker":"[2]"},{"why":"Introduces the deep Taylor decomposition and LRP-gamma rule, a hyperparameterized baseline that R-LRP aims to replace.","marker":"[20]"},{"why":"Provides the LRP overview and the practical guidance on hyperparameter mixing that motivates R-LRP's parameter-free design.","marker":"[21]"},{"why":"Defines the ResNet residual block architecture that R-LRP must handle, including skip connections and the linear projection shortcut.","marker":"[11]"},{"why":"Documents artifact problems in residual-block relevance propagation that motivate the normalization step in R-LRP's ResNet extension.","marker":"[16]"},{"why":"Supplies the transposed-convolution computation that R-LRP uses to implement the backward relevance sum efficiently for convolutional layers.","marker":"[7]"}],"fun_headline_variants":["R-LRP beats standard LRP on attribution benchmarks","New LRP variant avoids small-value divisions for cleaner explanations","Relative LRP: stable relevance maps without hyperparameter tuning","R-LRP gives stable feature attribution with no tuning required","R-LRP removes unstable divisions from neural network explanations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that a method is a better explanation if keeping only its highest-relevance inputs and zeroing the rest preserves classification accuracy.","fun_headline_variants_meta":{"raw":{"variants":["R-LRP beats standard LRP on attribution benchmarks","New LRP variant avoids small-value divisions for cleaner explanations","Relative LRP: stable relevance maps without hyperparameter tuning","R-LRP gives stable feature attribution with no tuning required","R-LRP removes unstable divisions from neural network explanations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000627,"raw_usage":{"total_tokens":2886,"prompt_tokens":915,"completion_tokens":1971,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":1892}},"tokens_in":531,"tokens_out":1971,"duration_ms":13702,"temperature":1.0,"reasoning_tokens":1892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:15:46.842813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled benchmark with known discriminative features (for example, a synthetic dataset where a specific image region determines the class) would settle the claim: if R-LRP's top-k pixels do not localize that region with higher precision than random selection or LRP0, the reported superiority of R-LRP would be an artifact of the masking protocol.","supporting_citations":[{"cited_title":"Unmasking Clever Hans predictors and assessing what machines really learn","cited_arxiv_id":null,"evidence_quote":"Introduces the deep Taylor decomposition and LRP-gamma rule, a hyperparameterized baseline that R-LRP aims to replace."},{"cited_title":"Explaining nonlinear classification decisions with deep Taylor decomposition","cited_arxiv_id":null,"evidence_quote":"Provides the LRP overview and the practical guidance on hyperparameter mixing that motivates R-LRP's parameter-free design."},{"cited_title":"A Visual Designer of Layer-wise Relevance Propagation Models","cited_arxiv_id":null,"evidence_quote":"Documents artifact problems in residual-block relevance propagation that motivate the normalization step in R-LRP's ResNet extension."}],"review_version":1}