{"id":"2ce69f49-c249-4e15-b6ec-9b2bb6f6b5f0","arxiv_id":"2506.18037","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new explanation method for ReLU neural networks that builds piecewise linear models from selected connected subsets of hidden units, called paths, to produce adjustable and decomposable input attributions.","lead":"This paper presents a way to explain a neural network's decision by tracing a specific chain of neurons, called a path, instead of using the whole network. The method can zoom in on particular features of an image, such as a fox's eyes or ears, and can break one explanation into smaller pieces.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's importance score does not measure path-term contribution, so the selected path may not be the decision path.","rationale":"I agree with the reader's conditional verdict. The theoretical contribution is real: ReLU networks decompose into path terms, and using connected subsets is novel. Propositions 1-2 are straightforward and appear correct. The weakness is in the bridge from decomposition to explanation: Algorithm 1's importance score is not derived from the magnitude of path terms. The softmax over classes destroys absolute scale, so high-importance units can be noise and small-importance units can be decisive. The authors do not report how much of the logit the selected path accounts for, nor compare against random or oracle path selection. This is addressable: a small-network enumeration or a logit-coverage statistic would settle it. The insertion/deletion gains are modest and lack error bars, but the main issue is the selection algorithm's validity. Therefore no verdict change is needed beyond the reader's CONDITIONAL; the paper should be accepted only if the authors add evidence that selected paths dominate the decomposition or otherwise justify the importance measure.","tokens_in":14425,"tokens_out":9410,"duration_ms":96851,"concrete_test":"On a small ReLU network (e.g., the Figure 5 network or a 3-layer MLP), enumerate all one-way complete paths and compute the exact f_p(X) from Eq. (2) for a set of inputs. Run Algorithm 1 (with the paper's alpha and a range of widths) and compare the selected path's f_p to the largest |f_p| among activated paths and to the target logit f(X). Report the fraction of inputs where the selected path is not within, say, the top decile of path contributions or has sign opposite to the target logit. If that fraction is non-negligible, Algorithm 1 is not selecting decision paths and the central explanation claim fails; if it is near zero, the heuristic is validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The mathematical path decomposition (Eqs. 2-7, Props. 1-2) is sound. The load-bearing weak point is Algorithm 1, which converts that decomposition into an actual explanation. The importance score imp_j = softmax_c( W_{target,j} h_j^(n) ) (lines 12-13) is a per-unit, class-contrast measure: it normalizes each unit's gradient-times-activation across classes, so it depends on relative class preference, not on the magnitude of the corresponding path term f_p(X) in Eq. (2). A unit with a tiny gradient and activation can have softmax probability near 1 if all other class responses are even smaller, and will pass the threshold alpha=1/|classes| even though every path through it contributes negligibly to the target logit. Conversely, a dominant path term can be discarded if its unit-level softmax contrast is low. The paper's central claim - that the selected path is the decision-making path and yields consistent, decomposable explanations - requires that the selected subset actually carries a substantial portion of the network's output. Algorithm 1 provides no such guarantee, and the paper does not report the contribution of the selected paths to the logit. If the selected path's f_p is small or negative on typical inputs, the 'pathwise explanation' is an arbitrary subset, not the cause of the decision. This is not merely a threshold sensitivity issue; the selection objective itself is not aligned with the path decomposition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a 'pathwise' explanation method for ReLU neural networks. Instead of using the activation states of all hidden units, it defines a path as a connected subset of hidden units and derives piecewise linear models for one-way and multi-way complete paths (Propositions 1 and 2). Algorithm 1 selects a subset of hidden units based on a softmax importance score, and the resulting linear model is used to produce input attribution maps. The paper claims that this pathwise explanation is clearer and more consistent than existing methods and that it outperforms them quantitatively and qualitatively. Experiments are reported on a curated 10-class ImageNet subset with VGG-16 and ResNet-18, and on CIFAR-10, using insertion and deletion metrics as well as qualitative visualizations.","tokens_in":14654,"tokens_out":12210,"duration_ms":114831,"significance":"The idea of explaining a ReLU network through a small, connected subset of hidden units is appealing, and the one-way complete path decomposition (Eqs. (2)-(4), Proposition 1) is mathematically straightforward and correctly grounded in the piecewise-linear structure of ReLU networks. The paper provides a public code repository and reports experiments across multiple architectures, which are strengths. However, the multi-way path theory is not correctly established, and the path-selection algorithm's importance score is not aligned with the path decomposition; consequently, the central claim that the selected path actually explains the decision is not yet supported. With substantial revision of the selection criterion and the theory, the approach could become a useful contribution to the interpretability literature.","major_comments":[{"comment":"The importance score imp_j = softmax_c(W_{target,j} h_j^(n)) used for path selection measures the relative contrast of unit j's contribution to the target class versus other classes, not the magnitude of the path term f_p(X) from Eq. (2). A unit with a tiny value of |W_{target,j} h_j| can have imp_j close to 1 if all other class responses are even smaller, and it will pass the threshold alpha=1/|classes| even though every path through it contributes negligibly to the target logit. Conversely, a unit with a dominant path-term contribution but lower class-contrast can be discarded. Moreover, the algorithm does not enforce that selected units are activated (phi(h)=1); since h_j^(n) can be negative, a deactivated unit can have high imp_j, making f_p=0 even though W_p X is used as the attribution. The paper never reports the fraction of the target logit that the selected path carries. Because the central claim is that Algorithm 1 identifies the decision-making path, this lack of alignment between the selection objective and the path decomposition is a load-bearing gap. Please report the logit contribution of the selected path on test inputs, or change the selection criterion to one based on the actual path-term contribution, and explicitly filter to activated units.","section":"Section 3.3, Algorithm 1, lines 10-19"},{"comment":"Proposition 2 states that for a multi-way complete path p, the piecewise linear model f_p(X) 'represents a summation of terms from Equation (2)'. As written, this is not correct. For a path containing two units h_i, h_j in the same hidden layer and a downstream unit h_k, the sum of the corresponding one-way terms from Eq. (2) is (W_{i,k} phi(h_i) + W_{j,k} phi(h_j)) phi(h_k), while f_p as defined in Eq. (5) is (W_{i,k}+W_{j,k}) phi(h_i) phi(h_j) phi(h_k). These coincide only when phi(h_i)=phi(h_j)=1. The proof in Appendix A.2 introduces a remainder alpha(h_i,h_j) but does not show that this remainder vanishes or otherwise complete the proof. This is not merely a cosmetic issue, because the multi-way path is used in all experiments with width>1, and the theoretical justification for the attribution map W_p X depends on the claim that f_p is the correct decomposition. The proposition should be restated with the appropriate activation-region qualification, and the proof should be completed, or the multi-way path explanation should be presented as a heuristic rather than as an exact decomposition.","section":"Section 3.2 and Appendix A.2, Proposition 2"},{"comment":"The quantitative support for the claim that the method 'outperforms others both quantitatively and qualitatively' is weak. In Table 1, the proposed method achieves the best insertion score (0.936) but is third-best among the compared methods on deletion (0.179, versus BlurIG 0.168 and IGs 0.176). No standard errors or significance tests are reported, and the experimental setup uses a curated 10-class subset of ImageNet without any description of the number of images per class or the train/test split. The Abstract and Conclusion overstate the results. Please report variance across multiple runs/samples, run appropriate significance tests (e.g., paired tests on the insertion/deletion AUC), and temper the claim to something like 'state-of-the-art or competitive on insertion, with competitive deletion performance' unless the additional analysis supports a stronger claim.","section":"Section 5.3, Table 1, and the Abstract"}],"minor_comments":[{"comment":"The main-text statement of Proposition 1 says f_p(X) is non-zero if and only if the path is activated, but the proof in Appendix A.1 adds the condition 'and (W_p X + b_p) is non-zero'. Please align the statement with the proof.","section":"Section 3.1, Proposition 1"},{"comment":"The softmax in lines 12-13 is applied over the class dimension of the matrix W diag(h^(n)), but the text says 'softmax(W^(N+1) h^(N))' without specifying the axis. Please clarify the axis and state the intended interpretation of the resulting importance score.","section":"Section 3.3, Algorithm 1, lines 12-13"},{"comment":"The hyperparameters depth and width used to produce the results in Table 1 (and the qualitative figures in Section 5.1-5.2) are not specified. Since the method's behavior depends strongly on these hyperparameters, please provide the exact values used.","section":"Section 5.3, Table 1"},{"comment":"The curated 10-class subset of ImageNet is not described (which classes, how many images per class, how the subset was split). This makes the experiments difficult to reproduce; please include this information.","section":"Section 5, experimental setup"},{"comment":"The term 'consistent' is used as a key advantage of the method, but it is never formally defined or quantified. Please provide a definition or a metric for consistency, or reframe the claim.","section":"Abstract and Section 4"}],"recommendation":"major_revision","confidential_remarks":"The core one-way path decomposition is sound, but the paper's central claims rely on two problematic pieces: the multi-way path proposition is not correctly proved, and Algorithm 1's importance score is not aligned with the path decomposition. Both are fixable in principle, but they require substantive revision rather than copy editing. The empirical overstatement in the abstract should also be corrected. I would encourage the editor to send the revised manuscript back to reviewers if the authors address these points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The mathematical core here is in good shape. The expansion of a ReLU network into a sum over one-way paths is standard, but the paper adds a useful formalism: paths as connected subsets of hidden units, linear models for one-way and multi-way complete paths, and a Taylor linearization for incomplete paths that is valid in the local activation region. Propositions 1 and 2 are correct as stated. The demonstration that you can decompose a prediction for a fox into separate paths that highlight the eyes and ears is genuinely nice, and the code is promised, which helps reproducibility.\n\nThe soft spot is Algorithm 1. The importance score in lines 12-13 is softmax over classes of W_target,j * h_j for each unit. That measures how much a unit prefers the target class relative to other classes, not the magnitude of that unit's contribution to the output. A unit with a tiny activation and tiny gradient can still get a high softmax probability if all other class responses are even smaller, so it passes the threshold alpha=1/|classes| even though every path through it contributes almost nothing to the logit. Conversely, a path term that dominates the output could be dropped if its unit-level class contrast is low. The paper never reports the actual contribution (the f_p value) of the selected path. Without that, the central claim that the selected path is 'the decision-making path' is not supported; it might be an arbitrary connected subset that happens to be class-preferring.\n\nThe empirical section doesn't fix this. The main ImageNet/VGG-16 table has no error bars, the exact depth/width configuration for the headline 'Ours' row isn't specified, and 'consistency' is never formalized or measured. The appendix gives ResNet and CIFAR10 numbers that show similar trends, but the selection measure remains unverified.\n\nThat said, this is a fixable problem. Replacing or augmenting the importance score with a measure of the path term's actual contribution (or at least reporting it for selected paths) would directly address the strongest objection. The idea of connected-subset explanations is worth a serious referee; it deserves to be developed properly.\n\nFor a reading group, it's a decent 'maybe' — the path-sum treatment is clean enough to be instructive, and the selection flaw is a good lesson in how an intuitive heuristic can drift from the stated objective.\n\nRecommendation: accept for peer review, with the expectation of substantive revision, mainly around Algorithm 1 and the evaluation.","headline":"Solid path-sum math, but the path-selection heuristic doesn't measure path contribution, so the 'decision path' claim is unsubstantiated.","tokens_in":15209,"tokens_out":3423,"would_cite":false,"duration_ms":31617,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that a ReLU neural network is better explained by a connected subset of hidden units, called a path, than by all of its activated units, and that pathwise explanations are more consistent, decomposable, and…","keywords":["ReLU networks","explainable AI","pathwise explanation","piecewise linear models","input attribution","unwrapping","insertion deletion evaluation","neural network interpretability"],"falsifier":"For a fixed test input, compute the path model $f_p(X)$ for the selected path and compare it with the network output $f(X)$; if $f_p$ is substantially different from $f(X)$ while all units in $p$ are active, or if ablating the units in $p$ leaves the prediction essentially unchanged, then the selected path is not the decision-making path and the method's central claim fails.","tokens_in":3790,"feed_emoji":"🧠","tokens_out":3370,"duration_ms":120869,"temperature":0.7,"pith_summary":"This paper argues that a ReLU neural network is better explained by a connected subset of hidden units, called a path, than by the whole set of activated units. For each such path the paper derives a piecewise linear model, so the explanation is literally the local linear function the network effectively uses along that path. Pathwise explanations are claimed to be more consistent with the input and can be decomposed, so a single decision can be split into components such as a fox's eyes and ears. On insertion and deletion tests over a curated ImageNet subset, pathwise attribution maps score at or above six standard attribution methods.","feed_headline":"Neuron paths explain ReLU networks better than whole networks","feed_subtitle":"Attributions built from connected neuron paths beat whole-network methods and can single out a fox's eyes and ears.","key_machinery":"A path is a connected set of hidden units in the network graph; a one-way complete path contains at most one unit per layer, and a multi-way complete path sums over all one-way complete paths contained in it. The object doing the work is the piecewise linear model $f_p(X) = (W_p X + b_p)\\prod_{h\\in p}\\phi(h)$, whose weight $W_p$ is the product of weights along the path, or the sum of such products for a multi-way path. Because $f_p$ is a genuine term in the unfolded network expansion, a nonzero $f_p$ means the path is active. Algorithm 1 selects the active units by a softmax importance score, top-$k$ filtering per layer, and a threshold of $1/|\\mathrm{classes}|$, with depth and width controlling the spatial scale of the explanation.","core_discovery":"The paper shows that a ReLU network's output can be unfolded into a sum of path terms, one per one-way complete path, and that a path's term is nonzero exactly when the path is activated. For a multi-way complete path, the weight and bias of the path model are obtained by summing the weights and biases of all one-way complete paths inside it. The central proposal is to explain a prediction with only the path, or several paths, that Algorithm 1 selects, rather than with the local linear model derived from all activated hidden units. The authors argue this avoids mixing conflicting class evidence and enables explanations that can be decomposed either by high-level features or by separate objects in the input.","pith_inferences":["The paper does not test this, but the unfolding theory suggests a purely path-based alternative to the softmax top-$k$ heuristic: greedily select the activated one-way complete paths with the largest $|W_pX + b_p|$, which would inherit the consistency guarantee directly from the expansion.","The observed common-path-per-class pattern suggests pathwise explanations could serve as a per-class probe for auditing which high-level units a model relies on, including detecting shortcut or spurious features, without additional training.","The framework should transfer to other piecewise-linear activations such as leaky ReLU, maxout, and quantized ReLU, because their indicator functions also admit a pathwise unfolding, though the paper does not test these cases.","A testable extension is counterfactual generation: since the selected path gives an explicit linear model, perturbing only the input directions it weights most should efficiently reveal minimal changes that flip the decision."],"forward_implications":["Explanations of a single input can be decomposed: distinct selected paths highlight separate features such as eyes and ears, or separate objects, so a user can see which input component each path supports.","Because each path's linear model is nonzero only when its units are activated, the attribution no longer mixes positive and negative class evidence, giving more consistent explanations than unwrapping with all units.","Varying depth and width in path construction continuously adjusts the explanation from whole-input attribution to local component attribution, offering a tunable scale not provided by fixed full-network methods.","On insertion and deletion metrics, pathwise attributions outperform six standard attribution methods in insertion and remain competitive in deletion across VGG-16, ResNet-18, and CIFAR-10 settings.","Misclassification diagnosis becomes explicit: explanations for the predicted and correct classes reveal the features, such as background details, that support the wrong answer."],"supporting_citations":[{"why":"Defines unwrapping, the all-hidden-units baseline that pathwise explanation directly contrasts with.","marker":"Sudjianto et al. (2020)"},{"why":"Extends unwrapping to other network structures and serves as a comparison point for the pathwise approach.","marker":"Villani and McBurney (2023)"},{"why":"Provides the Saliency gradient-based attribution baseline used in experiments.","marker":"Simonyan et al. (2013)"},{"why":"Provides the Integrated Gradients baseline, an attribution method with sensitivity and implementation invariance axioms.","marker":"Sundararajan et al. (2017)"},{"why":"Provides the Guided Backpropagation baseline in the comparison.","marker":"Springenberg et al. (2014)"},{"why":"Provides the Grad-CAM baseline for localization-style attributions.","marker":"Selvaraju et al. (2017)"},{"why":"Provides the Blur Integrated Gradients baseline in the insertion and deletion comparisons.","marker":"Xu et al. (2020)"},{"why":"Supplies the insertion and deletion causal metrics used to score attribution maps.","marker":"Petsiuk et al. (2018)"},{"why":"Provides ImageNet, the source of the curated ten-class evaluation data.","marker":"Russakovsky et al. (2015)"},{"why":"Provides the VGG-16 architecture used in the main experiments.","marker":"Simonyan and Zisserman (2015)"}],"fun_headline_variants":["Pathwise explanations beat full-network attributions","Explaining ReLU nets via activated paths, not all units","Selected neuron paths yield clearer, decomposable explanations","Path-based attributions outperform whole-network methods","ReLU network paths provide better, decomposable explanations"],"cache_read_input_tokens":17280,"weakest_assumption_plain":"The load-bearing premise is that Algorithm 1's heuristic, which ranks hidden units by softmax importance over the target class and keeps the top $k$ above the threshold $1/|\\mathrm{classes}|$, picks exactly the units that form the decision-making path; a different ranking or threshold would select different units, and the claimed consistency and decomposability are not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Pathwise explanations beat full-network attributions","Explaining ReLU nets via activated paths, not all units","Selected neuron paths yield clearer, decomposable explanations","Path-based attributions outperform whole-network methods","ReLU network paths provide better, decomposable explanations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000755,"raw_usage":{"total_tokens":3286,"prompt_tokens":803,"completion_tokens":2483,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":419,"completion_tokens_details":{"reasoning_tokens":2410}},"tokens_in":419,"tokens_out":2483,"duration_ms":18238,"temperature":1.0,"reasoning_tokens":2410,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:55:58.303793+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For a fixed test input, compute the path model $f_p(X)$ for the selected path and compare it with the network output $f(X)$; if $f_p$ is substantially different from $f(X)$ while all units in $p$ are active, or if ablating the units in $p$ leaves the prediction essentially unchanged, then the selected path is not the decision-making path and the method's central claim fails.","supporting_citations":[{"cited_title":"Unwrapping All ReLU Networks","cited_arxiv_id":"2305.09424","evidence_quote":"Extends unwrapping to other network structures and serves as a comparison point for the pathwise approach."},{"cited_title":"Axiomatic attribution for deep networks","cited_arxiv_id":null,"evidence_quote":"Provides the Integrated Gradients baseline, an attribution method with sensitivity and implementation invariance axioms."},{"cited_title":"RISE: randomized input sampling for explanation of black-box models","cited_arxiv_id":null,"evidence_quote":"Supplies the insertion and deletion causal metrics used to score attribution maps."},{"cited_title":"Very deep convolutional networks for large-scale image recognition","cited_arxiv_id":null,"evidence_quote":"Provides the VGG-16 architecture used in the main experiments."}],"review_version":2}