Pith. sign in

REVIEW 4 major objections 5 minor 45 references

Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that a CNN heatmap method that seeds layer-wise relevance propagation with the softmax gradient can isolate the pixels that justify a chosen class, and that it outperforms existing LRP-based attributions.

desk verdict Small, honest LRP variant with modest class-discrimination gains; the zero-sum initialization is under-justified and the quantitative evidence lacks error bars. read the letter →

arxiv 1908.04351 v3 pith:TT4M2MZC submitted 2019-08-06 cs.CV cs.LGcs.NE

Brian Kenji Iwana , Ryohei Kuroki , Seiichi Uchida This is my paper · ORCID
classification cs.CVcs.LGcs.NE
keywords softmaxgradientlayer-wiserelevancepropagationclass-discriminativeattributionCNNinterpretabilitydeepTaylordecompositioncontrastiveLRPVGG16pixel-wisemap
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes SGLRP (Softmax-Gradient Layer-wise Relevance Propagation), a way to produce pixel-level heatmaps showing which parts of an image drove a convolutional neural network's decision for a chosen class. It claims that starting layer-wise relevance propagation from the gradient of the softmax output, rather than from a one-hot target as in standard LRP or from a fixed uniform penalty as in contrastive LRP, makes the heatmaps class-discriminative: regions belonging to non-target objects are actively suppressed, with suppression weight proportional to how probable those classes are. On ImageNet with VGG16, the resulting relevance maps localize target objects more tightly and, in the paper's quantitative tests, outperform other LRP-based attributions. A sympathetic reader would care because class-discriminative attribution is what lets a heatmap answer not just 'where is the model looking' but 'which object in a cluttered scene is being classified.'

What carries the argument

The mechanism is the softmax-gradient initialization of the top-layer relevance. In a network with softmax outputs $\hat{y}_n$ and pre-softmax logits $z_n$, the paper sets $R_n^{(L)} = \partial \hat{y}_t / \partial z_n$, which evaluates to $\hat{y}_t(1-\hat{y}_t)$ on the target and $-\hat{y}_t\hat{y}_n$ on every other class. This signal is then pushed down through the usual Deep Taylor Decomposition and LRP propagation rules. The key property is that the negative terms are probability-weighted, so an object that the network actually treats as a competing class receives a larger subtractive relevance, whereas classes that are absent or unlikely are barely penalized.

What would settle it

A direct test would compare SGLRP's most relevant patch with human-annotated object masks: if the top patch frequently falls outside the target object's bounding box while masking it barely changes the target probability, whereas masking a patch inside the box changes it more, then the relevance ranking is not tracking the true cause. The paper's pointing game partially addresses this, but a causal counterfactual—systematically masking the top-ranked patch against the ground-truth mask—would settle it.

Watch

Extended reading notes

Core claim

The central claim is that the output-layer relevance seed should be the derivative of the target softmax probability with respect to each pre-softmax logit: $R_t^{(L)} = \hat{y}_t(1-\hat{y}_t)$ for the target node and $R_n^{(L)} = -\hat{y}_t\hat{y}_n$ for every other class. This replaces LRP's one-hot choice and CLRP's uniform penalty with a probability-weighted subtraction. The paper argues that this is more natural because it ties the propagated relevance directly to the probability that the image belongs to the target class, and it demonstrates empirically that SGLRP's maps are more class-discriminative and score better on patch-masking and pointing-game evaluations.

Load-bearing premise

The quantitative comparison assumes that the drop in the target class probability when the maximally relevant patch is masked is a faithful measure of attribution quality, and the paper does not calibrate that proxy against human judgments or causal interventions.

Editorial extensions

If this is right

  • If the central claim holds, SGLRP can be applied to any CNN with a softmax output layer without retraining or architectural changes.
  • Because the maps are class-discriminative, the method can target a class that is not the predicted class, revealing where a particular object sits in a multi-object scene.
  • Probability-weighted subtraction should scale more gracefully to datasets with many similar classes, where a uniform penalty over-penalizes all non-target classes equally.
  • SGLRP preserves the balance property that the target class relevance equals the sum of the subtractive relevance, so the propagated signal remains conservative in the same sense as LRP.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication not stated in the paper is that the same softmax-gradient seed could be combined with other propagation rules or with higher-order attribution schemes, potentially improving class-discriminative heatmaps beyond the LRP family.
  • The paper's failure cases (small targets, targets that fill the frame) suggest that SGLRP depends on the target being one among distinguishable objects; a natural extension would be to combine it with objectness priors or multi-scale relevance aggregation.
  • Because the softmax gradient is already computed during standard backpropagation, SGLRP adds very little computational overhead, making it a plausible routine debugging tool for large deployed classifiers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Softmax-Gradient Layer-wise Relevance Propagation (SGLRP), an extension of LRP in which the initial relevance at the output layer is set to the gradient of the target softmax probability with respect to the pre-softmax logits (Eq. 10), rather than LRP's one-hot initialization (Eq. 3) or CLRP's uniform penalty (Eq. 7). The relevance is then propagated to the input using the standard LRP rules (Eqs. 1 and 2), and the final map is the positive part (Eq. 9). The authors evaluate SGLRP against LRP, CLRP, Guided Grad-CAM, and random baselines on ImageNet/VGG16 using qualitative heatmaps, maximal patch masking of the ground-truth and second-most-probable classes, and the extended Pointing Game. They report that SGLRP outperforms LRP and CLRP and is competitive with Guided Grad-CAM, with an open-source implementation released.

Significance. If the central claims were fully established, SGLRP would be a simple, parameter-free modification of LRP that improves class-discriminative attribution without retraining, and the authors provide an open-source implementation using iNNvestigate. The proposal is well grounded in the LRP framework, and the qualitative examples do show sharper separation of target and distractor objects. The main limitation is that the quantitative evidence for the central claim is currently not conclusive: the figures lack error bars or significance tests, and the principal evaluation metric is an uncalibrated causal-impact proxy. With additional statistical rigor and a check of the attribution-theoretic interpretation, this could be a useful contribution to the interpretability literature.

major comments (4)
  1. [Section 4, Eqs. (9) and (10)] The output-layer relevance vector defined in Eq. (10) is a derivative signal, not a score, and it sums to zero over the output nodes. Because the propagation rules in Eqs. (1) and (2) are homogeneous linear operations in the incoming relevance, the input-layer relevance also sums to zero before the max(., 0) operation in Eq. (9); the negative entries are discarded rather than explained. The paper therefore does not show that the final positive map decomposes y_t or any other network output, and the statement in Section 4 that SGLRP propagates values "that relate directly to the probability" remains a heuristic. The authors should either provide a derivation of what the clipped positive map represents or reposition SGLRP as an empirical heuristic and support it with stronger quantitative evidence.
  2. [Section 5.3.1, Fig. 11] Fig. 11 reports average changes in y_t without error bars, confidence intervals, or significance tests, even though the evaluation is run on 50,000 test images and the differences among methods are small (e.g., for the ground-truth target, SGLRP is close to Guided Grad-CAM for patch sizes p >= 5). Without variance estimates, the claim that SGLRP outperforms CLRP across all patch sizes cannot be assessed reliably. The authors should report standard errors or confidence intervals and a paired significance test (e.g., paired bootstrap or Wilcoxon signed-rank test) over images.
  3. [Section 5.3.1, maximal patch masking] The maximal-patch-masking metric is an indirect causal proxy: masking the maximally attributed patch and measuring the drop in y_t rewards any heatmap that points at an input region whose removal lowers the probability, including correlated background patches that the classifier has learned to rely on rather than the target object. The paper never calibrates this proxy against human judgments or known causal structure, so the quantitative advantage of SGLRP may not reflect better attribution quality. I recommend adding a sanity check, such as comparing against a baseline that masks the region from an object detector or reporting agreement with human-annotated object regions, to show that the metric tracks explanation quality rather than merely model sensitivity.
  4. [Section 5.3.3, Fig. 13] In the Pointing Game, SGLRP is only the best method for energy levels E >= 67%, while Guided Grad-CAM achieves higher Pointing Accuracy for E < 67%; no error bars are shown and the crossing point is not statistically tested. This result is consistent with SGLRP producing more diffuse positive maps, so the claim that SGLRP "had the highest Pointing Accuracy out of all of the LRP-based methods" is accurate, but the comparison to Guided Grad-CAM is more nuanced than the text suggests and needs a significance test.
minor comments (5)
  1. [Section 3.1, Fig. 3 caption] The caption says that z_2^(2) contributes to both z_3^(2) and z_3^(2), which appears to be a typo; it should probably refer to two different output nodes.
  2. [Section 3.1] The sentence "z_2^(2) is not specifically relevant to neither class 1 nor class 2" contains a double negative; it should be "not specifically relevant to either class 1 or class 2."
  3. [Section 5.3.3, Eq. (12)] The definition of Energy is written as E = #(R_t^(1) >= tau) / #(R_t^(1) > 0); the subscript t on R^(1) is unexplained, since the input-layer relevance map already corresponds to the target class being explained.
  4. [Section 5.1] The accuracy 69.63% should be identified as top-1 accuracy at the point where it is first introduced, since the later discussion in Section 5.3.2 relies on this distinction.
  5. [Related Work] Integrated Gradients and DeepLIFT are mentioned in the related-work section but are not included in the empirical comparisons; a brief sentence explaining why they are excluded would help the reader position SGLRP.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SGLRP's softmax-gradient initialization is an explicit design choice, not a fitted input or self-referential prediction, and the quantitative evaluation uses external benchmarks.

full rationale

The paper's derivation chain is self-contained. SGLRP sets the output-layer relevance to the analytic gradient of the softmax probability with respect to the pre-softmax logits (Eq. 10: R_n^(L) = y_t(1 - y_t) for the target and -y_t y_n otherwise), then propagates this signal through the standard LRP/DTD rules of Eqs. (1) and (2). No parameter is fitted to any subset of the evaluation data, and no predicted quantity is defined in terms of the method's own output. The claim that the maps relate to the target-class probability is a heuristic interpretation of the gradient, not a circular reduction: the gradient is computed from the same network being explained, which is the intended explanatory target rather than an imported conclusion. The quantitative evaluation uses externally defined metrics—maximal patch masking measured by re-classification and the Pointing Game with ground-truth bounding boxes—and these are not equivalent by construction to the softmax-gradient initialization. The only self-cited prior work (reference [13]) is cited as an application of LRP and is not load-bearing for the proposed method or its evaluation. Concerns about the patch-drop proxy's validity as a faithfulness measure are correctness/robustness issues, not circularity. Therefore, no circular step is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method has no fitted free parameters and introduces no new physical or algorithmic entities. It relies on the standard assumptions of LRP as a faithful redistribution scheme, on the appropriateness of the softmax gradient as an output-layer relevance signal, and on the validity of the patch-masking metric as a proxy for attribution quality.

assumptions (4)
  • domain assumption LRP redistribution rules (Eqs. 1 and 2) faithfully propagate output-layer relevance through ReLU networks.
    SGLRP reuses Eq. 1 and 2 unchanged; the soundness of the method inherits DTD's Taylor decomposition assumptions, which are not re-derived for the softmax-gradient initialization.
  • ad hoc to paper The gradient of softmax can serve as the output-layer relevance vector, and its zero-sum property preserves conservation under LRP.
    Section 4, Eq. 10. This is the paper's central design choice; it is not derived from an optimization or game-theoretic principle, though it is a natural class-contrastive heuristic.
  • domain assumption Masking the most relevant patch and measuring the drop in y_t evaluates attribution quality.
    Section 5.3.1 states that regions with high relevance should cause a larger drop in the target probability when removed. This is the load-bearing evaluation assumption.
  • ad hoc to paper Positive relevance is sufficient for visualization and evaluation; negative relevance is discarded via max(., 0).
    Eq. 9 and Section 5.2: only positive contributions are shown and normalized. The paper does not analyze what information the negative part carries.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation." pith.science (2026). https://pith.science/paper/TT4M2MZC

@misc{pith2026190804351,
  author       = {Pith},
  title        = {Pith review of: Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TT4M2MZC}},
  note         = {Machine review of arXiv:1908.04351}
}
read the original abstract

Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and explainability of the predictions of CNNs for multi-class classification problems. Specifically, we propose a novel visualization method of pixel-wise input attribution called Softmax-Gradient Layer-wise Relevance Propagation (SGLRP). The proposed model is a class discriminate extension to Deep Taylor Decomposition (DTD) using the gradient of softmax to back propagate the relevance of the output probability to the input image. Through qualitative and quantitative analysis, we demonstrate that SGLRP can successfully localize and attribute the regions on input images which contribute to a target object's classification. We show that the proposed method excels at discriminating the target objects class from the other possible objects in the images. We confirm that SGLRP performs better than existing Layer-wise Relevance Propagation (LRP) based methods and can help in the understanding of the decision process of CNNs.

Figures

Figures reproduced from arXiv: 1908.04351 by the authors.

Figure 1
Figure 1. Our SGLRP targets a specific output node [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. In this example, z (2) 2 contributes to both z (3) 2 and z (3) 2 meaning that z (2) 2 is not specifically relevant to either. However, LRP deems z (2) 2 the most relevant irrespective to the contribution to the non-target node. The thickness of the arrows indicate the contribution to the output in (a) and the relevance propagated in (b). ⋯ ⋯ ⋯ Target 𝑎1 (𝑙) 𝑤1,1 +(𝑙) ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ 𝑧𝑡 (𝐿) 𝑎𝑛+1 (𝑙) 𝑎𝑛 (𝑙) 𝑎𝑁 (𝑙) 𝑤1,𝑡 … view at source ↗
Figure 4
Figure 4. Illustration of CLRP for the output layer. a (l) n is the post-activation output of node n in layer l and z (L) t is the pre-softmax value of target node t. The range [b (l) n , h(l) n ] represents the lower and the upper limits of z (l) n , respectively. Finally, w +(l) n,m and w −(l) n,m are: w +(l) n,m = max(w (l) n,m, 0), (4) and w −(l) n,m = min(w (l) n,m, 0), (5) respectively. Using Eq. (2) at the input layer … view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: In this example, the subtractive relevance of [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Visualizations of the relevance of “Zebra.” [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 8
Figure 8. Figure 8: Comparisons on other networks targeting “Zebra.” [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 7
Figure 7. Figure 7: Example relevance maps for various target classes. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 11
Figure 11. Figure 11: Change in the value of yˆt when masking the (a) ground truth class and the (b) class with the second highest probability. Lower is better. class is recorded. Second, relevance maps from LRP, CLRP, SGLRP, and Guided Grad-CAM is calculated. Third, the maximal relevance …
Figure 9
Figure 9. Figure 9: Examples of bad results of SGLRP. Average Image of Dataset Relevance Map of Bison Input Image Find Maxima 𝑦 1 𝑦 𝑡 𝑦 𝑁・・・ ・・・ VGG16 𝑦 𝑡 = 0.98 𝑦 𝑡 = 0.95 Step 1: Classify Step 3: Step 5: Re-Classify Step 2 [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Demonstration of the maximal patch masking evalua [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 12
Figure 12. Figure 12: Illustration of the Pointing Game. Provided a bounding [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 43 canonical work pages

  1. [1]

    Alber, S

    M. Alber, S. Lapuschkin, P. Seegerer, M. H ¨agele, K. T. Sch¨utt, G. Montavon, W. Samek, K.-R. M ¨uller, S. D ¨ahne, and P.-J. Kindermans. innvestigate neural networks! Jour- nal of Machine Learning Research, 20(93):1–8, 2019. 8

    work page 2019

  2. [2]

    Alipanahi, A

    B. Alipanahi, A. Delong, M. T. Weirauch, and B. J. Frey. Pre- dicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nature Biotechnology, 33(8):831– 838, 2015. 2

    work page 2015

  3. [3]

    S. Bach, A. Binder, G. Montavon, F. Klauschen, K. R. M¨uller, and W. Samek. On pixel-wise explanations for non- linear classifier decisions by layer-wise relevance propaga- tion. PLOS ONE, 10(7):1–46, 2015. 1, 2, 3, 5, 6

    work page 2015

  4. [4]

    Bazzani, A

    L. Bazzani, A. Bergamo, D. Anguelov, and L. Torresani. Self-taught object localization with deep networks. InWinter Conference on Applications of Computer Vision, 2016. 2

    work page 2016

  5. [5]

    Fukui, T

    H. Fukui, T. Hirakawa, T. Yamashita, and H. Fujiyoshi. At- tention branch network: Learning of attention mechanism for visual explanation. In Conference on Computer Vision and Pattern Recognition, pages 10705–10714, 2019. 3

    work page 2019

  6. [6]

    C. Gan, N. Wang, Y . Yang, D.-Y . Yeung, and A. G. Haupt- mann. DevNet: A deep event network for multimedia event detection and evidence recounting. In Conference on Com- puter Vision and Pattern Recognition, 2015. 2

    work page 2015

  7. [7]

    A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks

    F. Gr ¨un, C. Rupprecht, N. Navab, and F. Tombari. A tax- onomy and library for visualizing learned features in convo- lutional neural networks. arXiv preprint arXiv:1606.07757,

    work page Pith review arXiv

  8. [8]

    J. Gu, Y . Yang, and V . Tresp. Understanding individual de- cisions of cnns via contrastive backpropagation. In Asian Conference on Computer Vision, 2018. 2, 3, 4, 5, 6, 7, 8

    work page 2018

Show all 45 references
  1. [9]

    K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In International Conference on Computer Vi- sion, pages 1026–1034, 2015. 1

    2015

  2. [10]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition, 2016. 5

    2016

  3. [11]

    Huang, Z

    G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. In Conference on Computer Vision and Pattern Recognition, 2017. 5

    2017

  4. [12]

    Ide and S

    S. Ide and S. Uchida. How does a CNN manage different printing types? In International Conference on Document Analysis and Recognition, 2017. 3

    2017

  5. [13]

    Jolly, B

    S. Jolly, B. K. Iwana, R. Kuroki, and S. Uchida. How do convolutional neural networks learn design? InInternational Conference on Pattern Recognition, 2018. 2, 3

    2018

  6. [14]

    Kindermans, K

    P.-J. Kindermans, K. T. Sch ¨utt, M. Alber, K.-R. M ¨uller, D. Erhan, B. Kim, and S. D ¨ahne. Learning How to Explain Neural Networks: PatternNet and PatternAttribution. In In- ternational Conference on Learning Representations , 2018. 2

    2018

  7. [15]

    LeCun, L

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner. Gradient- based learning applied to document recognition. Proceed- ings of IEEE, 86(11):2278–2324, 1998. 1

    1998

  8. [16]

    M. Lin, Q. Chen, and S. Yan. Network In Network. In Inter- national Conference on Learning Representations, 2014. 1, 2

    2014

  9. [17]

    S. M. Lundberg and S.-I. Lee. A unified approach to in- terpreting model predictions. In I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information Process- ing Systems, pages 4765–4774, 2017. 2

    2017

  10. [18]

    Mahendran and A

    A. Mahendran and A. Vedaldi. Visualizing deep convolu- tional neural networks using natural pre-images. Interna- tional Journal of Computer Vision , 120(3):233–255, 2016. 2

    2016

  11. [19]

    Montavon, S

    G. Montavon, S. Lapuschkin, A. Binder, W. Samek, and K. R. M ¨uller. Explaining nonlinear classification deci- sions with deep taylor decomposition. Pattern Recognition, 65:211–222, 2017. 1, 2

    2017

  12. [20]

    C. Olah, A. Mordvintsev, and L. Schubert. Feature visual- ization. Distill, 2017. 1, 2

    2017

  13. [21]

    C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev. The building blocks of inter- pretability. Distill, 2018. 3

    2018

  14. [22]

    Z. Qi, S. Khorram, and F. Li. Visualizing deep networks by optimizing with integrated gradients. In CVPR Workshops,

  15. [23]

    P. E. Rauber, S. G. Fadel, A. X. Falcao, and A. C. Telea. Visualizing the hidden activity of artificial neural networks. IEEE Transactions on Visualization and Computer Graphics, 23(1):101–110, 2017. 3

    2017

  16. [24]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. Imagenet large scale visual recog- nition challenge. International Journal of Computer Vision, 115(3):211–252, 2015. 5

    2015

  17. [25]

    Schmidhuber

    J. Schmidhuber. Deep learning in neural networks: An overview. Neural Networks, 61:85–117, 2015. 1

    2015

  18. [26]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-CAM: visual explanations from deep networks via gradient-based localization. In In- ternational Conference on Computer Vision, pages 618–626,

  19. [27]

    Shrikumar, P

    A. Shrikumar, P. Greenside, and A. Kundaje. Learning im- portant features through propagating activation differences. In International Conference on Machine Learning , pages 3145–3153, 2017. 2

    2017

  20. [28]

    Simonyan, A

    K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034,

    arXiv

  21. [29]

    Simonyan and A

    K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 1, 5

    2014 arXiv

  22. [30]

    J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Ried- miller. Striving for simplicity: The all convolutional net. In International Conference on Learning Representations ,

  23. [31]

    Sturm, S

    I. Sturm, S. Lapuschkin, W. Samek, and K.-R. Mller. Inter- pretable deep neural networks for single-trial EEG classifica- tion. Journal of Neuroscience Methods, 274:141–145, 2016. 2, 3

    2016

  24. [32]

    Sundararajan, A

    M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319–3328, 2017. 2

    2017

  25. [33]

    Szegedy, V

    C. Szegedy, V . Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. In Conference on Computer Vision and Pattern Recognition,

  26. [34]

    P. Tang, X. Wang, A. Wang, Y . Yan, W. Liu, J. Huang, and A. Yuille. Weakly supervised region proposal network and object detection. In Europen Conference on Computer Vi- sion, pages 370–386, 2018. 2

    2018

  27. [35]

    E. Teh, M. Rochan, and Y . Wang. Attention networks for weakly supervised object localization. In British Machine Vision Conference, 2016. 2

    2016

  28. [36]

    Uchida, S

    S. Uchida, S. Ide, B. K. Iwana, and A. Zhu. A further step to perfect accuracy by training CNN with larger data. In In- ternational Conference on Frontiers in Handwriting Recog- nition, pages 405–410, 2016. 1

    2016

  29. [37]

    L. Wan, M. Zeiler, S. Zhang, Y . L. Cun, and R. Fergus. Regu- larization of neural networks using dropconnect. In Interna- tional Conference on Machine Learning , pages 1058–1066,

  30. [38]

    Watanabe, K

    C. Watanabe, K. Hiramatsu, and K. Kashino. Modular rep- resentation of layered neural networks. Neural Networks , 97:62–73, 2018. 3

    2018

  31. [39]

    S. Xie, D. Chen, R. Zhang, and H. Xue. Deep features anal- ysis with attention networks. In AAAI Workshop on Network Interpretability for Deep Learning, 2019. 3

    2019

  32. [40]

    Yosinski, J

    J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson. Understanding neural networks through deep visualization. In ICLR Workshops, 2015. 1, 3

    2015

  33. [41]

    M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In European Conference on Com- puter Vision, pages 818–833, 2014. 1, 2, 3

    2014

  34. [42]

    Zhang, Z

    J. Zhang, Z. Lin, J. Brandt, X. Shen, and S. Sclaroff. Top- down neural attention by excitation backprop. In European Conference on Computer Vision, pages 543–559, 2016. 6, 8

    2016

  35. [43]

    B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Object detectors emerge in deep scene cnns. InInternational Conference on Learning Representations, 2014. 2

    2014

  36. [44]

    B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Tor- ralba. Learning deep features for discriminative localization. In Conference on Computer Vision and Pattern Recognition, pages 2921–2929, 2016. 1, 2

    2016

  37. [45]

    Zhou and O

    J. Zhou and O. G. Troyanskaya. Predicting effects of non- coding variants with deep learning–based sequence model. Nature Methods, 12(10):931–934, 2015. 2

    2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.