Computing Linear Restrictions of Neural Networks

Aditya V. Thakur; Matthew Sotoudeh

REVIEW 4 major objections 4 minor 41 references

Computing Linear Restrictions of Neural Networks

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For any piecewise-linear neural network and any input line segment, ExactLine computes an exact partition into affine pieces, yielding exact decision boundaries, exact integrated gradients, and a falsification of the linear explanation of…

desk verdict ExactLine is a genuinely useful primitive with a new exact integrated-gradients result, but the paper overstates its universality: degenerate lines and floating-point error are waved at rather than handled. read the letter →

arxiv 1908.06214 v2 pith:NC4OV3NT submitted 2019-08-17 cs.LG stat.ML

Matthew Sotoudeh , Aditya V. Thakur This is my paper · ORCID

classification cs.LGstat.ML

keywords linearrestrictionspiecewise-linearneuralnetworksExactLineReLUintegratedgradientsadversarialexamplesdecisionboundariesnetworkverification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces ExactLine, an algorithm that, for any piecewise-linear neural network and any line segment in the input, partitions the segment into pieces on each of which the network is an affine function. Because every partition endpoint is computed rather than sampled, the representation describes the network's behavior on all infinitely many points of the segment. The authors use this primitive to determine decision boundaries of an aircraft collision-avoidance network exactly, to compute integrated gradients exactly for the first time, and to show that approximating those gradients with left Riemann sums carries 25–45% relative error on CIFAR10 networks. They also use it to test the linear explanation of adversarial examples, finding that adversarial directions cross more linear partitions than random directions, contradicting a core assumption of that hypothesis.

What carries the argument

The central object is the linear partitioning $\mathcal{P}(f\restriction QR)$, a tuple of endpoints $P_1=Q,\ldots,P_n=R$ such that $f$ is affine on each segment $P_iP_{i+1}$. Once the partition is known, the network's output at any point on the line follows from affine interpolation $f(x)=(1-\alpha)f(P_i)+\alpha f(P_{i+1})$, and gradients and integrals become constants on each piece. The algorithm's workhorse is a ratio-pullback identity: when an affine map $g$ sends segment $P_iP_{i+1}$ to $g(P_i)g(P_{i+1})$, a partition point $y$ in the output line is pulled back to the input through $P_i + \frac{y-g(P_i)}{g(P_{i+1})-g(P_i)}(P_{i+1}-P_i)$. ReLU splits are found by solving $Q_i+\alpha(R_i-Q_i)=0$, producing at most one crossing per coordinate per line.

What would settle it

Compute ExactLine on a ReLU network with a rational-arithmetic implementation and compare partition endpoints and integrated-gradient errors to the floating-point results; any material shift in endpoints or in the reported 25–45% approximation-error range shows the exactness claims are not numerically supported.

Watch

Extended reading notes

Core claim

The central claim is that the linear restriction of a piecewise-linear network can be represented exactly and efficiently enough for practical analysis: for a fixed number of layers, the number of affine segments along a line is polynomial, and the algorithm runs in seconds on networks with tens of thousands of units. The proof mechanism is to track a line through each layer: affine layers add no splits, ReLU layers split wherever a coordinate crosses zero, MaxPool layers split when the maximizer changes, and the composition theorem pulls partition points back through affine pieces by ratio preservation. With this representation, integrated gradients become a finite sum of constant-gradient terms rather than an integral needing approximation. The paper further claims that exact decision boundaries for ACAS Xu reveal behaviors invisible to finite sampling, and that partition density along FGSM attacks falsifies the linear explanation of adversarial examples while robustly trained networks show fewer partitions.

Load-bearing premise

The central exactness claims assume that every partition endpoint is computed exactly; the implementation uses floating-point arithmetic, and Appendix L concedes endpoints may be slightly inaccurate, which would make all reported exact results approximate rather than exact.

Editorial extensions

If this is right

Decision-boundary analysis along any line is exact, so safety evaluations can avoid the arbitrary sampling densities that miss regions like the 'strong left' island in ACAS Xu.
Integrated gradients can be computed exactly in seconds on small-to-medium networks, making attribution checks auditable instead of heuristic.
The standard left-Riemann integrated-gradient approximation carries 25–45% relative error; switching to trapezoidal sampling cuts the samples needed for 5% error by 20–40%.
FGSM adversarial examples are associated with unusually non-linear directions, not linear ones, so the linear explanation's core premise is empirically false.
Adversarially trained networks, especially DiffAI-trained ones, have up to roughly five times fewer linear partitions along tested directions, indicating flatter, more linear neighborhoods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because ExactLine's complexity is polynomial for fixed depth, the same exact line-restriction primitive could serve as a fast pre-filter inside complete verifiers: check infinite line segments for counterexamples and return them immediately before a slower full-dimensional solver runs.
The partition-density metric suggests a testable causal hypothesis the paper leaves open: if one regularizes networks to stabilize ReLU patterns and reduce partitions, adversarial robustness should improve if linearity causes robustness; otherwise the observed correlation is an effect, not a cause.
The exact-integrated-gradients machinery should transfer to other path-integral attribution measures such as neuron conductance, replacing sampling error with the same finite partition sum; the paper names this as future work, and the mechanism is already in place.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should read this one if you care about exact analysis of piecewise-linear networks. The core idea is simple and solid: restrict the network to a line, track where each layer's linear regions change, and you get an exact piecewise-affine description of the network along that line. Composing per-layer partitionings into a whole-network partitioning is done cleanly, and the worst-case segment bounds (O(dl) for ReLU networks) are a real improvement over prior region-based enumeration. The exact integrated gradients application is the standout: it is new, it is useful, and the 25-45% error numbers for the standard approximation are worth knowing. The code is public, which is good practice and makes the empirical claims checkable.

The soft spots are real but not fatal. Theorem 2's formula divides by R_i - Q_i, so constant coordinates are undefined; that includes the black-baseline IG case where input pixels are zero along the whole interpolation path. You can patch it by ignoring dimensions that never cross zero, but the paper does not say so. Theorem 3 has the same issue: it divides by g(P_{i+1}) - g(P_i) and only mentions the degenerate case in a parenthetical, not in the theorem statement or algorithm. Both are fixable, but the universal "any line segment" claim is not actually proven as written. The stress-test note is fair on this point.

Also fair: Appendix L admits floating-point computations are used and gives no error bound. Calling the results "exact" is therefore too strong. The errors are probably small in practice, but the paper should either bound them or soften the language. The adversarial-examples section is suggestive rather than conclusive—no significance tests, and the FGSM/random density ratios overlap at the 25th percentile for MNIST. The authors do report medians and quartiles and also measure gradient error, which helps, but "falsifies" is a strong word for what is essentially an empirical probe on a few networks. Minor but annoying: several citation placeholders like [? ] appear in the text.

The central contribution holds up. The gaps are technical, not conceptual, and a careful revision could close them. I would send this to peer review, and I would expect a competent referee to ask for: (1) explicit non-degeneracy handling in Theorems 2 and 3, (2) an error discussion or bound for the floating-point implementation, and (3) softened or better-supported claims about falsifying the linear explanation. Despite those issues, the exact IG result alone justifies engaging with the paper, and the ExactLine primitive will likely be cited by people building on exact one-dimensional analysis.

Recommendation: accept with major revision, assuming the degeneracy gaps get fixed and the exactness language is made honest.

Referee Report

4 major / 4 minor

Summary. The paper introduces ExactLine, an algorithm that, for a piecewise-linear ReLU network f and a line segment QR, computes a tuple of points partitioning QR into finitely many subsegments on each of which f is affine. The algorithm composes per-layer partitions for affine, ReLU, MaxPool, and batch-normalization layers, and the authors claim worst-case polynomial time for fixed network depth. They apply ExactLine to three tasks: exact characterization of ACAS Xu decision boundaries along lines; exact computation of integrated gradients, reporting 25-45% error for the standard Riemann-sum approximation; and an empirical test of the linear explanation of adversarial examples, reporting that FGSM directions cross more linear partitions than random directions and that adversarially-trained networks are more linear. Source code is released.

Significance. If the algorithmic claims are made precise, ExactLine is a useful primitive: one-dimensional restrictions and path integrals over ReLU networks become exactly computable, and the quantified error of integrated-gradient approximations has direct practical value for interpretability tooling. The paper is honest about several limitations and includes formal statements and proofs for the per-layer algorithms, a composition theorem, and complexity bounds, with no fitted parameters used to derive the results. The adversarial-example experiments test a falsifiable hypothesis and are the kind of empirical contribution that can change how the community views the linear-explanation hypothesis. However, the exactness claims currently outrun the formal support: degenerate line segments are not handled in the stated theorems, the composition formula contains an undefined vector division and an unhandled collapse case, and the implementation admits unquantified floating-point error while the paper uses the word 'exact.' These issues are load-bearing for the central claims and require repair.

major comments (4)

[Sec. 2, Thm. 2 (App. E)] Theorem 2 defines D = {-Q_i/(R_i-Q_i) | 1 <= i <= d}; this expression is undefined whenever R_i = Q_i. Definition 1 and the paper's statements apply to arbitrary line segments QR, and the integrated-gradients experiment with a black baseline x'=0 makes R_i=Q_i=0 for any zero-valued input pixel a real case. As written, the theorem does not prove that ExactLine partitions such segments. Please either state a non-degeneracy assumption and verify that the applications satisfy it, or handle sign-constant coordinates explicitly in the theorem statement and in the algorithm.
[Sec. 2, Thm. 3 (App. I)] The composition formula in Theorem 3 uses (y - g(P_i))/(g(P_{i+1}) - g(P_i)), but both numerator and denominator are vectors and the intended scalar interpolation parameter is not defined. More seriously, the formula divides by g(P_{i+1}) - g(P_i), which can be zero on an intermediate partition even when g(R) != g(Q); for example, a ReLU layer whose units are all negative on a segment maps that entire segment to a point. The proof's parenthetical 'in the degenerate case, we can take I_i^j = P_i, I_i^{j+1} = P_{i+1}' is not reflected in the theorem statement or in an algorithmic branch. Since the universal claim that ExactLine partitions any line segment depends on this theorem, please define the scalar ratio, add a non-degeneracy condition covering all intermediate segments, and give an explicit degenerate-case construction with proof.
[Sec. 4, Thm. 4 (App. J)] Theorem 4 states that P(f restricted to QR) is computed 'according to Equation 3,' but Equation 3 is the algorithm for a single ReLU layer, not for a composed network. The proof argues that signs of internal activations are constant on the partitions, but it does not explicitly show that the partitions produced by composing per-layer ExactLine computations have this property for all layers simultaneously, nor does it address the measure-zero boundaries where a pre-activation is exactly zero. Since the exact integrated-gradients application relies on gradient constancy inside each partition, this gap should be closed with a proof for the full composed algorithm, not just for one ReLU layer.
[App. L] Appendix L states that the implementation uses floating-point computations and that 'there may be some slight inaccuracies in our computations of each Pi,' with only the informal claim that errors are small and unlikely to cause meaningful issues. The manuscript's central claims are that ExactLine exactly computes a linear partitioning and exactly computes integrated gradients; without an error bound or exact arithmetic, a miscomputed endpoint can place a point on the wrong side of a sign boundary, changing the reported partition, the integrated-gradient value, and the partition-density statistics. Please either implement exact arithmetic (for example, rationals or arbitrary precision) or provide a formal error analysis that ties floating-point error to the reported conclusions, and otherwise weaken the 'exact' claims to 'up to floating-point error.'

minor comments (4)

[App. C] The complexity statements 'O((d + ws)l+m)' and 'O(dl)' appear to be missing exponents; as written they are linear in l and m, contradicting the combinatorial nature of the segment count. Please clarify whether these are O((d + ws)^{l+m}) and O(d^l), or give the intended bound.
[Sec. 4 / App. J] Theorem 4 and its proof contain cross-reference inconsistencies: the statement refers to Equation 3 while the proof refers to 'Equation 4' when describing the partition property; the equation numbers should be corrected and the statement should reference the full ExactLine algorithm for composed networks.
[Sec. 2 / App. K] Several citations are still placeholders, including '[11?]' and '[? ]' in Section 2 and '[? ]' in Appendix K; these should be resolved to the intended references.
[App. F] The assignment 'D <- Q - Q_m / (R_i - Q_i) - (R - Q)' in Algorithm 1 is syntactically unclear; please specify the intended element-wise operations and parenthesization so that the proof and algorithm can be checked.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ExactLine derives its representations from the network's layer structure and evaluates them against external data, with no fitted parameters or load-bearing self-citations.

full rationale

The paper's central contribution is an algorithm that computes a linear partitioning of a piecewise-linear neural network restricted to a line segment. The partitioning is constructed by composing per-layer computations (Theorems 1-3), each derived from the layer's mathematical definition rather than from the quantities that the paper later reports. No network parameter is fitted to the decision-boundary, integrated-gradient, or adversarial-example results, and no 'prediction' is obtained by renaming a fitted input. The exact integrated-gradient computation uses the network's own gradient, but that is the object being computed, not a parameter tuned to produce a desired conclusion. The comparisons against left/right Riemann sums and trapezoidal rules are external to the derivation and report measured errors; the adversarial-example falsification counts empirical partition densities along FGSM and random directions and does not assume the hypothesis under test. The paper contains no load-bearing self-citation and does not import a uniqueness theorem from prior work by the same authors. Appendix B's uniqueness claim is proved in the text. The noted issues about degenerate line segments (division by zero when R_i = Q_i or when g(P_i) = g(P_{i+1})) and about floating-point inaccuracies (Appendix L) are potential soundness or precision gaps, not circularity: they concern whether the stated algorithm handles all cases or computes exact arithmetic endpoints, not whether the derivation reduces to its own assumptions. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central algorithm relies on the standard piecewise-linear assumption of ReLU networks and on exact arithmetic. No free parameters are fit, and no new physical or mathematical entities are postulated.

assumptions (2)

domain assumption The network is exactly piecewise-linear and each layer can be partitioned into convex polytopes on which it is affine.
The ExactLine algorithm in Section 2 and Appendix H depends on this property for ReLU, MaxPool, batch norm, and fully-connected/convolutional layers; networks with smooth activations are outside the method's scope.
domain assumption Floating-point arithmetic produces errors small enough not to change the conclusions.
Appendix L explicitly states the implementation uses floating point and may miscompute partition endpoints Pi; the paper provides no formal error bound connecting these numerical errors to the exact IG or decision-boundary claims.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Computing Linear Restrictions of Neural Networks." pith.science (2026). https://pith.science/paper/NC4OV3NT

@misc{pith2026190806214,
  author       = {Pith},
  title        = {Pith review of: Computing Linear Restrictions of Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NC4OV3NT}},
  note         = {Machine review of arXiv:1908.06214}
}

read the original abstract

A linear restriction of a function is the same function with its domain restricted to points on a given line. This paper addresses the problem of computing a succinct representation for a linear restriction of a piecewise-linear neural network. This primitive, which we call ExactLine, allows us to exactly characterize the result of applying the network to all of the infinitely many points on a line. In particular, ExactLine computes a partitioning of the given input line segment such that the network is affine on each partition. We present an efficient algorithm for computing ExactLine for networks that use ReLU, MaxPool, batch normalization, fully-connected, convolutional, and other layers, along with several applications. First, we show how to exactly determine decision boundaries of an ACAS Xu neural network, providing significantly improved confidence in the results compared to prior work that sampled finitely many points in the input space. Next, we demonstrate how to exactly compute integrated gradients, which are commonly used for neural network attributions, allowing us to show that the prior heuristic-based methods had relative errors of 25-45% and show that a better sampling method can achieve higher accuracy with less computation. Finally, we use ExactLine to empirically falsify the core assumption behind a well-known hypothesis about adversarial examples, and in the process identify interesting properties of adversarially-trained networks.

Figures

Figures reproduced from arXiv: 1908.06214 by the authors.

**Figure 1.** Computing the linear restriction of f (Equation 1) using EXACTLINE. The input line segment QR is divided into three linear partitions such that the transformation from input space to output space (left plot to right plot) is affine (Equation 2). Tick marks (on the left) are used in figures throughout this paper to indicate the partition endpoints (P1, P2, . . . , Pn). Other Network Analysis Techniques Compared to pr… view at source ↗

**Figure 2.** (a)–(d) Understanding the decision boundary of an ACAS Xu aircraft avoidance network [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** “Integrated Gradients” is a powerful method of neural network attribution. IG relies on [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Comparing the density of linear partitions from a test image to FGSM and random [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Comparing the density of linear partitions from a test image to random baselines for normal [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 32 canonical work pages

[1]

Deep learning

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016

2016
[2]

Re- thinking the inception architecture for computer vision

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition CVPR, 2016

work page 2016
[3]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classiﬁcation with deep convolutional neural networks. Commun. ACM, 60(6):84–90, 2017

work page 2017
[4]

BERT: pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018

arXiv 2018
[5]

Deep neural network compression for aircraft collision avoidance systems

Kyle D Julian, Mykel J Kochenderfer, and Michael P Owen. Deep neural network compression for aircraft collision avoidance systems. Journal of Guidance, Control, and Dynamics, 42(3): 598–608, 2018

work page 2018
[6]

Goodfellow, and Rob Fergus

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, ICLR, 2014

work page 2014
[7]

Goodfellow, Jonathon Shlens, and Christian Szegedy

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In International Conference on Learning Representations, ICLR, 2015

work page 2015
[8]

Deepfool: A simple and accurate method to fool deep neural networks

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016, pages 2574–2582, 2016

work page 2016

Show all 41 references

[9]

Audio adversarial examples: Targeted attacks on speech- to-text

Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech- to-text. In 2018 IEEE Security and Privacy Workshops (SPW), pages 1–7. IEEE, 2018

2018
[10]

Deep neural networks are easily fooled: High conﬁdence predictions for unrecognizable images

Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High conﬁdence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015, pages 427–436, 2015

2015
[11]

Nori, and Antonio Criminisi

Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya V . Nori, and Antonio Criminisi. Measuring neural net robustness with constraints. InAdvances in Neural Information Processing Systems, 2016

2016
[12]

Reluplex: An efﬁcient smt solver for verifying deep neural networks

Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efﬁcient smt solver for verifying deep neural networks. In International Conference on Computer Aided Veriﬁcation, 2017

2017
[13]

Formal veriﬁcation of piece-wise linear feed-forward neural networks

Ruediger Ehlers. Formal veriﬁcation of piece-wise linear feed-forward neural networks. In International Symposium on Automated Technology for Veriﬁcation and Analysis ATVA, 2017

2017
[14]

Safety veriﬁcation of deep neural networks

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. Safety veriﬁcation of deep neural networks. In International Conference on Computer Aided Veriﬁcation CAV, 2017

2017
[15]

Timon Gehr, Matthew Mirman, Dana Drachsler-Cohen, Petar Tsankov, Swarat Chaudhuri, and Martin T. Vechev. AI2: safety and robustness certiﬁcation of neural networks with abstract interpretation. In 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21-23 May 201...

2018
[16]

Bunel, Ilker Turkaslan, Philip H

Rudy R. Bunel, Ilker Turkaslan, Philip H. S. Torr, Pushmeet Kohli, and Pawan Kumar Mudigonda. A uniﬁed view of piecewise linear neural network veriﬁcation. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018,...

2018
[17]

Boning, and Inderjit S

Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane S. Boning, and Inderjit S. Dhillon. Towards fast computation of certiﬁed robustness for relu networks. In International Conference on Machine Learning, ICML, 2018

2018
[18]

Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. An abstract domain for certifying neural networks. PACMPL, 3(POPL):41:1–41:30, 2019

2019
[19]

Optimization and abstrac- tion: A synergistic approach for analyzing neural network robustness

Greg Anderson, Shankara Pailoor, Isil Dillig, and Swarat Chaudhuri. Optimization and abstrac- tion: A synergistic approach for analyzing neural network robustness. CoRR, abs/1904.09959, 2019

1904 arXiv
[20]

Zeiler and Rob Fergus

Matthew D. Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I, pages 818–833, 2014

2014
[21]

Fuchs, and Hod Lipson

Jason Yosinski, Jeff Clune, Anh Mai Nguyen, Thomas J. Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. CoRR, abs/1506.06579, 2015

2015 arXiv
[22]

Network dissection: Quantifying interpretability of deep visual representations

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, 2017

2017
[23]

How to explain individual classiﬁcation decisions

David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller. How to explain individual classiﬁcation decisions. Journal of Machine Learning Research, 11:1803–1831, 2010

2010
[24]

Not just a black box: Learning important features through propagating activation differences

Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences. CoRR, abs/1605.01713, 2016

2016 arXiv
[25]

Axiomatic attribution for deep networks

Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International Conference on Machine Learning ICML, 2017

2017
[26]

Cai, James Wexler, Fernanda B

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCA V). InInternational Conference on Machine Learning, ICML, 2018

2018
[27]

A survey of methods for explaining black box models

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5):93, 2018

2018
[28]

Himmelstein, Brett K

Travers Ching, Daniel S. Himmelstein, Brett K. Beaulieu-Jones, Alexandr A. Kalinin, Brian T. Do, Gregory P. Way, Enrico Ferrero, Paul-Michael Agapow, Michael Zietz, Michael M. Hoffman, Wei Xie, Gail L. Rosen, Benjamin J. Lengerich, Johnny Israeli, Jack Lanchantin, Stephen Wolo...

2018
[29]

Riccardo Miotto, Fei Wang, Shuang Wang, Xiaoqian Jiang, and Joel T. Dudley. Deep learning for healthcare: review, opportunities and challenges. Brieﬁngs in Bioinformatics, 19(6):1236– 1246, 2018

2018
[30]

Artiﬁcial intelligence in radiology

Ahmed Hosny, Chintan Parmar, John Quackenbush, Lawrence H Schwartz, and Hugo JWL Aerts. Artiﬁcial intelligence in radiology. Nature Reviews Cancer, page 1, 2018. 11

2018
[31]

Artiﬁcial intelligence in breast imaging: potentials and limitations

Ellen B Mendelson. Artiﬁcial intelligence in breast imaging: potentials and limitations. Ameri- can Journal of Roentgenology, 212(2):293–299, 2019

2019
[32]

Formal security analysis of neural networks using symbolic intervals

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Formal security analysis of neural networks using symbolic intervals. In 27th USENIX Security Symposium, USENIX Security 2018, Baltimore, MD, USA, August 15-17, 2018., pages 1599–1614, 2018

2018
[33]

Differentiable abstract interpretation for provably robust neural networks

Matthew Mirman, Timon Gehr, and Martin Vechev. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning ICML , 2018

2018
[34]

Reachable set computation and safety veriﬁcation for neural networks with relu activations

Weiming Xiang, Hoang-Dung Tran, and Taylor T Johnson. Reachable set computation and safety veriﬁcation for neural networks with relu activations. arXiv preprint arXiv:1712.08163, 2017

2017 arXiv
[35]

Rosenfeld, and Taylor T

Weiming Xiang, Hoang-Dung Tran, Joel A. Rosenfeld, and Taylor T. Johnson. Reachable set estimation and safety veriﬁcation for piecewise linear systems with neural network controllers. In 2018 Annual American Control Conference, ACC, 2018

2018
[36]

Output range analysis for deep feedforward neural networks

Souradeep Dutta, Susmit Jha, Sriram Sankaranarayanan, and Ashish Tiwari. Output range analysis for deep feedforward neural networks. In NASA Formal Methods Symposium, pages 121–138. Springer, 2018

2018
[37]

Interpretable deep learning in drug discovery

Kristina Preuer, Günter Klambauer, Friedrich Rippmann, Sepp Hochreiter, and Thomas Un- terthiner. Interpretable deep learning in drug discovery. CoRR, abs/1903.02788, 2019

1903 arXiv
[38]

https://github.com/eth-sri/ eran, 2019

ETH robustness analyzer for neural networks (ERAN). https://github.com/eth-sri/ eran, 2019. Accessed: 2019-05-01

2019
[39]

How important is a neuron? arXiv preprint arXiv:1805.12233, 2018

Kedar Dhamdhere, Mukund Sundararajan, and Qiqi Yan. How important is a neuron? arXiv preprint arXiv:1805.12233, 2018

2018 arXiv
[40]

A boundary tilting persepective on the phenomenon of adversarial examples

Thomas Tanay and Lewis Grifﬁn. A boundary tilting persepective on the phenomenon of adversarial examples. arXiv preprint arXiv:1608.07690, 2016

2016 arXiv
[41]

matter of seconds

A collection of pre-trained, state-of-the-art models in the onnx format. https://github.com/ onnx/models, 2019. Accessed: 2019-05-01. 12 Computing Linear Restrictions of Neural Networks: Supplemental A Speciﬁcation of Evaluation Hardware Although we do not claim particular per...

2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Deep learning

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016

2016

[2] [2]

Re- thinking the inception architecture for computer vision

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition CVPR, 2016

work page 2016

[3] [3]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classiﬁcation with deep convolutional neural networks. Commun. ACM, 60(6):84–90, 2017

work page 2017

[4] [4]

BERT: pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018

arXiv 2018

[5] [5]

Deep neural network compression for aircraft collision avoidance systems

Kyle D Julian, Mykel J Kochenderfer, and Michael P Owen. Deep neural network compression for aircraft collision avoidance systems. Journal of Guidance, Control, and Dynamics, 42(3): 598–608, 2018

work page 2018

[6] [6]

Goodfellow, and Rob Fergus

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, ICLR, 2014

work page 2014

[7] [7]

Goodfellow, Jonathon Shlens, and Christian Szegedy

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adver- sarial examples. In International Conference on Learning Representations, ICLR, 2015

work page 2015

[8] [8]

Deepfool: A simple and accurate method to fool deep neural networks

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016, pages 2574–2582, 2016

work page 2016