Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations

Andras Rozsa; Terrance E. Boult

REVIEW 3 major objections 5 minor 39 references

Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing monotonic activations with bounded tent activations makes classifiers robust to six white-box attacks, beating PGD adversarial training on both MNIST and CIFAR-10.

desk verdict The tent activation is a clean idea, but the reported robustness numbers almost certainly owe as much to zero-gradient saturation as to any real boundary robustness. read the letter →

arxiv 1908.02435 v1 pith:GCDLJDP3 submitted 2019-08-07 cs.CV cs.LG

Andras Rozsa , Terrance E. Boult This is my paper · ORCID

classification cs.CVcs.LG

keywords tentactivationfunctionopenspaceriskadversarialrobustnessexamplesmonotonicPGDtrainingMNISTCIFAR-10

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that adversarial examples succeed because conventional activation functions like ReLU have unbounded "open space risk": an adversary can keep pushing an activation input in the direction that increases its response, and with monotonic activations that response grows without limit. The authors introduce the tent activation $f(x;\delta)=\max(0,\delta-|x|)$, a bounded, learnable activation, and prove that standard monotonic activations have unbounded open space risk while tents have bounded risk. On six white-box attacks, tent networks average 91.8% accuracy on MNIST and 73.5% on CIFAR-10, compared with 76.8% and 41.8% for PGD adversarial training, with no additional training cost. If correct, simply changing the activation function offers a cheaper path to robustness.

What carries the argument

The load-bearing object is the tent activation function $f(x;\delta)=\max(0,\delta-|x|)$, a learnable bump that is zero outside $[-\delta,\delta]$, reaches height $\delta$ at the origin, and has derivative $-\operatorname{sgn}(x)$ inside the interval and zero outside. A normalized tent $\hat{f}(x;\delta)=f(x;\delta)/\delta^2$ acts as a compact probability abating model in the open-set recognition sense, which is what makes its open set risk finite; Theorem 1 of the paper shows no monotonic activation can be bounded this way. In the trained networks, every ReLU is replaced by batch normalization plus a tent, with tents shared across channels, initialized at $\delta=1$, and squeezed by weight decay so that adversarial perturbations pushing activation inputs beyond the tent produce zero output instead of larger responses.

What would settle it

Train the same architecture with a bounded activation that is nonzero across the same interval but has no flat zero-gradient region, such as a smoothed tent, and attack it with the same PGD and Carlini-Wagner budgets; if accuracy collapses relative to the hard tent, the flat tails, not the bounded output, carried the defense.

Watch

Extended reading notes

Core claim

The paper's central claim is that replacing monotonic activation functions with tent activations yields significantly improved adversarial robustness compared to the state of the art. It proves that every standard monotonic activation function acts as a weak classifier with unbounded open space risk, then shows that the tent activation $f(x;\delta)=\max(0,\delta-|x|)$ has bounded open space risk. Replacing each ReLU with a tent fed by batch normalization and trained with weight decay on the learnable width $\delta$ produces classifiers whose average accuracy across six white-box attacks reaches 91.8% on MNIST and 73.5% on CIFAR-10, versus 76.8% and 41.8% for PGD adversarial training. The paper presents this as support for its hypothesis that open space risk is a major contributing factor to the success of adversarial attacks.

Load-bearing premise

The explanation assumes that adversarial attacks succeed chiefly because monotonic activations can be pushed to arbitrarily large responses, so capping the response is what makes networks robust; if the accuracy gains actually come from the zero gradients on the tent's flat tails, the explanatory claim would be unsupported even with the accuracy numbers intact.

Editorial extensions

If this is right

Adversarial robustness can be improved at no extra training cost: tents replace ReLUs without the extra forward and backward passes that PGD adversarial training requires.
Robustness is more even across attack types: tent classifiers are strong against DeepFool and the Carlini-Wagner attacks, the two families where PGD adversarial training is weakest.
The trade-off between clean accuracy and adversarial accuracy is gentler on smaller models, as the narrow WRN-28-1 keeps near-82% clean accuracy with tents while adversarial training drops it below 74%.
The learned tent widths $\delta$ act as a robustness dial: shrinking them with weight decay improves adversarial accuracy until the tents become too small and clean accuracy falls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not test combining tents with adversarial training; since tents cap activation output and adversarial training augments the input distribution, the two defenses target different failure modes and may stack, a testable extension.
Because the same bounded activations that resist attacks also limit the range of internal features, tent networks are a natural candidate for out-of-distribution rejection, an experiment the paper's open-space framing invites but does not run.
The uniform weight decay applied to all $\delta$ parameters is admittedly crude; per-layer schedules or a learned budget on total open space risk could push the reported accuracies higher.
The tent's flat zero-gradient tails could in principle be exploited by adaptive attacks; a smoothed bounded activation would separate the effect of bounded range from the effect of zero gradients.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about this paper. First, the tent activation f(x;δ)=max(0,δ−|x|) is a genuinely simple and original proposal, and the paper's formal point is correct: standard monotonic activations, including bounded ones like tanh, have unbounded open-space risk in the Scheirer–Boult sense because they don't decay to zero, while the tent does. Second, the headline result — state-of-the-art white-box robustness without adversarial training — is not supported by the evidence as presented. The high accuracies look like an artifact of zero gradients, not improved decision boundaries.

The paper deserves credit for a clean theoretical observation and for running six white-box and six black-box attacks on two architectures. The activation-by-activation distribution plots in Appendix B are a nice piece of diagnostic work. The proof sketch that ReLU-family activations have infinite open-space risk is straightforward and valid, and the tent's compact support is the right way to make the integral finite.

The soft spots are significant. Most of the model's units are saturated for |x|>δ, and Appendix B shows learned δ values around 0.1–0.2 on MNIST. FGSM, BIM, and PGD are exactly the attacks that get stuck when local gradients vanish. The clearest tell is Table 2: on CIFAR-10, WRN-28-10 tent(0.003) reports 84.6% FGSM, 83.3% BIM, 83.3% PGD, 83.6% DeepFool, but 24.1% against CW-l2. A robust classifier should not show that gap. The paper needs an adaptive attack — e.g., using a straight-through gradient estimator, or attacking a surrogate with similar activations, or running PGD with random restarts in the active regime — before any SOTA claim is credible. There are also no error bars, no multiple seeds, and no comparison against existing bounded activations like hard tanh. The open-space-risk explanation is offered as a hypothesis, which is honest, but experiments that are consistent with it don't prove it.

Who should read this? Anyone working on adversarial defenses, mainly as a cautionary example of gradient masking. The idea is worth a serious referee, because if the robustness survives adaptive attacks it would be a real low-cost defense, and if it doesn't, the paper still demonstrates how easily gradient-based evaluations can be fooled. I'd send it to review, but the authors need to run the adaptive attacks and the hard-tanh baseline before the central claim is publishable.

Referee Report

3 major / 5 minor

Summary. The paper proposes the tent activation function f(x;δ)=max(0,δ−|x|), argues that monotonic activations such as ReLU have unbounded open space risk while tents have bounded open space risk, and reports that replacing ReLUs with tents yields substantially improved white-box adversarial robustness. On MNIST, the tent network achieves an average accuracy of 91.8% across six white-box attacks versus 76.8% for PGD adversarial training; on CIFAR-10 the corresponding numbers are 73.5% versus 41.8%. The experiments also include black-box transfer results, activation-input distribution analyses, and a formal section connecting the tent to the open-set-risk framework of Scheirer et al.

Significance. If the empirical claim is correct, this is an important result: it suggests that a simple bounded activation function trained at no extra computational cost can match or beat PGD adversarial training on standard benchmarks, and it offers a conceptual link between open space risk and adversarial vulnerability. The paper is also commendable for evaluating six attack types, reporting several weight-decay settings, and including an analysis of activation distributions under attack. However, the central robustness claim is not yet established because the reported white-box accuracies may reflect zero-gradient saturation of the tent activation rather than genuine decision-boundary robustness, and the open-space-risk explanation is not isolated from confounding changes in the training protocol.

major comments (3)

[§4.2, Tables 1–2; Eq. (2)] The white-box accuracies that support the state-of-the-art claim are not trustworthy without an adaptive attack. The tent activation in Eq. (1) has zero derivative for |x|>δ (Eq. 2), and Appendix B reports learned δ values (e.g., 0.058–0.234 on MNIST) that leave a large fraction of activation inputs in the saturated regime. Under a standard PGD/BIM attack, gradients vanish in those regions, so the attack can terminate at a point of zero local gradient; the reported pattern, with FGSM/BIM/PGD accuracies above 83% but CW-l2 accuracy only 23.65 for WRN-28-10 tent (0.004), is exactly the signature of gradient masking. Please evaluate with an adaptive attack that handles the zero-gradient regions (e.g., BPDA with a straight-through surrogate, or random/ensemble PGD restarts), or provide a certified robustness bound, before the claim of significantly improved adversarial robustness compared to the state of the art can be accepted.
[§4.1, Tables 1–2; §5] The causal claim that reducing open space risk is the reason for the observed robustness is not identified by the experiments. The tent networks differ from the ReLU baselines in several confounded ways: for MNIST, the primary comparison includes the batch-normalization addition, and for CIFAR-10 the tent models use Adam while the ReLU and adversarial-training models use SGD; in all tent models, weight-decay is applied to the δ parameters rather than to the network weights. Bounding the output range alone, or changing the optimization trajectory, could explain the results without the open-space-risk mechanism. To support the attribution, the authors should compare against other bounded activations (e.g., clipped ReLU, hard tanh, tanh, sigmoid) under a matched training protocol, and ideally vary the bound while holding everything else fixed.
[§3, Theorems 1–2; §5] The theoretical analysis is at the level of a scalar activation function, but the robustness claim concerns classifiers over input images. Theorem 2 establishes that a tent activation has finite open space risk when viewed as a local recognition function over its own input; it does not establish a relationship between that quantity and the success of adversarial perturbations in image space. The conclusion's assertion that the open space risk model explains many issues for adversarial example generation is therefore stronger than what is proved. Please state the connection as a hypothesis and test it directly, for example by measuring the change in feature-space open space risk under attacks for tent versus ReLU networks.

minor comments (5)

[Abstract and §5] The state-of-the-art claim is supported only by a comparison to Madry et al. [6]; please either broaden the comparison to other defenses available at the time of submission or temper the wording.
[§4.1, Tables 1–2] No error bars or multiple-seed results are reported; given that the paper makes a state-of-the-art claim, reporting the mean and variance over at least three runs would strengthen the conclusions.
[§3, Theorem 1] The proof of Theorem 1 is described as a sketch, although the contributions section says the paper proves the result; please provide a complete proof with explicit assumptions on the activation function and training distribution.
[Appendix A] The appendix states that quantitative evaluation of adversarial robustness solely based upon accuracies may not be sufficient; this undercuts the accuracy-only evaluation in the main body and should be reconciled in the text.
[Figures 2 and 3 captions] Figure 3's caption has a typo, 'class indeces' should be 'class indices', and both captions could clarify that the displayed images are adversarial examples generated on the source network and then evaluated on the target network.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the robustness claims are measured against external white-box attacks, and the open-space-risk analysis is a mathematical property of the tent activation, not an input to the attack numbers.

full rationale

The main robustness claims (91.8% on MNIST and 73.5% on CIFAR-10) are empirical accuracies from Tables 1 and 2 against six external attack implementations (FGSM, BIM, PGD, DeepFool, CW-l2, CW-l∞), compared with a PGD adversarially trained baseline under the same ART settings. These numbers are not derived from the open-space-risk formalism; the weight-decay hyperparameters on δ are varied and reported, and the best models are selected on a validation set rather than fitted to reproduce attack accuracies. The theoretical section shows that monotonic activations have unbounded open-space risk and that the tent f(x;δ)=max(0,δ−|x|) has bounded range and finite integral, which is a direct property of Eq. 1 rather than an imported conclusion. The citation to [8] (Boult is a coauthor) supplies the open-space-risk definitions and the compact-abating-model theorem, but the boundedness of tents is independently evident, and the cited framework is not what produces the reported attack accuracies. The skeptical concern that high white-box accuracies may reflect zero-gradient saturation (gradient masking) is a correctness and adaptive-attack question, not a circularity: it challenges whether the numbers reflect true robustness, not whether the derivation reduces to its inputs. No load-bearing step equates a fitted parameter with a prediction, and no uniqueness result is imported from the authors' prior work to force the conclusion.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the tent activation function, which is introduced, and on the open-space-risk hypothesis that is not independently established. The experimental numbers depend on tuned weight-decay hyperparameters.

free parameters (2)

δ (tent size) = learned, initialized at 1, bounded [0.05, 1.0]
Each layer's tent size is learned; its value determines the bounded input/output range of the activation and is central to the open-space-risk argument.
weight-decay on δ = 0.12 (MNIST), 0.003 or 0.004 (CIFAR-10 WRN-28-10)
Hyperparameters tuned per dataset to balance clean accuracy and robustness; the reported state-of-the-art averages depend on these specific choices.

assumptions (3)

domain assumption Open space risk definitions from Scheirer et al. [8] apply to activation functions as weak classifiers.
The paper reuses the open set recognition framework without re-deriving it; the mapping to activations is the paper's own conceptual claim.
domain assumption Each activation function acts as a weak classifier and the network is a cascade or ensemble of them.
Stated in Section 1; this is the bridge from bounding activation responses to improving network robustness.
domain assumption Reducing open space risk at activation level reduces adversarial vulnerability.
This is the paper's hypothesis, not proven; the experiments are consistent with it but do not establish causality.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations." pith.science (2026). https://pith.science/paper/GCDLJDP3

@misc{pith2026190802435,
  author       = {Pith},
  title        = {Pith review of: Improved Adversarial Robustness by Reducing Open Space Risk via Tent Activations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GCDLJDP3}},
  note         = {Machine review of arXiv:1908.02435}
}

read the original abstract

Adversarial examples contain small perturbations that can remain imperceptible to human observers but alter the behavior of even the best performing deep learning models and yield incorrect outputs. Since their discovery, adversarial examples have drawn significant attention in machine learning: researchers try to reveal the reasons for their existence and improve the robustness of machine learning models to adversarial perturbations. The state-of-the-art defense is the computationally expensive and very time consuming adversarial training via projected gradient descent (PGD). We hypothesize that adversarial attacks exploit the open space risk of classic monotonic activation functions. This paper introduces the tent activation function with bounded open space risk and shows that tents make deep learning models more robust to adversarial attacks. We demonstrate on the MNIST dataset that a classifier with tents yields an average accuracy of 91.8% against six white-box adversarial attacks, which is more than 15 percentage points above the state of the art. On the CIFAR-10 dataset, our approach improves the average accuracy against the six white-box adversarial attacks to 73.5% from 41.8% achieved by adversarial training via PGD.

Figures

Figures reproduced from arXiv: 1908.02435 by the authors.

**Figure 2.** Adversaries on MNIST. We show clean samples followed by their perturbed versions [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

**Figure 3.** Adversaries on CIFAR-10. We show clean samples followed by their perturbed versions [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Activation Inputs: This figure shows the distributions of activation’s inputs per layer for the [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 37 canonical work pages

[1]

Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus

Christian J. Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representation (ICLR), 2014

work page 2014
[2]

Goodfellow, Jonathon Shlens, and Christian Szegedy

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representation (ICLR), 2015

work page 2015
[3]

Deepfool: a simple and accurate method to fool deep neural networks

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016

work page 2016
[4]

Towards evaluating the robustness of neural networks

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy (SP). IEEE, 2017

work page 2017
[5]

Goodfellow, and Samy Bengio

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representation (ICLR), 2017

work page 2017
[6]

To- wards deep learning models resistant to adversarial attacks

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representation (ICLR), 2018

work page 2018
[7]

Attacks meet interpretability: Attribute- steered detection of adversarial samples

Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretability: Attribute- steered detection of adversarial samples. In Advances in Neural Information Processing Systems (NIPS), 2018

work page 2018
[8]

Scheirer, L.P

W.J. Scheirer, L.P. Jain, and T.E. Boult. Probability models for open set recognition. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36, 2014

work page 2014

Show all 39 references

[9]

Out-of-distribution detection using an ensemble of self supervised leave-out classiﬁers

Apoorv Vyas, Nataraj Jammalamadaka, Xia Zhu, Dipankar Das, Bharat Kaul, and Theodore L Willke. Out-of-distribution detection using an ensemble of self supervised leave-out classiﬁers. In Proceedings of the European Conference on Computer Vision (ECCV), 2018

2018
[10]

A simple uniﬁed framework for detecting out- of-distribution samples and adversarial attacks

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple uniﬁed framework for detecting out- of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[11]

Out-of-distribution detection using multiple semantic label representations

Gabi Shalev, Yossi Adi, and Joseph Keshet. Out-of-distribution detection using multiple semantic label representations. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[12]

Extremeweather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events

Evan Racah, Christopher Beckham, Tegan Maharaj, Samira Ebrahimi Kahou, Mr Prabhat, and Chris Pal. Extremeweather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. In Advances in Neural Information Processin...

2017
[13]

A loss framework for calibrated anomaly detection

Aditya Krishna Menon and Robert C Williamson. A loss framework for calibrated anomaly detection. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. Curran Associates Inc., 2018

2018
[14]

Generative probabilistic novelty detection with adversarial autoencoders

Stanislav Pidhorskyi, Ranya Almohsen, and Gianfranco Doretto. Generative probabilistic novelty detection with adversarial autoencoders. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[15]

Concrete dropout

Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. In Advances in Neural Information Processing Systems (NIPS), 2017

2017
[16]

Simple and scalable predictive uncertainty estimation using deep ensembles

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NIPS), 2017

2017
[17]

Evidential deep learning to quantify classiﬁcation uncertainty

Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classiﬁcation uncertainty. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[18]

Predictive uncertainty estimation via prior networks

Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[19]

Open set ﬁngerprint spoof detection across novel fabrication materials

Ajita Rattani, Walter J Scheirer, and Arun Ross. Open set ﬁngerprint spoof detection across novel fabrication materials. IEEE Transactions on Information Forensics and Security (TIFS), 10(11), 2015. 9

2015
[20]

Sparse representation-based open set recognition

He Zhang and Vishal M Patel. Sparse representation-based open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 39(8):1690–1696, 2016

2016
[21]

Open set recognition for automatic target classiﬁcation with rejection

Matthew D Scherreik and Brian D Rigling. Open set recognition for automatic target classiﬁcation with rejection. IEEE Transactions on Aerospace and Electronic Systems, 52(2), 2016

2016
[22]

Odn: Opening the deep network for open-set action recognition

Yemin Shi, Yaowei Wang, Yixiong Zou, Qingsheng Yuan, Yonghong Tian, and Yu Shu. Odn: Opening the deep network for open-set action recognition. In International Conference on Multimedia and Expo (ICME). IEEE, 2018

2018
[23]

Open set incremental learning for automatic target recognition

Sihang Dang, Zongjie Cao, Zongyong Cui, Yiming Pi, and Nengyuan Liu. Open set incremental learning for automatic target recognition. Transactions on Geoscience and Remote Sensing, 2019

2019
[24]

Combining clustering and active learning for the detection and learning of new image classes

Luiz FS Coletta, Moacir Ponti, Eduardo R Hruschka, Ayan Acharya, and Joydeep Ghosh. Combining clustering and active learning for the detection and learning of new image classes. Neurocomputing, 2019

2019
[25]

Rectiﬁed linear units improve restricted Boltzmann machines

Vinod Nair and Geoffrey E Hinton. Rectiﬁed linear units improve restricted Boltzmann machines. In International Conference on Machine Learning (ICML), 2010

2010
[26]

Batch normalization: Accelerating deep network training by reducing internal covariate shift

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning (ICML), 2015

2015
[27]

Noisy activation functions

Caglar Gulcehre, Marcin Moczulski, Misha Denil, and Yoshua Bengio. Noisy activation functions. In International Conference on Machine Learning (ICML), 2016

2016
[28]

Rectiﬁer nonlinearities improve neural network acoustic models

Andrew L Maas, Awni Y Hannun, and Andrew Y Ng. Rectiﬁer nonlinearities improve neural network acoustic models. In International Conference on Machine Learning (ICML), 2013

2013
[29]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016

2016
[30]

Self-normalizing neural networks

Günter Klambauer, Thomas Unterthiner, Andreas Mayr, and Sepp Hochreiter. Self-normalizing neural networks. In Advances in Neural Information Processing Systems (NIPS), 2017

2017
[31]

Performance analysis of various activation functions in generalized mlp architectures of neural networks

Bekir Karlik and A Vehbi Olgac. Performance analysis of various activation functions in generalized mlp architectures of neural networks. International Journal of Artiﬁcial Intelligence and Expert Systems, 1(4), 2011

2011
[32]

A comparison of activation functions in artiﬁcial neural networks

Cenk Bircano˘glu and Naﬁz Arıca. A comparison of activation functions in artiﬁcial neural networks. In 26th Signal Processing and Communications Applications Conference (SIU). IEEE, 2018

2018
[33]

Efﬁcient formal safety analysis of neural networks

Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. Efﬁcient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems (NIPS), 2018

2018
[34]

The MNIST database of handwritten digits, 1998

Yann LeCun, Corinna Cortes, and Christopher JC Burges. The MNIST database of handwritten digits, 1998

1998
[35]

Learning multiple layers of features from tiny images, 2009

Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009. Technical report. University of Toronto

2009
[36]

Adversarial robustness toolbox v0.10.0

Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial robustness toolbox v0.10.0. CoRR, 1807.01069, 2018

arXiv
[37]

Distillation as a defense to adversarial perturbations against deep neural networks

Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. In Symposium on Security and Privacy (SP). IEEE, 2016

2016
[38]

Wide residual networks

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference (BMVC), 2016

2016
[39]

Sara Sabour, Yanshuai Cao, Fartash Faghri, and David J. Fleet. Adversarial manipulation of deep representations. In International Conference on Learning Representation (ICLR), 2016. 10 APPENDIX A: Visualization of Adversarial Examples In the paper, similar to Madry et al. [6],...

2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus

Christian J. Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representation (ICLR), 2014

work page 2014

[2] [2]

Goodfellow, Jonathon Shlens, and Christian Szegedy

Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representation (ICLR), 2015

work page 2015

[3] [3]

Deepfool: a simple and accurate method to fool deep neural networks

Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016

work page 2016

[4] [4]

Towards evaluating the robustness of neural networks

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy (SP). IEEE, 2017

work page 2017

[5] [5]

Goodfellow, and Samy Bengio

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning at scale. In International Conference on Learning Representation (ICLR), 2017

work page 2017

[6] [6]

To- wards deep learning models resistant to adversarial attacks

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. To- wards deep learning models resistant to adversarial attacks. In International Conference on Learning Representation (ICLR), 2018

work page 2018

[7] [7]

Attacks meet interpretability: Attribute- steered detection of adversarial samples

Guanhong Tao, Shiqing Ma, Yingqi Liu, and Xiangyu Zhang. Attacks meet interpretability: Attribute- steered detection of adversarial samples. In Advances in Neural Information Processing Systems (NIPS), 2018

work page 2018

[8] [8]

Scheirer, L.P

W.J. Scheirer, L.P. Jain, and T.E. Boult. Probability models for open set recognition. Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 36, 2014

work page 2014