Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

Emre Gursoy; Ka-Ho Chow; Ling Liu; Margaret Loper; Stacey Truex; Wenqi Wei; Yanzhao Wu

REVIEW 4 major objections 6 minor 26 references

Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that diverse DNN ensembles block a dozen adversarial attacks without retraining.

desk verdict A competent survey of ensemble diversity wrapped in an unsupported robustness claim; the paper's own Section V concedes the threat model that would be needed to support it. read the letter →

arxiv 1908.11091 v1 pith:HVOJSZDH submitted 2019-08-29 cs.LG stat.ML

Ling Liu , Wenqi Wei , Ka-Ho Chow , Margaret Loper , Emre Gursoy , Stacey Truex , Yanzhao Wu This is my paper · ORCID

classification cs.LGstat.ML

keywords ensemblediversityadversarialrobustnessDNNensembleskappastatistictransferabilityconsensusmodelblack-boxattacks

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that adversarial robustness can be bought by composition rather than retraining: instead of hardening a single deep network, surround a target model with a team of individually accurate models that disagree on their mistakes, and let a consensus rule decide the final label. It distinguishes three sources of diversity—model construction, output disagreement, and training-time decorrelation—and concentrates on the first two. The proposed recipe is to collect pre-trained models, score every candidate team by pairwise disagreement, rank teams by the kappa statistic, and combine member outputs by majority or weighted averaging. The authors report that such teams outperform representative baselines against a dozen attacks on MNIST, CIFAR-10, and ImageNet. If this is right, a defender can raise robustness at deployment time using off-the-shelf models, without modifying the target's weights.

What carries the argument

The load-bearing object is the diversity-ranked ensemble team. Models are first gathered into a pool by structural variety (architecture, training data, hyperparameters), then every subset of a given size is scored by pairwise disagreement measures—chiefly the kappa statistic, with Q-statistic, rho-statistic, binary disagreement, and entropy as alternatives—and teams with high disagreement are selected. The target model is a member of every team. Consensus methods (majority voting, plurality, simple or confidence-weighted averaging, and supra-Bayesian or stacked combinations) turn the members' outputs into one prediction. The kappa score is doing the work of predicting which teams will be robust: low average kappa means errors are spread across different members, so a consensus can overrule a member fooled by an adversarial example.

What would settle it

Run the same twelve attacks on the full ensemble—for example, optimizing the perturbation against the committee's averaged loss while treating all members as white-box—and compare accuracy with the target-only attacks. If the examples fool the target and the teammates alike, the disagreement-based consensus cannot repair the damage, and the defense's advantage over baselines should shrink.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a diversity-optimized ensemble—built from structurally different pre-trained models selected for pairwise disagreement—can defend a target deep network against a known suite of twelve adversarial attacks more effectively than adversarial training, defense distillation, or input-transformation defenses. The paper defines three types of ensemble diversity, proposes a creation pipeline for type 1 and type 2 diversity teams, and reports that the best kappa-ranked team outperforms random teams and the target on targeted and untargeted attacks, with additional gains when a rotation-denoising input transform is added to the consensus. It also argues that ensemble accuracy is not strongly tied to team size, so small diverse teams can be both accurate and cheap. The robustness argument rests on adversarial examples being inconsistent across models: an example that fools the target need not fool its diverse teammates.

Load-bearing premise

The reported robustness numbers assume that adversarial examples generated over the target model will not transfer to the other ensemble members, so the consensus can correct the target's mistake; the paper only evaluates attacks generated from the target model, never from the ensemble itself.

Editorial extensions

If this is right

A defender can improve robustness by selecting pre-trained models from a model zoo and combining them, without retraining the target model.
Small, highly diverse teams can deliver the same accuracy as larger teams, reducing test-time cost and storage.
Because diversity is attack-independent, ensembles selected this way may generalize to attacks not present during selection.
Adding an input-transformation member to a kappa-selected team can further improve robustness, but applying the transform before the ensemble can hurt because denoising errors propagate.
Under white- or grey-box online attacks, a fixed committee is just another target, but randomly selecting a defense team per query improves robustness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: a decisive evaluation would attack the full committee rather than the target alone; the paper's own results only use attacks generated from the target model, so how much robustness remains against adaptive attacks is untested here.
Beyond the paper: kappa is measured on benign examples, so an adversary aware of the selection rule could try to make the target and its teammates err together, converting low benign disagreement into high attack-time correlation.
Beyond the paper: the proposed per-query random team selection is a moving-target defense in disguise; quantifying how often the ensemble changes per query and how much that raises the attacker's cost could turn the heuristic into a measurable security property.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this is a survey/position paper with a plausible recipe—select pretrained models by disagreement diversity and combine them—but the headline robustness claim is not backed by the experiments as designed, and the paper's own Section V admits the core limitation.

What's new: not much. The three-type diversity taxonomy is a re-labeling of categories already in Brown et al. and Kuncheva and Whitaker. All five metrics are standard. The only genuinely new piece is the idea of applying kappa-ranked team selection to a pool of public pretrained DNNs, and the "random per-query ensemble selection" is a design sketch, not an evaluation. The survey parts are competent and would be useful to someone new to the area.

What's good: the paper is honest enough to state in Section V that a fixed ensemble defense is "just another target model" and that existing attacks remain effective. That admission is the right instinct, and it's exactly what kills the main claim. The preliminary results in Table 3 are at least described with enough detail to see what was done—attacks generated on the target model, then measured against the ensemble. That's a transferability study, not an ensemble robustness study.

Where it falls down: the abstract and introduction claim "stronger robustness and higher defensibility" than adversarial training, distillation, and ensemble transformation, but Table 3 compares only the target model, a baseline defense model, and a few ensemble variants on ImageNet, with no error bars, no attack parameters, no code, and an inconsistency in the number of attacks (12 in text, 15 in caption). No comparison against the named defense baselines appears anywhere. And as noted, the attacks are generated against the target model alone, so they test transfer failure, not resistance to an adversary who can query the ensemble. The paper's own Section V concession applies directly to the evaluated fixed ensembles. The later suggestion of random per-query selection is supported only by an unpublished tech report.

Proportion: the central comparative claim has no supporting evidence under a realistic threat model, but the underlying idea—using disagreement diversity as a cheap way to get transfer resistance—is reasonable and worth studying properly.

Who this is for: someone wanting a quick overview of diversity measures and a pointer to a possible defense direction. It is not a completed research result.

Recommendation: if I were the editor, I'd send it to reviewers, not desk-reject, because the topic is relevant and the paper can be revised into an honest survey plus a clearly-labeled preliminary study. But as submitted, it should be rejected or major-revised: the robustness comparisons need adaptive attacks, baselines, and uncertainty estimates.

Referee Report

4 major / 6 minor

Summary. The paper proposes a framework for building deep neural network ensembles that are accurate and robust against adversarial examples. It defines three types of ensemble diversity—structural (type 1), disagreement-based (type 2), and hypothesis-space/training diversity (type 3)—and describes pairwise and non-pairwise diversity metrics, ensemble creation algorithms built on pre-trained model pools, and consensus methods such as majority/plurality voting, averaging, weighted averaging, and supra-Bayesian and stacked generalization. The authors report preliminary experimental results on MNIST, CIFAR, and ImageNet, and claim in Section I that the combination of type 1 and type 2 diversity creates ensembles with 'stronger robustness and higher defensibility' than adversarial training, defense distillation, and ensemble transformation. The paper also discusses offline versus online and black-box versus white-box threat models, and proposes randomized per-query ensemble selection as a defense against white-box online attacks.

Significance. If the central claim were substantiated, the paper would offer an attractive and cheap defense: select diverse pre-trained models by disagreement metrics and use a consensus rule rather than retraining with adversarial training or distillation. The paper has several genuine strengths: it gives a clear taxonomy of diversity types, reviews standard diversity metrics and consensus methods, explicitly distinguishes threat models, and constructs pools of pre-trained models for MNIST, CIFAR-10, and ImageNet. However, the load-bearing empirical claim—that these ensembles are more robust than existing representative defenses—is not supported by the reported experiments. The only comparative table, Table 3, evaluates ImageNet with adversarial examples generated from the target model, which measures cross-model transferability rather than robustness of the ensemble as a defended system, and Section V concedes that a fixed ensemble is just another target model. No adaptive, ensemble-aware attacks are evaluated, no confidence intervals or error bars are provided, and the attack count is inconsistent between the text and the table caption.

major comments (4)

[Section I and Section III.C, Table 3] The headline claim of 'stronger robustness and higher defensibility than those offered by existing representative defense methods' is not supported by the experimental evidence. Table 3 is the only comparison against adversarial training, distillation, and ensemble transformation, and its caption states that 'the adversarial examples are generated from the target model.' This evaluates whether target-model-specific adversarial examples transfer to the ensemble members; it does not evaluate the robustness of the ensemble as a defended system. A adversary who knows the ensemble can generate perturbations directly against the consensus rule or averaged logits, using the ensemble attack methods cited in the paper itself (Refs. [10-14]). Without such adaptive attacks, the comparative robustness claim is unsupported.
[Section V] The paper concedes that 'if a fixed ensemble committee is chosen as the defense protection of a target model, then this protected prediction model is just another target model, and existing attack algorithms remain to be effective.' This admission applies directly to the fixed ensembles evaluated in Table 3, so the reported results are at best evidence of limited transferability, not of robustness against a competent adversary. The subsequent proposal of randomized per-query ensemble selection is a design sketch, not evaluated data, and is supported only by an unpublished technical report [17]. The central robustness claim therefore lacks evidence under the paper's own stated threat-model taxonomy.
[Table 3 and Section III.C] The experimental reporting is incomplete in ways that undermine the empirical conclusions. The text says the authors evaluate '12 attacks' (Section III.C), while the caption of Table 3 says '15 attacks.' Table 3 reports no confidence intervals, error bars, or repeated runs, and no attack parameters (e.g., perturbation budgets or iteration counts) are given. The table appears to cover only ImageNet, with no comparative table for MNIST or CIFAR-10, despite the paper's broad claims about three benchmark datasets. This makes it impossible to assess the variance or statistical significance of the reported accuracy improvements.
[Section II.B and Section III.C] The paper assumes, rather than establishes, the load-bearing premise that higher disagreement diversity implies lower adversarial transferability and hence higher ensemble robustness. Section II.B states that greater diversity is 'highly correlated' with ensemble accuracy and that negatively correlated classifiers reduce error further, but no formal argument or independent empirical test connects the proposed kappa-based diversity selection to robustness against targeted or untargeted adversarial examples. The authors note in Section VI that several issues 'are not yet well defined with theoretical formulation,' including how to correlate increase in diversity with increase in ensemble robustness. Given that the entire defense proposal rests on this link, it needs either a formal justification or a systematic empirical validation across models, datasets, and attack types.

minor comments (6)

[Throughout] The manuscript contains several typographical errors and garbled formulas, e.g., 'parafigm' in Section II.A, 'Carlini & Wagber' in Section III.C, and the equations in Section II.C appear with mangled symbols in the text. A careful proofreading and typesetting pass is needed.
[Section III.C] The description of the attack set is confusing: the text lists 'two untargeted attacks: FGSM, BIM, and ten targeted attacks: targeted FGSM, targeted BIM, JSMA, Carlini & Wagner attacks (CW0, CW2, CWy).' The count of ten targeted attacks does not obviously follow from the listed algorithms, and the caption of Table 3 uses a different total of 15. The attack count and the mapping of algorithms to targeted/untargeted categories should be made explicit.
[Section III] Tables 1 and 2 are referenced repeatedly (e.g., 'Table 1 shows ten base models for MNIST, eight base models for CIFAR-10 and five models for ImageNet-1000'), but the actual table contents are not included in the manuscript text, only captions. The reader cannot verify the model pools, the kappa values, or the ensemble team compositions.
[Section III.C] The discussion of the random ensemble results is difficult to follow: the sentence 'the random ensemble of team size 3 in this case outperforms the random Kappa ensemble' appears to compare two similar-sounding baselines without a clear definition of either. Please define RandBase and Randκ explicitly and explain the difference between them.
[Section V] The paper uses 'black-box' and 'black box' inconsistently (e.g., 'back-box online attack' appears in Section V). The threat-model terminology should be used uniformly, since the distinctions are central to the paper's argument.
[References] Reference [17] is an unpublished technical report that supports the only proposed mechanism for white-box online robustness; the paper should either include its key results or mark the claim as speculative. Also, Reference [32] is cited as 'Patridge' but the standard spelling is 'Partridge.'

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on unproven assumptions about adversarial transferability, the predictive power of Kappa diversity, and the independence of pretrained model zoos. No new physical or conceptual entities are introduced; the only free parameters are an undisclosed diversity threshold and manually chosen input transformation settings.

free parameters (2)

minimum diversity threshold = not reported
The ensemble creation algorithm in Section III requires a pre-defined minimum diversity threshold to filter candidate teams, but the paper never specifies its value, so all experimental results depend on an undisclosed hyperparameter.
input transformation parameters = not reported (rotation angles, smoothing kernel sizes)
Type 1 diversity classifiers are created using input transformations such as rotation -12 to 12 and smoothing filters (Section III.C); these manually chosen parameters affect the reported robustness but are not systematically specified.

assumptions (4)

standard math Ensemble committee error will not exceed average member error under error-averaging (Cauchy-Schwarz).
Invoked in Sections I and II.A as the theoretical basis for ensemble learning, without proof or qualification.
domain assumption Adversarial examples generated on the target model do not transfer consistently to other ensemble members.
The entire defense evaluation in Section III.C relies on this low-transferability premise; adversarial examples are generated from the target model only, not from the ensemble.
domain assumption Low pairwise Kappa on benign examples is a usable proxy for ensemble robustness against adversarial attacks.
Section III.B selects type 2 diversity ensembles by Kappa scores; the paper itself notes that low Kappa does not guarantee the best performance, yet this proxy is load-bearing for the team-selection algorithm.
domain assumption Publicly available pretrained models are independently trained and sufficiently diverse from the target model.
Section III.A builds the base pool from public model zoos, assuming their errors are decorrelated from the target model's errors.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness." pith.science (2026). https://pith.science/paper/HVOJSZDH

@misc{pith2026190811091,
  author       = {Pith},
  title        = {Pith review of: Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HVOJSZDH}},
  note         = {Machine review of arXiv:1908.11091}
}

read the original abstract

Ensemble learning is a methodology that integrates multiple DNN learners for improving prediction performance of individual learners. Diversity is greater when the errors of the ensemble prediction is more uniformly distributed. Greater diversity is highly correlated with the increase in ensemble accuracy. Another attractive property of diversity optimized ensemble learning is its robustness against deception: an adversarial perturbation attack can mislead one DNN model to misclassify but may not fool other ensemble DNN members consistently. In this paper we first give an overview of the concept of ensemble diversity and examine the three types of ensemble diversity in the context of DNN classifiers. We then describe a set of ensemble diversity measures, a suite of algorithms for creating diversity ensembles and for performing ensemble consensus (voted or learned) for generating high accuracy ensemble output by strategically combining outputs of individual members. This paper concludes with a discussion on a set of open issues in quantifying ensemble diversity for robust deep learning.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 17 canonical work pages

[17]

Strategic Teaming of Input Ensemble and Output Ensemble Against Deception

Wenqi Wei, Ling Liu. "Strategic Teaming of Input Ensemble and Output Ensemble Against Deception", May, 2019, Technology Report, Georgia Institute of Technology

work page 2019
[1]

Defense against the dark arts: An overview of adversarial example security research and future research directions

Ian Goodfellow. Defense against the dark arts: An overview of adversarial example security research and future research directions. CVPR 2018 workshop on challenges and opportunities for privacy and security,

work page 2018
[2]

Adversarial Examples in Deep Learning: Characterization and Divergence

Wenqi Wei, Ling Liu, Margaret Loper, Stacey Truex, Lei Yu, Mehmet Emre Gursoy, and Yanzhao Wu. Adversarial examples in deep learning: Characterization and divergence. arXiv preprint arXiv:1807.00051,

work page Pith review arXiv
[3]

Intriguing properties of neural networks

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199,

arXiv
[4]

Explaining and harnessing adversarial examples (2014)

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples (2014). arXiv preprint arXiv:1412.6572,

arXiv 2014
[5]

Transferability in machine learning: from phenomena to black-box attacks using adversarial samples

Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv:1605.07277,

arXiv
[6]

The limitations of deep learning in adversarial settings

Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. 2016 IEEE EuroS&P, pp 372–387

work page 2016
[7]

Training Ensembles to Detect Adversarial Examples

Alexander Bagnall, Razvan Bunescu and Gordon Stewart. Training Ensembles to Detect Adversarial Examples. arXiv:1712.04006v1, 11 Dec

arXiv

Show all 26 references

[8]

Ensemble Methods as a Defense to Adversarial Perturbations against Deep Neural Networks

Thilo Strauss, Markus Hanselmann, Andrej Junginger and Holger Ulmer. Ensemble Methods as a Defense to Adversarial Perturbations against Deep Neural Networks. arXiv: 1709.03423v2, 8 Feb

arXiv
[9]

T. Pang, K. Xu, C. Du, N. Chen, and J. Zhu, Improving adversarial robustness via promoting ensemble diversity, arXiv:1901.08846,

1901 arXiv
[12]

Y. Li, S. Bai, Y. Zhou, C. Xie, Z. Zhang, and A. Yuille, Learning transferable adversarial examples via ghost networks, arXiv:1812.03413,

arXiv
[13]

Y. Li, S. Bai, C. Xie, Z. Liao, X. Shen, and A. L. Yuille, Regional homogeneity: Towards learning transferable universal adversarial perturbations against defenses, arXiv:1904.00979,

1904 arXiv
[14]

Transferable adversarial perturbations,

W. Zhou, X. Hou, Y. Chen, M. Tang, X. Huang, X. Gan, and Y. Yang, “Transferable adversarial perturbations,” in ECCV 2018, pp. 452–467

2018
[15]

Towards evaluating the robustness of neural networks

Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. 2017 IEEE S&P, pp 39–57

2017
[16]

Meng and H

D. Meng and H. Chen. MagNet: A two-pronged defense against adversarial examples. ACM SIGSAC Conference on Computer and Communication Security (CCS 2017)

2017
[18]

Madry, A

A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. ICLR 2018

2018
[19]

Tramer, A

F. Tramer, A. Kurakin, N. Papernot, D. Boneh, P. McDaniel. Ensemble adversarial training: Attacks and Defenses. arXiv: 1705.07204 (2017)

2017 arXiv
[20]

Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks

Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, Ananthram Swami. Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks. https://arxiv.org/abs/1511.04508. V1 (2015)

2015 arXiv
[21]

C. Xie, J. Wang, Z. Zhang, Z. Ren and A.Yuille. Mitigating adversarial effects through randomization. International Conference on Learning Representations (ICLR 2018)

2018
[23]

Brown, J Wyatt, R

G. Brown, J Wyatt, R. Harris, X. Yao. Diversity creation methods: A survey and categorization. Information Fusion, 6(1): pp5-21. (2005)

2005
[28]

Goodfellow, and Samy Bengio

Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. ICLR 2017 Workshop

2017
[31]

Domingos

P. Domingos. A Unified Bias-Variance Decomposition and its Applications. ICML 2000, pp231-238

2000
[32]

Patridge and W.J

D. Patridge and W.J. Krzanowski. Software diversity: practical statistics for its measurement and exploitation. Information & software Technologies, 39, pp707-717 (1997)

1997
[33]

J. Kittler. Combining Classifiers: A Theoretical Framework, Patten Analysis and App. (1998) 18-27

1998
[35]

Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks, arXiv:1908.07667,

Ka Ho Chow, Wenqi Wei, Yanzhao Wu, Ling Liu. Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks, arXiv:1908.07667,

1908 arXiv
[36]

Demystifying Membership Inference Attacks in Machine Learning as a Service, IEEE Transactions on Services Computing

Stacey Truex, Ling Liu, Emre Gursoy, Lei Yu, Wenqi Wei. Demystifying Membership Inference Attacks in Machine Learning as a Service, IEEE Transactions on Services Computing. arXiv: 1807.09173

arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [17]

Strategic Teaming of Input Ensemble and Output Ensemble Against Deception

Wenqi Wei, Ling Liu. "Strategic Teaming of Input Ensemble and Output Ensemble Against Deception", May, 2019, Technology Report, Georgia Institute of Technology

work page 2019

[2] [1]

Defense against the dark arts: An overview of adversarial example security research and future research directions

Ian Goodfellow. Defense against the dark arts: An overview of adversarial example security research and future research directions. CVPR 2018 workshop on challenges and opportunities for privacy and security,

work page 2018

[3] [2]

Adversarial Examples in Deep Learning: Characterization and Divergence

Wenqi Wei, Ling Liu, Margaret Loper, Stacey Truex, Lei Yu, Mehmet Emre Gursoy, and Yanzhao Wu. Adversarial examples in deep learning: Characterization and divergence. arXiv preprint arXiv:1807.00051,

work page Pith review arXiv

[4] [3]

Intriguing properties of neural networks

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199,

arXiv

[5] [4]

Explaining and harnessing adversarial examples (2014)

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples (2014). arXiv preprint arXiv:1412.6572,

arXiv 2014

[6] [5]

Transferability in machine learning: from phenomena to black-box attacks using adversarial samples

Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv:1605.07277,

arXiv

[7] [6]

The limitations of deep learning in adversarial settings

Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. 2016 IEEE EuroS&P, pp 372–387

work page 2016

[8] [7]

Training Ensembles to Detect Adversarial Examples

Alexander Bagnall, Razvan Bunescu and Gordon Stewart. Training Ensembles to Detect Adversarial Examples. arXiv:1712.04006v1, 11 Dec

arXiv