REVIEW 5 major objections 5 minor 13 references
A Comprehensive Review of Adversarial Attacks on Machine Learning
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read The paper claims that defensive distillation restores correct predictions for 71% of adversarial bone-fracture images, outperforming all seven preprocessing defenses it tested.
desk verdict Competent tutorial survey; the only new result (defensive distillation at 71%) is invalid due to train/test overlap, and the rest is a rehash of known attacks and defenses. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is defensive distillation: a student network trained on the teacher model's soft class probabilities rather than hard labels, which smooths the loss landscape and reduces the gradient signal an attacker can exploit. In the paper, the student is trained through the Adversarial Robustness Toolbox on 100 original bone-fracture samples, and the same toolbox is used to generate adversarial examples from FGSM, PGD, BIM, SIMBA, Carlini-Wagner L2, DeepFool, and Pixel attacks. The student's restored-prediction rate across those attacks is the paper's main evidence for distillation's effectiveness.
What would settle it
Train the defensive-distillation student on a set of bone-fracture images disjoint from the ones used to generate adversarial test examples, then measure the restored-prediction rate; if it falls to the same 15-45% range as the preprocessing defenses, the paper's central claim that distillation is an effective defense would be undercut.
Extended reading notes
Core claim
On its own terms, the central discovery is that adversarial attacks generated with the Adversarial Robustness Toolbox reliably mislead both a DETR-ResNet50 object detector and a roughly 99%-accurate CNN for bone-fracture classification, and that defensive distillation—training a student network to mimic the teacher's soft probability outputs—restores the original prediction on 71% of adversarial bone-fracture images across SIMBA, BIM, DeepFool, FGSM, Carlini-Wagner L2, and Pixel attacks. The same evaluation shows that preprocessing defenses are largely ineffective: total variation denoising tops out at a 42.7% success rate, and feature squeezing at 14%. For object detection, none of the preprocessing defenses reached 1% restoration, which the authors take as evidence that detection models need defense strategies beyond simple input preprocessing.
Load-bearing premise
The student model in defensive distillation was trained on the same 100 original images used to create the adversarial test samples, so the 71% success rate could be inflated by overfitting to those specific images rather than reflecting performance on new inputs.
Editorial extensions
If this is right
- Practitioners evaluating input-preprocessing defenses for medical image classifiers should expect limited restoration (at most roughly 43% in these experiments) and should consider distillation-based training instead.
- The 71% defensive-distillation restoration rate suggests that retraining a student on soft labels can absorb much of the perturbation noise that defeats preprocessing.
- Object detection models, at least in this setup, remain almost entirely unprotected by preprocessing, so defenses for detection need a different design.
- The reported mean perturbation values provide a rough ranking of attack stealthiness, with the Pixel attack being the most imperceptible on the bone-fracture model.
Reading between the lines
- Because the student was trained on the same 100 images later used to create the adversarial test examples, the 71% figure is likely an upper bound; a held-out evaluation could show a substantially lower restoration rate.
- The paper's attack-success results (100% for PGD and BIM on the object detector) are consistent with the broader pattern that gradient-based attacks transfer across architectures, so the specific model choice may matter less than the perturbation budget.
- A natural next experiment the paper does not run is to combine defensive distillation with adversarial training; the two mechanisms may compound and push restoration rates higher.
- The claim that preprocessing defenses are insufficient should be read as specific to these attacks and hyperparameters; techniques like feature squeezing are often more effective when paired with adversarial detection rather than simple correction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript positions itself as a comprehensive review of adversarial attacks on machine learning and complements this with two empirical studies using the Adversarial Robustness Toolbox (ART). The first study applies FGSM, PGD, and BIM to a DETR-ResNet50 object detector on 100 sampled self-driving car images. The second applies BIM, FGSM, Simba, C&W L2, DeepFool, and Pixel attacks to a CNN bone-fracture classifier, then evaluates seven preprocessing defenses and defensive distillation. The paper's central positive claim is that defensive distillation achieves a 71% success rate in restoring correct predictions on adversarial bone-fracture images, while preprocessing defenses are largely ineffective.
Significance. If the defensive distillation claim were supported by a valid out-of-sample evaluation, it would be a practically relevant result for medical-image classifiers. The paper is transparent about ART attack parameters and explicitly acknowledges the small sample size in its Limitations section, which is a strength. However, the central empirical contribution is currently not supported: the distillation evaluation is circular because the student model is trained on the same 100 images that seed the adversarial test set, attack and defense success rates are reported without confidence intervals or repeated resampling, object-detection 'success' is not crisply defined, and the review's reference list contains several incorrect citations. The reported defense rankings therefore cannot be accepted as out-of-sample evidence.
major comments (5)
- [Defensive Distillation] The 71% defensive distillation success rate is not an out-of-sample estimate. The section states 'we trained a model using the ART library on a limited dataset of 100 original samples,' and the adversarial images evaluated against the student model are perturbations of those same 100 samples. A student model could memorize the training labels and appear robust on perturbations of its own training images without generalizing to new bone-fracture images. No held-out split, cross-validation, or repeated resampling is described. This circularity directly undermines the central positive claim of the paper.
- [Evaluation Metrics and attack/defense tables] All attack and defense success rates (e.g., 95%, 99%, 47%, 71%) are reported as point estimates without confidence intervals, standard deviations, or repeated resampling over the 100-image sample. Given n=100, binomial confidence intervals are wide; for example, the 71% defense success rate has an approximate 95% confidence interval of roughly 61-80%. Consequntly, the rankings of defenses (e.g., median filtering at 35.7% vs. Gaussian blur at 30.3%) and the relative effectiveness of attacks are not statistically supported as stated.
- [Attacks on Object Detection Use Case] Object-detection attack success is loosely defined. In the FGSM example, a successful attack is described as 'either no object detection or incorrect identifications,' which conflates missing an object with misclassifying a detected object and does not specify the detection confidence threshold or matching procedure. No mAP, precision/recall, or per-class metrics are reported, so the 99-100% success rates for FGSM, PGD, and BIM on the object detector are ambiguous and not comparable with standard object-detection evaluations.
- [Conclusion] The blanket statement that 'preprocessing techniques proved ineffective in mitigating these attacks' is contradicted by the paper's own defense tables. For example, median filtering and spatial smoothing each report 98% success on Pixel-attack images, TV denoising reports 83% on Carlini-Wagner L2 images, and Gaussian blur reports 97% on Pixel-attack images. The aggregated preprocessing rates are below 50%, but the per-attack results show that several preprocessing defenses are highly effective against specific attacks, so the conclusion is not supported without qualification.
- [References] The literature review is undermined by multiple incorrect citations. Reference [9] is cited for 'certified robustness (Wong et al., 2018)' but points to a 3D segmentation paper; Reference [10] is cited for 'adversarial detection (Ma et al., 2019)' but points to a bias-variance trade-off paper; References [6] and [14] duplicate the same 'Making an Invisibility Cloak' paper with different arXiv identifiers. These errors affect the credibility of the review aspect of the manuscript and need correction.
minor comments (5)
- [Throughout] There are frequent typos and formatting inconsistencies, such as 'FSGM' instead of 'FGSM' and duplicated equation text in the FGSM definition (e.g., 'ψ(x,y)=−e⋅sign(∇xL(x,y))ψ(x,y)=−e⋅sign(∇xL(x,y))'). A careful proofread is needed.
- [Figures] Figures are referenced in text but are not numbered, and many figure captions are missing or too terse to interpret (e.g., the before/after prediction labels are only in the body text). Figure numbering and captions should be added.
- [Experimental Setup] The random selection of 100 representative samples is not described with a random seed or selection protocol, so the experiments cannot be reproduced exactly.
- [Defensive Distillation] No training hyperparameters for the student model are provided: temperature, student architecture, optimizer, number of epochs, and loss function are all omitted. This makes the distillation experiment not reproducible even if a held-out split were added.
- [References] Reference [11] is given as a Hugging Face URL but is also cited as the source for several standard attacks, and Reference [12] is a bare Kaggle URL with no title or access date. These should be clarified and completed.
Circularity Check
Defensive distillation's 71% success rate is measured on adversarial versions of the same 100 original samples used to train the student model, so the result is a same-distribution fit rather than an out-of-sample prediction.
-
fitted input called prediction
[Defensive Distillation section (p. 25) and 'Performance Metrics of Defensive distillation Defense Method Against Adversarial Attacks' (p. 25)]
"To explore the effectiveness of defensive distillation, we trained a model using the ART library on a limited dataset of 100 original samples. ... The model demonstrated a 71% success rate against a diverse range of adversarial attacks considered in this research, highlighting its effectiveness in enhancing the robustness of the neural network."
Every adversarial test image in the bone-break use case is a perturbation of one of the same 100 original samples on which the defensive-distillation student model was trained. The paper states that each attack 'was applied to 100 sample images' and that the student was trained 'on a limited dataset of 100 original samples.' Thus the reported 71% defense success rate is computed on adversarial inputs derived from the student's own training images, with no held-out split or repeated resampling. A model that memorizes those 100 clean images and their labels can appear robust to small perturbations of them even if it has no ability to classify unseen images.
full rationale
This paper is a review with a limited experimental component, not a formal derivation, so there is no equation-level circularity in the survey portions. The review material is a synthesis of standard, externally attributable attacks and defenses, and the attack-success experiments are ordinary demonstrations of known vulnerabilities. The one load-bearing positive result is the claim that defensive distillation achieves a 71% success rate. That claim is circular in evaluation design: the student model is trained on the same 100 original bone-fracture images that seed all adversarial evaluation examples, as stated in the Experimental Setup and Defensive Distillation sections. Consequently, the 71% figure cannot be interpreted as robustness on new images; it is at least partly an artifact of training and test sharing the same source images. The Limitations section acknowledges the small sample size but does not acknowledge this train/test overlap. No self-citation chain or imported uniqueness theorem is involved, so the circularity is localized to the defensive-distillation evaluation rather than pervading the whole paper. A score of 5 reflects that the paper's only original empirical claim is partially circular, while the rest of the manuscript is an independent literature review.
Assumptions & free parameters
free parameters (2)
- Manual attack parameters =
per-attack values (e.g., FGSM eps=0.1, eps_step=0.0005; PGD eps=0.1; BIM eps=0.1, eps_step=0.05)
- Random 100-image subset =
unspecified seed
assumptions (3)
- domain assumption ART library implements the attacks and defenses as described
- domain assumption The CNN for bone-break classification achieves ~99% accuracy
- domain assumption The Kaggle self-driving dataset is appropriate for DETR pretrained on COCO
Cite this review
Pith. "Pith review of A Comprehensive Review of Adversarial Attacks on Machine Learning." pith.science (2026). https://pith.science/paper/MM3UODIS
@misc{pith2026241211384,
author = {Pith},
title = {Pith review of: A Comprehensive Review of Adversarial Attacks on Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/MM3UODIS}},
note = {Machine review of arXiv:2412.11384}
}
read the original abstract
This research provides a comprehensive overview of adversarial attacks on AI and ML models, exploring various attack types, techniques, and their potential harms. We also delve into the business implications, mitigation strategies, and future research directions. To gain practical insights, we employ the Adversarial Robustness Toolbox (ART) [1] library to simulate these attacks on real-world use cases, such as self-driving cars. Our goal is to inform practitioners and researchers about the challenges and opportunities in defending AI systems against adversarial threats. By providing a comprehensive comparison of different attack methods, we aim to contribute to the development of more robust and secure AI systems.
Figures
Reference graph
Works this paper leans on
-
[9]
3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes
K. C. L. Wong, "3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes, " arxiv.org/pdf/1809.00076, 2018
work page Pith review arXiv 2018
-
[10]
Reconciling modern machine learning practice and the bias-variance trade-off,
M. Belkina, "Reconciling modern machine learning practice and the bias-variance trade-off, " arxiv.org/pdf/1812.11118, 2019
arXiv 2019
-
[6]
Making an Invisibility Cloak: Real World,
Z. Wu, "Making an Invisibility Cloak: Real World, " arxiv.org/pdf/1611.02770, 2020
arXiv 2020
-
[14]
Making an Invisibility Cloak: Real World,
Z. Wu, "Making an Invisibility Cloak: Real World, " arxiv.org/pdf/1910.14667, 2020
arXiv 1910
-
[1]
Adversarial Robustness Toolbox (ART),
"Adversarial Robustness Toolbox (ART), " [Online]. Available: https://github.com/Trusted- AI/adversarial-robustness-toolbox
-
[2]
Intriguing properties of neural networks,
C. Szegedy, "Intriguing properties of neural networks, " arxiv.org/pdf/1312.6199, 2014
arXiv 2014
-
[3]
Explaining and Harnessing Adversarial Examples,
I. J. Goodfellow, "Explaining and Harnessing Adversarial Examples, " arxiv.org/pdf/1412.6572, 2015
arXiv 2015
-
[4]
Towards Evaluating the Robustness of Neural Networks,
N. Carlini, "Towards Evaluating the Robustness of Neural Networks, " arxiv.org/pdf/1608.04644, 2017
arXiv 2017
Show all 13 references
-
[7]
Discovering Attributable Signals in Adversarial ML Attack,
M. Dotter, "Discovering Attributable Signals in Adversarial ML Attack, " arxiv.org/pdf/2101.02899, 2021
2021 arXiv
-
[8]
Towards Deep Learning Models Resistant to Adversarial Attacks,
A. Madry, "Towards Deep Learning Models Resistant to Adversarial Attacks, " arxiv.org/pdf/1706.06083, 2019
2019 arXiv
-
[11]
Available: https://huggingface.co/facebook/detr-resnet-50
[Online]. Available: https://huggingface.co/facebook/detr-resnet-50
-
[12]
Available: https://www.kaggle.com/datasets/alincijov/self-driving-cars
[Online]. Available: https://www.kaggle.com/datasets/alincijov/self-driving-cars
-
[13]
Adversarial Robustness Toolbox v1.0.0,
M.-I. Nicolae, "Adversarial Robustness Toolbox v1.0.0, " arxiv.org, p. 34, 2019
2019
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.