REVIEW 5 major objections 5 minor 20 references
Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization
T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read Adversarial attacks should maximize the gap between the top non-true-label probability and the true-label probability; SDM claims to beat PGD, C&W, APGD, and AutoAttack on strength and cost.
desk verdict A genuinely new staged attack, but the ℓ2 evaluation is undefined and the baselines look weak; the idea is worth engaging, the numbers are not. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the three-layer cycle-stage-step schedule with a per-stage loss switch. In stage one the loss is L_y = -P_y, which only suppresses the true label. In each later stage the loss is the Directional Probability Difference Ratio, L_DPDR^(n) = (Pτ - Py) / (δ - sign(Pτ - Py)·(Pτ - P̂_n) + ζ), where δ = 0.5·max(Pτ - P̂_n), ζ = 1e-10, and P̂_n is the n-th largest non-true probability. The sign term keeps the optimization direction consistent whether the attack has succeeded or not, and the denominator is constructed to stay positive. Stage by stage, the same update rule compresses the 2nd, 3rd, ..., N-th largest probabilities, raising the top non-true probability Pτ whil
What would settle it
Run SDM and APGD-DLR with identical step budgets on a different architecture and dataset, and for each generated example record whether a high DPDR value (Pτ - Py) actually corresponds to a prediction other than the true label; if SDM's advantage shrinks on other models or a high-DPDR example still predicts the true label, the central claim fails.
Extended reading notes
Core claim
The paper's central claim is that the conventional objective of minimizing the true label's logit or probability is not aligned with actual attack success. It documents a failed attack with true-label probability 30.25% and cross-entropy loss 1.196, alongside a successful attack with higher true-label probability 34.74% and lower loss 1.057, as evidence that a low true-label probability and a high loss do not guarantee adversariality. The replacement objective is maximizing Pτ - Py, where Pτ is the largest non-true-label probability and Py is the true-label probability. SDM solves this by splitting optimization into identical cycles, each containing N ordered stages of iterative gradient-asc
Load-bearing premise
The load-bearing assumption is that the fixed staged schedule of SDM reliably finds better solutions to the new objective than direct optimization, even though the paper gives no convergence proof and tests only one model on CIFAR.
Editorial extensions
If this is right
- SDM gives robustness evaluators a stronger attack at the same perturbation budget, so defenses that survive PGD or APGD still need to be checked against SDM.
- Because SDM is more cost-effective, high-quality attack evaluation can be done with fewer iterations.
- Replacing the PGD-generated examples in adversarial training with SDM-generated examples improves both clean accuracy and robust error by about 2% on CIFAR-10, so SDM can strengthen defenses rather than only attack them.
- Plugging SDM into AutoAttack as ASDM raises the ensemble's average attack success rate by roughly 1.18%, improving composite evaluation.
Reading between the lines
- If DPDR is closer to true attack success than cross-entropy or DLR, similar staged objectives could improve transfer-based or query-based attacks, although the paper only tests white-box settings.
- The reported gains are on one architecture and two small datasets; a natural test is whether SDM's advantage persists on larger models and ImageNet-scale inputs, where the loss landscape differs.
- The high-loss non-adversarial example phenomenon suggests a diagnostic metric: measure the gap between loss value and attack success across iterations; a loss family with a monotone relation would likely be more efficient.
- A direct ablation varying the number of stages N or the order of probability compression could show whether the staging itself, rather than just the DPDR loss, drives the improvement; the paper does not include that ablation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Sequential Difference Maximization (SDM), a white-box adversarial attack that reconstructs the attack objective as maximizing the difference between the highest non-true-label probability and the true-label probability. SDM uses a three-level "cycle-stage-step" optimization: an initial stage minimizes the true-label probability, followed by stages optimizing a Directional Probability Difference Ratio (DPDR) loss that suppresses the largest non-true labels' probabilities. Experiments on WideResNet-28-10 with CIFAR-10/100 and five defenses (AT, TRADES, MART, HAT, LOAT) claim that SDM achieves higher attack success and better cost-effectiveness than PGD, C&W, APGD, and AutoAttack, and that using SDM in adversarial training improves robust accuracy.
Significance. If the empirical claims hold, the paper offers a simple, practical improvement over standard strong attacks and a new perspective on attack objectives. The manuscript is explicit about the algorithm (Algorithm 1) and makes code available, which aids reproducibility. It evaluates against a diverse set of defenses and also probes the use of the attack as a defensive training component. However, the evidence as presented is not yet sufficient to support the SOTA claim: the ℓ2 protocol is underspecified, the C&W baseline appears unrepresentatively weak, all results are single-run, and no ablation validates the staged design.
major comments (5)
- [Sec. 4.2 / Algorithm 1] The ℓ2 evaluation is underspecified. Algorithm 1's update is x'_t = x + clamp(x'_{t-1} - x + alpha * sgn(Grad), -epsilon, epsilon), which is an ℓ∞-style sign step with box projection. For ℓ2 = 1.0, no projection or gradient normalization is stated; if used verbatim, the resulting perturbation may exceed the ℓ2 budget. All ℓ2 columns in Tables 3 and 4 are therefore not reproducible as written. Please state the exact ℓ2 update and projection, and report the achieved average ℓ2 norms.
- [Table 3, ℓ2 columns] C&W, an ℓ2-designed attack, scores 52.31 and 48.35 on TRADES and MART, while PGD reaches 59.63 and 59.02. This 7–10 point gap is inconsistent with a standard C&W-Box implementation at a comparable budget and suggests a non-representative baseline configuration (e.g., no binary search, fixed step, mismatched iteration count). The paper should report the exact C&W setup or replace it with a faithful implementation; as it stands, the claimed SOTA gains over previous methods are not established.
- [Sec. 4.2, Tables 3–5] All results are single-run with no standard deviations, seeds, or significance tests. The average advantage over APGD2 and AA is small (1.57% and 1.18%). Given the variability of adversarial attack results across restarts and network evaluations, these differences may be within noise. Please provide mean ± std over at least 5 independent runs and/or paired significance tests, particularly for the headline comparisons.
- [Sec. 3.2, Eq. (6)] There is an inconsistency in the DPDR denominator. With sign(Pτ − Py) = −1, Eq. (6) gives denominator δ − (−1)(Pτ − P̂_n − δ) + ζ = 2δ + Pτ − P̂_n + ζ, but the text states the denominator is Pτ − P̂_n + ζ. Also, δ is defined as 0.5 × max(Pτ − P̂_n) without specifying the range of the max. This makes the loss definition ambiguous; please correct the formula or the text and define max explicitly.
- [Sec. 3.2, Table 2; Sec. 4.2] The staged scheme is the central algorithmic contribution, but no ablation isolates the effect of the stages or the cycle/step schedule. The hyperparameters N=5, the C/T schedule in Table 2, δ, and ζ are hand-selected. It is possible that the observed gains come from the total step count and restart structure rather than from the sequential stage ordering. Add ablations (e.g., single-stage DPDR with the same total steps, shuffled stage order, varying N) to validate the 'sequential' claim.
minor comments (5)
- [Sec. 4.3, Fig. 2] The cost-effectiveness figure is shown only for AT on CIFAR-10. The text generalizes to 'each total iteration step' without stating that this is a single defense/dataset; please qualify.
- [Sec. 4.1] The sentence 'For APGD1 and APGD2, the restart count is set to 5, with the total iteration step calculated as the product of restart count and iteration steps' is ambiguous: specify whether the reported 1000 total steps is the per-restart iteration count or the sum over restarts.
- [General / Notation] The notation `P_n` in Eq. (6) uses a backtick that is not formally defined. The descending-order notation `P should be defined in a display equation or clearly explained.
- [Introduction / Related Work] There is no dedicated related-work section. The paper jumps from the introduction to motivation; a brief review of existing gradient-based and objective-reformulation attacks would improve positioning.
- [Table 1] The layout of Table 1 is hard to follow; the columns for 'output k', 'loss', 'pred. label', and 'att. result' are crowded. Please reformat for readability.
Circularity Check
No significant circularity: SDM's optimization objective and loss are internally consistent, and the reported gains are evaluated against external baselines and held-out defenses.
full rationale
The paper's central derivation is the reconstruction of the adversarial-example objective as maximizing P_tau - P_y (Eq. 4), followed by the design of the DPDR loss (Eq. 6) and the staged algorithm (Algorithm 1). The DPDR loss is indeed constructed to directly increase P_tau and decrease competing-class probabilities, so the method optimizes the objective it sets out to optimize. This is a design choice, not circularity: the paper's empirical claims are that SDM outperforms PGD, C&W, APGD, and AutoAttack on CIFAR-10/100 under multiple defenses (Tables 3-5, Fig. 2), which are external comparisons not derived from the construction. No parameter is fitted to the reported outcomes; hyperparameters such as N=5, T, and the cycle-stage-step schedule are hand-selected but not tuned to the evaluation results. The paper contains no self-citations and does not invoke any uniqueness theorem or prior work by the same authors. The motivation in Sec. 2 uses an empirical observation ('non-adversarial examples with high loss values') rather than a definitional equivalence. Concerns about baseline strength or the underspecified l2 update are correctness risks, not instances of circular reasoning. Therefore, no circular step meets the evidentiary bar required by the analysis.
Assumptions & free parameters
free parameters (4)
- Number of optimization stages N =
5
- Cycle/step schedule (C, T) =
e.g., C=5, T=40 at 1000 steps
- Bias term delta =
0.5 * max(P_tau - P_n)
- Numerical stability term zeta =
1e-10
assumptions (4)
- standard math The classifier outputs logits that can be normalized with softmax to well-defined probabilities (Eq. 3).
- domain assumption White-box access to model gradients enables gradient-based optimization.
- ad hoc to paper The staged optimization heuristic (first minimize true-label probability, then compress non-true probabilities) improves convergence to the margin objective.
- ad hoc to paper The single motivating example in Table 1 is representative of a general failure mode of CE loss.
Cite this review
Pith. "Pith review of Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization." pith.science (2026). https://pith.science/paper/POGTQMPQ
@misc{pith2026250900826,
author = {Pith},
title = {Pith review of: Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/POGTQMPQ}},
note = {Machine review of arXiv:2509.00826}
}
read the original abstract
Efficient adversarial attack methods are critical for assessing the robustness of computer vision models. In this paper, we reconstruct the optimization objective for generating adversarial examples as "maximizing the difference between the non-true labels' probability upper bound and the true label's probability," and propose a gradient-based attack method termed Sequential Difference Maximization (SDM). SDM establishes a three-layer optimization framework of "cycle-stage-step." The processes between cycles and between iterative steps are respectively identical, while optimization stages differ in terms of loss functions: in the initial stage, the negative probability of the true label is used as the loss function to compress the solution space; in subsequent stages, we introduce the Directional Probability Difference Ratio (DPDR) loss function to gradually increase the non-true labels' probability upper bound by compressing the irrelevant labels' probabilities. Experiments demonstrate that compared with previous SOTA methods, SDM not only exhibits stronger attack performance but also achieves higher attack cost-effectiveness. Additionally, SDM can be combined with adversarial training methods to enhance their defensive effects. The code is available at https://github.com/X-L-Liu/SDM.
Figures
Reference graph
Works this paper leans on
-
[1]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (May 2015), 436–444. doi:10.1038/nature14539
-
[2]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 770–778. doi:10.1109/CVPR.2016.90
-
[3]
Qian Chen, Ze Liu, Yi Zhang, Keren Fu, Qijun Zhao, and Hongwei Du. 2021. RGB- D Salient Object Detection via 3D Convolutional Neural Networks. InThirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, Virtual Event, February 2-9, 2021 . AAAI Press, 1063–1071. doi:10.1609/AAAI.V35I2.16191
- [4]
-
[5]
Costas Neocleous and Christos N. Schizas. 2014. On the Claim for the Existence of "Adversarial Examples" in Deep Learning Neural Networks. In NCTA 2014 - Proceedings of the International Conference on Neural Computation Theory and Applications, part of IJCCI 2014, Rome, Italy, 22 - 24 October, 2014 , Kurosh Madani and Joaquim Filipe (Eds.). SciTePress, 30...
-
[6]
Goodfellow, Jonathon Shlens, and Christian Szegedy
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In 2015 International Conference on Learning Representations (ICLR), Yoshua Bengio and Yann LeCun (Eds.)
work page 2015
-
[7]
Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial ro- bustness with an ensemble of diverse parameter-free attacks. In2020 International Conference on Machine Learning (ICML) , Vol. 119. PMLR, 2206–2216
work page 2020
-
[8]
Xinyi Wang, Zhibo Jin, Zhiyu Zhu, Jiayu Zhang, and Huaming Chen. 2024. Improving Adversarial Transferability via Frequency-Guided Sample Relevance Attack. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024. ACM, 2410–2419. doi:10.1145/3627673.3679858
arXiv 2024
Show all 20 references
-
[9]
Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. 2021. Threat of Adversarial Attacks on Deep Learning in Computer Vision: Survey II. CoRR abs/2108.00401 (2021). arXiv:2108.00401
2021 arXiv
-
[10]
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In 2018 International Conference on Learning Representations (ICLR) . OpenReview.net
2018
-
[11]
Goodfellow, and Rob Fergus
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conf...
2014 arXiv
-
[12]
Goodfellow, and Samy Bengio
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In2017 International Conference on Learning Representations (ICLR). OpenReview.net
2017
-
[13]
Nicholas Carlini and David A. Wagner. 2017. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017 . IEEE Computer Society, 39–57. doi:10.1109/ SP.2017.49
2017
-
[14]
J.C. Spall. 1992. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automat. Control 37, 3 (1992), 332–341. doi:10.1109/9.119632
1992 doi
-
[15]
Sergey Zagoruyko and Nikos Komodakis. 2016. Wide Residual Networks. In BMVC, York, UK, September 19-22 . https://bmva-archive.org.uk/bmvc/2016/ papers/paper087/index.html
2016
-
[16]
Krizhevsky and G
A. Krizhevsky and G. Hinton. 2009. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto (2009). https://www.cs.utoronto.ca/~kriz/cifar.html
2009
-
[17]
Xing, Laurent El Ghaoui, and Michael I
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. 2019. Theoretically Principled Trade-off between Robustness and Accuracy. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long ...
2019
-
[18]
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. 2020. Improving Adversarial Robustness Requires Revisiting Misclassified Examples. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, April 26-30, 2020 . OpenReview....
2020
-
[19]
Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. 2022. Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off. In The Tenth In- ternational Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https://openre...
2022
-
[20]
Xiangyu Yin and Wenjie Ruan. 2024. Boosting Adversarial Training via Fisher- Rao Norm-Based Regularization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024 . IEEE, 24544–24553. doi:10.1109/CVPR52733.2024.02317
2024
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.