Pith. sign in

REVIEW 5 major objections 5 minor 20 references

Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization

T0 review · 5 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read Adversarial attacks should maximize the gap between the top non-true-label probability and the true-label probability; SDM claims to beat PGD, C&W, APGD, and AutoAttack on strength and cost.

desk verdict A genuinely new staged attack, but the ℓ2 evaluation is undefined and the baselines look weak; the idea is worth engaging, the numbers are not. read the letter →

arxiv 2509.00826 v1 pith:POGTQMPQ submitted 2025-08-31 cs.CV cs.AIcs.LG

classification cs.CVcs.AIcs.LG
keywords adversarialexampleswhite-boxattackgradient-basedmulti-stageoptimizationprobabilitydifferenceratiotrainingrobustnessevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Current gradient attacks usually aim to lower the true label's probability, but the paper shows this can produce high-loss examples that are not adversarial at all: the model still predicts the true label. It reconstructs the goal as maximizing the difference between the highest non-true-label probability and the true-label probability, and introduces Sequential Difference Maximization (SDM) to solve it. SDM first suppresses the true-label probability, then applies a Directional Probability Difference Ratio (DPDR) loss in ordered stages to push up the top non-true probability while shrinking the competing labels. On CIFAR-10/100 with WideResNet-28-10 and five defense methods, SDM reports the highest attack error rates and the best attack performance per iteration step, and using SDM inside adversarial training lowers clean and robust error by about two percentage points.

What carries the argument

The load-bearing mechanism is the three-layer cycle-stage-step schedule with a per-stage loss switch. In stage one the loss is L_y = -P_y, which only suppresses the true label. In each later stage the loss is the Directional Probability Difference Ratio, L_DPDR^(n) = (Pτ - Py) / (δ - sign(Pτ - Py)·(Pτ - P̂_n) + ζ), where δ = 0.5·max(Pτ - P̂_n), ζ = 1e-10, and P̂_n is the n-th largest non-true probability. The sign term keeps the optimization direction consistent whether the attack has succeeded or not, and the denominator is constructed to stay positive. Stage by stage, the same update rule compresses the 2nd, 3rd, ..., N-th largest probabilities, raising the top non-true probability Pτ whil

What would settle it

Run SDM and APGD-DLR with identical step budgets on a different architecture and dataset, and for each generated example record whether a high DPDR value (Pτ - Py) actually corresponds to a prediction other than the true label; if SDM's advantage shrinks on other models or a high-DPDR example still predicts the true label, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the conventional objective of minimizing the true label's logit or probability is not aligned with actual attack success. It documents a failed attack with true-label probability 30.25% and cross-entropy loss 1.196, alongside a successful attack with higher true-label probability 34.74% and lower loss 1.057, as evidence that a low true-label probability and a high loss do not guarantee adversariality. The replacement objective is maximizing Pτ - Py, where Pτ is the largest non-true-label probability and Py is the true-label probability. SDM solves this by splitting optimization into identical cycles, each containing N ordered stages of iterative gradient-asc

Load-bearing premise

The load-bearing assumption is that the fixed staged schedule of SDM reliably finds better solutions to the new objective than direct optimization, even though the paper gives no convergence proof and tests only one model on CIFAR.

Editorial extensions

If this is right

  • SDM gives robustness evaluators a stronger attack at the same perturbation budget, so defenses that survive PGD or APGD still need to be checked against SDM.
  • Because SDM is more cost-effective, high-quality attack evaluation can be done with fewer iterations.
  • Replacing the PGD-generated examples in adversarial training with SDM-generated examples improves both clean accuracy and robust error by about 2% on CIFAR-10, so SDM can strengthen defenses rather than only attack them.
  • Plugging SDM into AutoAttack as ASDM raises the ensemble's average attack success rate by roughly 1.18%, improving composite evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If DPDR is closer to true attack success than cross-entropy or DLR, similar staged objectives could improve transfer-based or query-based attacks, although the paper only tests white-box settings.
  • The reported gains are on one architecture and two small datasets; a natural test is whether SDM's advantage persists on larger models and ImageNet-scale inputs, where the loss landscape differs.
  • The high-loss non-adversarial example phenomenon suggests a diagnostic metric: measure the gap between loss value and attack success across iterations; a loss family with a monotone relation would likely be more efficient.
  • A direct ablation varying the number of stages N or the order of probability compression could show whether the staging itself, rather than just the DPDR loss, drives the improvement; the paper does not include that ablation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Sequential Difference Maximization (SDM), a white-box adversarial attack that reconstructs the attack objective as maximizing the difference between the highest non-true-label probability and the true-label probability. SDM uses a three-level "cycle-stage-step" optimization: an initial stage minimizes the true-label probability, followed by stages optimizing a Directional Probability Difference Ratio (DPDR) loss that suppresses the largest non-true labels' probabilities. Experiments on WideResNet-28-10 with CIFAR-10/100 and five defenses (AT, TRADES, MART, HAT, LOAT) claim that SDM achieves higher attack success and better cost-effectiveness than PGD, C&W, APGD, and AutoAttack, and that using SDM in adversarial training improves robust accuracy.

Significance. If the empirical claims hold, the paper offers a simple, practical improvement over standard strong attacks and a new perspective on attack objectives. The manuscript is explicit about the algorithm (Algorithm 1) and makes code available, which aids reproducibility. It evaluates against a diverse set of defenses and also probes the use of the attack as a defensive training component. However, the evidence as presented is not yet sufficient to support the SOTA claim: the ℓ2 protocol is underspecified, the C&W baseline appears unrepresentatively weak, all results are single-run, and no ablation validates the staged design.

major comments (5)
  1. [Sec. 4.2 / Algorithm 1] The ℓ2 evaluation is underspecified. Algorithm 1's update is x'_t = x + clamp(x'_{t-1} - x + alpha * sgn(Grad), -epsilon, epsilon), which is an ℓ∞-style sign step with box projection. For ℓ2 = 1.0, no projection or gradient normalization is stated; if used verbatim, the resulting perturbation may exceed the ℓ2 budget. All ℓ2 columns in Tables 3 and 4 are therefore not reproducible as written. Please state the exact ℓ2 update and projection, and report the achieved average ℓ2 norms.
  2. [Table 3, ℓ2 columns] C&W, an ℓ2-designed attack, scores 52.31 and 48.35 on TRADES and MART, while PGD reaches 59.63 and 59.02. This 7–10 point gap is inconsistent with a standard C&W-Box implementation at a comparable budget and suggests a non-representative baseline configuration (e.g., no binary search, fixed step, mismatched iteration count). The paper should report the exact C&W setup or replace it with a faithful implementation; as it stands, the claimed SOTA gains over previous methods are not established.
  3. [Sec. 4.2, Tables 3–5] All results are single-run with no standard deviations, seeds, or significance tests. The average advantage over APGD2 and AA is small (1.57% and 1.18%). Given the variability of adversarial attack results across restarts and network evaluations, these differences may be within noise. Please provide mean ± std over at least 5 independent runs and/or paired significance tests, particularly for the headline comparisons.
  4. [Sec. 3.2, Eq. (6)] There is an inconsistency in the DPDR denominator. With sign(Pτ − Py) = −1, Eq. (6) gives denominator δ − (−1)(Pτ − P̂_n − δ) + ζ = 2δ + Pτ − P̂_n + ζ, but the text states the denominator is Pτ − P̂_n + ζ. Also, δ is defined as 0.5 × max(Pτ − P̂_n) without specifying the range of the max. This makes the loss definition ambiguous; please correct the formula or the text and define max explicitly.
  5. [Sec. 3.2, Table 2; Sec. 4.2] The staged scheme is the central algorithmic contribution, but no ablation isolates the effect of the stages or the cycle/step schedule. The hyperparameters N=5, the C/T schedule in Table 2, δ, and ζ are hand-selected. It is possible that the observed gains come from the total step count and restart structure rather than from the sequential stage ordering. Add ablations (e.g., single-stage DPDR with the same total steps, shuffled stage order, varying N) to validate the 'sequential' claim.
minor comments (5)
  1. [Sec. 4.3, Fig. 2] The cost-effectiveness figure is shown only for AT on CIFAR-10. The text generalizes to 'each total iteration step' without stating that this is a single defense/dataset; please qualify.
  2. [Sec. 4.1] The sentence 'For APGD1 and APGD2, the restart count is set to 5, with the total iteration step calculated as the product of restart count and iteration steps' is ambiguous: specify whether the reported 1000 total steps is the per-restart iteration count or the sum over restarts.
  3. [General / Notation] The notation `P_n` in Eq. (6) uses a backtick that is not formally defined. The descending-order notation `P should be defined in a display equation or clearly explained.
  4. [Introduction / Related Work] There is no dedicated related-work section. The paper jumps from the introduction to motivation; a brief review of existing gradient-based and objective-reformulation attacks would improve positioning.
  5. [Table 1] The layout of Table 1 is hard to follow; the columns for 'output k', 'loss', 'pred. label', and 'att. result' are crowded. Please reformat for readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SDM's optimization objective and loss are internally consistent, and the reported gains are evaluated against external baselines and held-out defenses.

full rationale

The paper's central derivation is the reconstruction of the adversarial-example objective as maximizing P_tau - P_y (Eq. 4), followed by the design of the DPDR loss (Eq. 6) and the staged algorithm (Algorithm 1). The DPDR loss is indeed constructed to directly increase P_tau and decrease competing-class probabilities, so the method optimizes the objective it sets out to optimize. This is a design choice, not circularity: the paper's empirical claims are that SDM outperforms PGD, C&W, APGD, and AutoAttack on CIFAR-10/100 under multiple defenses (Tables 3-5, Fig. 2), which are external comparisons not derived from the construction. No parameter is fitted to the reported outcomes; hyperparameters such as N=5, T, and the cycle-stage-step schedule are hand-selected but not tuned to the evaluation results. The paper contains no self-citations and does not invoke any uniqueness theorem or prior work by the same authors. The motivation in Sec. 2 uses an empirical observation ('non-adversarial examples with high loss values') rather than a definitional equivalence. Concerns about baseline strength or the underspecified l2 update are correctness risks, not instances of circular reasoning. Therefore, no circular step meets the evidentiary bar required by the analysis.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard softmax math, white-box gradient access, and a heuristic staged optimization. The free parameters are structural hyperparameters (N, C, T) and loss-construction constants, none fitted to the reported outcomes. No new physical or conceptual entities are introduced.

free parameters (4)
  • Number of optimization stages N = 5
    Across all total-step budgets the paper fixes N=5 (Table 2) without reporting sensitivity or ablation, so the choice is hand-selected.
  • Cycle/step schedule (C, T) = e.g., C=5, T=40 at 1000 steps
    The decomposition of total steps into cycles and iteration steps (Table 2) is chosen by the authors; no ablation justifies this allocation.
  • Bias term delta = 0.5 * max(P_tau - P_n)
    Eq. 6 defines delta as half the current max difference; it is a construction choice rather than a fitted constant.
  • Numerical stability term zeta = 1e-10
    Default value used to stabilize the denominator in Eq. 6; not fitted to data.
assumptions (4)
  • standard math The classifier outputs logits that can be normalized with softmax to well-defined probabilities (Eq. 3).
    Used throughout, e.g., Eq. 3 and the probability notation in Sec. 3.2.
  • domain assumption White-box access to model gradients enables gradient-based optimization.
    Algorithm 1 line 10 requires gradient of loss with respect to input; this is the standard white-box threat model.
  • ad hoc to paper The staged optimization heuristic (first minimize true-label probability, then compress non-true probabilities) improves convergence to the margin objective.
    Sec. 3.2 claims compressing the solution space 'can reduce the solution space and facilitate subsequent optimization' without formal proof or ablation.
  • ad hoc to paper The single motivating example in Table 1 is representative of a general failure mode of CE loss.
    Sec. 2 generalizes from one example to the claim that existing objectives are 'unreasonable'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization." pith.science (2026). https://pith.science/paper/POGTQMPQ

@misc{pith2026250900826,
  author       = {Pith},
  title        = {Pith review of: Sequential Difference Maximization: Generating Adversarial Examples via Multi-Stage Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/POGTQMPQ}},
  note         = {Machine review of arXiv:2509.00826}
}
read the original abstract

Efficient adversarial attack methods are critical for assessing the robustness of computer vision models. In this paper, we reconstruct the optimization objective for generating adversarial examples as "maximizing the difference between the non-true labels' probability upper bound and the true label's probability," and propose a gradient-based attack method termed Sequential Difference Maximization (SDM). SDM establishes a three-layer optimization framework of "cycle-stage-step." The processes between cycles and between iterative steps are respectively identical, while optimization stages differ in terms of loss functions: in the initial stage, the negative probability of the true label is used as the loss function to compress the solution space; in subsequent stages, we introduce the Directional Probability Difference Ratio (DPDR) loss function to gradually increase the non-true labels' probability upper bound by compressing the irrelevant labels' probabilities. Experiments demonstrate that compared with previous SOTA methods, SDM not only exhibits stronger attack performance but also achieves higher attack cost-effectiveness. Additionally, SDM can be combined with adversarial training methods to enhance their defensive effects. The code is available at https://github.com/X-L-Liu/SDM.

Figures

Figures reproduced from arXiv: 2509.00826 by the authors.

Figure 1
Figure 1. Overall structure of SDM. Algorithm 1 Detailed algorithm for SDM. Input: A multi-class classifier 𝑓 with the parameter 𝜃, a clean example 𝒙 with the true label 𝑦, a adversarial perturbation budget 𝜖 and step size 𝛼 under the ℓ𝑝 -norm constraint, the number of optimization cycles 𝐶, the number of optimization stages 𝑁, and the number of iteration steps 𝑇 . Output: The adversarial example 𝒙 ′ . 1: Set the initial valu… view at source ↗
Figure 2
Figure 2. Cost-effectiveness comparison of different attack [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 12 canonical work pages

  1. [1]

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (May 2015), 436–444. doi:10.1038/nature14539

  2. [2]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . 770–778. doi:10.1109/CVPR.2016.90

  3. [3]

    Qian Chen, Ze Liu, Yi Zhang, Keren Fu, Qijun Zhao, and Hongwei Du. 2021. RGB- D Salient Object Detection via 3D Convolutional Neural Networks. InThirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, Virtual Event, February 2-9, 2021 . AAAI Press, 1063–1071. doi:10.1609/AAAI.V35I2.16191

  4. [4]

    Ge Jin, Chuancai Liu, and Xu Chen. 2021. Adversarial network integrating dual attention and sparse representation for semi-supervised semantic segmentation. Inf. Process. Manag. 58, 5 (2021), 102680. doi:10.1016/J.IPM.2021.102680

  5. [5]

    Adversarial Examples

    Costas Neocleous and Christos N. Schizas. 2014. On the Claim for the Existence of "Adversarial Examples" in Deep Learning Neural Networks. In NCTA 2014 - Proceedings of the International Conference on Neural Computation Theory and Applications, part of IJCCI 2014, Rome, Italy, 22 - 24 October, 2014 , Kurosh Madani and Joaquim Filipe (Eds.). SciTePress, 30...

  6. [6]

    Goodfellow, Jonathon Shlens, and Christian Szegedy

    Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In 2015 International Conference on Learning Representations (ICLR), Yoshua Bengio and Yann LeCun (Eds.)

  7. [7]

    Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial ro- bustness with an ensemble of diverse parameter-free attacks. In2020 International Conference on Machine Learning (ICML) , Vol. 119. PMLR, 2206–2216

  8. [8]

    Xinyi Wang, Zhibo Jin, Zhiyu Zhu, Jiayu Zhang, and Huaming Chen. 2024. Improving Adversarial Transferability via Frequency-Guided Sample Relevance Attack. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, ID, USA, October 21-25, 2024. ACM, 2410–2419. doi:10.1145/3627673.3679858

Show all 20 references
  1. [9]

    Naveed Akhtar, Ajmal Mian, Navid Kardan, and Mubarak Shah. 2021. Threat of Adversarial Attacks on Deep Learning in Computer Vision: Survey II. CoRR abs/2108.00401 (2021). arXiv:2108.00401

  2. [10]

    Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In 2018 International Conference on Learning Representations (ICLR) . OpenReview.net

  3. [11]

    Goodfellow, and Rob Fergus

    Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conf...

  4. [12]

    Goodfellow, and Samy Bengio

    Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In2017 International Conference on Learning Representations (ICLR). OpenReview.net

  5. [13]

    Nicholas Carlini and David A. Wagner. 2017. Towards Evaluating the Robustness of Neural Networks. In 2017 IEEE Symposium on Security and Privacy, SP 2017, San Jose, CA, USA, May 22-26, 2017 . IEEE Computer Society, 39–57. doi:10.1109/ SP.2017.49

  6. [14]

    J.C. Spall. 1992. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automat. Control 37, 3 (1992), 332–341. doi:10.1109/9.119632

  7. [15]

    Sergey Zagoruyko and Nikos Komodakis. 2016. Wide Residual Networks. In BMVC, York, UK, September 19-22 . https://bmva-archive.org.uk/bmvc/2016/ papers/paper087/index.html

  8. [16]

    Krizhevsky and G

    A. Krizhevsky and G. Hinton. 2009. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto (2009). https://www.cs.utoronto.ca/~kriz/cifar.html

  9. [17]

    Xing, Laurent El Ghaoui, and Michael I

    Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. 2019. Theoretically Principled Trade-off between Robustness and Accuracy. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long ...

  10. [18]

    Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. 2020. Improving Adversarial Robustness Requires Revisiting Misclassified Examples. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, April 26-30, 2020 . OpenReview....

  11. [19]

    Rahul Rade and Seyed-Mohsen Moosavi-Dezfooli. 2022. Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off. In The Tenth In- ternational Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https://openre...

  12. [20]

    Xiangyu Yin and Wenjie Ruan. 2024. Boosting Adversarial Training via Fisher- Rao Norm-Based Regularization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, W A, USA, June 16-22, 2024 . IEEE, 24544–24553. doi:10.1109/CVPR52733.2024.02317

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.