Pith. sign in

REVIEW 4 major objections 4 minor 10 references

Binary Iterative Method finds stronger non-targeted adversarial examples than FGSM, BIM and VAM by repeatedly halving the step size to approach local loss minima.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:21 UTC pith:IZDPISOL

load-bearing objection BIM with geometric step decay and restarts, sold with a false binary-search optimality claim; thin empirical edge over weak baselines only. the 4 major comments →

arxiv 2607.04145 v1 pith:IZDPISOL submitted 2026-07-05 cs.LG cs.CV

Binary Iterative Method for Non-targeted Adversarial Attack

Naman Goyal , Milan Chaudhari This is my paper
classification cs.LG cs.CV
keywords adversarial attacksnon-targeted attackBinary Iterative Methodgradient-based attackImageNetadversarial robustnessstep-size search
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that fixed-epsilon ball searches used by FGSM, BIM and VAM cannot reliably reach local minima of the non-targeted attack objective, even after many iterations. Binary Iterative Method (BinIM) instead treats the step size as a binary-search parameter: it starts large, updates the image by the signed gradient, then halves the step each iteration, and restarts over multiple rounds to pick the strongest local minimum. On 1000 randomly sampled ImageNet images the resulting attacks drive Inception-v3 adversarial accuracy down to 0.009 and true-class probabilities to roughly 2e-9 while wrong-class confidences reach 0.995–1.0. Stronger, systematically generated attacks of this kind matter because they supply better data for adversarial training and give a stricter test of model robustness.

Core claim

Standard epsilon-ball gradient attacks cannot guarantee proximity to a local minimum of the non-targeted attack objective. Binary Iterative Method replaces the constant step with a divide-and-conquer schedule that initialises a large eps_iter, repeatedly halves it after each signed-gradient update, and selects the best result across restarts. Evaluated on 1000 ImageNet images, BinIM lowers Inception-v3 accuracy to 0.009 (versus 0.025 for BIM) and can push true-label probability to 2.21e-9 while raising wrong-class confidence above 0.995.

What carries the argument

Binary Iterative Method (BinIM): an iterative signed-gradient update whose step size is initialised large and halved every iteration (eps_iter ← eps_iter/2), with optional multi-round restarts; the schedule is presented as binary search over an assumed sorted one-dimensional gradient landscape, thereby approaching a local minimum of the attack loss more closely than constant-step epsilon-ball search.

Load-bearing premise

The method assumes that the input-loss gradient behaves like a sorted one-dimensional array so that repeatedly halving the step size is genuine binary search guaranteed to approach a local minimum.

What would settle it

If, on the same 1000 ImageNet images and identical iteration budget, a constant-step BIM or a simple linearly decaying step schedule matches or beats BinIM’s 0.009 adversarial accuracy and the extreme probability collapses shown in Table 4, the claimed advantage of the binary-search schedule is refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • BinIM can generate stronger adversarial training sets that improve model robustness.
  • Robustness scores measured with BinIM will be lower (stricter) than those measured with FGSM, BIM or VAM on the same models.
  • The binary-search step schedule can be extrapolated to targeted attacks.
  • Once a gradient is available the method applies to any differentiable classifier.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The binary-search justification requires the directional loss along the signed-gradient ray to be unimodal or sorted; typical high-dimensional neural losses violate this, so BinIM may simply be an effective decreasing-step heuristic rather than true binary search.
  • If the observed gains are real, the same half-step schedule could be grafted onto momentum or projected-gradient attacks to tighten their local minima.
  • Transfer results (attacks crafted on Inception-v3 still degrade Inception-v2 and ResNet) suggest the schedule may strengthen black-box attacks without white-box access to the victim.
  • A natural next measurement is whether BinIM achieves the same success rate at smaller L∞ or L2 perturbation norms; the paper does not report norms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Binary Iterative Method (BinIM), a non-targeted gradient attack that starts from BIM and replaces a fixed step size with a geometric schedule eps_iter ← eps_iter/2, framed as binary search over a divide-and-conquer paradigm. The authors argue that ∇_x Loss behaves like a sorted 1-D array (positive on one side, negative on the other), so halving the step is guaranteed to approach a local (or, in concave regions, global) minimum of the attack objective, and that this is superior to ε-ball search used by FGSM, BIM, and VAM. On 1000 randomly sampled ImageNet images, attacks generated from Inception-v3 gradients are evaluated on Inception-v3, Inception-v2, and ResNet-v2-152; BinIM reports the lowest adversarial accuracy (0.009 on Inception-v3 vs BIM 0.0252, FGSM 0.356, VAM 0.628) and sample true-label probabilities driven near zero with wrong-class confidences up to ~1.0 (Tables 2 and 4).

Significance. A correctly justified, stronger non-targeted iterative attack would be useful for adversarial training and robustness evaluation. The reported numbers on Inception-v3 are competitive with the weak baselines shown. However, the claimed optimality rests on a false premise about the geometry of neural losses, so the method reduces to BIM with restarts and a fixed geometric step schedule—an already-known heuristic. Without a sound principle or comparisons to modern iterative attacks (PGD, MI-FGSM, etc.), the contribution does not advance the technical state of the art even if the empirical edge over FGSM/BIM/VAM holds on this subsample.

major comments (4)
  1. [Methodology; Appendix] Methodology (paragraphs on binary search; Figure 3; Appendix Figures 4–5 and the O(log n) claim): The load-bearing premise is that ∇_x Loss “behaves like a sorted 1-D array” so that repeatedly setting eps_iter ← eps_iter/2 is binary search and is therefore guaranteed to approach a local (or, in concave regions, global) minimum. Neural-network losses are high-dimensional and non-convex; the sign of the gradient along the signed-gradient direction does not form a sorted 1-D sequence. The geometric schedule therefore has no optimality property beyond ordinary step-size decay. This premise is required for the claim that BinIM is principled rather than an ad-hoc decreasing-step BIM variant, and it is false.
  2. [Algorithm 1] Algorithm 1: The algorithm states “Update the eps_iter_r until binary search objective is maximised” but never defines that objective, the acceptance criterion, or how the best local minimum is selected across rounds. The only concrete update is eps_iter ← eps_iter/2 after a signed-gradient step. Without a defined search objective, the procedure is not binary search; it is BIM with a geometric schedule and optional restarts. The algorithm as written is incomplete and does not implement the claimed method.
  3. [Experiment; Table 2] Experiment / Table 2: Fair comparison requires matched budgets (same total iterations, same max L_∞ radius, same number of restarts). The paper does not report the values of eps, E, R, or the iteration counts used for FGSM/BIM/VAM. VAM results are missing for Inception-v2 and ResNet-v2-152. Related Work cites PGD and momentum iterative attack, yet neither is included as a baseline. The reported superiority is therefore only relative to weak or incompletely specified baselines and does not support the abstract claim of outperforming “all other gradient-based methods.”
  4. [Appendix] Appendix (ε-ball vs binary search; Figures 4–5): The complexity comparison O(n) for ε-ball vs O(log n) for binary search is misapplied. Continuous adversarial optimization is not a discrete sorted-array search; the radius of an ε-ball is a constraint, not an unbounded sequential scan. The appendix therefore does not establish that BinIM is closer to a local minimum in the same number of epochs.
minor comments (4)
  1. [Throughout] Numerous typos and grammar issues (e.g., “the useful of new”, “Please not”, “main contributions if are our paper”, “maxmima”, “misclassifciations”, “state-of-the neural networks”).
  2. [Table 3] Table 3 is referenced as sample outputs but is only a caption pair; no quantitative distortion metrics (L_∞, L_2, or perceptual) are reported for the modified images.
  3. [Experiment; Table 1] Clean accuracies in Table 1 and the 1000-image subsample protocol are underspecified (which ImageNet split, preprocessing, whether labels match the pretrained models’ label space).
  4. [Related Work] Related Work lists several attacks (JSMA, DeepFool, EAD, distributional attacks) that are never used in the experiments or discussed relative to BinIM.

Circularity Check

0 steps flagged

No circularity: empirical non-targeted attack method whose success metrics are measured independently on held-out ImageNet images against fixed pretrained classifiers.

full rationale

The paper proposes BinIM (BIM with a geometric eps_iter schedule plus restarts) and evaluates it by generating attacks from Inception-v3 gradients then measuring classification accuracy and label probabilities on 1000 ImageNet images for three fixed pretrained networks (Tables 1-4). Attack success is an external empirical quantity, not forced by any fitted constant or definitional identity. The binary-search optimality argument (Methodology; Appendix Figs. 4-5) is a (flawed) geometric claim about the sign pattern of abla x Loss, but it does not equate the reported accuracies or confidences to any quantity defined as the method’s own input; the derivation chain therefore remains self-contained against external benchmarks. No self-definitional equations, no fitted-then-predicted quantities, no load-bearing self-citations, and no uniqueness theorems imported from the authors appear.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 1 invented entities

The work sits entirely inside the standard white-box gradient-attack framework of Goodfellow et al. and iterative FGSM/BIM. Load-bearing free choices are the initial eps, the halving schedule, restart count, epoch budget, and the 1000-image sample. The only non-standard modeling claim is that input gradients are ordered so binary search applies—an ad hoc assumption, not a theorem. No new physical entities are introduced.

free parameters (4)
  • initial eps / eps_iter_1
    Starting step size for the attack; chosen by the authors and halved each iteration; not derived from data or theory in the paper.
  • R (number of restart rounds)
    Multi-start count in Algorithm 1; free experimental choice that affects which local attack is returned.
  • E (epochs per round)
    Iteration budget per restart; free and not justified by a stopping criterion beyond the binary-search narrative.
  • 1000-image ImageNet subsample
    Evaluation set size and sampling procedure are author-chosen; no full validation set or confidence intervals.
axioms (4)
  • domain assumption Deep networks are vulnerable to small input perturbations found via ∇_x Loss (piecewise linearity / linear explanation of adversarial examples).
    Taken from Goodfellow, Shlens & Szegedy 2014 and used throughout Preliminaries and Methodology as the reason signed gradients work.
  • domain assumption Non-targeted attack success can be measured by drop in top-1 accuracy and true-class probability on fixed pretrained ImageNet classifiers.
    Standard evaluation protocol assumed in Experiment and Tables 1–4.
  • ad hoc to paper ∇_x Loss along the attack path is ordered like a sorted array so that halving the step size is binary search toward a zero-gradient local minimum.
    Stated in Methodology and Appendix with a 1-D diagram; required for the claim that BinIM is guaranteed closer to a local minimum than ε-ball methods.
  • standard math Standard calculus and iterative signed-gradient updates (FGSM/BIM form).
    Equations (1)–(7) and (9) use ordinary gradient ascent on the input.
invented entities (1)
  • Binary Iterative Method (BinIM) no independent evidence
    purpose: Named attack procedure: multi-round BIM with eps_iter halved each step, claimed to approximate multiple local minima of the non-targeted objective.
    The only new construct; it is a schedule on top of BIM rather than a new mathematical object with independent evidence outside this paper.

pith-pipeline@v1.1.0-grok45 · 11989 in / 3331 out tokens · 33820 ms · 2026-07-11T21:21:56.394667+00:00 · methodology

0 comments
read the original abstract

Adversarial attacks guide and provide additional training and test data for both adversarial training and adversarial robustness validation, and expose the 'piecewise linearity' of deep learning based models. Since adversarial attacks and adversarial robustness are mathematically defined problems that can be optimised directly with end-to-end differentiable search, adversarial robustness is more widely applicable than other robustness metrics such as corruption and perturbation robustness, and new kinds of adversarial attacks are beneficial for robustness testing. Attacks are targeted or non-targeted depending on whether the image is modified to misclassify to a particular class or to any incorrect class; we focus on the non-targeted setting. Finding the optimal input data points and hyper-parameters for generating non-targeted adversarial attacks remains a challenge for current methods like the Fast Gradient Method, Basic Iterative Method and Virtual Adversarial Method. We propose a new method, the "Binary Iterative Method" (BinIM), which uses a divide-and-conquer paradigm to optimise parameters and hyper-parameters for the generation of non-targeted attacks. We compare our method to other gradient-based adversarial attacks evaluated over pre-trained networks (InceptionV3, InceptionV2, ResNet V2 152) on classification tasks. On 1000 randomly-sampled images from the standard ImageNet dataset, the Binary Iterative Method outperforms all other gradient-based methods, qualitatively making the classifier misclassify with confidence up to 0.995 while reducing the probability of the true label to 2.21e-09 (approximately 0).

Figures

Figures reproduced from arXiv: 2607.04145 by Milan Chaudhari, Naman Goyal.

Figure 1
Figure 1. Figure 1: Explaining and Harnessing Adversarial Examples [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Method of transforming the image Algorithm 1 BinIM algorithm (Non-targeted adversarial at￾tack) repeated over multiple rounds Require: A known classifier K to generate non-targeted adversarial attack for unknown classifier, R number of rounds to re-initialize; permissible epochs per round; E maximum permissible epochs per round; Optimisation Objective Find an optimal value of eps starting with eps iter1 us… view at source ↗
Figure 3
Figure 3. Figure 3: Maximising the error using binary search to ap [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Searching using Binary Search. As one moves [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

10 extracted references · 7 linked inside Pith

  1. [1]

    Chen, P.-Y.; Sharma, Y.; Zhang, H.; Yi, J.; and Hsieh, C.-J. 2017. Ead: elastic-net attacks to deep neural networks via adversarial examples. arXiv preprint arXiv:1709.04114

    Pith/arXiv arXiv 2017

  2. [2]

    Dong, Y.; Liao, F.; Pang, T.; Su, H.; Zhu, J.; Hu, X.; and Li, J. 2018. Boosting adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, 9185--9193

    2018

  3. [3]

    J.; Shlens, J.; and Szegedy, C

    Goodfellow, I. J.; Shlens, J.; and Szegedy, C. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572

    Pith/arXiv arXiv 2014

  4. [4]

    Gu, S.; and Rigazio, L. 2014. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068

    Pith/arXiv arXiv 2014

  5. [5]

    Hu, W.; and Tan, Y. 2017. Generating adversarial malware examples for black-box attacks based on gan. arXiv preprint arXiv:1702.05983

    Pith/arXiv arXiv 2017

  6. [6]

    Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; and Vladu, A. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083

    Pith/arXiv arXiv 2017

  7. [7]

    Miyato, T.; Maeda, S.-i.; Koyama, M.; Nakae, K.; and Ishii, S. 2015. Distributional smoothing with virtual adversarial training. arXiv preprint arXiv:1507.00677

    Pith/arXiv arXiv 2015

  8. [8]

    Moosavi-Dezfooli, S.-M.; Fawzi, A.; and Frossard, P. 2016. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2574--2582

    2016

  9. [9]

    Papernot, N.; Goodfellow, I.; Sheatsley, R.; Feinman, R.; and McDaniel, P. 2016. cleverhans v1.0.0: an adversarial machine learning library. arXiv preprint arXiv:1610.00768

    Pith/arXiv arXiv 2016

  10. [10]

    Zheng, T.; Chen, C.; and Ren, K. 2019. Distributionally adversarial attack. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, 2253--2260

    2019