Pith. sign in

REVIEW 3 major objections 5 minor 15 cited by

Torchattacks: A PyTorch Repository for Adversarial Attacks

T0 review · 3 major / 5 minor · reviewed 2026-08-27 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Torchattacks packages ten published adversarial attacks behind a common PyTorch interface.

desk verdict A useful software manual with correct equations and zero verification: the library is real, but the paper needs a reproducibility appendix before it can be trusted as a citable artifact. read the letter →

arxiv 2010.01950 v3 pith:BWTCW26M submitted 2020-09-24 cs.LG cs.AIcs.CR

Hoki Kim This is my paper · ORCID
classification cs.LGcs.AIcs.CR
keywords adversarialattacksPyTorchexamplesrobustnessevaluationFGSMPGDdeeplearningsecuritysoftwarelibrary
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Torchattacks is a PyTorch library that implements ten published adversarial-attack methods and presents them through a single attack interface so a user can generate adversarial examples with a few lines of code. The paper's claim is that the repository at the stated URL contains working implementations of FGSM, BIM, CW, R+FGSM, PGD, EOTPGD, TPGD, FFGSM, MIFGSM, and MultiAttack, plus utilities for targeted modes and saving results. A sympathetic reading treats the report as a usage manual: it states the optimization problem each attack solves and gives code snippets for each method. If the code works as described, the collection would let model developers check robustness without reimplementing each attack.

What carries the argument

The load-bearing object is the Attack base class. It owns the model, the perturbation bound, and the preconditions that inputs be scaled to $[0,1]$ and that models return a single $(N,C)$ logit tensor; every listed attack is a subclass implementing a specific update rule, typically $x'_{t+1} = \operatorname{clip/proj}_{B(x,\epsilon)}(x'_t + \alpha \operatorname{sgn}(\nabla \ell))$, with CW replacing the sign rule by Adam on a tanh reparameterization. This shared wrapper is what makes the uniform call `atk(examples, labels)` work across all ten attacks.

What would settle it

Install the package from the linked repository, load a standard pretrained classifier on a fixed test set, and run the PGD example with eps=8/255, checking that the call runs without error, each output lies in the $\ell_\infty$ ball of radius 8/255, and classification accuracy on the adversarial examples drops below the original accuracy. Failure on any of these three checks would settle that the claim is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that Torchattacks is a usable library that generates adversarial examples and verifies the robustness of deep learning models. All attacks share the same wrapper: instantiate with a model and a perturbation budget, then call the object on data to get adversarial examples. The report frames each method as a solver for maximizing $\ell(f(x+\delta), y)$ over $\delta \in B(x, \epsilon)$: FGSM takes one gradient-sign step, BIM and PGD iterate with clipping or projection, CW optimizes in tanh-space with Adam, EOTPGD averages gradients over randomized forward passes, and MIFGSM accumulates momentum. MultiAttack combines attacks so an example succeeds if any component attack succeeds.

Load-bearing premise

The central claim stands or falls on whether the public code in the linked repository actually matches the documented formulas and examples and runs as advertised, since the report itself gives no commit hash, test outputs, or benchmark results to verify this.

Editorial extensions

If this is right

  • A model developer can produce adversarial examples for all ten attacks with the same calling convention and feed them directly into robustness-evaluation scripts.
  • The targeted and least-likely modes let users probe specific failure classes rather than only untargeted misclassification.
  • MultiAttack combines attacks into a stronger test, since an adversarial example succeeds if any of its constituent attacks defeats the model.
  • Because reverse-normalization is intentionally excluded from the attacks, users with normalized models must add a normalization layer inside the model, keeping input ranges explicit.
  • The same API can be reused for attack generation, saving outputs from a DataLoader, and later reloading them for analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the code is maintained and matches the documented API, this kind of single-interface collection could standardize adversarial-robustness comparisons, but the report itself provides no benchmark tables to demonstrate that role.
  • Beyond the paper: a minimal validation, such as attack success rates for each method on a standard dataset with the stated epsilon values, would turn the library claim into a measured one; no such measurements appear in the report.
  • Beyond the paper: the deliberate choice to leave input normalization out of the attack process moves a common source of error into user code, and a future version could detect or wrap normalization layers automatically.
  • Beyond the paper: the MultiAttack combinator could be extended into a test-time ensemble of defenses, though the paper only presents it as a way to strengthen attacks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes Torchattacks, a PyTorch library implementing a collection of adversarial attack algorithms: FGSM, BIM, CW, R+FGSM, PGD, EOTPGD, TPGD, FFGSM, MIFGSM, and a MultiAttack composition wrapper. It states the standard adversarial-example optimization problem, then for each attack gives the algorithm, the parameters of its implementation, and a short usage example. The central claim is that the code at https://github.com/Harry24k/adversarial-attacks-pytorch provides these attacks and that the documented usage works. The paper also documents inherited utility methods for changing attack mode, return type, and saving outputs.

Significance. If the code artifact is as described, this is a useful software contribution: it bundles well-known attacks behind a uniform, simple API, includes EOT and MultiAttack composition, and is concise enough to serve as user documentation. The algorithm descriptions in Sections 2.1 to 2.9 are faithful to the standard forms in the cited literature, and the usage examples are easy to follow. The main weakness is that the manuscript provides no version pin, no test suite, no sample outputs, and no benchmark results, so the paper's central claim currently rests on an unverified software artifact. The contribution is therefore conditional on the actual state of the repository.

major comments (3)
  1. [Abstract and Section 2 (all subsections)] The central claim is that the repository at https://github.com/Harry24k/adversarial-attacks-pytorch implements the listed attacks as described, but the manuscript pins no commit hash, release version, or snapshot date and reports no test suite, sample output, or minimal verification script. Without a fixed artifact, the equations alone do not establish that the code works, and the paper cannot be independently checked. Please add version/commit information and a minimal reproducible verification, such as a short script that runs each attack on a small model and prints one adversarial example or its success status.
  2. [Section 2.4 (R+FGSM)] Equation (5) describes a non-iterative two-step procedure, yet the Implementation list and usage example expose a 'steps' parameter and set steps=2. This is a load-bearing API inconsistency: either the implementation iterates Equation (5), which would differ from the standard R+FGSM and is not described, or the 'steps' parameter is spurious. Please align the equation, the parameter list, and the example.
  3. [Section 2.9 (MIFGSM)] The Implementation list specifies 'alpha' as a parameter, but the usage example is 'atk = torchattacks.MIFGSM(model, eps=8/255, steps=5, decay=1.0)', which omits 'alpha' and instead uses 'steps', a parameter not listed. As written, the effective step size in the documented API is unspecified. Please correct either the parameter list or the usage example.
minor comments (5)
  1. [Title] The title contains an extraneous space in 'Attack s'; the running head has the same issue.
  2. [Section 2.5 (PGD)] The text says that both L∞ and L2 are supported, but Equation (6) uses the sign of the gradient, which is the L∞ update; the L2 variant would require a normalized gradient and projection onto the L2 ball. Please either provide the L2 update or state explicitly that PGDL2 uses a separate update rule not shown in Equation (6).
  3. [Section 2.3 (CW)] The sentence 'The larger c, the stronger adversarial example will produced' is ungrammatical; also, the role of κ as a confidence margin could be stated more precisely.
  4. [Section 3.2 (MultiAttack)] The phrase 'more powerful adversarial adversarial example' repeats 'adversarial'.
  5. [Section 1 (Precautions)] The note that inputs should be scaled to [0, 1] and that reverse-normalization is not included is helpful; it would be even clearer to give a one-line example of adding a normalization layer to a model before wrapping it with an attack.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the report restates externally referenced attack algorithms and documents a software artifact; no derivation or prediction reduces to its own inputs.

full rationale

This paper is a software/technical report, not a derivation-driven paper. It lists standard adversarial attacks (FGSM, BIM, CW, R+FGSM, PGD, EOTPGD, TPGD, FFGSM, MIFGSM, MultiAttack), restates their published update equations with citations to the original external sources, and documents the Torchattacks library API. There is no self-citation chain, no fitted parameter that is later renamed as a prediction, and no uniqueness theorem imported from the authors' prior work. The attack formulas in Sections 2.1-2.9 are presented as summaries of prior publications rather than as new results derived in this paper. The central claim that the public repository implements these attacks is not circular; it is an empirical claim about a software artifact, whose verification would be a reproducibility check (e.g., running the code with a specific commit and test data), not a logical reduction. The internal inconsistencies noted in the manuscript (e.g., R+FGSM listing a 'steps' argument despite Eq. 5 being a two-step procedure, and MIFGSM usage omitting 'alpha' while the implementation list includes it) are documentation-quality concerns that do not make the argument circular. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. It relies on standard domain assumptions about input scaling, model output shape, and loss function, stated in Section 1 and Section 2.

assumptions (3)
  • domain assumption Inputs are scaled to [0, 1] before attacks are applied.
    Stated in Section 1 (Precautions); the attacks assume this normalization and reverse-normalization is not included.
  • domain assumption Models return a single vector of shape (·, C), where C is the number of classes.
    Stated in Section 1 (Precautions); the library only supports this model output format.
  • standard math The loss function ℓ is cross-entropy unless specified otherwise.
    Stated in Section 2 before Eq. (1); this is a standard choice in the cited attacks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Torchattacks: A PyTorch Repository for Adversarial Attacks." pith.science (2026). https://pith.science/paper/BWTCW26M

@misc{pith2026201001950,
  author       = {Pith},
  title        = {Pith review of: Torchattacks: A PyTorch Repository for Adversarial Attacks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWTCW26M}},
  note         = {Machine review of arXiv:2010.01950}
}
read the original abstract

Torchattacks is a PyTorch library that contains adversarial attacks to generate adversarial examples and to verify the robustness of deep learning models. The code can be found at https://github.com/Harry24k/adversarial-attacks-pytorch.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Two Sides of the Same Coin: Learning the Backdoor to Remove the Backdoor

    cs.LG 2026-07 conditional novelty 6.5 of 10

    Learning a backdoored reference model as a poisonous-sample oracle enables near-perfect training-time backdoor removal with negligible natural-accuracy loss.

  2. Test Case Prioritization for DNNs via Neural Collapse Instability

    cs.LG 2026-07 conditional novelty 6.0 of 10

    DNN test inputs ranked by prediction instability across late-training checkpoints find faults earlier than confidence-based ranking in most benchmark settings.

  3. Canonical Latent Representations in Conditional Diffusion Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Projecting out the top Jacobian singular directions in a conditional diffusion model's latent space yields class prototypes that, when used for distillation, improve classifier robustness and generalization.

  4. Towards Robust Stability Prediction in Smart Grids: GAN-based Approach under Data Constraints and Adversarial Challenges

    cs.CR 2025-01 conditional novelty 6.0 of 10

    A GAN trained only on stable grid data can flag unstable states and adversarial attacks, reaching 98.1% stability accuracy on the augmented UCI grid dataset.

  5. HEM: a margin-based loss for visual categorisation tasks

    cs.LG 2025-01 conditional novelty 6.0 of 10

    A new margin-based loss, HEM, trains image classifiers that are more robust to unknown and adversarial inputs and better at continual learning and segmentation than cross-entropy-trained models.

  6. Foveation-Guided Dynamic Token Selection for Robust and Efficient Vision Transformers

    cs.CV 2026-07 conditional novelty 5.0 of 10

    FDT adds foveation and binary fixation modules to DeiT so multi-scale tokens are selected dynamically in one pass, improving ImageNet100 accuracy, MACs, and robustness without adversarial training.

  7. QShield: Securing Neural Networks Against Adversarial Attacks using Quantum Circuits

    cs.CR 2026-04 unverdicted novelty 5.0 of 10

    Hybrid quantum-classical models using structured entanglement keep high accuracy on MNIST, OrganAMNIST and CIFAR-10 while lowering adversarial attack success rates and raising the computational cost of generating attacks.

  8. FedAPT: Federated Adversarial Prompt Tuning for Vision-Language Models

    cs.CV 2025-09 conditional novelty 5.0 of 10

    FedAPT improves adversarial robustness of federated prompt tuning for CLIP by generating visual prompts from text prompts under a global-label beacon, with reported gains of up to 11.49% under PGD-100.

  9. Prediction Inconsistency Helps Achieve Generalizable Detection of Adversarial Examples

    cs.CR 2025-06 conditional novelty 5.0 of 10

    A new black-box adversarial example detector, PID, uses the confidence of an auxiliary model on the primary model's predicted label (1 - g_y(x)) to separate adversarial from normal inputs.

  10. SemSegBench & DetecBench: Benchmarking Reliability and Generalization Beyond Classification

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A large-scale benchmark of 76 segmentation and 61 detection models shows that robustness to attacks and corruptions does not reliably track clean accuracy, and that transformer backbones generalize better under shift.

  11. Human Aligned Compression for Robust Models

    cs.CV 2025-04 conditional novelty 5.0 of 10

    On Imagenette and a 1000-image ImageNet sample, learned compression (HiFiC, ELIC) defends image classifiers against adversarial examples better than JPEG, especially for Vision Transformers, and sequential compression...

  12. Topological Signatures of Adversaries in Multimodal Alignments

    cs.LG 2025-01 conditional novelty 5.0 of 10

    Adversarial images induce monotonic changes in persistent-homology-based losses on CLIP/BLIP image-text alignments, and gradient features from these losses modestly improve MMD-based adversarial detection.

  13. Adversarial Robustness of Bottleneck Injected Deep Neural Networks for Task-Oriented Communication

    cs.LG 2024-12 conditional novelty 5.0 of 10

    Shallow variational bottleneck compression is more vulnerable to adversarial attacks than deep variational bottleneck compression, and generative decoders widen the attack surface in task-oriented communication.

  14. TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models

    cs.CV 2024-11 reject novelty 5.0 of 10

    TAPT adapts CLIP's prompts per test image using entropy minimization and ImageNet-based embedding alignment, but it is evaluated only against non-adaptive attacks.

  15. BEARD: Benchmarking the Adversarial Robustness for Dataset Distillation

    cs.CV 2024-11 reject novelty 5.0 of 10

    BEARD is a benchmark and leaderboard evaluating adversarial robustness of dataset distillation methods with new normalized metrics across multiple datasets and images-per-class settings.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 15 Pith papers

  1. [3]

    Adam: A Method for Stochastic Optimization

    Adam: A method for stochastic o ptimization. arXiv preprint arXiv:1412.6980 . Kurakin, A.; Goodfellow, I.; and Bengio, S

    work page Pith review arXiv

  2. [6]

    Towards Deep Learning Models Resistant to Adversarial Attacks

    Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 . Papernot, N.; McDaniel, P .; Sinha, A.; and Wellman, M

    work page Pith review arXiv

  3. [7]

    Towards the Science of Security and Privacy in Machine Learning

    Towards the science of security and privacy in machine learning. arXiv preprint arXiv:1611.03814 . Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Ch anan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al

    work page Pith review arXiv

  4. [9]

    Ensemble Adversarial Training: Attacks and Defenses

    Ensemble adversarial training: Attacks and defenses. arXiv preprint arXiv:1705.07204 . Wong, E.; Rice, L.; and Kolter, J. Z

    work page Pith review arXiv

  5. [12]

    Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"

    Comment on" Adv-BNN: Improved Adver sarial Defense through Robust Bayesian Neural Network". arXiv preprint arXiv:1907.00895 . 6

    work page Pith review arXiv 1907

  6. [2013]

    Intriguing properties of neural networks

    Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 . Tramèr, F.; Kurakin, A.; Papernot, N.; Goodfellow, I.; Bone h, D.; and McDaniel, P

    work page Pith review arXiv

  7. [2014]

    Explaining and Harnessing Adversarial Examples

    Explain ing and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 . 5 T orchattacks : A PyT orch Repository for Adversarial Attacks Kingma, D. P .; and Ba, J

    work page Pith review arXiv

  8. [2016]

    Adversarial examples in the physical world

    Adversari al examples in the physical world. arXiv preprint arXiv:1607.02533 . Liu, X.; Li, Y .; Wu, C.; and Hsieh, C.-J

    work page Pith review arXiv

Show all 12 references
  1. [2017]

    In 2017 ieee symposium on security and privacy (sp) , 39–57

    Towards evaluating the rob ustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , 39–57. IEEE. Dong, Y .; Liao, F.; Pang, T.; Su, H.; Zhu, J.; Hu, X.; and Li, J

    2017

  2. [2018]

    arXiv preprint arXiv:1810.01279

    Adv-bnn: Impro ved adversarial defense through robust bayesian neural network. arXiv preprint arXiv:1810.01279 . Madry, A.; Makelov, A.; Schmidt, L.; Tsipras, D.; and Vladu, A

    arXiv

  3. [2019]

    arXiv preprint arXiv:1901.08573

    Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv:1901.08573 . Zimmermann, R. S

    1901 arXiv

  4. [2020]

    arXiv preprint arXiv:2001.03994

    Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03994 . Zhang, H.; Y u, Y .; Jiao, J.; Xing, E. P .; Ghaoui, L. E.; and Jor dan, M. I

    2001 arXiv

Pith tools

Reviewed August 27, 2026 · model on record in the stance chip above.