REVIEW 2 major objections 4 minor 37 references
Once a MAN: Towards Multi-Target Attack via Learning Multi-Target Adversarial Network Once
T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read One conditional network can generate adversarial examples for any target class of a classifier, the paper claims, replacing thousands of per-class generators with a single training run.
desk verdict A useful multi-target attack extension whose headline numbers are inflated by counting the ground-truth label as a valid target; the central claim survives but needs a cleaner evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the conditional adversarial generator: an encoder-decoder whose image-feature stream is fused with a label-embedding stream. In MANc the one-hot target label is expanded spatially and concatenated with the image features before a convolution; in MANr the label goes through a two-layer MLP with sigmoid activation and multiplies every channel of the image feature map, a squeeze-and-excitation-style recalibration. The target label is sampled at random per training image, so one forward pass of the same model can target any of the $K$ classes, and the frozen attacked classifier acts as the loss oracle that injects target information into the perturbation.
What would settle it
For a fixed trained MAN, evaluate attack success separately for each of the $K$ target classes on a held-out set: if any class's success rate drops far below the reported average, or if the label embedding $t'$ is nearly constant across different targets, then the single-model multi-target claim fails.
Extended reading notes
Core claim
On the paper's own terms, the central claim is that target-conditional adversarial generation can be made multi-target: a single network $F_\theta(x,t)$ trained once with random target labels learns to map any input image $x$ and any target class $t$ to an adversarial image $x^*$ such that $\arg\max_i H_i(F_\theta(x,t)) = t$ for a fixed attacked classifier $H$. The paper demonstrates this with two fusion designs, one that concatenates the one-hot label map to the image features (MANc) and one that recalibrates image feature channels by multiplying them with a sigmoid-activated MLP label embedding (MANr), optimized by cross-entropy classification loss through the frozen attacked model plus an $L_2$ reconstruction loss with a distance cap. It reports attack success above 98% in white-box settings on ImageNet and CIFAR-10, stronger black-box transfer than the single-target baselines, and improved robustness when the generated samples are used to fine-tune the target classifier. The paper also reports that the multi-target model shows better transferability than a single-target model trained for the same number of iterations, attributing this to competition between target labels promoting more generalizable features.
Load-bearing premise
The load-bearing premise is that a single shared encoder-decoder, guided only by a random-target training objective, can learn to produce distinct, reliable adversarial perturbations for every one of the $K$ target classes at once without forgetting, mode collapse, or interference among classes; the paper offers empirical support but no capacity or interference analysis.
Editorial extensions
If this is right
- Targeted attacks against classifiers with hundreds or thousands of classes no longer need one generator per class; a single MAN model covers all classes after one training run.
- The reported black-box transfer gains imply that MAN samples generalize across architectures better than single-target generator baselines, which matters when the attacker does not know the victim model.
- Because MAN produces diverse adversarial samples with random targets at feed-forward speed, adversarial training can be scaled to many target classes at a fraction of the training and storage cost of per-class generators.
- The same conditional-generation idea extends beyond classification to any differentiable victim model that can provide a loss signal, though the paper only demonstrates image classifiers.
Reading between the lines
- The paper leaves implicit that the conditional design also benefits the defense side: one MAN can generate a far larger and more diverse set of adversarial examples for data augmentation than a handful of per-class models, and the reported comparison of training with one, five, and ten per-class generators suggests diversity itself, not just target count, drives robustness gains.
- A testable extension is to report per-target attack success separately: the paper's averages over random targets could hide tail classes where the shared model underperforms, and a per-class breakdown would show whether the label branch truly conditions the output or merely biases toward easy classes.
- If the capacity-interference premise holds, the approach should scale to much larger label spaces, such as fine-grained datasets with thousands of classes, and to conditional attacks on other tasks such as detection or retrieval; if it fails there, a hybrid of shared features with class-specific heads may be needed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Multi-target Adversarial Network (MAN), a conditional adversarial-sample generator that takes an image x and a target label t as input and outputs a perturbed image x* via an encoder-decoder architecture. Two integration variants are presented: MANc, which concatenates image features with a one-hot label map, and MANr, which re-calibrates image feature channels through an MLP-based label encoder. The network is trained with a cross-entropy classification loss on the attacked model plus an L2 reconstruction loss, and can be used with random target labels to produce attacks against any class with a single model. Experiments cover white-box and black-box single-target attack on ImageNet and CIFAR-10, multi-target attack, use of MAN-generated samples for adversarial training, and ablations on loss weight, perturbation budget, transferability, and model ensembling. The paper claims that MAN is the first multi-target adversarial network and that it achieves competitive or better attack success and transferability than ATN and GAP, while requiring far fewer trained models to cover all classes.
Significance. If the central claim is supported, MAN would be a useful efficiency advance over training one generator per target class, and the transferability and adversarial-training results would be of interest to the adversarial ML community. The single-target comparisons in Tables 1 and 2 are internally consistent, the parameter-efficiency analysis in Table 5 is a clear strength, and the two conditional-integration variants are simple and clearly described. However, the multi-target evaluation on CIFAR-10 assigns all ten labels to every sample, including the ground-truth label, and reports only aggregate success rates; this directly affects the paper's headline claim that a single model can attack any category at the reported success rate. The manuscript is well organized and the method is conceptually reproducible, though no code is released.
major comments (2)
- [Section 4.1, Table 4] The CIFAR-10 multi-target evaluation assigns all ten labels to each sample, which includes the ground-truth label y. The problem definition in Section 3 explicitly requires t != y, and for t = y the classification loss in Eq. (2) is already satisfied by the original image while the reconstruction loss encourages x* approximately equal to x, so such successes are trivial. Because Table 4 reports only aggregate success over the ten targets, the reported white-box rate of 99.50% and black-box rates of 89.74-95.47% can be inflated by the trivially successful t = y cases and can hide substantially lower per-target rates for t != y. The central claim that MAN can attack any category at the reported rate is therefore not yet established. Please rerun the multi-target evaluation excluding t = y, and likewise ensure that the ImageNet random labels exclude the ground-truth label, and report per-target success rates or a source-by-target confusion matrix.
- [Section 4.1, training protocol, and Eq. (2)] The multi-target training phase assigns a random target label to each training image. If targets are drawn uniformly without excluding y, then on CIFAR-10 roughly 10% of training pairs have t = y, and for those pairs the classification term in Eq. (2) is near-zero while the reconstruction term dominates, producing reconstruction-only gradients with no attack signal. This is exactly the label-dilution mechanism that the shared-encoder design must overcome, and the current aggregate results do not show whether the model has truly learned all K distinct target maps. Please state whether t = y was excluded during training; if it was not, retrain or re-evaluate with y excluded and report how the aggregate multi-target numbers change. The same caveat also applies to the adversarial-training experiments in Section 4.2, where MAN-generated samples with random target labels are used to fine-tune the attacked model.
minor comments (4)
- [Section 3.1, Eq. (1)] The symbol t is used both for the target label index and for the one-hot vector in R^K, which makes the dimensions of W1 in Eq. (1) ambiguous; please use separate notation, e.g., e_t, for the one-hot vector.
- [Section 4.1, Table 3] The text says that for ImageNet the authors 'randomly assign ten labels for each sample,' but it does not state whether the ten labels are distinct or whether the ground-truth label is excluded; please specify.
- [Section 4.3, Figure 4(a)] The x-axis labels in Figure 4(a) are missing the symbol alpha (e.g., they should read alpha=600, alpha=800, and so on), and the caption should state that the results are on CIFAR-10 against the pretrained VGG16 model.
- [References] Reference [13] is cited as 'Jie et al.' in the text, but the correct author list is Hu et al.; please correct this citation.
Circularity Check
No circularity found: the multi-target attack network is an empirical system validated by held-out attack rates and external black-box models, with no prediction that reduces to a fitted input or self-citation.
full rationale
MAN is an empirical construction, not a derivation. The objective in Eq. (2) combines a cross-entropy classification loss on the attacked model's output with an L2 reconstruction loss; success is measured by feeding generated samples through pretrained VGG/ResNet models and by transfer to untouched architectures. No parameter is fitted to the reported success rates, no uniqueness theorem is imported from the authors' prior work, and the cited baselines (ATN, GAP, SENet) are external works. The only in-scope concern is evaluation protocol: the CIFAR-10 multi-target test assigns all ten labels including the ground-truth class, so the aggregate rate includes a trivial target; however, this is an experimental validity issue, not circularity, because the central claim is not derived from this metric by construction and the white-box and black-box numbers remain externally falsifiable. The paper therefore scores 0.
Assumptions & free parameters
free parameters (2)
- alpha (loss weight) =
100 on ImageNet, 800 on CIFAR10
- epsilon perturbation budget =
10*sqrt(N), where N=C*H*W, in main tables
assumptions (4)
- domain assumption The attacked model H is differentiable with respect to its input, so classification loss gradients can train the generator.
- ad hoc to paper A one-hot label vector combined with image features provides sufficient conditioning for target-specific perturbations.
- domain assumption Pixel-space L2 distance and test-time rescaling to a fixed L2 budget preserve attack validity and visual similarity.
- domain assumption Standard cross-entropy loss on the target classifier is a good proxy for attack success.
Cite this review
Pith. "Pith review of Once a MAN: Towards Multi-Target Attack via Learning Multi-Target Adversarial Network Once." pith.science (2026). https://pith.science/paper/VEO62IFJ
@misc{pith2026190805185,
author = {Pith},
title = {Pith review of: Once a MAN: Towards Multi-Target Attack via Learning Multi-Target Adversarial Network Once},
year = {2026},
howpublished = {\url{https://pith.science/paper/VEO62IFJ}},
note = {Machine review of arXiv:1908.05185}
}
read the original abstract
Modern deep neural networks are often vulnerable to adversarial samples. Based on the first optimization-based attacking method, many following methods are proposed to improve the attacking performance and speed. Recently, generation-based methods have received much attention since they directly use feed-forward networks to generate the adversarial samples, which avoid the time-consuming iterative attacking procedure in optimization-based and gradient-based methods. However, current generation-based methods are only able to attack one specific target (category) within one model, thus making them not applicable to real classification systems that often have hundreds/thousands of categories. In this paper, we propose the first Multi-target Adversarial Network (MAN), which can generate multi-target adversarial samples with a single model. By incorporating the specified category information into the intermediate features, it can attack any category of the target classification model during runtime. Experiments show that the proposed MAN can produce stronger attack results and also have better transferability than previous state-of-the-art methods in both multi-target attack task and single-target attack task. We further use the adversarial samples generated by our MAN to improve the robustness of the classification model. It can also achieve better classification accuracy than other methods when attacked by various methods.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Adversarial transformation networks: Learning to generate adversarial examples
Shumeet Baluja and Ian Fischer. Adversarial transformation networks: Learning to generate adversarial examples. arXiv preprint arXiv:1703.09387, 2017
arXiv 2017
-
[2]
Towards evaluating the robustness of neural networks
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39–57. IEEE, 2017
2017
-
[3]
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E. Kounavis, and Duen Horng Chau. Keeping the Bad Guys Out: Protecting and Vaccinat- ing Deep Learning with JPEG Compression. arXiv e-prints, page arXiv:1705.02900, May 2017
arXiv 2017
-
[4]
Imagenet: A large-scale hierarchical im- age database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. In Computer Vision and Pattern Recognition,
-
[5]
Boosting adversarial attacks with momentum
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Xi- aolin Hu, Jianguo Li, and Jun Zhu. Boosting adversarial attacks with momentum. arXiv preprint arXiv:1710.06081, 2017
arXiv 2017
-
[6]
Practical methods of optimization
Roger Fletcher. Practical methods of optimization . John Wiley & Sons, 2013
work page 2013
-
[7]
Convolutional sequence to sequence learning
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03122, 2017
arXiv 2017
-
[8]
Fast r-cnn
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448, 2015
2015
Show all 37 references
-
[9]
Explaining and harnessing adversarial examples
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Inter- national Conference on Learning Representations, 2015
2015
-
[10]
Goodfellow, Jonathon Shlens, and Christian Szegedy
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. arXiv e- prints, page arXiv:1412.6572, Dec 2014
2014 arXiv
-
[11]
Towards Deep Neural Net- work Architectures Robust to Adversarial Examples
Shixiang Gu and Luca Rigazio. Towards Deep Neural Net- work Architectures Robust to Adversarial Examples. arXiv e-prints, page arXiv:1412.5068, Dec 2014
2014 arXiv
-
[12]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016
2016
-
[13]
Squeeze-and-excitation net- works
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. arXiv preprint arXiv:1709.01507, 7, 2017
2017 arXiv
-
[14]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014
2014 arXiv
-
[15]
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, Cite- seer, 2009
2009
-
[16]
Imagenet classification with deep convolutional neural net- works
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural net- works. In Advances in neural information processing sys- tems, pages 1097–1105, 2012
2012
-
[17]
Ad- versarial examples in the physical world
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Ad- versarial examples in the physical world. arXiv preprint arXiv:1607.02533, 2016
2016 arXiv
-
[18]
Ad- versarial Machine Learning at Scale
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Ad- versarial Machine Learning at Scale. arXiv e-prints, page arXiv:1611.01236, Nov 2016
2016 arXiv
-
[19]
Defense against Adversarial At- tacks Using High-Level Representation Guided Denoiser
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu. Defense against Adversarial At- tacks Using High-Level Representation Guided Denoiser. arXiv e-prints, page arXiv:1712.02976, Dec 2017
2017 arXiv
-
[20]
Feature pyramid networks for object detection
Tsung-Yi Lin, Piotr Doll ´ar, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. Feature pyramid networks for object detection. In CVPR, volume 1, page 4, 2017
2017
-
[21]
Deepfool: a simple and accurate method to fool deep neural networks
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 2574–2582, 2016
2016
-
[22]
Rectified linear units im- prove restricted boltzmann machines
Vinod Nair and Geoffrey E Hinton. Rectified linear units im- prove restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML- 10), pages 807–814, 2010
2010
-
[23]
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Anh Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 427–436, 2015
2015
-
[24]
Osadchy, J
M. Osadchy, J. Hernandez-Castro, S. Gibson, O. Dunkel- man, and D. Prez-Cabo. No bot expects the deepcaptcha! introducing immutable adversarial examples, with applica- tions to captcha generation. IEEE Transactions on Informa- tion Forensics and Security, 12(11):2640–2653, Nov 2017
2017
-
[25]
Berkay Celik, and Ananthram Swami
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Prac- tical Black-Box Attacks against Machine Learning. arXiv e-prints, page arXiv:1602.02697, Feb 2016
2016 arXiv
-
[26]
Practi- cal black-box attacks against machine learning
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practi- cal black-box attacks against machine learning. In Proceed- ings of the 2017 ACM on Asia Conference on Computer and Communications Security, pages 506–519. ACM, 2017
2017
-
[27]
Towards the Science of Security and Privacy in Machine Learning
Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. Towards the Science of Security and Privacy in Machine Learning. arXiv e-prints , page arXiv:1611.03814, Nov 2016
2016 arXiv
-
[28]
Generative adversarial perturbations
Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Be- longie. Generative adversarial perturbations. arXiv preprint arXiv:1712.02328, 2017
2017 arXiv
-
[29]
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information pro- cessing systems, pages 91–99, 2015
2015
-
[30]
Very deep convo- lutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[31]
One pixel attack for fooling deep neural networks
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 2019
2019
-
[32]
Intriguing properties of neural networks
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013
2013 arXiv
-
[33]
Deepface: Closing the gap to human-level perfor- mance in face verification
Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level perfor- mance in face verification. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 1701–1708, 2014
2014
-
[34]
Ensemble Adversarial Training: Attacks and Defenses
Florian Tram `er, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. arXiv e-prints, page arXiv:1705.07204, May 2017
2017 arXiv
-
[35]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008, 2017
2017
-
[36]
Generating adversarial examples with adversarial networks
Chaowei Xiao, Bo Li, Jun-Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610, 2018
2018 arXiv
-
[2009]
IEEE Conference on , pages 248–255
CVPR 2009. IEEE Conference on , pages 248–255. Ieee, 2009
2009
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.