REVIEW 4 major objections 5 minor 35 references
Explore the vulnerability of black-box models via diffusion models
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A black-box image classifier can be approximated and attacked using synthetic images generated from benign text prompts, with only a few thousand queries to the target.
desk verdict A useful empirical finding about diffusion-synthetic pretraining for substitute models, but the general data-free claim is unsupported and the baseline comparison is stacked. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the synthetic-image generation step: the attacker feeds class-level prompts $p_i$ of the form 'a photo of a $c_i$' to a text-to-image diffusion API, which produces $s_i = G(z_i, e_i)$, an image obtained by denoising a noise vector $z_i$ under the prompt embedding $e_i$. This yields a 200,000-image pre-training corpus per dataset at zero query cost. The second mechanism is a pretrain-then-distill schedule: the substitute model $S$ is pre-trained on synthetic data, then fine-tuned by cross-entropy distillation $L_{dis} = \mathrm{CE}(S(\hat{x}, \theta_S), y_T)$ on hard labels $y_T = T(\hat{x})$ obtained from one query per image to the target $T$. The authors' ablation attributes most of the query savings to the pre-training stage: removing it drops CIFAR-10 accuracy from 81.8% to 47.9%.
What would settle it
Train the pipeline on a benchmark of fine-grained or rare classes where the chosen diffusion model visibly fails to render the class, then check whether substitute accuracy and adversarial transfer success drop toward the no-pretraining baseline; a large drop would falsify the claim that benign class-name prompts provide adequate pre-training coverage.
Extended reading notes
Core claim
The paper's central claim is that an attacker can obtain a high-performing substitute for a black-box classifier using only synthetic images generated from class names by diffusion model APIs, plus a very small number of hard-label queries to the target. The authors report that this pipeline outperforms prior data-free model-extraction baselines by an average of 27.37% while using one hundredth of the query budget, reaching top-1 accuracy of 81.8% on CIFAR-10 with 5,000 queries and a 98.68% adversarial attack success rate in the soft-label setting. The mechanism is a two-stage training scheme: pre-training on 200,000 synthetic images establishes a knowledge base, and knowledge distillation with a single query per sampled image aligns the substitute with the target. The paper argues this works because the synthetic images are high-resolution and diverse enough that the substitute's decision boundary lands close to the target's, making adversarial examples transfer.
Load-bearing premise
The method assumes that a diffusion model's synthetic images for a given class name are close enough to the target model's private training images that pre-training on them transfers; if a class is niche, visually ambiguous, or outside the diffusion model's training distribution, the substitute will not approximate the target and the query savings disappear.
Editorial extensions
If this is right
- A classifier that returns only top-1 labels can be approximated to roughly 80% accuracy on CIFAR-10 with 5,000 queries and no access to its training data.
- Adversarial examples crafted against the substitute transfer to the target at high rates, up to 98.68% success in the soft-label scenario, so the substitute is not just a copy but an attack surface.
- The query budget advantage is large enough that even allowing prior methods 100 times more queries, the paper reports a 21.4% accuracy gap in its favor on ImageNette.
- The attack requires no real data and no gradients, and it uses only top-1 hard labels from the target, so the attack surface is wider than methods that need logits or training sets.
Reading between the lines
- This suggests a defense should focus on detecting or watermarking synthetic images before they enter fine-tuning pipelines, since query throttling alone cannot stop the zero-query generation stage.
- The attack's ceiling is set by how well the generative model's distribution covers the target classes; for rare or proprietary classes, the pre-training advantage should shrink, and the paper's results may overstate the threat for such domains.
- The same pattern should extend to other modalities, such as text or audio classifiers, whenever a public generative model can produce class-conditional samples from class names.
- A direct test of the paper's core assumption is to vary class rarity or domain shift and measure both substitute accuracy and adversarial transfer success, expecting a decline as coverage worsens.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method for model extraction and transfer-based adversarial attacks on black-box classifiers. It uses Stable Diffusion APIs with class-name prompts to generate 200k synthetic images per dataset, pre-trains a substitute model on them, then fine-tunes with a small number of hard-label queries from the target. The authors report large improvements over existing data-free extraction baselines on CIFAR-10, CIFAR-100, and several ImageNet subsets, with attack success rates up to 98.68% and claims of a 0.01x query budget relative to baselines.
Significance. If the premise of distribution match holds, the query-efficiency gains are substantial, and the paper ships reproducible ablations across architectures and seeds, which is a definite strength. The main experiments are internally consistent, and the observation that removing pre-training degrades CIFAR-10 accuracy by 33.9 points is informative. However, the generality of the central claim is not established: all evaluated target domains are within Stable Diffusion's training distribution, and the target model contributes only about 1 point of accuracy over zero-query pre-training on CIFAR-10. The paper would be considerably stronger with out-of-distribution target experiments, a fair calibration of baselines at their design budgets, and a corrected theoretical discussion.
major comments (4)
- [§3.2, Appendix A.0.1, and all experiments] The central claim of a data-free attack is only evaluated on target distributions that are already well represented in Stable Diffusion's LAION training data: CIFAR-10/100 classes are ImageNet classes, ImageNette/ImageSquawk/ImageFruit/ImageYellow are ImageNet subsets, and Tiny-ImageNet is ImageNet-derived. Since the substitute is pre-trained on 200k synthetic samples from class-name prompts, and since Fig. 3 shows CIFAR-10 accuracy of 80.6% with zero target queries versus 81.51% after 5k queries, the target model contributes almost nothing beyond the public distribution the diffusion model has memorized. The paper therefore does not demonstrate extraction of a private decision rule; it demonstrates that a model trained on SD data transfers to ImageNet-family targets. To support the stated generalization, the authors should test on targets whose training distribution is not covered by LAION (e.g., medical, satellite, or any domain constructed by the authors) and should report zero-query accuracy on those targets.
- [Appendix A.0.6] The derivation supporting query efficiency is circular and contains an algebraic sign error. The inequality T(f_real_sub -> f_target) <= T(f_syn_sub -> f_target) is asserted without proof and is exactly the conclusion that synthetic pretraining improves transferability. Moreover, the text says 'due to reduced transferability T(f_syn_sub -> f_target), the expected number of queries decreased,' which contradicts the subsequent equation E[Q] = 1/T: if T is higher, the expected query count is lower, not higher. This section should be removed or rewritten as an explicit hypothesis rather than an analytical proof.
- [Tables 1 and 2; Fig. 3] The headline comparison against 'state-of-the-art methods' uses query budgets of 5k or less for methods such as MAZE, DFME, ZSDB3, DFMS-HL, and DisGuide that are designed and previously validated with millions of queries. At these budgets the baselines all yield near-random accuracy (roughly 10-13%), so the comparison establishes only that the baselines are not adapted to the few-query regime. The paper's scaled-up experiments (Fig. 1 and Fig. 3) are the right direction, but the text does not report the baseline asymptotic accuracies at their design budgets for all datasets, making it impossible to judge whether the proposed method remains superior when the baselines are fairly configured. Please report both the per-method query budget needed to reach the accuracy reported in the original papers and the accuracy gap at that budget.
- [§3.2, Appendix A.0.1, and cost claims] The 200,000 synthetic-image generations per dataset are not counted in the query budget. Since the abstract and introduction claim a 0.01x query budget and 'lower attack deployment costs,' the total API cost should be reported: for CIFAR-10 the method makes 200k diffusion calls plus 5k target calls, which is 41 times the baseline's 5k target calls. Even if diffusion calls are cheaper, the claims should be restated as low target-model query count, not low total API usage, and the monetary/time cost of synthetic generation should be quantified.
minor comments (5)
- [Section 4.1.1] The phrase 'and more training detial' contains a typo and should read 'and more training details.'
- [Fig. 4 caption] The caption contains the string '双击此处编辑', which appears to be leftover template text from a document editor.
- [References] References [11] and [13] are missing full author lists and appear in an inconsistent format ('Kariyappa' and 'Sanyal' are used as if they were the paper titles); these should be corrected to standard citation form.
- [Appendix A.0.12] The statement that D_syn is 'designed to approximate' D_real is an assumption that underlies the whole method, not a consequence of the training objective; it should be stated explicitly as an assumption to be verified empirically.
- [Tables 6 and 9] Tables 6 and 9 are identical and one of them should be removed to avoid duplication.
Circularity Check
The query-efficiency claim is partially circular: Appendix A.0.6 asserts the transferability inequality that the conclusion restates, and Appendix A.0.12 assumes the distribution match it claims to prove.
-
other
[Appendix A.0.6 (Impact Analysis of Query Efficiency in Adversarial Transfer Attack)]
"SinceT(f real sub →f target)≤T(f syn sub →f target)implies: E[Q(f syn sub →f target)]≤E[Q(f real sub →f target)]. Conclusion Pretraining the substitute model requires fewer queries to achieve an adversarial transfer attack."
The inequality T(f_real)≤T(f_syn) is exactly the claim that synthetic pretraining improves transferability and hence reduces queries; it is asserted without measurement or derivation. The preceding equations merely define T and invert it to E[Q]=1/T, so substituting the assumed inequality yields the conclusion mechanically. The text even says 'due to reduced transferability T(f_syn_sub→f_target)', contradicting the inequality direction, which confirms the step is an unproven assumption. The query-efficiency prediction therefore reduces to its own premise.
-
other
[Appendix A.0.12 (Effect Investigation of Pre-training model in Model Extraction)]
"Given that the synthetic data distributionD syn is designed to approximate the real data distributionD real, the pretrained parametersθ pretrain sub are expected to be closely align to the target model's parametersθ target."
The conclusion that synthetic pretraining makes fine-tuning need fewer queries is derived from the premise that D_syn approximates D_real, which is precisely the assumption that pretraining on synthetic data transfers to the target. The subsequent small-gradient argument merely restates this premise as 'small gradient'; no independent evidence connects Stable Diffusion outputs to the target's private distribution. Thus the claimed query-efficiency advantage is the assumed distribution match, not a demonstrated consequence, making this part of the derivation circular.
full rationale
The paper's central experimental claims—substitute model accuracy and high transfer attack success rates—are empirically grounded and are not circular: Table 1, Table 3, and the ablations provide concrete measurements. However, the formal justification for the headline query-efficiency result is circular in two appendix derivations. In A.0.6, the paper defines transferability T and expected query count E[Q]=1/T, then simply asserts T(f_real)≤T(f_syn); the conclusion E[Q(f_syn)]≤E[Q(f_real)] is algebraically identical to that assertion, so the 'proof' is the premise restated. In A.0.12, the paper assumes D_syn approximates D_real and θ_pretrain is close to θ_target to conclude that fewer queries are needed; this is the same transfer assumption the method must establish. These steps are load-bearing because the abstract and introduction advertise 'minimal queries' and '0.01× of the query budget' as the main advance. The experimental numbers remain meaningful evidence, but the claimed query reduction is partly by construction. Separately, the 'data-free' premise is satisfied by construction in all seven benchmarks: CIFAR and all ImageNet subsets are drawn from ImageNet/LAION classes that Stable Diffusion was trained on, so the synthetic distribution is not independent of the target's public training distribution; this is a validity-scope concern rather than a circular derivation. No load-bearing self-citation chain was found.
Assumptions & free parameters
free parameters (5)
- Synthetic sample count per dataset =
200,000
- Target query budget Q per dataset =
5k (CIFAR-10), 1k (ImageNette), 30 (ImageSquawk), 130 (ImageFruit), 50 (ImageYellow), 150k (CIFAR-100), 200k…
- Stable Diffusion inference steps =
50
- Adversarial perturbation budget epsilon and step size alpha =
epsilon = 8/255, alpha = 2/255
- Distillation training hyperparameters =
SGD momentum 0.9, weight decay 5e-4, initial LR 0.1, cosine schedule
assumptions (4)
- domain assumption Synthetic data distribution D_syn approximates the real data distribution D_real for the target classes.
- domain assumption Hard labels from the target model on a small queried subset are sufficient to align the substitute with the target.
- ad hoc to paper T(f_real) <= T(f_syn), i.e., a synthetic-pretrained substitute transfers at least as well as a real-data-trained substitute.
- ad hoc to paper The cost of generating 200,000 images via the diffusion API can be excluded from the query budget.
Cite this review
Pith. "Pith review of Explore the vulnerability of black-box models via diffusion models." pith.science (2026). https://pith.science/paper/B74QDN6P
@misc{pith2026250607590,
author = {Pith},
title = {Pith review of: Explore the vulnerability of black-box models via diffusion models},
year = {2026},
howpublished = {\url{https://pith.science/paper/B74QDN6P}},
note = {Machine review of arXiv:2506.07590}
}
read the original abstract
Recent advancements in diffusion models have enabled high-fidelity and photorealistic image generation across diverse applications. However, these models also present security and privacy risks, including copyright violations, sensitive information leakage, and the creation of harmful or offensive content that could be exploited maliciously. In this study, we uncover a novel security threat where an attacker leverages diffusion model APIs to generate synthetic images, which are then used to train a high-performing substitute model. This enables the attacker to execute model extraction and transfer-based adversarial attacks on black-box classification models with minimal queries, without needing access to the original training data. The generated images are sufficiently high-resolution and diverse to train a substitute model whose outputs closely match those of the target model. Across the seven benchmarks, including CIFAR and ImageNet subsets, our method shows an average improvement of 27.37% over state-of-the-art methods while using just 0.01 times of the query budget, achieving a 98.68% success rate in adversarial attacks on the target model.
Reference graph
Works this paper leans on
-
[8]
Dast: Data-free substitute training for adversarial attacks,
Mingyi Zhou, Jing Wu, Yipeng Liu, Shuaicheng Liu, and Ce Zhu, “Dast: Data-free substitute training for adversarial attacks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2020, pp. 234–243
work page 2020
-
[1]
Explore the vulnerability of black-box models via diffusion models
INTRODUCTION Black box machine learning models are susceptible to vari- ous attacks, including model extraction [1, 2] and adversarial transfer attacks [3, 4]. In a model extraction attack, an at- tacker attempts to replicate a target model’s functionality by querying it and using the responses to reconstruct a substitute model. On the other hand, an adve...
work page Pith review arXiv 2025
-
[2]
RELATED WORK Diffusion Model. Diffusion-based generative models, known for generating high-fidelity and diverse synthetic images, are central to our research, particularly stable and latent diffu- sion models. While these models are powerful, their security and privacy implications warrant critical examination. Previ- ous studies [9] address security conc...
-
[3]
PROPOSED METHOD 3.1. Overview The objective of our approach is to utilize off-the-shelf gener- ative models to synthesize high-quality images and efficiently train robust substitute models for deploying two types of at- tacks. The entire framework is divided into three parts: (i) In the data generation phase as shown in Fig. 2, we transi- tion from tradit...
-
[4]
Experiments on Model Extraction 4.1.1
EXPERIMENTS 4.1. Experiments on Model Extraction 4.1.1. Performance Comparison on Small Datasets Query budget is a critical factor in evaluating the efficiency of adversarial methods. To ensure a fair comparison, we initially fixed the query budget to a small value across all methods. As shown in Tab. 1, our method consistently outperforms baselines, achi...
-
[5]
CONCLUSION In this work, we identified that attackers could exploit dif- fusion model APIs to generate high-resolution, diverse syn- thetic images for training substitute models to facilitate model extraction and adversarial transfer attacks. We developed a method for training robust substitute models in a data-free, hard-label, and query-limited setting....
-
[6]
Jean-Baptiste Truong, Pratyush Maini, Robert J Walls, and Nicolas Papernot, “Data-free model extraction,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4771–4780
work page 2021
-
[7]
Disguide: Disagreement-guided data-free model extraction,
Jonathan Rosenthal, Eric Enouen, Hung Viet Pham, and Lin Tan, “Disguide: Disagreement-guided data-free model extraction,” 2023
work page 2023
Show all 35 references
-
[9]
Towards efficient data free black-box adversarial attack,
Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, and Chao Wu, “Towards efficient data free black-box adversarial attack,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15115–15125
2022
-
[10]
Why do adversarial attacks transfer? explaining transferability of evasion and poi- soning attacks,
Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita- Rotaru, and Fabio Roli, “Why do adversarial attacks transfer? explaining transferability of evasion and poi- soning attacks,” in28th USENIX security symposium (USENIX sec...
2019
-
[11]
Catastrophic forgetting and mode collapse in gans,
Hoang Thanh-Tung and Truyen Tran, “Catastrophic forgetting and mode collapse in gans,” in2020 inter- national joint conference on neural networks (ijcnn). IEEE, 2020, pp. 1–10
2020
-
[12]
Adversarial attacks against deep generative models on data: a survey,
Hui Sun, Tianqing Zhu, Zhiqiu Zhang, Dawei Jin, Ping Xiong, and Wanlei Zhou, “Adversarial attacks against deep generative models on data: a survey,”IEEE Trans- actions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3367–3388, 2021
2021
-
[13]
Data-free black- box attack based on diffusion model,
Mingwen Shao, Lingzhuang Meng, Yuanjian Qiao, Lixu Zhang, and Wangmeng Zuo, “Data-free black- box attack based on diffusion model,”arXiv preprint arXiv:2307.12872, 2023
2023 arXiv
-
[14]
Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models,
Patrick Schramowski, Manuel Brack, Bj ¨orn Deiseroth, and Kristian Kersting, “Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models,” inPro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2023, pp. 22522–22531
2023
-
[15]
Fitnets: Hints for thin deep nets,
Adriana Romero, Nicolas Ballas, Samira Ebrahimi Ka- hou, Antoine Chassang, Carlo Gatta, and Yoshua Ben- gio, “Fitnets: Hints for thin deep nets,”arXiv preprint arXiv:1412.6550, 2014
2014 arXiv
-
[16]
Maze: Data-free model stealing attack us- ing zeroth-order gradient estimation,
Kariyappa, “Maze: Data-free model stealing attack us- ing zeroth-order gradient estimation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 13814–13823
2021
-
[17]
Zero-shot knowledge distillation from a decision-based black-box model,
Zi Wang, “Zero-shot knowledge distillation from a decision-based black-box model,” inInternational Con- ference on Machine Learning. PMLR, 2021, pp. 10675– 10685
2021
-
[18]
Towards data-free model stealing in a hard label setting,
Sanyal, “Towards data-free model stealing in a hard label setting,” inProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2022, pp. 15284–15293
2022
-
[19]
Vidmodex: Interpretable and efficient black box model extrac- tion for high-dimensional spaces,
Somnath Sendhil Kumar, Yuvaraj Govindarajulu, Pa- van Kulkarni, and Manojkumar Parmar, “Vidmodex: Interpretable and efficient black box model extrac- tion for high-dimensional spaces,”arXiv preprint arXiv:2408.02140, 2024
2024 arXiv
-
[20]
Dualcos: Query-efficient data-free model stealing with dual clone networks and optimal samples,
Yunfei Yang, Xiaojun Chen, Yuexin Xuan, and Zhen- dong Zhao, “Dualcos: Query-efficient data-free model stealing with dual clone networks and optimal samples,” in2024 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 2024, pp. 1–6
2024
-
[21]
Practical black-box attacks against machine learning,
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami, “Practical black-box attacks against machine learning,” inProceedings of the 2017 ACM on Asia conference on computer and communications security, 2017, pp. 506– 519
2017
-
[22]
Knockoff nets: Stealing functionality of black-box models,
Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz, “Knockoff nets: Stealing functionality of black-box models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4954–4963
2019
-
[23]
Difattack: Query-efficient black-box adversarial at- tack via disentangled feature space,
Jun Liu, Jiantao Zhou, Jiandian Zeng, and Jinyu Tian, “Difattack: Query-efficient black-box adversarial at- tack via disentangled feature space,” inProceedings of the AAAI Conference on Artificial Intelligence, 2024, vol. 38, pp. 3666–3674
2024
-
[24]
Derd: data-free adversarial robustness distillation through self-adversarial teacher group,
Yuhang Zhou, Yushu Zhang, Leo Yu Zhang, and Zhongyun Hua, “Derd: data-free adversarial robustness distillation through self-adversarial teacher group,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 10055–10064
2024
-
[25]
High-resolution im- age synthesis with latent diffusion models,
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer, “High-resolution im- age synthesis with latent diffusion models,” inProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684–10695
2022
-
[26]
Learning multiple layers of features from tiny images,
Alex Krizhevsky, Geoffrey Hinton, et al., “Learning multiple layers of features from tiny images,” 2009
2009
-
[27]
Imagenet: A large-scale hierarchical image database,
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255
2009
-
[28]
Tiny imagenet visual recogni- tion challenge,
Ya Le and Xuan Yang, “Tiny imagenet visual recogni- tion challenge,”CS 231N, vol. 7, no. 7, pp. 3, 2015
2015
-
[29]
Imagenet classification with deep convolutional neural networks,
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hin- ton, “Imagenet classification with deep convolutional neural networks,”Advances in neural information pro- cessing systems, vol. 25, 2012
2012
-
[30]
Very deep convolutional networks for large-scale image recogni- tion,
Karen Simonyan and Andrew Zisserman, “Very deep convolutional networks for large-scale image recogni- tion,”arXiv preprint arXiv:1409.1556, 2014
2014 arXiv
-
[31]
Wide resid- ual networks,
Sergey Zagoruyko and Nikos Komodakis, “Wide resid- ual networks,”arXiv preprint arXiv:1605.07146, 2016
2016 arXiv
-
[32]
Deep residual learning for image recognition,
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778
2016
-
[33]
Adversarial machine learning at scale,
Alexey Kurakin, Ian Goodfellow, and Samy Bengio, “Adversarial machine learning at scale,”arXiv preprint arXiv:1611.01236, 2016
2016 arXiv
-
[34]
Adversarial examples in the physical world,
Kurakin Alexey, “Adversarial examples in the physical world,”arXiv preprint arXiv: 1607.02533, 2016
2016 arXiv
-
[35]
Towards deep learning models resistant to adversarial attacks,
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2017. A. APPENDIX A.0.1. A. Query Budget and Training Settings for Data-Free Model Extrac...
2017 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.