REVIEW 3 major objections 5 minor 40 references
Stealix: Model Stealing via Prompt Evolution
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Stealix steals hard-label image classifiers with no class names and no human-written prompts, beating prompt-based baselines by up to 22.2 points.
desk verdict Solid empirical attack paper with a genuinely new combination, but the headline gains shrink exactly where the threat model matters most, and the PC proxy is validated only against the same victim model it is meant to approximate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is prompt consistency (PC), the fraction of M = 10 synthesized images from a given prompt that the victim classifies as the target class; it serves as the fitness function in the genetic algorithm and as a proxy for how close the synthesized images are to the victim data. Around it, prompt refinement minimizes a triplet contrastive loss between the embedding of a prompt and the embeddings of the seed image, a positively classified image, and a negatively classified image: positive images are pulled toward the prompt and negative images are pushed away. The reproduction step uses tournament selection, one elite retained per generation, crossover on image triplets, and mutation by resampling from the accumulated seed, positive, and negative sets. These three mechanisms convert a single seed image per class into a growing, labelled pool of query images without ever asking the attacker to describe the class.
What would settle it
Train a victim classifier that is overconfident on out-of-distribution synthetic images, so that prompts far from the victim's real data still get high prompt consistency. If a surrogate trained on those high-PC images fails to improve test accuracy relative to a random-prompt baseline, the paper's proxy-metric assumption is falsified.
Extended reading notes
Core claim
The paper's central claim, stated sympathetically, is that the victim model's own predictions contain enough information to replace human prompt engineering for model stealing. A prompt earns a high score (called prompt consistency, or PC) when the images it synthesizes are consistently classified into the target class by the victim, and the authors show statistically that high PC correlates with small feature-space distance to the victim's real data, with Spearman rho between -0.63 and -0.88 across the four datasets. Around that score the method builds an evolutionary loop: contrastive prompt refinement keeps the prompt close to the seed and to positively classified synthetic images while pushing it away from negatively classified ones, while tournament selection, crossover, and mutation recombine the best image triplets and retain the best prompt as an elite. The discovery is thus that victim feedback, not human knowledge, is the workable source of prompt quality for data synthesis in model stealing.
Load-bearing premise
The load-bearing premise is that a prompt which makes the victim consistently output the target class on synthesized images produces images close to the victim's actual class data; the paper supports this only by a correlation computed on the same victim models that supply the labels.
Editorial extensions
If this is right
- A hard-label-only API no longer blocks prompt-based model stealing: top-1 predictions alone drive the entire prompt evolution loop.
- Attacker accuracy improves by up to 22.2 percentage points over class-name or human-prompt baselines under a 500-query-per-class budget.
- One seed image per class suffices across satellite imagery, PASCAL VOC objects, CIFAR-10, and six DomainNet domains.
- Higher prompt consistency translates into a more accurate stolen model, so the attack improves as the query budget grows.
- Because synthesis and surrogate training are decoupled, the same synthetic query images can be reused to tune the attacker's architecture or hyperparameters without further queries.
Reading between the lines
- The PC loop can be read as a black-box probe of what a model's decisions reveal about its training distribution; model stealing is one application, and measuring a model's exposure to synthetic data is another.
- The medical-dataset results imply that the ceiling is set by the generative prior: when the diffusion model cannot draw the domain, PC loses its meaning, so an attacker with a domain-tuned generator could extend the attack to niche fields.
- If a defender could detect query images whose rate of victim agreement is abnormally high relative to their distance from known data, that would break the PC signal; the current hard-label defense does not attempt this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Stealix, a model stealing attack that requires no human-crafted prompts or class names. The attacker starts from one seed image per class, uses a vision-language model to iteratively refine hard prompts with a contrastive loss, and employs a genetic algorithm whose fitness is a proposed 'prompt consistency' (PC) metric: the fraction of synthetic images, generated by a frozen Stable Diffusion model, that the victim classifies into the target class. The evolved prompts generate synthetic images that are pseudo-labeled by the victim and used to train a surrogate classifier. The authors validate the approach on EuroSAT, PASCAL VOC, CIFAR-10, and DomainNet, report improvements of up to 22.2 accuracy points over baselines under a 500-query-per-class budget, and include additional experiments on two medical datasets and a proprietary NSFW detector.
Significance. If the results hold, Stealix is a meaningful advance in model stealing: it demonstrates that hard-label black-box models can be stolen without prompt engineering or class-name knowledge, using only one seed image per class and a modest query budget. The paper ships a full algorithmic description, ablations, an analysis of the PC proxy, a diversity comparison, and a real-world NSFW-model case study. Strengths include the use of a public generative model and VLM, hard-label-only feedback, and explicit three-repeat runs. The main weaknesses are the endogenous validation of PC, the absence of variance reporting in the main tables, and the mismatch between the broad threat-model motivation (specialized high-value domains) and the fact that the largest gains appear on datasets where Stable Diffusion already has strong priors.
major comments (3)
- [Section 5.2, Table 2] The validation of the prompt consistency (PC) proxy is circular and therefore does not establish that PC tracks distributional similarity to the victim data. PC (Eq. 5) is defined by querying the victim model, and the L2 feature distance is computed from the same victim model's penultimate layer. A prompt that places generated images on the victim's side of the decision boundary can achieve high PC without generating images close to the true data manifold. The paper's ultimate external check is the attacker's accuracy on the victim test set, but the paper does not separately validate that high-PC prompts are the ones that transfer. I ask the authors to validate PC against an independent feature space (e.g., a pre-trained foundation model not used anywhere in the attack) or to show that high-PC prompts outperform low-PC prompts on the victim test set in a domain-shifted setting. The concern is not merely theoretical: in Appendix L, on PCAM and RetinaMNIST, where Stable Diffusion has weak priors, Stealix's advantage over DA-Fusion shrinks to 0.7 and 1.3 accuracy points, consistent with the proxy overfitting the victim boundary.
- [Section 5.1, Tables 1 and 12, Figure 4] All reported accuracies are means over only three repeated runs, and neither Table 1 nor Table 12 includes standard deviations, confidence intervals, or significance tests. Figure 4, cited as containing confidence intervals, is not shown with visible error bars in the manuscript. The absence of variance reporting is load-bearing for the comparisons that are close: in Table 12, Stealix's 62.2% versus DA-Fusion's 61.5% on PCAM, and 58.0% versus 56.7% on RetinaMNIST, could easily be within run-to-run noise. The authors should report per-run values or error bars and perform a paired significance test (or at least state the variance) for the headline claims, including the 22.2% improvement on CIFAR-10.
- [Section 1 and Appendix L] The central claim that Stealix 'significantly outperforms other methods ... across diverse image domains' and 'improving attacker model accuracy by up to 22.2%' is overstated relative to the evidence. The 22.2% figure is from CIFAR-10, a dataset where class names are strong and Stable Diffusion has a good prior; on EuroSAT the improvement over the second-best baseline is 6.9 points, and in the medical domains in Appendix L it is below 1.5 points. Section 1 motivates the attack specifically with 'specialized fields, where high-value models are the most common', yet the specialized-domain results are relegated to an appendix and show only marginal gains. The contributions and conclusion should either scope the claim to domains with adequate generative priors or present the medical results prominently in the main text, with a clear statement of where the method does and does not help.
minor comments (5)
- [Section 5.1 and Appendix L] The paper does not state the random seeds or the exact seed-image selection procedure, and the DomainNet experiment uses only 10 of 345 classes; please report which classes are used and how seed images are sampled across the three repeats.
- [Appendix A, Algorithm 3] Line 20 of Algorithm 3 ends with an incomplete set-union expression ('S t+1'); please fix the notation so that the update is fully specified.
- [Section 4.3] The text should clarify whether the same M synthetic images used to compute PC are also added to the positive/negative sets and used for training the attacker, and whether this double use is intentional under the query budget.
- [Section 5.3] The NSFW experiment uses a dataset of only 200 images and reports a single point estimate (73.0% accuracy); please provide variation across repeats or a note about the small evaluation set.
- [General] The project page is mentioned but the manuscript does not provide code or a data-release link; for an attack paper that claims reproducibility through three repeats, a code release or a detailed implementation appendix would substantially help the community.
Circularity Check
No significant circularity: the attack's fitness signal and final evaluation are distinct, with final accuracy measured on external victim test data.
full rationale
Stealix's derivation chain is not circular. The core loop in Algorithm 1 and Section 4.3 queries the victim with synthesized images, labels them with V(x), and uses PC = (1/M) * sum I(V(x_i)=c) as a fitness signal to evolve prompts; this is inherent to hard-label model stealing, not a prediction derived from the same data. The final claim is evaluated by training a ResNet-18 attacker on the synthesized, victim-labeled images and measuring accuracy on the victim's held-out test set, an external benchmark that is not an input to prompt optimization. The PC proxy is validated in Section 5.2 via Spearman correlation against L2 distance in the victim's penultimate layer; this validation is self-referential in the sense that both quantities depend on the same victim model, so it is weaker than an independent distributional check, but it is an empirical correlation rather than a definitional identity, and the central result does not reduce to it. The paper's comparisons against Real Guidance, DA-Fusion, ASPKD, Knockoff, DFME, and KD are all external baselines; no fitted parameter is renamed as a prediction, and no load-bearing uniqueness claim or self-citation is used. The medical-domain appendix (Appendix L) further shows the method's gains shrink when Stable Diffusion lacks domain priors, which is consistent with the claim's scope rather than a circular artifact.
Assumptions & free parameters
free parameters (9)
- Prompt length L =
16
- Prompt optimization steps s =
500
- Prompt optimization learning rate gamma =
0.1
- Population size N =
10
- Number of parents Np =
5
- Tournament size k =
5
- Mutation probability p_m =
0.6
- PC batch size M =
10
- Stable Diffusion guidance scale =
9
assumptions (5)
- domain assumption The attacker has one real seed image per class and no class names.
- domain assumption The victim model returns only the top-1 hard label for each query and does not detect the attack.
- domain assumption Prompt consistency is a valid proxy for proximity to the victim data distribution.
- domain assumption Stable Diffusion-v2 and OpenCLIP-ViT/H have enough domain knowledge to generate useful images for the target classes.
- standard math Standard gradient-based optimization of the PEZ-style hard prompt projection converges to useful prompts.
Cite this review
Pith. "Pith review of Stealix: Model Stealing via Prompt Evolution." pith.science (2026). https://pith.science/paper/QSCMUGVZ
@misc{pith2026250605867,
author = {Pith},
title = {Pith review of: Stealix: Model Stealing via Prompt Evolution},
year = {2026},
howpublished = {\url{https://pith.science/paper/QSCMUGVZ}},
note = {Machine review of arXiv:2506.05867}
}
read the original abstract
Model stealing poses a significant security risk in machine learning by enabling attackers to replicate a black-box model without access to its training data, thus jeopardizing intellectual property and exposing sensitive information. Recent methods that use pre-trained diffusion models for data synthesis improve efficiency and performance but rely heavily on manually crafted prompts, limiting automation and scalability, especially for attackers with little expertise. To assess the risks posed by open-source pre-trained models, we propose a more realistic threat model that eliminates the need for prompt design skills or knowledge of class names. In this context, we introduce Stealix, the first approach to perform model stealing without predefined prompts. Stealix uses two open-source pre-trained models to infer the victim model's data distribution, and iteratively refines prompts through a genetic algorithm, progressively improving the precision and diversity of synthetic images. Our experimental results demonstrate that Stealix significantly outperforms other methods, even those with access to class names or fine-grained prompts, while operating under the same query budget. These findings highlight the scalability of our approach and suggest that the risks posed by pre-trained generative models in model stealing may be greater than previously recognized.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Learning multiple layers of features from tiny images
Alex, K. Learning multiple layers of features from tiny images. https://www. cs. toronto. edu/kriz/learning-features-2009-TR. pdf, 2009
work page 2009
-
[3]
Ba, J. and Caruana, R. Do deep nets really need to be deep? In Advances in Neural Information Processing Systems (NeurIPS), 2014
work page 2014
-
[4]
Beetham, J., Kardan, N., Mian, A. S., and Shah, M. Dual student networks for data-free model stealing. In International Conference on Learning Representations (ICLR), 2022
work page 2022
-
[5]
D., Steinke, T., Hayase, J., Cooper, A
Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., Wallace, E., Rolnick, D., and Tram\` e r, F. Stealing part of a production language model. In International Conference on Machine Learning (ICML), 2024
work page 2024
-
[6]
Reproducible scaling laws for contrastive language-image learning
Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., and Jitsev, J. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023
work page 2023
-
[7]
Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B. A., Fung, P., and Hoi, S. C. H. Instructblip: Towards general-purpose vision-language models with instruction tuning. In Advances in Neural Information Processing Systems (NeurIPS), 2023
work page 2023
-
[8]
An image is worth 16x16 words: Transformers for image recognition at scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021
2021
Show all 40 references
-
[9]
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. The pascal visual object classes (voc) challenge. International Journal of Computer Vision (IJCV), 2010
2010
-
[10]
Data-free adversarial distillation
Fang, G., Song, J., Shen, C., Wang, X., Chen, D., and Song, M. Data-free adversarial distillation. arXiv preprint arXiv:1912.11006, 2019
1912 arXiv
-
[11]
H., Chechik, G., and Cohen-Or, D
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An image is worth one word: Personalizing text-to-image generation using textual inversion. In International Conference on Learning Representations (ICLR), 2023
2023
-
[12]
Is synthetic data from generative models ready for image recognition? In International Conference on Learning Representations (ICLR), 2023
He, R., Sun, S., Yu, X., Xue, C., Zhang, W., Torr, P., Bai, S., and Qi, X. Is synthetic data from generative models ready for image recognition? In International Conference on Learning Representations (ICLR), 2023
2023
-
[13]
Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification
Helber, P., Bischke, B., Dengel, A., and Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019
2019
-
[14]
Distilling the knowledge in a neural network
Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015
2015 arXiv
-
[15]
and Ionescu, R
Hondru, V. and Ionescu, R. T. Towards few-call model stealing via active self-paced knowledge distillation and diffusion-based image generation. arXiv preprint arXiv:2310.00096, 2023
2023 arXiv
-
[16]
S., Parikh, A
Krishna, K., Tomar, G. S., Parikh, A. P., Papernot, N., and Iyyer, M. Thieves on sesame street! model extraction of bert-based apis. In International Conference on Learning Representations (ICLR), 2020
2020
-
[17]
Improved precision and recall metric for assessing generative models
Kynk \"a \"a nniemi, T., Karras, T., Laine, S., Lehtinen, J., and Aila, T. Improved precision and recall metric for assessing generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2019
2019
-
[18]
Defending against machine learning model stealing attacks using deceptive perturbations
Lee, T., Edwards, B., Molloy, I., and Su, D. Defending against machine learning model stealing attacks using deceptive perturbations. In IEEE Security and Privacy Workshops (SPW), 2019
2019
-
[19]
Not-safe-for-work dataset
Lewis, Z. Not-safe-for-work dataset. https://huggingface.co/datasets/zanderlewis/nsfw_detection_large/viewer/default/train?p=1, 2024. Accessed: 2024-11-30
2024
-
[20]
G., Fenu, S., and Starner, T
Lopes, R. G., Fenu, S., and Starner, T. Data-free knowledge distillation for deep neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2017
2017
-
[21]
How to steer your adversary: Targeted and efficient model stealing defenses with gradient redirection
Mazeika, M., Li, B., and Forsyth, D. How to steer your adversary: Targeted and efficient model stealing defenses with gradient redirection. In International Conference on Machine Learning (ICML), 2022
2022
-
[22]
and Storkey, A
Micaelli, P. and Storkey, A. J. Zero-shot knowledge transfer via adversarial belief matching. In Advances in Neural Information Processing Systems (NeurIPS), 2019
2019
-
[23]
I know what you trained last summer: A survey on stealing machine learning models and defences
Oliynyk, D., Mayer, R., and Rauber, A. I know what you trained last summer: A survey on stealing machine learning models and defences. ACM Computing Surveys, 2023
2023
-
[24]
Knockoff nets: Stealing functionality of black-box models
Orekondy, T., Schiele, B., and Fritz, M. Knockoff nets: Stealing functionality of black-box models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019
2019
-
[25]
Moment matching for multi-source domain adaptation
Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., and Wang, B. Moment matching for multi-source domain adaptation. In International Conference on Computer Vision (ICCV), 2019
2019
-
[26]
High-resolution image synthesis with latent diffusion models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022
2022
-
[27]
Sanyal, S., ini, Addepalli, S., and Babu, R. V. Towards data-free model stealing in a hard label setting. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022
2022
-
[28]
Data-free black-box attack based on diffusion model
Shao, M., Meng, L., Qiao, Y., Zhang, L., and Zuo, W. Data-free black-box attack based on diffusion model. arXiv preprint arXiv:2307.12872, 2023
2023 arXiv
-
[29]
Medical multimodal model stealing attacks via adversarial domain alignment
Shen, Y., Zhuang, Z., Yuan, K., Nicolae, M.-I., Navab, N., Padoy, N., and Fritz, M. Medical multimodal model stealing attacks via adversarial domain alignment. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025
2025
-
[30]
Not-safe-for-work image detection
Team, F. Not-safe-for-work image detection. https://huggingface.co/Falconsai/nsfw_image_detection, 2023. Accessed: 2024-11-30
2023
-
[31]
Effective data augmentation with diffusion models
Trabucco, B., Doherty, K., Gurinas, M., and Salakhutdinov, R. Effective data augmentation with diffusion models. In International Conference on Learning Representations (ICLR), 2024
2024
-
[32]
K., and Ristenpart, T
Tram \`e r, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. Stealing machine learning models via prediction \ APIs \ . In USENIX Security, 2016
2016
-
[33]
J., and Papernot, N
Truong, J.-B., Maini, P., Walls, R. J., and Papernot, N. Data-free model extraction. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021
2021
-
[34]
S., Linmans, J., Winkens, J., Cohen, T., and Welling, M
Veeling, B. S., Linmans, J., Winkens, J., Cohen, T., and Welling, M. Rotation equivariant cnns for digital pathology. In Medical Image Computing and Computer Assisted Intervention (MICCAI), 2018
2018
-
[35]
and Gong, N
Wang, B. and Gong, N. Z. Stealing hyperparameters in machine learning. In IEEE Symposium on Security and Privacy (IEEE S&P), 2018
2018
-
[36]
Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery
Wen, Y., Jain, N., Kirchenbauer, J., Goldblum, M., Geiping, J., and Goldstein, T. Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. In Advances in Neural Information Processing Systems (NeurIPS), 2024
2024
-
[37]
J., Jordan, M., and Duchi, J
Wibisono, A., Wainwright, M. J., Jordan, M., and Duchi, J. C. Finite sample convergence rates of zero-order stochastic optimization methods. In Advances in Neural Information Processing Systems (NeurIPS), 2012
2012
-
[38]
Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification
Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L., Ke, B., Pfister, H., and Ni, B. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 2023
2023
-
[39]
Genetic algorithms in search, optimization and machine learning
Zames, G. Genetic algorithms in search, optimization and machine learning. Inf Tech J, 3 0 (1): 0 301, 1981
1981
-
[40]
Stealthy imitation: Reward-guided environment-free policy stealing
Zhuang, Z., Nicolae, M.-I., and Fritz, M. Stealthy imitation: Reward-guided environment-free policy stealing. In International Conference on Machine Learning (ICML), 2024
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.