Pith. sign in

REVIEW 5 major objections 6 minor 57 references

Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that public generative models can supply synthetic data standing in for a target model's training set, enabling model extraction, membership inference, and model inversion attacks in a data-free, black-box setting with…

desk verdict A timely, plausible data-free attack pipeline using public generators, but the theory is invalid, the 'comparable to white-box' claim is overstated, and the private-dataset test does not actually test the key distribution-coverage assumption. read the letter →

arxiv 2501.16671 v1 pith:MO5QSP23 submitted 2025-01-28 cs.CR cs.AI

Dayong Ye , Tianqing Zhu , Shang Wang , Bo Liu , Leo Yu Zhang , Wanlei Zhou , Yang Zhang This is my paper · ORCID
classification cs.CRcs.AI
keywords modelextractionmembershipinferenceinversiondata-freeattackblack-boxgenerativeAIsyntheticdatadiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generative AI is usually studied as a victim of privacy attacks, but this paper argues it can be the weapon. The authors try to establish that an adversary who knows only the task and class labels of a deployed classifier, with no access to its training data, parameters, or architecture, can use a public diffusion model or large language model to synthesize a substitute dataset and then run model extraction, membership inference, and model inversion attacks that approach the performance of white-box baselines. The significance is that today's public generators may already lower the barrier for attacking real deployed models, turning a capability problem into a defense problem. The paper evaluates the pipeline on CIFAR10, MNIST, SkinCancer, BBCNews, and IMDB, plus a newly collected PET dataset, and shows that hand-crafted prompts, boundary augmentation, and inter-class filtering bring the synthetic data close enough to the target's training distribution.

What carries the argument

The load-bearing mechanism is a three-stage data synthesis pipeline. First, prompted generation: the adversary asks a public generative model for samples of each target class, keeping only those the target classifies correctly. Second, boundary augmentation: starting from a kept sample, the adversary adds Gaussian noise in growing spherical shells and collects samples just inside the target's decision boundary, which expands coverage of the sample space. Third, inter-class filtering: samples are embedded through the target's output vectors, class centroids are computed, and any sample in class $k$ whose distance to another class's centroid exceeds three standard deviations is removed. The paper argues, via a Kullback-Leibler divergence inequality, that this filtering reduces the distribution shift between the generated set and the target's training distribution.

What would settle it

Train a target classifier on a genuinely private dataset whose classes are absent from any public generator's training distribution (for example, specialized medical or industrial imagery), run the three-step pipeline, and compare the stolen model's accuracy and agreement to the random-search baseline. If they collapse to that baseline, the distribution-coverage premise is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a black-box adversary can replace the target model's private training set with generated data from a public generative model and still obtain a stolen model whose accuracy and agreement approach those of a thief who has the real training set and architecture. For example, on CIFAR10 the stolen model reaches 82.6% accuracy against a target at 88.7% and a white-box baseline at 85.3%. For membership inference, the adversary trains a shadow model on the generated data and transfers the resulting attack model to the target. For model inversion, the adversary trains a generative inversion model on pairs of target output vectors and generated inputs, then reconstructs inputs from confidence vectors. The paper also claims the same pipeline works in label-only settings, where boundary-crossing noise becomes a proxy for membership and class-representative samples stand in for exact inversion.

Load-bearing premise

The pipeline only works if the public generative model's training data already covers the target model's task domain, because the generated samples are supposed to stand in for the target's private training data.

Editorial extensions

If this is right

  • A deployed image or text classifier that exposes output confidence vectors can be functionally copied with no access to its training data, provided a public generator exists for the same task.
  • Membership inference can be run without collecting a shadow dataset matching the target's domain: synthetic data plus a mimic model yields attack accuracy, F1, AUC, and TPR@1%FPR comparable to a white-box shadow-model baseline.
  • The same generated dataset supports model inversion by training an inversion model on pairs of target output vectors and generated inputs, with reconstruction quality close to the baseline that uses the real training set.
  • Label-only access does not stop the attacks: boundary-crossing noise can signal membership, and class-representative synthetic samples serve as inversion outputs.
  • Synthetic-data detection and output perturbation are only partial defenses: detection can be bypassed by generating more data, and perturbation leaves membership inference and inversion largely intact.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The PET experiment only partially tests the distribution-coverage assumption, because its cat/dog/bird classes are common and several videos were downloaded from YouTube, which may already be inside the generator's training data; a target trained on a genuinely novel domain remains the untested case.
  • The KL-divergence argument shows filtering reduces the distance to the target distribution only under the three-sigma model; in practice the same filtering could remove informative boundary samples, so the attack's edge may depend on how cleanly class centroids separate in output space.
  • As generators improve for non-image, non-text modalities such as audio, video, or structured medical records, the same three-step pipeline would likely carry over, making the threat broader than the two modalities tested here.
  • Because automated prompt generation already comes close to manual prompts in extraction accuracy, the attack appears automatable end-to-end, so the practical barrier for attackers is mostly compute and API access rather than expert prompt design.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a data-free, black-box framework for model-related attacks (model extraction, membership inference, and model inversion) that uses public generative models (Stable Diffusion and GPT-4) to synthesize a surrogate dataset, augments it by probing the target model's decision boundary, and then filters the data with an inter-class three-sigma rule. The authors evaluate the pipeline on CIFAR10, MNIST, SkinCancer, BBCNews, IMDB, and a newly collected PET dataset, and claim that the attacks achieve performance comparable to white-box baselines that use the target's training data and architecture. The paper also includes hyperparameter studies, ablations with random-search baselines and alternative generative models, a discussion of defenses, and a release of source code.

Significance. If validated, the paper would be a useful contribution to the security/privacy literature: it extends data-free model stealing and related attacks to a setting where the adversary uses only a public generative model and black-box query access, and it is one of the few studies to cover both image and text domains under one pipeline. The authors also provide ablations and a code link, which supports reproducibility. The central empirical claim, however, currently rests on an untested distribution-coverage assumption, and the evaluation protocol for membership inference and model extraction does not, as written, measure what the abstract claims it measures. The theoretical analysis in Section 4 is not sound. These issues preclude acceptance in the current form.

major comments (5)
  1. [Section 4, Eq. (4)] The proof that inter-class filtering reduces the KL divergence between the target training distribution and the generated distribution is invalid. The first integral in the second line is not itself a KL divergence and can be negative, so the statement 'any KL divergence value is larger than or equal to 0' does not justify dropping it. The subsequent claim that P'_gen(x)/P_gen(x) > 1 pointwise is not established; P'_gen is a distribution obtained by deleting samples and can be smaller than P_gen on some points or zero where P_gen is positive. The line 'Pgen(x < mu + 3sigma) < 1' also treats a probability distribution as a random variable. As written, the theoretical result is unproved. I recommend either supplying a correct proof or explicitly labeling the three-sigma filter as a heuristic supported only by the experiments.
  2. [Section 3.3 and Section 5.2] The membership inference attack is trained and evaluated using the auxiliary dataset's own train/test split (dDtrainaux versus dDtestaux), not using samples whose membership in the target model's training set is known. Tables 2 and 3 therefore report how well the attack model distinguishes the surrogate's generated training samples from its generated test samples. This does not establish that the attack transfers to the target model T. The paper does not describe a ground-truth membership test set for T or report how often samples from T's actual Dtrain are flagged as members. Please specify the evaluation protocol and report attack accuracy/F1/AUC on the target model's actual members versus non-members.
  3. [Section 5.2, Table 1] Model extraction accuracy and agreement are computed on dDtestaux, a subset of the same generated, T-filtered dataset used to train the stolen model. Because both the target and stolen models are evaluated on samples that T has already classified consistently during the filtering step, the target accuracy and the agreement metric are inflated by construction. The paper should additionally evaluate both models on the original held-out test sets (e.g., the standard CIFAR10 test split, the original BBCNews/IMDB splits) or on an independently collected set, and report target versus stolen performance there.
  4. [Section 3.1 and Section 5.1] The central premise of the paper is that a public generative model's training distribution 'is highly likely to encompass' the target model's training distribution. None of the main datasets tests this premise: CIFAR10, MNIST, SkinCancer, BBCNews, and IMDB are public and plausibly present in the training corpora of Stable Diffusion or GPT-4. The PET dataset does not resolve the issue: its dog and bird videos are listed as YouTube URLs in the paper, and 'cat' is a common image category, so the generator can produce in-domain samples without having seen the specific PET videos. Please test the pipeline on a genuinely private or niche domain where the generator cannot rely on memorized or near-memorized content, or otherwise quantify the distribution overlap between the generator's output and the target's training set. Without such a test, the data-free claim is only demonstrated for public-distribution targets.
  5. [Section 5.1 and Tables 1-4] The experiments report averages over three runs but provide no error bars, confidence intervals, or statistical comparisons, yet the abstract and Section 5.5 make the strong claim of performance 'comparable' to white-box baselines. Some of the observed gaps are substantial (e.g., Table 2 CIFAR10 AUC 0.54 versus 0.67; Table 1 CIFAR10 agreement 83.7 versus 89.4). Please report standard deviations and either bootstrap confidence intervals or a paired significance test for the headline accuracy/agreement/AUC comparisons.
minor comments (6)
  1. [Section 4, Eq. (1)] The text says 'I is the unit vector' but the context requires the identity matrix; please correct the notation.
  2. [Algorithm 2] In lines 2-3, the algorithm refers to samples xk_j in class i, but the centroid is computed over xi_j; the subscript should be i throughout for consistency.
  3. [Table 11] The table header says 'AUC' but the values are TPR@1%FPR; the header should be updated to match the rows.
  4. [Section 1, Contributions] There is a typo: 'utilitizing' should be 'utilizing'.
  5. [Section 3.3] The label-only variants of membership inference and model inversion are described but not evaluated in the experiments; please clarify whether these variants are part of the claims or defer them to future work.
  6. [Section 5.1] The description of the comparison baselines would benefit from more detail on training hyperparameters and the amount of data used, since the baseline's access to the full target training set is a key point in the comparison.

Circularity Check

0 steps flagged · score 2.0 of 10

No meaningful circularity: the attack pipeline is an empirical construction, the distribution-coverage premise is an explicit assumption rather than a derived prediction, and the only self-citation ([57]) is background.

full rationale

No load-bearing step in this paper reduces to its own inputs by construction. The central premise (Section 3.1) is explicitly an assumption: "the distribution space of training data for generative models is highly likely to encompass the distribution space of the training set for the target model if the target model shares a similar task with the generative model." This is a domain-coverage hypothesis, not a quantity fitted from the target outputs and then renamed as a prediction; the experiments test it on public datasets and on the newly created PET dataset. The filtering and augmentation steps use target-model outputs as an oracle, but that is the attack mechanism (the adversary is allowed black-box queries), not a self-definitional derivation. The stolen model's accuracy is measured against the baseline rather than being forced by a fitted parameter. The one apparent author self-citation is [57] in the introductory list of model inversion attacks; the paper's label-only inversion procedure (Section 3.3) does not import a load-bearing theorem or fitted value from that reference. Section 4's KL-divergence proof appears mathematically invalid (Eq. (4) drops the negative integral over the removed region), and the PET baseline is omitted because it "did not adequately support" the baseline; these are correctness and external-validity concerns, not circularity. The paper is self-contained in the sense that its claims are evaluated against white-box baselines on external datasets, so any circularity score above 2 would not be justified.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or model entities beyond the standard attack components (mimic model, attack model, inversion model). The main load-bearing choices are the free hyperparameters and the domain assumption that the generator's training data overlaps the target's task distribution.

free parameters (6)
  • delta_0 (initial noise) = not reported in main text
    Hyperparameter in Algorithm 1; determines starting distance for boundary exploration.
  • epsilon (noise step size) = not reported in main text
    Hyperparameter in Algorithm 1; controls step size when searching the decision boundary.
  • N (samples per augmentation round) = not reported in main text
    Hyperparameter in Algorithm 1; number of noise-added samples collected at each radius.
  • generated samples per class = 100 to 500 per dataset
    Chosen per dataset complexity; affects all attack results.
  • augmented samples per class = 1000 to 6000 per dataset
    Chosen per dataset; augmentation volume is a free choice.
  • manual prompt phrasing = e.g., 'Generate a single class name, in a realistic style, with a clear background'
    Prompt design is manually crafted and dataset-specific; the paper notes automated prompts underperform.
assumptions (4)
  • domain assumption Generative models are trained on data whose distribution covers the target model's training domain.
    Stated in Section 3.1 as the basis for using a public generator as a data source.
  • domain assumption The adversary knows the class labels and meanings of the target model.
    Stated in the threat model (Section 2); needed to craft prompts.
  • domain assumption Target model output vectors reflect learned data distribution, so three-sigma filtering in output space reduces distribution shift.
    Introduced in Section 3.2, Step 3; the flawed proof in Section 4 is meant to justify this.
  • standard math Standard probability and KL divergence properties as used in Section 4.
    The proof of distribution-shift mitigation invokes these, though it misapplies them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI." pith.science (2026). https://pith.science/paper/MO5QSP23

@misc{pith2026250116671,
  author       = {Pith},
  title        = {Pith review of: Data-Free Model-Related Attacks: Unleashing the Potential of Generative AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MO5QSP23}},
  note         = {Machine review of arXiv:2501.16671}
}
read the original abstract

Generative AI technology has become increasingly integrated into our daily lives, offering powerful capabilities to enhance productivity. However, these same capabilities can be exploited by adversaries for malicious purposes. While existing research on adversarial applications of generative AI predominantly focuses on cyberattacks, less attention has been given to attacks targeting deep learning models. In this paper, we introduce the use of generative AI for facilitating model-related attacks, including model extraction, membership inference, and model inversion. Our study reveals that adversaries can launch a variety of model-related attacks against both image and text models in a data-free and black-box manner, achieving comparable performance to baseline methods that have access to the target models' training data and parameters in a white-box manner. This research serves as an important early warning to the community about the potential risks associated with generative AI-powered attacks on deep learning models.

Figures

Figures reproduced from arXiv: 2501.16671 by the authors.

Figure 1
Figure 1. Exploration of the decision boundary in the target [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Data distribution of CIFAR10, generated data, and [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Inversion results on MNIST, CIFAR10 and Skin [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Inversion results on BBCNews and IMDB [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Inversion results on PET. eter study focuses on two aspects: the quantity of generated data and the extent of augmentation applied to these data. 5.3.1 The Number of Generated Data We assess how varying the number of generated data points for each class impacts the per…
Figure 6
Figure 6. Figure 6: Inversion results on MNIST, CIFAR10 and Skin [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Inversion results on BBCNews and IMDB with [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 9
Figure 9. Figure 9: Inversion results on BBCNews and IMDB with and [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 11
Figure 11. Figure 11: Inversion results on BBCNews and IMDB using [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: Inversion results on CIFAR10 with additional [PITH_FULL_IMAGE:figures/full_fig_p015_12.png]
Figure 13
Figure 13. Figure 13: Inversion results on IMDB with additional genera [PITH_FULL_IMAGE:figures/full_fig_p015_13.png]
Figure 14
Figure 14. Figure 14: Comparison of images generated using manual [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Inversion results on CIFAR10 with and without [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Inversion results on IMDB with and without de [PITH_FULL_IMAGE:figures/full_fig_p016_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 56 canonical work pages

  1. [1]

    Fast Stable Diffusion XL on TPU v5e, 2023

    work page 2023

  2. [2]

    Imdb, https://www.kaggle.com/datasets/atulanandjha/imdb- 50k-movie-reviews-test-your-bert

    Atul Anand. Imdb, https://www.kaggle.com/datasets/atulanandjha/imdb- 50k-movie-reviews-test-your-bert

    work page

  3. [3]

    RatGPT: Turn- ing online LLMs into Proxies for Malware Attacks, 2023

    M Beckerich, L Plein, and S Coronado. RatGPT: Turn- ing online LLMs into Proxies for Malware Attacks, 2023

    work page 2023

  4. [4]

    Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text? ACM SIGKDD Explorations Newsletter, 25(2):14–21, 2024

    A Bhattacharjee and H Liu. Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text? ACM SIGKDD Explorations Newsletter, 25(2):14–21, 2024

    work page 2024

  5. [5]

    Membership Inference Attacks From First Principles

    N Carlini, S Chien, M Nasr, S Song, A Terzis, and F Tramer. Membership Inference Attacks From First Principles. In Proc. of IEEE S & P, pages 1897–1914, 2022

    work page 1914

  6. [6]

    Extracting Training Data from Large Language Models

    N Carlini, F Tramer, E Wallace, M Jagielski, A Herbert- V oss, K Lee, A Roberts, T Brown, D Song, U Erlingsson, A Oprea, and C Raffel. Extracting Training Data from Large Language Models. In Proc. of USENIX Security Symposium, page 2633–2650, 2021

    work page 2021

  7. [7]

    Can LLM-Generated Misinforma- tion Be Detected? In Proc

    C Chen and K Shu. Can LLM-Generated Misinforma- tion Be Detected? In Proc. of ICLR, pages 1–17, 2024

    work page 2024

  8. [8]

    Choquette-Choo, F Tramer, N Carlini, and N Pa- pernot

    C A. Choquette-Choo, F Tramer, N Carlini, and N Pa- pernot. Label-Only Membership Inference Attacks. In Proc. of ICML, pages 1964–1974, 2021

    work page 1964

Show all 57 references
  1. [9]

    MeaeQ: Mount Model Extraction Attacks with Efficient Queries

    C Dai, M Lv, K Li, and W Zhou. MeaeQ: Mount Model Extraction Attacks with Efficient Queries. In Proc. of EMNLP, pages 12671–12684, 2023

    2023

  2. [10]

    Dwork and A

    C. Dwork and A. Roth. The Algorithmic Foundations of Differential Privacy. Foundations and Trends in The- oretical Computer Science, 9(3-4):211–407, 2014

    2014

  3. [11]

    A comprehensive survey and analysis of generative mod- els in machine learning

    H GM, M K Gourisaria, M Pandey, and S S Rautaray. A comprehensive survey and analysis of generative mod- els in machine learning. Computer Science Review , 38:100285:1–29, 2022

    2022

  4. [12]

    Goodfellow, J Pouget-Abadie, M Mirza, B Xu, D Warde-Farley, S Ozair, A Courville, and Y Bengio

    I J. Goodfellow, J Pouget-Abadie, M Mirza, B Xu, D Warde-Farley, S Ozair, A Courville, and Y Bengio. Generative Adversarial Nets. In Proc. of NeurIPS, 2014

    2014

  5. [13]

    Gemini 1.5 Flash, 2024

    Google. Gemini 1.5 Flash, 2024. 17

    2024

  6. [14]

    Are GAN Generated Images Easy to De- tect? A Critical Analysis of the State-Of-The-Art

    D Gragnaniello, D Cozzolino, F Marra, G Poggi, and L Verdoliva. Are GAN Generated Images Easy to De- tect? A Critical Analysis of the State-Of-The-Art. In Proc. of ICME, 2021

    2021

  7. [15]

    How close is chatgpt to human experts? comparison corpus, evaluation, and detection, 2023

    B Guo, X Zhang, Z Wang, M Jiang, J Nie, Y Ding, J Yue, and Y Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection, 2023

    2023

  8. [16]

    From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy

    M Gupta, C Akiri, K Aryal, E Parker, and L Praharaj. From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy. IEEE Access, 11:80218 – 80245, 2023

    2023

  9. [17]

    Reinforcement Learning-Based Black-Box Model Inversion Attacks

    G Han, J Choi, H Lee, and J Kim. Reinforcement Learning-Based Black-Box Model Inversion Attacks. In Proc. of CVPR, pages 20504–20513, 2023

    2023

  10. [18]

    Getting pwn’d by AI: Penetration Testing with Large Language Models

    A Happe and J Cito. Getting pwn’d by AI: Penetration Testing with Large Language Models. In Proc. of the 31st ACM Joint European Software Engineering Con- ference and Symposium on the Foundations of Software Engineering, page 2082–2086, 2023

    2023

  11. [19]

    Spear Phishing With Large Language Models, 2023

    J Hazell. Spear Phishing With Large Language Models, 2023

    2023

  12. [20]

    Deep Residual Learn- ing for Image Recognition

    K He, X Zhang, S Ren, and J Sun. Deep Residual Learn- ing for Image Recognition. In Proc. of CVPR, pages 770–778, 2015

    2015

  13. [21]

    Denoising Diffusion Proba- bilistic Models

    J Ho, A Jain, and P Abbeel. Denoising Diffusion Proba- bilistic Models. In Proc. of NeurIPS, 2020

    2020

  14. [22]

    Fraud GPT: A new face of cyber threats of the future, 2023

    SharkStriker Inc. Fraud GPT: A new face of cyber threats of the future, 2023

    2023

  15. [23]

    Melanoma skin cancer dataset, 2022

    M H Javid. Melanoma skin cancer dataset, 2022

    2022

  16. [24]

    OpenAI Debuts DALL-E for Generating Images from Text, 2021

    K Johnson. OpenAI Debuts DALL-E for Generating Images from Text, 2021

    2021

  17. [25]

    Label-Only Model Inversion Attacks via Boundary Repulsion

    M Kahla, S Chen, H A Just, and R Jia. Label-Only Model Inversion Attacks via Boundary Repulsion. In Proc. of CVPR, pages 15045–15053, 2022

    2022

  18. [26]

    MAZE: Data- Free Model Stealing Attack Using Zeroth-Order Gradi- ent Estimation

    S Kariyappa, A Prakash, and M Qureshi. MAZE: Data- Free Model Stealing Attack Using Zeroth-Order Gradi- ent Estimation. In Proc. of CVPR, pages 13814–13823, 2021

    2021

  19. [27]

    An Introduction to Vari- ational Autoencoders

    D P Kingma and M Welling. An Introduction to Vari- ational Autoencoders. Foundations and Trends in Ma- chine Learning, 2019

    2019

  20. [28]

    The cifar-10 dataset, 2014

    A Krizhevsky, V Nair, and G Hinton. The cifar-10 dataset, 2014

    2014

  21. [29]

    The mnist database of handwritten digits, 1998

    Y LeCun. The mnist database of handwritten digits, 1998

    1998

  22. [30]

    Multi-step Jailbreaking Privacy Attacks on ChatGPT

    H Li, D Guo, W Fan, M Xu, J Huang, F Meng, and Y Song. Multi-step Jailbreaking Privacy Attacks on ChatGPT. In Proc. of EMNLP, page 4138–4153, 2023

    2023

  23. [31]

    Membership Leakage in Label-Only Exposures

    Z Li and Y Zhang. Membership Leakage in Label-Only Exposures. In Proc. of ICML, page 880–895, 2021

    2021

  24. [32]

    ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

    Y Liu, R Wen, X He, A Salem, Z Zhang, M Backes, E De Cristofaro, M Fritz, and Y Zhang. ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models. In Proc. of USENIX Security, pages 1–18, 2022

    2022

  25. [33]

    Membership Infer- ence Attacks against Language Models via Neighbour- hood Comparison

    J Mattern, F Mireshghallah, Z Jin, B Schoelkopf, M Sachan, and T Berg-Kirkpatrick. Membership Infer- ence Attacks against Language Models via Neighbour- hood Comparison. In Proc. of ACL, page 11330–11343, 2023

    2023

  26. [34]

    Regularizing and Optimizing LSTM Language Models

    S Merity, N S Keskar, and R Socher. Regularizing and Optimizing LSTM Language Models. In Proc. of ICLR, pages 1–13, 2018

    2018

  27. [35]

    Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

    M Nasr, R Shokri, , and A Houmansadr. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In Proc. of IEEE S & P , pages 739–753, 2019

    2019

  28. [36]

    Re-thinking Model Inversion Attacks Against Deep Neural Networks

    N Nguyen, K Chandrasegaran, M Abdollahzadeh, and N Cheung. Re-thinking Model Inversion Attacks Against Deep Neural Networks. In Proc. of CVPR , pages 16384–16393, 2023

    2023

  29. [37]

    I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences

    D Oliynyk, R Mayer, and A Rauber. I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences. ACM Computing Sur- veys, 55(14s):324:1–41, 2023

    2023

  30. [38]

    DALL·E API Now Available in Public Beta, 2022

    OpenAI. DALL·E API Now Available in Public Beta, 2022

    2022

  31. [39]

    GPT-4 Technical Report, 2023

    OpenAI. GPT-4 Technical Report, 2023

    2023

  32. [40]

    Bbc news, https://www.kaggle.com/dsv/7633851

    G Preda. Bbc news, https://www.kaggle.com/dsv/7633851

  33. [41]

    Open-journey, 2024

    PromptHero. Open-journey, 2024

    2024

  34. [42]

    AI Prompt Optimizer, 2024

    PromptPerfect. AI Prompt Optimizer, 2024

    2024

  35. [43]

    Deep- steal: Advanced model extractions leveraging efficient weight stealing in memories

    A S Rakin, M H I Chowdhuryy, F Yao, and D Fan. Deep- steal: Advanced model extractions leveraging efficient weight stealing in memories. In Proc. of IEEE S&P , pages 1157–1174, 2022

    2022

  36. [44]

    Photorealistic Text-to-Image Dif- fusion Models with Deep Language Understanding

    C Saharia and et al. Photorealistic Text-to-Image Dif- fusion Models with Deep Language Understanding. In Proc. of NeurIPS, 2022. 18

    2022

  37. [45]

    Prompt stealing attacks against large language models, 2024

    Z Sha and Y Zhang. Prompt stealing attacks against large language models, 2024

    2024

  38. [46]

    Model Stealing Attacks Against Inductive Graph Neural Networks

    Y Shen, X He, Y Han, and Y Zhang. Model Stealing Attacks Against Inductive Graph Neural Networks. In Proc. of IEEE S & P, page 1175–1192, 2022

    2022

  39. [47]

    Shokri, M

    R. Shokri, M. Stronati, C. Song, and Vi. Shmatikov. Membership Inference Attacks Against Machine Learn- ing Models. In Proc. of IEEE S & P, pages 3–18, 2017

    2017

  40. [48]

    Very Deep Convolu- tional Networks for Large-Scale Image Recognition

    K Simonyan and A Zisserman. Very Deep Convolu- tional Networks for Large-Scale Image Recognition. In Proc. of ICLR, pages 1–10, 2015

    2015

  41. [49]

    Agent SCA: Advanced Physical Side Channel Analysis Agent with LLMs

    F Yaman. Agent SCA: Advanced Physical Side Channel Analysis Agent with LLMs. PhD thesis, North Carolina State University, 2023

    2023

  42. [50]

    Detect- ing fake images by identifying potential texture differ- ence

    J Yang, S Xiao, A Li, G Lan, and H Wang. Detect- ing fake images by identifying potential texture differ- ence. Future Generation Computer Systems, 125:127– 135, 2021

    2021

  43. [51]

    Diffusion Models: A Comprehensive Survey of Methods and Applications

    L Yang, Z Zhang, Y Song, S Hong, R Xu, Y Zhao, W Zhang, B Cui, and M Yang. Diffusion Models: A Comprehensive Survey of Methods and Applications. ACM Computing Surveys, 56(4):105:1–39, 2023

    2023

  44. [52]

    Prsa: Prompt reverse stealing attacks against large language models, 2024

    Y Yang, X Zhang, Y Jiang, X Chen, H Wang, S Ji, and Z Wang. Prsa: Prompt reverse stealing attacks against large language models, 2024

    2024

  45. [53]

    Neural Net- work Inversion in Adversarial Setting via Background Knowledge Alignment

    Z Yang, J Zhang, E Chang, and Z Liang. Neural Net- work Inversion in Adversarial Setting via Background Knowledge Alignment. In Proc. of CCS, page 225–240, 2019

    2019

  46. [54]

    A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly, 2023

    Y Yao, J Duan, K Xu, Y Cai, Z Sun, and Y Zhang. A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly, 2023

    2023

  47. [55]

    J. Ye, A. Maddi, S. K. Murakonda, V . Bindschaedler, and R. Shokri. Enhanced Membership Inference Attacks against Machine Learning Models. In Proc. of CCS, 2022

    2022

  48. [56]

    X Zhao, W Zhang, X Xiao, and B Y . Lim. Exploiting Explanations for Model Inversion Attacks. In Proc. of ICCV, 2021

    2021

  49. [57]

    Label-only model inversion attacks: Attack with the least informa- tion

    T Zhu, D Ye, S Zhou, B Liu, and W Zhou. Label-only model inversion attacks: Attack with the least informa- tion. IEEE Transactions on Information Forensics and Security, 18:991–1005, 2023. Appendix 1 Model Architecture, Sample Complexity, and Computation Cost Model Architectur...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.