Pith. sign in

REVIEW 4 major objections 5 minor 78 references

SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmark

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper gives data reconstruction attacks a formal definition and evaluation metrics, and argues from a ten-attack benchmark that model memorization is the main driver of how much training data leaks.

desk verdict Useful formalism and benchmark, but the headline memorization claim rests on an unvalidated LLM judge and a protocol that leaks target information. read the letter →

arxiv 2506.07888 v1 pith:ZZG7U57Q submitted 2025-06-09 cs.CR cs.LG

Rui Wen , Yiyong Liu , Michael Backes , Yang Zhang This is my paper
classification cs.CRcs.LG
keywords datareconstructionattacksmodelinversionmemorizationevaluationmetricscoverageLLM-basedprivacyleakagebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Data reconstruction attacks — efforts to recover the training data of a machine learning model — have no shared definition of success and no agreed evaluation metric, which this systematization paper sets out to fix. It proposes a formal taxonomy of attack settings (static vs. dynamic training, black-box vs. white-box model access, and no, similar, or same-distribution auxiliary data), formal definitions of exact and approximate reconstruction, and two complementary metrics: a dataset-level distance (D-Dis, computed as Fréchet Inception Distance) and a sample-level distance with a coverage score (S-Dis and $\alpha$) that measures how much of the dataset is reconstructed, not just how well. Benchmarking ten attacks across image and text tasks, the paper's central empirical claim is that a model's memorization of its training samples is the main driver of reconstruction success: models trained on smaller data memorize more and, judged by GPT-4o pairwise visual comparisons, leak far more recognizable content, a trend that pixel-level metrics such as FID, SSIM, PSNR, and MSE often miss or even reverse. If correct, the finding makes memorization a measurable vulnerability signal and points to memorization-reducing defenses — notably model pruning, which the paper finds costs under 1% accuracy — as more practical than differential privacy.

What carries the argument

The load-bearing mechanism is the pairing of a model-level memorization score with an LLM-based similarity judgment. The memorization score extends the standard sample-level memorization definition to the whole model: model-mem is the average increase, over the first 1,000 training samples, in the probability that a sample is predicted correctly because it was included in training. Reconstruction is measured on two levels: D-Dis, a Fréchet Inception Distance that treats both datasets as Gaussian distributions, and the pair (S-Dis, $\alpha$), the averaged nearest-sample distance and the coverage fraction of target samples matched. The bridge between the two is a GPT-4o protocol in which, for 1,000 target identities, the six reconstructions produced from models of different training-set sizes are compared pairwise and the model reports which is most similar to the target; majority votes, unanimous votes, and overall selection rates ('pred rate') are tallied. Because these LLM judgments track the memorization score while pixel-level metrics do not, the protocol is what carries the central conclusion.

What would settle it

Repeat the Table 3 pairwise protocol with human raters: for each of the 1,000 target identities, ask raters which of the six reconstructions best resembles the target, and compare human selection rates with GPT-4o's. If humans do not show the same majority preference for reconstructions from the 1,000-sample model, or if that preference disappears when the small-dataset reconstructions are blurred to match the sharpness of the 20,000-sample ones, the central claim is refuted. A secondary check: the memorization account predicts monotone decreasing pred rates as dataset size grows, but Table 3 already shows non-monotone bumps (PLGMI's pred rate rises at 10,000), so a statistical test of monotonicity across the six sizes would also adjudicate the strength of the claim.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that data reconstruction attacks should be defined by what they achieve at the dataset level and understood as a memorization-driven phenomenon. A reconstruction algorithm $A$ takes a target model $m$ and extra knowledge $k$ (specifying training type, model access, and dataset access) and returns a dataset of the same size as the target set; a reconstruction is $\mu$-exact when the chosen metric equals zero and $(\varepsilon,\mu)$-approximate when it falls below $\varepsilon$. Using their metrics — D-Dis, plus the pair (S-Dis, $\alpha$) where $\alpha$ is the fraction of target samples matched — the authors show no attack achieves exact reconstruction, that dataset- and sample-level metrics correlate only weakly, and that quantitative scores frequently contradict human perception, especially in the high-quality regime. Substituting GPT-4o for the human viewer in forced pairwise similarity judgments (cross-checked with InternVL 2.5 and Claude 3.7), they find that reconstructions from high-memorization models — training sets of 1,000 images, memorization score near 1.0 — are consistently judged closer to the target identities than reconstructions from low-memorization models (score around 0.3), with the same pattern in text reconstruction and in Swin and MAE transformer architectures. The paper concludes that model memorization is strongly correlated with reconstruction vulnerability, that fine-tuning pre-trained models amplifies this exposure, and that pruning reduces it with negligible accuracy loss.

Load-bearing premise

The load-bearing premise is that GPT-4o's forced 'which image is more similar' choices faithfully reproduce human visual judgment of reconstruction quality; the paper provides no human validation and no control for the possibility that the LLM prefers blurrier, less detailed reconstructions for reasons unrelated to privacy leakage, and if that preference is an artifact, the claimed memorization–vulnerability correlation collapses.

Editorial extensions

If this is right

  • Higher memorization implies higher reconstruction exposure: across GAN-based image attacks, text reconstruction (Vec2Text, Complete), and Swin/MAE transformer models, smaller training sets that memorize more produce reconstructions judged significantly closer to the targets.
  • Defenses should target memorization: the paper shows model pruning weakens reconstruction performance (higher FID) at under 1% accuracy cost, while differential privacy pays a larger accuracy price, so memorization-aware defenses can be cheaper than DP.
  • Fine-tuning pre-trained models is riskier than training from scratch: pre-trained models memorize sample-specific features more efficiently, so fine-tuning datasets reconstruct better and should be treated as an exposure risk.
  • Parameters that store training statistics are a primary leak channel: DeepInversion's reconstruction quality collapses when BatchNorm statistics are fixed rather than updated, showing that releasing such parameters amplifies reconstruction risk.
  • Evaluation must report coverage alongside fidelity: under the paper's definition an attack that perfectly recovers a few samples is a small-coverage leak, not a dataset reconstruction, and generating extra samples to inflate coverage is excluded by the same-size requirement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the memorization–reconstruction curve could be inverted into a cheap privacy audit — estimate a model's memorization score and read off its reconstruction exposure, without running an attack at all.
  • Editorial: the benchmark's numbers are tied to the specific LLM judge, so pred-rate tables would need re-running when GPT-4o-class models are updated; freezing the judge version is a reproducibility constraint the paper leaves implicit.
  • Editorial: the backdoor-trigger experiment implies reconstruction can double as a diagnostic for what a model has memorized, such as poisoned triggers, giving the attack methodology a use beyond privacy breach.
  • Editorial: a blur-matched control (blurring small-dataset reconstructions to the sharpness of large-dataset ones before LLM comparison) would separate genuine identity leakage from softness preference, sharpening the memorization–vulnerability claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a systematization of data reconstruction attacks against machine learning models in the vision domain. It contributes a taxonomy over training type, model access, and dataset access; formal definitions of reconstruction algorithms and of exact/approximate reconstruction; a set of quantitative evaluation metrics (a dataset-level FID-based D-Dis and sample-level S-Dis and coverage alpha); and an LLM-based visual evaluation protocol. Using these, it benchmarks ten reconstruction attacks across CelebA, CIFAR10, and MNIST on VGG16, MobileNetV2, and ResNet-18, with varying training set sizes, and also studies the influence of pre-training, BatchNorm statistics, bottleneck width, defenses (DP, pruning, MID), and additional data modalities. The central empirical claim, stated in Section 8, is that model memorization is strongly correlated with reconstruction performance, i.e., models with higher memorization are more vulnerable to reconstruction attacks.

Significance. If the central claim were fully supported, this would be a valuable SoK: the formal definitions give the field a common vocabulary, the quantitative metrics correct known failures of accuracy-based evaluation (with concrete counterexamples in Figure 7), and the benchmark is broad and internally documented, with public code, ablations for BatchNorm, pre-training, bottleneck width, and defenses, and a text-domain extension. The proposal to use LLMs as a scalable human surrogate for visual similarity is also potentially important. However, the memorization-vulnerability conclusion currently rests on an LLM evaluation protocol that is unvalidated and that contradicts the paper's own quantitative metrics; the formal framing is therefore ahead of the evidence behind the paper's headline claim.

major comments (4)
  1. [§6.1, Definition 3.2] The protocol in Section 6.1 selects, for each target image, the "1,000 results with the highest PSNR score relative to their targets" before asking GPT-4o to compare reconstructions. This selection uses information about the target dataset, which Section 3.2 explicitly prohibits: "it is crucial that the selection procedure does not involve any information about the target dataset." Since the central conclusion in Section 8 relies entirely on this PSNR-against-target selection, the evaluation as reported is inconsistent with the paper's own formal definition of a reconstruction algorithm, and the main empirical claim is not established by the given protocol.
  2. [§6.1, Tables 3, 8, 12] The claim in Section 8 of a "clear trend" of memorization being "strongly correlated" with reconstruction performance is not supported by the data presented. Even within Table 3, the pred rates are non-monotonic in training size for all three attacks (e.g., Revealer: 0.335, 0.185, 0.194, ...; KEDMI: 0.281, 0.169, 0.217, ...; PLGMI: 0.283, 0.142, 0.174, 0.200, ...). The transformer results contradict the trend more sharply: in Table 8 (Swin) the maximum # of Major occurs at 2,000 samples, and in Table 12 (MAE) it occurs at 5,000 samples. The paper's own quantitative metrics in Table 2 show the opposite relation: for GAN-based attacks, FID, SSIM, PSNR, and MSE all improve monotonically as training size grows and memorization falls. The authors acknowledge this discrepancy in Section 6, but the substitute evidence does not currently support the strong conclusion drawn in Section 8.
  3. [§6.1] GPT-4o is used as a substitute for human judgment in pairwise similarity comparisons, but the paper provides no human validation of this surrogate on the actual reconstruction task. The reconstructions from the 1,000-sample model are described as "blurrier," and no control is performed for the possibility that the LLM prefers blurred or more generic-looking faces for reasons unrelated to identity leakage. A concrete remedy would be to run the same pairwise protocol with human raters on a subset of the 1,000 identities and to include a control condition where sharp reconstructions are artificially blurred or where generic face templates are compared against identity-specific reconstructions. Without such controls, the LLM-based metric cannot be taken as evidence of privacy leakage in the high-quality regime where the quantitative metrics disagree.
  4. [§4.3, Definition 4.4] Definition 4.4 defines (ε, µ)-approximate reconstruction as µ(A_k(m), D_tar) ≤ ε, but Definition 4.2 states that the sample-level metric µ returns the pair (S-Dis, α), where higher α means better coverage. For a tuple-valued µ, the inequality µ ≤ ε is not well-defined, and for coverage a lower bound rather than an upper bound would be required. This is a formal gap in the proposed framework; it is fixable by defining a scalarization or by requiring separate ε thresholds for each component, but as written the definitions are not fully rigorous.
minor comments (5)
  1. [§5.1 vs. Definition 4.2] The text in Section 5.1 says coverage is computed by identifying the nearest pair "within the same class," but Definition 4.2 defines f as a mapping to the target dataset without restricting to the same class; these two statements should be reconciled.
  2. [Figure 13] The y-axis label in Figure 13 reads "Measeure" instead of "Measure."
  3. [Table 1] In Table 1, Inv-Alignment appears in both the "Similar Distribution" and "Same Distribution" columns; the table would be clearer if the entries indicated which attacks use which dataset access level exactly, especially because the text states that attacks without model information require data from the same distribution.
  4. [§5.1, Figure 2] The statement that "attacks achieving higher performance at the dataset level also tend to perform well on sample-level metrics" is too strong given Figure 2d, which shows weak correlations (e.g., FID vs. PSNR = 0.117); the text immediately qualifies this, but the summary sentence should not overstate the relationship.
  5. [Section 2.4, Eq. (2)] In Eq. (2), the notation D \ i is used inside the expectation over x_i in D, and the probability is over f_θ ∼ A(D \ i); the composition is clear but the notation could be made more explicit, e.g., by writing the index set or conditioning on the random subset.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the formal definitions, proposed metrics, and benchmark are self-contained, and the minor self-citations appear only in background or related-work contexts.

full rationale

The paper's central contributions are a formal definition of data reconstruction, new quantitative and LLM-based evaluation metrics, and a benchmark of existing attacks. None of these reduce to their own inputs. The quantitative metrics (FID, SSIM, PSNR, MSE, coverage) are defined independently of the attacks being evaluated, and the memorization score is computed from the target model's behavior, not from the reconstruction outputs. The GPT-4o evaluation is an empirical measurement procedure, and while it lacks human validation and arguably conflicts with the paper's own Definition 3.2 because it selects candidate reconstructions using target PSNR values, that is a validity or consistency concern rather than a circularity concern. The paper explicitly acknowledges that its quantitative metrics and the LLM-based evaluation disagree for high-quality reconstructions, which shows the LLM result is not definitionally forced by the metrics. The self-citations (e.g., ML-Doctor, Membership-Doctor, Wen et al.) appear in background and related-work sections and are not load-bearing for the paper's main claims. No fitted parameter is renamed as a prediction, and no uniqueness or ansatz is imported from the authors' prior work. The derivation chain is therefore self-contained, and any weaknesses lie in external validity, not circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on five stated assumptions about the FID metric, the memorization proxy, the LLM judge, the Inception feature space, and nearest-neighbor matching. No free parameters or invented entities are introduced; the paper's contribution is a measurement framework and benchmark rather than a new mechanism.

assumptions (5)
  • standard math FID assumes the feature distributions of both datasets are Gaussian (Definition 4.1).
    The paper approximates Dtar and reconstruction distributions as Gaussians to compute FID; with small datasets this approximation may be poor, so the metric may mis-rank attacks.
  • domain assumption The first 1,000 samples of the training set are a representative proxy for the model's overall memorization (Section 6, Equation 2).
    Model-mem is defined as an expectation over the training distribution, but the implementation uses only the first 1,000 samples; no analysis shows this subset is unbiased.
  • domain assumption GPT-4o's pairwise visual similarity judgments correspond to human-perceived reconstruction quality (Section 6.1).
    The entire memorization-vulnerability conclusion depends on the LLM judge, but no human annotation study or calibration is provided.
  • domain assumption InceptionV3 features are a meaningful distance space for CelebA, CIFAR10, and MNIST reconstructions (Definition 4.1 and Section 4.3).
    FID uses pre-trained InceptionV3 regardless of dataset domain; for faces and digits, the ImageNet-trained features may emphasize different attributes than human perception.
  • standard math Nearest-neighbor matching with the target dataset is an adequate measure of sample-level precision (Definition 4.2).
    S-Dis and coverage assume each reconstruction should be matched to one closest target; duplicate target images or near-duplicates can inflate coverage without true diversity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmark." pith.science (2026). https://pith.science/paper/ZZG7U57Q

@misc{pith2026250607888,
  author       = {Pith},
  title        = {Pith review of: SoK: Data Reconstruction Attacks Against Machine Learning Models: Definition, Metrics, and Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZZG7U57Q}},
  note         = {Machine review of arXiv:2506.07888}
}
read the original abstract

Data reconstruction attacks, which aim to recover the training dataset of a target model with limited access, have gained increasing attention in recent years. However, there is currently no consensus on a formal definition of data reconstruction attacks or appropriate evaluation metrics for measuring their quality. This lack of rigorous definitions and universal metrics has hindered further advancement in this field. In this paper, we address this issue in the vision domain by proposing a unified attack taxonomy and formal definitions of data reconstruction attacks. We first propose a set of quantitative evaluation metrics that consider important criteria such as quantifiability, consistency, precision, and diversity. Additionally, we leverage large language models (LLMs) as a substitute for human judgment, enabling visual evaluation with an emphasis on high-quality reconstructions. Using our proposed taxonomy and metrics, we present a unified framework for systematically evaluating the strengths and limitations of existing attacks and establishing a benchmark for future research. Empirical results, primarily from a memorization perspective, not only validate the effectiveness of our metrics but also offer valuable insights for designing new attacks.

Figures

Figures reproduced from arXiv: 2506.07888 by the authors.

Figure 1
Figure 1. Visualization of existing reconstruction attacks. For each attack, the left two images are reconstructed from the target model [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Relationship between sample-level metrics and the dataset-level metric, we plot the correlation heatmap using the absolute [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Visualization of PLGMI on different target models. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Influence of auxiliary information for PLGMI. Exper [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 7
Figure 7. Figure 7: Counterexamples of accuracy-based metrics. The left examples show that using random-like samples can train a well [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: t-SNE visualization of reconstructions and corresponding target datasets for CelebA. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: t-SNE visualization of reconstructions and corresponding target datasets for CIFAR10. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: t-SNE visualization of reconstructions and corresponding target datasets for MNIST. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Relationship between coverages calculated with different sample-level metrics and the dataset-level metric for CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Relationship between coverages calculated with different sample-level metrics and the dataset-level metric for CelebA, we [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Influence of reconstruction quantity. For ease of presentation, values are normalized to fit the same y-axis. [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: More results on the influence of auxiliary information. [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 66 canonical work pages

  1. [4]

    Deep Learning with Differential Privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. InACM SIGSAC Conference on Computer and Communica- tions Security (CCS), pages 308–318. ACM, 2016. 4, 20

    work page 2016

  2. [5]

    Mancini, Angelo Spog- nardi, Antonio Villani, Domenico Vitali, and Giovanni Felici

    Giuseppe Ateniese, Luigi V . Mancini, Angelo Spog- nardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers.Int. J. Secur. Networks, 2015. 22

    work page 2015

  3. [6]

    Re- constructing Training Data with Informed Adversaries

    Borja Balle, Giovanni Cherubin, and Jamie Hayes. Re- constructing Training Data with Informed Adversaries. InIEEE Symposium on Security and Privacy (S&P), pages 1138–1156. IEEE, 2022. 3

    work page 2022

  4. [7]

    Semi-supervised Learning with Constraints for Person Identification in Multimedia Data

    Martin Bäuml, Makarand Tapaswi, and Rainer Stiefel- hagen. Semi-supervised Learning with Constraints for Person Identification in Multimedia Data. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 3602–3609. IEEE, 2013. 1

    work page 2013

  5. [8]

    Jens Behrmann, Will Grathwohl, Ricky T. Q. Chen, David Duvenaud, and Jörn-Henrik Jacobsen. Invert- ible Residual Networks. InInternational Conference on Machine Learning (ICML), pages 573–582. PMLR,

    work page

  6. [9]

    Membership Inference Attacks From First Principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership Inference Attacks From First Principles. InIEEE Sym- posium on Security and Privacy (S&P), pages 1897–

  7. [10]

    The Privacy Onion Effect: Memorization is Relative

    Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramèr. The Privacy Onion Effect: Memorization is Relative. In Annual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2022. 4, 10

    work page 2022

  8. [11]

    Brown, Dawn Song, Úl- far Erlingsson, Alina Oprea, and Colin Raffel

    Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-V oss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úl- far Erlingsson, Alina Oprea, and Colin Raffel. Ex- tracting Training Data from Large Language Models. InUSENIX Security Symposium (USENIX Security), pages 2633–2650. USENIX, 2021. 13, 20, 21

    work page 2021

Show all 78 references
  1. [12]

    Big Data Analysis for Financial Risk Management.Journal of Big Data,

    Paola Cerchiello and Paolo Giudici. Big Data Analysis for Financial Risk Management.Journal of Big Data,

  2. [13]

    Harsh Chaudhari, John Abascal, Alina Oprea, Matthew Jagielski, Florian Tramèr, and Jonathan R. Ullman. SNAP: Efficient Extraction of Private Properties with Poisoning. InIEEE Symposium on Security and Pri- vacy (S&P), pages 1935–1952. IEEE, 2023. 22

    1935

  3. [14]

    Knowledge-Enriched Distributional Model Inversion Attacks

    Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi. Knowledge-Enriched Distributional Model Inversion Attacks. InIEEE International Conference on Com- puter Vision (ICCV), pages 16158–16167. IEEE, 2021. 2, 3, 6

    2021

  4. [15]

    Choquette Choo, Florian Tramèr, Nicholas Carlini, and Nicolas Papernot

    Christopher A. Choquette Choo, Florian Tramèr, Nicholas Carlini, and Nicolas Papernot. Label-Only Membership Inference Attacks. InInternational Con- ference on Machine Learning (ICML), pages 1964–

    1964

  5. [16]

    Deep Learning for Classical Japanese Literature.CoRR abs/1812.01718, 2018

    Tarin Clanuwat, Mikel Bober-Irizar, Asanobu Kita- moto, Alex Lamb, Kazuaki Yamamoto, and David Ha. Deep Learning for Classical Japanese Literature.CoRR abs/1812.01718, 2018. 12

    2018 arXiv

  6. [17]

    On the Privacy Risk of In-context Learning

    Haonan Duan, Adam Dziedzic, Mohammad Yaghini, Nicolas Papernot, and Franziska Boenisch. On the Privacy Risk of In-context Learning. InWorkshop on Trustworthy Natural Language Processing (TrustNLP),

  7. [18]

    Now Publishers Inc., 2014

    Cynthia Dwork and Aaron Roth.The Algorithmic Foundations of Differential Privacy. Now Publishers Inc., 2014. 4

    2014

  8. [19]

    Does Learning Require Memoriza- tion? A Short Tale about a Long Tail

    Vitaly Feldman. Does Learning Require Memoriza- tion? A Short Tale about a Long Tail. InAnnual ACM Symposium on Theory of Computing (STOC), pages 954–959. ACM, 2020. 4

    2020

  9. [20]

    Model Inversion Attacks that Exploit Confidence Infor- mation and Basic Countermeasures

    Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model Inversion Attacks that Exploit Confidence Infor- mation and Basic Countermeasures. InACM SIGSAC Conference on Computer and Communications Secu- rity (CCS), pages 1322–1333. ACM, 2015. 1, 2, 5, 6, 7, 18

    2015

  10. [21]

    Gunter, and Nikita Borisov

    Karan Ganju, Qi Wang, Wei Yang, Carl A. Gunter, and Nikita Borisov. Property Inference Attacks on Fully Connected Neural Networks using Permutation Invari- ant Representations. InACM SIGSAC Conference on Computer and Communications Security (CCS), pages 619–633. ACM, 2018. 22

    2018

  11. [22]

    Generative Adversar- ial Nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative Adversar- ial Nets. InAnnual Conference on Neural Information Processing Systems (NIPS), pages 2672–2680. NIPS,

  12. [23]

    Reconstructing Training Data from Trained Neural Networks

    Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing Training Data from Trained Neural Networks. InAnnual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2022. 2, 3 15

    2022

  13. [24]

    Song Han, Jeff Pool, John Tran, and William J. Dally. Learning both Weights and Connections for Efficient Neural Networks.CoRR abs/1506.02626, 2015. 13

    2015 arXiv

  14. [25]

    Girshick

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked Autoen- coders Are Scalable Vision Learners. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 15979–15988. IEEE, 2022. 13, 21

    2022

  15. [26]

    Deep Residual Learning for Image Recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. IEEE, 2016. 19

    2016

  16. [27]

    Membership-Doctor: Comprehensive As- sessment of Membership Inference Against Machine Learning Models.CoRR abs/2208.10445, 2022

    Xinlei He, Zheng Li, Weilin Xu, Cory Cornelius, and Yang Zhang. Membership-Doctor: Comprehensive As- sessment of Membership Inference Against Machine Learning Models.CoRR abs/2208.10445, 2022. 1

    2022 arXiv

  17. [28]

    Node-Level Membership Infer- ence Attacks Against Graph Neural Networks.CoRR abs/2102.05429, 2021

    Xinlei He, Rui Wen, Yixin Wu, Michael Backes, Yun Shen, and Yang Zhang. Node-Level Membership Infer- ence Attacks Against Graph Neural Networks.CoRR abs/2102.05429, 2021. 1, 2

    2021 arXiv

  18. [29]

    Sanjay Kariyappa, Atul Prakash, and Moinuddin K. Qureshi. MAZE: Data-Free Model Stealing Attack Us- ing Zeroth-Order Gradient Estimation. InIEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), pages 13814–13823. IEEE, 2021. 22

    2021

  19. [30]

    A Style- Based Generator Architecture for Generative Adversar- ial Networks

    Tero Karras, Samuli Laine, and Timo Aila. A Style- Based Generator Architecture for Generative Adversar- ial Networks. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4401–4410. IEEE, 2019. 12

    2019

  20. [31]

    Kingma and Prafulla Dhariwal

    Diederik P. Kingma and Prafulla Dhariwal. Glow: Generative Flow with Invertible 1x1 Convolutions. In Annual Conference on Neural Information Process- ing Systems (NeurIPS), pages 10236–10245. NeurIPS,

  21. [32]

    Application of Machine Learning in Disease Prediction

    Pahulpreet Singh Kohli and Shriya Arora. Application of Machine Learning in Disease Prediction. InInterna- tional Conference on Computing Communication and Automation (ICCCA), pages 1–4. IEEE, 2018. 1

    2018

  22. [33]

    Se- qMIA: Sequential-Metric Based Membership Inference Attack.CoRR abs/2407.15098, 2024

    Hao Li, Zheng Li, Siyuan Wu, Chengrui Hu, Yutong Ye, Min Zhang, Dengguo Feng, and Yang Zhang. Se- qMIA: Sequential-Metric Based Membership Inference Attack.CoRR abs/2407.15098, 2024. 1

    2024 arXiv

  23. [34]

    Mem- bership Inference Attack Against Masked Image Mod- eling.CoRR abs/2408.06825, 2024

    Zheng Li, Xinlei He, Ning Yu, and Yang Zhang. Mem- bership Inference Attack Against Masked Image Mod- eling.CoRR abs/2408.06825, 2024. 1

    2024 arXiv

  24. [35]

    Auditing Membership Leak- ages of Multi-Exit Networks

    Zheng Li, Yiyong Liu, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. Auditing Membership Leak- ages of Multi-Exit Networks. InACM SIGSAC Con- ference on Computer and Communications Security (CCS), pages 1917–1931. ACM, 2022. 1, 2

    1917

  25. [36]

    Membership Leakage in Label-Only Exposures

    Zheng Li and Yang Zhang. Membership Leakage in Label-Only Exposures. InACM SIGSAC Conference on Computer and Communications Security (CCS), pages 880–895. ACM, 2021. 1, 2, 22

    2021

  26. [37]

    PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps

    Ruixuan Liu, Tianhao Wang, Yang Cao, and Li Xiong. PreCurious: How Innocent Pre-Trained Language Models Turn into Privacy Traps. InACM SIGSAC Conference on Computer and Communications Secu- rity (CCS), pages 3511–3524. ACM, 2024. 13

    2024

  27. [38]

    Membership Inference Attacks by Exploit- ing Loss Trajectory

    Yiyong Liu, Zhengyu Zhao, Michael Backes, and Yang Zhang. Membership Inference Attacks by Exploit- ing Loss Trajectory. InACM SIGSAC Conference on Computer and Communications Security (CCS), pages 2085–2098. ACM, 2022. 2, 22

    2022

  28. [39]

    ML-Doctor: Holis- tic Risk Assessment of Inference Attacks Against Ma- chine Learning Models

    Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristo- faro, Mario Fritz, and Yang Zhang. ML-Doctor: Holis- tic Risk Assessment of Inference Attacks Against Ma- chine Learning Models. InUSENIX Security Sympo- sium (USENIX Security), page...

  29. [40]

    Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. InIEEE International Conference on Computer Vision (ICCV), pages 9992–10002. IEEE,

  30. [41]

    Deep Learning Face Attributes in the Wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep Learning Face Attributes in the Wild. InIEEE International Conference on Computer Vision (ICCV), pages 3730–3738. IEEE, 2015. 18

    2015

  31. [42]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y . Ng, and Christopher Potts. Learning Word Vectors for Sentiment Analysis. InAn- nual Meeting of the Association for Computational Lin- guistics (ACL), pages 142–150. ACL, 2011. 20

    2011

  32. [43]

    Property Inference from Poisoning

    Saeed Mahloujifar, Esha Ghosh, and Melissa Chase. Property Inference from Poisoning. InIEEE Sympo- sium on Security and Privacy (S&P), pages 1120–1137. IEEE, 2022. 22

    2022

  33. [44]

    Exploiting Unintended Feature Leakage in Collaborative Learning

    Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting Unintended Feature Leakage in Collaborative Learning. InIEEE Sympo- sium on Security and Privacy (S&P), pages 497–512. IEEE, 2019. 1

    2019

  34. [45]

    Morris, V olodymyr Kuleshov, Vitaly Shmatikov, and Alexander M

    John X. Morris, V olodymyr Kuleshov, Vitaly Shmatikov, and Alexander M. Rush. Text Em- beddings Reveal (Almost) As Much As Text. In Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 12448–12460. Association for Computational Linguistics, 2023. 13, 20

    2023

  35. [46]

    Scalable Private Learning with PATE

    Nicolas Papernot, Shuang Song, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Úlfar Erlingsson. Scalable Private Learning with PATE. InInternational Conference on Learning Representations (ICLR), 2018. 4 16

    2018

  36. [47]

    Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learn- ing

    Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learn- ing. InUSENIX Security Symposium (USENIX Secu- rity), pages 1291–1308. USENIX, 2020. 1, 2, 3, 5, 6

    2020

  37. [48]

    SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning

    Ahmed Salem, Giovanni Cherubin, David Evans, Boris Köpf, Andrew Paverd, Anshuman Suri, Shruti Tople, and Santiago Zanella Béguelin. SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning. InIEEE Symposium on Security and Privacy (S&...

  38. [49]

    ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In Network and Distributed System Security Symposium (NDSS). Internet Society, ...

    2019

  39. [50]

    Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen

    Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mo- bileNetV2: Inverted Residuals and Linear Bottlenecks. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4510–4520. IEEE, 2018. 19

    2018

  40. [51]

    Venkatesh Babu

    Sunandini Sanyal, Sravanti Addepalli, and R. Venkatesh Babu. Towards Data-Free Model Stealing in a Hard Label Setting.CoRR abs/2204.11022, 2022. 22

    2022 arXiv

  41. [52]

    Membership Inference Attacks Against Machine Learning Models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vi- taly Shmatikov. Membership Inference Attacks Against Machine Learning Models. InIEEE Symposium on Se- curity and Privacy (S&P), pages 3–18. IEEE, 2017. 1, 2, 22

    2017

  42. [53]

    Very Deep Convolutional Networks for Large-Scale Image Recog- nition

    Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recog- nition. InInternational Conference on Learning Rep- resentations (ICLR), 2015. 19

    2015

  43. [54]

    Manning, Andrew Y

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y . Ng, and Christopher Potts. Recursive Deep Models for Seman- tic Compositionality Over a Sentiment Treebank. In Conference on Empirical Methods in Natural Language Processing (EMNLP), page...

    2013

  44. [55]

    Machine Learning Models that Remember Too Much

    Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. Machine Learning Models that Remember Too Much. InACM SIGSAC Conference on Computer and Communications Security (CCS), pages 587–601. ACM, 2017. 22

    2017

  45. [56]

    Overlearning Reveals Sensitive Attributes

    Congzheng Song and Vitaly Shmatikov. Overlearning Reveals Sensitive Attributes. InInternational Confer- ence on Learning Representations (ICLR), 2020. 22

    2020

  46. [57]

    Mint- Net: Building Invertible Neural Networks with Masked Convolutions

    Yang Song, Chenlin Meng, and Stefano Ermon. Mint- Net: Building Invertible Neural Networks with Masked Convolutions. InAnnual Conference on Neural Infor- mation Processing Systems (NeurIPS), pages 11002– 11012. NeurIPS, 2019. 3

    2019

  47. [58]

    Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks

    Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correia, Antonia Adler, and Kristian Kersting. Plug & Play Attacks: Towards Robust and Flexible Model Inversion Attacks. InInternational Conference on Machine Learning (ICML), pages 20522–20545. PMLR, 2022. 2, 5, 6

    2022

  48. [59]

    Formalizing and Estimating Distribution Inference Risks.CoRR abs/2109.06024, 2021

    Anshuman Suri and David Evans. Formalizing and Estimating Distribution Inference Risks.CoRR abs/2109.06024, 2021. 22

    2021 arXiv

  49. [60]

    Deep Learn- ing and the Information Bottleneck Principle.CoRR abs/1503.02406, 2015

    Naftali Tishby and Noga Zaslavsky. Deep Learn- ing and the Information Bottleneck Principle.CoRR abs/1503.02406, 2015. 11

    2015 arXiv

  50. [61]

    Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets

    Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets. InACM SIGSAC Conference on Computer and Communica- tions Security (CCS). ACM, 2022. 4, 10, 22

    2022

  51. [62]

    Re- iter, and Thomas Ristenpart

    Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Re- iter, and Thomas Ristenpart. Stealing Machine Learn- ing Models via Prediction APIs. InUSENIX Secu- rity Symposium (USENIX Security), pages 601–618. USENIX, 2016. 22

    2016

  52. [63]

    Walls, and Nicolas Papernot

    Jean-Baptiste Truong, Pratyush Maini, Robert J. Walls, and Nicolas Papernot. Data-Free Model Extraction. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4771–4780. IEEE, 2021. 6, 12, 22

    2021

  53. [64]

    Zemel, and Alireza Makhzani

    Kuan-Chieh Wang, Yan Fu, Ke Li, Ashish Khisti, Richard S. Zemel, and Alireza Makhzani. Varia- tional Model Inversion Attacks. InAnnual Conference on Neural Information Processing Systems (NeurIPS), pages 9706–9719. NeurIPS, 2021. 2, 6

    2021

  54. [65]

    Improv- ing Robustness to Model Inversion Attacks via Mu- tual Information Regularization

    Tianhao Wang, Yuheng Zhang, and Ruoxi Jia. Improv- ing Robustness to Model Inversion Attacks via Mu- tual Information Regularization. InAAAI Conference on Artificial Intelligence (AAAI), pages 11666–11673. AAAI, 2021. 20

    2021

  55. [66]

    Under- standing Data Importance in Machine Learning At- tacks: Does Valuable Data Pose Greater Harm? In Network and Distributed System Security Symposium (NDSS)

    Rui Wen, Michael Backes, and Yang Zhang. Under- standing Data Importance in Machine Learning At- tacks: Does Valuable Data Pose Greater Harm? In Network and Distributed System Security Symposium (NDSS). Internet Society, 2025. 22

    2025

  56. [67]

    Membership Inference Attacks Against In- Context Learning

    Rui Wen, Zheng Li, Michael Backes, and Yang Zhang. Membership Inference Attacks Against In- Context Learning. InACM SIGSAC Conference on Computer and Communications Security (CCS). ACM,

  57. [68]

    Last One Standing: A Com- parative Analysis of Security and Privacy of Soft 17 Prompt Tuning, LoRA, and In-Context Learning.CoRR abs/2310.11397, 2023

    Rui Wen, Tianhao Wang, Michael Backes, Yang Zhang, and Ahmed Salem. Last One Standing: A Com- parative Analysis of Security and Privacy of Soft 17 Prompt Tuning, LoRA, and In-Context Learning.CoRR abs/2310.11397, 2023. 20

    2023 arXiv

  58. [69]

    Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models

    Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini. Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models. InAnnual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2024. 13

    2024

  59. [70]

    Quanti- fying Privacy Risks of Prompts in Visual Prompt Learn- ing

    Yixin Wu, Rui Wen, Michael Backes, Pascal Berrang, Mathias Humbert, Yun Shen, and Yang Zhang. Quanti- fying Privacy Risks of Prompts in Visual Prompt Learn- ing. InUSENIX Security Symposium (USENIX Secu- rity). USENIX, 2024. 1

    2024

  60. [71]

    Neural Network Inversion in Adversarial Set- ting via Background Knowledge Alignment

    Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural Network Inversion in Adversarial Set- ting via Background Knowledge Alignment. InACM SIGSAC Conference on Computer and Communica- tions Security (CCS), page 225–240. ACM, 2019. 1, 2, 3, 5, 6

    2019

  61. [72]

    Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven C. H. Hoi. Deep Learning for Person Re-Identification: A Survey and Outlook.IEEE Trans- actions on Pattern Analysis and Machine Intelligence,

  62. [73]

    Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K

    Hongxu Yin, Pavlo Molchanov, Jose M. Alvarez, Zhizhong Li, Arun Mallya, Derek Hoiem, Niraj K. Jha, and Jan Kautz. Dreaming to Distill: Data-Free Knowl- edge Transfer via DeepInversion. InIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8712–8721. IEEE, ...

    2020

  63. [74]

    Pseudo Label- Guided Model Inversion Attack via Conditional Gen- erative Adversarial Network

    Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang. Pseudo Label- Guided Model Inversion Attack via Conditional Gen- erative Adversarial Network. InAAAI Conference on Artificial Intelligence (AAAI). AAAI, 2023. 3

    2023

  64. [75]

    SecretGen: Privacy Recovery on Pre-trained Models via Distribution Discrimination

    Zhuowen Yuan, Fan Wu, Yunhui Long, Chaowei Xiao, and Bo Li. SecretGen: Privacy Recovery on Pre-trained Models via Distribution Discrimination. InEuropean Conference on Computer Vision (ECCV), pages 139–

  65. [76]

    Generated Distributions Are All You Need for Membership Inference Attacks Against Gen- erative Models

    Minxing Zhang, Ning Yu, Rui Wen, Michael Backes, and Yang Zhang. Generated Distributions Are All You Need for Membership Inference Attacks Against Gen- erative Models. InWinter Conference on Applications of Computer Vision (WACV), pages 4827–4837. IEEE,

  66. [77]

    Character-level Convolutional Networks for Text Clas- sification

    Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level Convolutional Networks for Text Clas- sification. InAnnual Conference on Neural Informa- tion Processing Systems (NIPS), pages 649–657. NIPS,

  67. [78]

    The Secret Re- vealer: Generative Model-Inversion Attacks Against Deep Neural Networks

    Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The Secret Re- vealer: Generative Model-Inversion Attacks Against Deep Neural Networks. InIEEE Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 250–258. IEEE, 2020. 1, 2, 3, 5, 6

    2020

  68. [79]

    poi- soned

    Ligeng Zhu, Zhijian Liu, and Song Han. Deep Leak- age from Gradients. InAnnual Conference on Neu- ral Information Processing Systems (NeurIPS), pages 14747–14756. NeurIPS, 2019. 1, 2, 3, 5, 6 A Counterexamples We generate counterexamples in Figure 7a which contribute a lot to ...

    2019

  69. [155]

    Springer, 2022. 5, 6

    2022

  70. [1914]

    2, 3, 22

    IEEE, 2022. 2, 3, 22

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.