ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:7EQGDF6Brecord.jsonopen to challenge →
read the original abstract
Machine learning (ML) has become a core component of many real-world applications and training data is a key factor that drives current progress. This huge success has led Internet companies to deploy machine learning as a service (MLaaS). Recently, the first membership inference attack has shown that extraction of information on the training set is possible in such MLaaS settings, which has severe security and privacy implications. However, the early demonstrations of the feasibility of such attacks have many assumptions on the adversary, such as using multiple so-called shadow models, knowledge of the target model structure, and having a dataset from the same distribution as the target model's training data. We relax all these key assumptions, thereby showing that such attacks are very broadly applicable at low cost and thereby pose a more severe risk than previously thought. We present the most comprehensive study so far on this emerging and developing threat using eight diverse datasets which show the viability of the proposed attacks across domains. In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.
This paper has not been read by Pith yet.
Forward citations
Cited by 19 Pith papers
-
Cross-Flow Correlations Survive Synthesis: Measuring Source-Level Privacy Leakage in Synthetic Network Traces
Synthetic network generators preserve cross-flow correlations enabling source-level membership inference, shown via the TraceBleed attack across five datasets and six generators.
-
Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models
Entity-level membership inference determines whether information about a target real-world entity was used in LLM training, using only black-box generated text and achieving AUC up to 0.97 on person entities.
-
MRMMIA: Membership Inference Attacks on Memory in Chat Agents
MRMMIA is a multi-recall-probe membership inference attack that extracts signals from chat agent memory and outperforms baselines in black-, gray-, and white-box settings.
-
Single-Sample Black-Box Membership Inference Attack against Vision-Language Models via Cross-modal Semantic Alignment
A cross-modal alignment attack achieves AUC 0.821 for single-sample black-box membership inference on VLMs such as LLaVA-1.5 by quantifying image-generated caption similarity.
-
ReMIA: a Powerful and Efficient Alternative to Membership Inference Attacks against Synthetic Data Generators
ReMIA offers a practical privacy metric for synthetic data by training two generators and using a classifier to detect source dataset membership, achieving sensitivity comparable to standard MIAs with far less computation.
-
DistractMIA: Black-Box Membership Inference on Vision-Language Models via Semantic Distraction
DistractMIA performs output-only black-box membership inference on vision-language models by inserting semantic distractors and measuring shifts in generated text responses.
-
Revisiting Privacy Leakage in Machine Unlearning: Membership Inference Beyond the Forgotten Set
TC-UMIA is a population-level attack using pre- and post-unlearning predictions to infer membership across forget, retain, and unseen sets, revealing added privacy leakage to retained data.
-
Revisiting Privacy Leakage in Machine Unlearning: Membership Inference Beyond the Forgotten Set
Unlearning increases privacy leakage for the retain set, and a new tri-class membership inference attack distinguishes forget, retain, and unseen data using pre- and post-unlearning model outputs.
-
Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models
Introduces noise aggregation analysis with single-step small-noise injection to enable efficient and accurate membership inference attacks on diffusion models.
-
Bayesian Membership Privacy for Graph Neural Networks
Introduces Bayesian Membership Privacy (BMP) as a sampling-aware node-level privacy definition for GNNs quantified by posterior membership probability, plus an auditing method and benchmark experiments.
-
idSCD: Identifying Training Datasets through Semantic Correlation Descriptors
idSCD uses semantic correlation descriptors to perform dataset membership inference by comparing learned semantic structures, outperforming baselines in NLI, emotion, and medical text experiments.
-
A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning
Presents a systematic framework for evaluating MIAs across the full ML pipeline with standardized threat models and complementary metrics for different cost scenarios.
-
On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference
An attack aligns differently shuffled intermediate activations from secure Transformer inference queries to recover model weights with low error using roughly one dollar of queries.
-
A Dual Perspective on Synthetic Trajectory Generators: Utility Framework and Privacy Vulnerabilities
A new framework evaluates utility of synthetic mobility trajectories while a membership inference attack reveals privacy vulnerabilities in generative models thought to be safe.
-
Towards Reliable Testing of Machine Unlearning
Causal fuzzing with budgeted interventions can detect residual direct and indirect influence of unlearned data that standard attribution methods miss due to proxies, cancellations, and masking.
-
Jellyfish: Zero-Shot Federated Unlearning Scheme with Knowledge Disentanglement
Jellyfish enables zero-shot federated unlearning through synthetic proxy data generation, channel-restricted knowledge disentanglement, and a composite loss with repair to forget target data while retaining model utility.
-
Mitigating Membership Inference in Intermediate Representations with Differentially Private Training
LM-DP-SGD estimates layer-specific MIA risks from shadow models and reweights gradients to give stronger protection to vulnerable layers, improving the privacy-utility trade-off over uniform DP-SGD.
-
Revisiting Privacy Amplification by Subsampling in Selective Release DPSGD
DPSR-CG corrects the privacy accounting for selective release in DPSGD by addressing sampling probability variation and reports strong empirical results on MNIST, CIFAR-10, IMDB, and FMNIST while claiming strict privacy.
-
Forgetting to Witness: Efficient Federated Unlearning and Its Visible Evaluation
A complete pipeline for federated unlearning via knowledge distillation for efficient removal and a GAN-integrated classifier for visual evaluation of forgetting capacity.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.