REVIEW 7 cited by
On the Importance of Difficulty Calibration in Membership Inference Attacks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
On the Importance of Difficulty Calibration in Membership Inference Attacks
read the original abstract
The vulnerability of machine learning models to membership inference attacks has received much attention in recent years. However, existing attacks mostly remain impractical due to having high false positive rates, where non-member samples are often erroneously predicted as members. This type of error makes the predicted membership signal unreliable, especially since most samples are non-members in real world applications. In this work, we argue that membership inference attacks can benefit drastically from \emph{difficulty calibration}, where an attack's predicted membership score is adjusted to the difficulty of correctly classifying the target sample. We show that difficulty calibration can significantly reduce the false positive rate of a variety of existing attacks without a loss in accuracy.
Forward citations
Cited by 7 Pith papers
-
A Unified Perspective on Adversarial Membership Manipulation in Vision Models
Adversarial perturbations reliably fabricate membership signals in vision-model MIAs, separated by a gradient-norm collapse trajectory that enables robust detection and inference.
-
Reference-Based Distillation Detection in LLMs
Reference-based membership inference recovers the true teacher of a distilled LLM with near-perfect accuracy when an earlier same-lineage checkpoint is available.
-
Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models
Entity-level membership inference determines whether information about a target real-world entity was used in LLM training, using only black-box generated text and achieving AUC up to 0.97 on person entities.
-
DCMI: A Differential Calibration Membership Inference Attack Against Retrieval-Augmented Generation
DCMI infers RAG database membership by subtracting the system's yes-probability on a perturbed query from the original query, cancelling the interference of non-member retrieved documents.
-
The False Promise of Imitating Proprietary LLMs
Finetuning open LMs on ChatGPT outputs creates models that mimic style and fool human raters but fail to close the performance gap to proprietary systems on tasks not well-represented in the imitation data.
-
Generalization and Membership Inference Attack a Practical Perspective
Advanced generalization techniques such as augmentation and early stopping can reduce membership inference attack success rates by up to 100 times, confirmed across more than 1,000 models.
-
Membership Inference Attacks with False Discovery Rate Control
A post-hoc wrapper, MIAFdR, converts any membership inference attack scores into conformal p-values and applies a Benjamini-Hochberg correction, guaranteeing that the expected proportion of non-members among flagged m...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.