The reviewed record of science sign in
Pith

arxiv: 2111.08440 · v2 · pith:IGD6SV6J · submitted 2021-11-15 · cs.CR · cs.LG

On the Importance of Difficulty Calibration in Membership Inference Attacks

Lauren Watson , Chuan Guo , Graham Cormode , Alex Sablayrolles This is my paper

Reviewed by Pithpith:IGD6SV6Jopen to challenge →

classification cs.CR cs.LG
keywords attacksmembershipdifficultycalibrationinferencepredictedexistingfalse
0
0 comments X
read the original abstract

The vulnerability of machine learning models to membership inference attacks has received much attention in recent years. However, existing attacks mostly remain impractical due to having high false positive rates, where non-member samples are often erroneously predicted as members. This type of error makes the predicted membership signal unreliable, especially since most samples are non-members in real world applications. In this work, we argue that membership inference attacks can benefit drastically from \emph{difficulty calibration}, where an attack's predicted membership score is adjusted to the difficulty of correctly classifying the target sample. We show that difficulty calibration can significantly reduce the false positive rate of a variety of existing attacks without a loss in accuracy.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. A Unified Perspective on Adversarial Membership Manipulation in Vision Models

    cs.CV 2026-04 conditional novelty 8.0

    Adversarial perturbations reliably fabricate membership signals in vision-model MIAs, separated by a gradient-norm collapse trajectory that enables robust detection and inference.

  2. Have You Ever Seen Them? Entity-level Membership Inference through Interrogating Large Language Models

    cs.CL 2026-06 unverdicted novelty 7.0

    Entity-level membership inference determines whether information about a target real-world entity was used in LLM training, using only black-box generated text and achieving AUC up to 0.97 on person entities.

  3. The False Promise of Imitating Proprietary LLMs

    cs.CL 2023-05 conditional novelty 6.0

    Finetuning open LMs on ChatGPT outputs creates models that mimic style and fool human raters but fail to close the performance gap to proprietary systems on tasks not well-represented in the imitation data.

  4. Generalization and Membership Inference Attack a Practical Perspective

    cs.LG 2026-04 unverdicted novelty 4.0

    Advanced generalization techniques such as augmentation and early stopping can reduce membership inference attack success rates by up to 100 times, confirmed across more than 1,000 models.