Pith. sign in

REVIEW 5 major objections 5 minor 82 references

Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Targeted attacks expose hidden privacy leaks in inexact unlearning.

desk verdict RULI's privacy attack is a real contribution, but the efficacy measurement is not implemented as described—the dual-objective claim doesn't survive Algorithm 1. read the letter →

arxiv 2506.13009 v1 pith:BO7CPVTP submitted 2025-06-16 cs.CR cs.LG

Nima Naderloui , Shenao Yan , Binghui Wang , Jie Fu , Wendy Hui Wang , Weiran Liu , Yuan Hong This is my paper · ORCID
classification cs.CRcs.LG
keywords machineunlearningmembershipinferenceattackprivacyleakageefficacylikelihoodratiotesttargetedcanarysamplesshadowmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that standard ways of measuring whether machine unlearning worked are too weak. Existing evaluations mostly average membership-inference performance over random samples and compare aggregate accuracy to a retrained model; RULI instead runs per-sample likelihood-ratio tests on targeted, highly memorized 'vulnerable' samples. On CIFAR-10/100, TinyImageNet, and WikiText-103 the targeted attacks succeed far more often, which the authors take to show that privacy leakage has been systematically underestimated and that most inexact unlearning methods do not closely approximate retraining.

What carries the argument

The load-bearing object is a pair of likelihood ratios built from shadow-model distributions. Privacy leakage is $\Lambda(z) = p(\theta_U | Q_u(z)) / p(\theta_U | Q_h(z))$, comparing the unlearned model's output to distributions from samples that were included then unlearned versus samples never trained on; efficacy is $\Psi(z) = p(\theta_T | Q_u(z)) / p(\theta_T | Q_R(z))$, where $\theta_T$ is a test model that returns unlearned-model output for unlearned samples and retrained-model output for held-out ones. Kernel density estimation smooths the confidence-score distributions, and targets are selected as 'vulnerable' samples identified by high membership likelihood on the original model, injected as canaries into the forget set.

What would settle it

Run RULI twice on the same unlearning method: once targeting samples ranked vulnerable by the original model, once targeting samples re-ranked by the unlearned model. If the second run finds no elevation over random-sample attacks, the reported privacy leakage is an artifact of target selection rather than a property of unlearning.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that inexact unlearning should be evaluated per sample, on the samples most likely to be memorized, using two distinct likelihood-ratio tests: one for privacy leakage that compares an unlearned query's confidence score against held-out (never-trained) shadow distributions, and one for efficacy that compares it against distributions from models retrained without the sample. Applied to Scrub, gradient ascent, negative-gradient, and sparsity-based unlearning, this reveals TPR@1% FPR far above average-case attacks and above U-LiRA, and shows that unlearned models remain distinguishable from retrained models on individual samples even when aggregate accuracy matches. The authors conclude that prior privacy claims for these unlearning methods gave a false sense of privacy.

Load-bearing premise

The attack assumes that the samples most vulnerable to membership inference before unlearning are still the samples most vulnerable after unlearning; if unlearning reorders which samples are memorized, RULI measures only the pre-selected set and could miss the true worst-case leakage.

Editorial extensions

If this is right

  • Privacy reports for inexact unlearning that rely on average-case MIAs should be read as lower bounds; RULI shows the true leakage can be several times higher and is concentrated on specific samples.
  • Unlearning efficacy cannot be read off from accuracy deltas; per-sample test models separate 'forgot the requested sample' from 'behaves like retraining', and most methods fail that test on vulnerable samples.
  • Target selection matters more than the attack itself: swapping U-LiRA's random class-based targets for vulnerable canaries raises its TPR@1% FPR on Scrub from 1.13% to 8.6%.
  • The methodology transfers to text generation: on WikiText-103 7-gram unlearning, RULI reaches up to 54% TPR@1% FPR for privacy leakage and up to 90% attack accuracy for efficacy on language models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If RULI's findings generalize, unlearning benchmarks should report per-sample worst-case or canary-based privacy metrics alongside accuracy, since aggregate accuracy masks order-of-magnitude differences in sample-level leakage.
  • The paper's own observation that unlearning degrades accuracy on remaining vulnerable samples implies a new failure mode: inexact unlearning can over-forget, destroying memorization that was never requested to be removed, a cost invisible to average-case accuracy.
  • A natural next test is whether the vulnerability ranking itself is stable: re-ranking samples on the unlearned model and seeing whether RULI's success persists would clarify whether the reported leakage is a property of the unlearned model or an artifact of the pre-unlearning selection.
  • Since RULI can distinguish unlearned from retrained outputs, it offers an empirical audit tool for certified unlearning claims: if a certified method's guaranteed region cannot be distinguished by RULI on targeted samples, that is evidence the certificate is meaningful in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes RULI, a framework for evaluating machine unlearning through per-sample likelihood-ratio inference attacks. It identifies three pitfalls in existing evaluations: average-case metrics, random target selection, and incomplete comparisons with retraining. RULI introduces two games, Game 2 (targeted privacy MIA) and Game 3 (efficacy MIA via a test model), and reports experiments on CIFAR-10/100, TinyImageNet, and WikiText-103 showing higher attack success rates than existing baselines. The central claim is that average-case, random-sample evaluations substantially underestimate privacy leakage, and that most inexact unlearning methods do not closely approximate retraining.

Significance. If the claims hold, this would be a valuable contribution to machine unlearning evaluation. The paper correctly identifies that average-case and random-sample evaluations can miss per-sample vulnerabilities, and the targeted canary-injection perspective is a meaningful step. The conceptual separation of privacy (unlearned vs held-out) from efficacy (unlearned vs retrained) is important, and the authors provide code and broad empirical coverage. However, the current manuscript does not fully establish these claims because the reported efficacy measurement, as described in Algorithm 1 and Game 3, does not match the formal test model defined in Eq. (2), and the headline quantitative results lack error bars or an explicit multi-run protocol. The target-selection stability issue further weakens the generality of the privacy-leakage claims.

major comments (5)
  1. [Section 4.3.2, Algorithm 1] Algorithm 1 does not implement the test model of Eq. (2) and Game 3. Line 23, 'f_T <- Equation 2', is not an executable assignment, and line 24 computes O_fT <- phi(f_U(z)) for every target sample, never querying theta_R. Consequently, the statistic Psi in line 26 equals p(theta_U(z)|Qu)/p(theta_U(z)|QR), which is exactly the U-LiRA-style Qu-vs-QR comparison that the paper says it supersedes. The held-out branch of Eq. (2) is therefore unused. The efficacy results in Figures 5, 6, and 8 and Table 4 do not test the indistinguishability between unlearned and retrained models as defined, so the central claim that RULI provides a dual-objective measurement is currently unsupported. Please correct the pseudocode to branch on target membership and query theta_R for held-out samples, or provide evidence from the released code that the actual implementation follows Eq. (2).
  2. [Section 4.1, Game 3] Game 3 as written is inconsistent with Eq. (2) and with the surrounding text. In step 4, the tail branch specifies that the query result is f_theta_I(·) (the original trained model), whereas Eq. (2) and Section 4.3.1 define the held-out branch as f_theta_R(·) (the retrained model). These are different distributions unless theta_I and theta_R coincide, which is precisely the property that efficacy is meant to test. The game therefore formalizes a privacy-like comparison between unlearned and original models, not the claimed efficacy comparison between unlearned and retrained models. This needs to be reconciled before the theoretical foundation can support the experimental claims.
  3. [Table 2 caption] The caption states: 'Inferences are performed in a single training and unlearning run, selecting the best unlearning instance for evaluation.' This introduces a cherry-picking risk, and no confidence intervals are reported for the majority of the numbers in Tables 2, 3, 5, 10, and Figure 5. Appendix C.2 further shows that small hyperparameter changes can shift TPR@1%FPR from the reported value to 24.5%. The paper's quantitative claim that average-case attacks 'underestimate' privacy leakage by large margins requires a multi-run protocol with means and variances, and an explicit definition of what 'best' means (best for the attack or best for the unlearning method). All methods should be compared on the same matched instances.
  4. [Section 5.2 and Appendix A.3] Target samples are chosen as 'vulnerable' by running LiRA on the original trained model, and the unlearned model is then evaluated on those same samples. The manuscript does not investigate whether per-sample vulnerability is stable under unlearning. If unlearning changes which samples are most memorized, RULI may be targeting a stale ranking rather than the true high-risk samples of the unlearned model, which would overstate the privacy leakage of the unlearned model. Please add an experiment that recomputes the vulnerability ranking from unlearned shadow models (or otherwise tests stability) and compares attack success on top-vulnerable samples from the original versus the unlearned model.
  5. [Section 6.3 and Section 6.4] The conclusion that 'most inexact unlearning methods cannot closely approximate retraining' rests entirely on the efficacy attack. Given that the efficacy attack in Algorithm 1 is not the Game 3 test, this conclusion is currently unsupported. Once the test model is properly implemented, the efficacy experiments (Figures 5, 6, 8, and Table 4) should be re-run, and the comparison with U-LiRA should use matched target sets and matched training/unlearning instances. The current Table 4 reports gains that conflate the target-selection change with the attack change, so the individual contribution of each component cannot be separated without matched settings.
minor comments (5)
  1. [Section 6.4] In the paragraph on differences in target design and testing strategy, the text says 'RULI uses Qh, a held-out distribution QR'; this should read 'a held-out distribution Qh' or similar, as QR denotes the retrained distribution.
  2. [Algorithm 1] The pseudocode loops 'for N iterations' but the text describes grouping target samples into three roles with N/3 samples per iteration and repeating three times. The pseudocode does not show this rotation, making it unclear how each target sample receives the claimed N/3 observations per distribution.
  3. [Table 2] Several rows are missing the delta ACC(Df) value (e.g., the 'Vulnerable + Protected' rows for Scrub and GA+), and some entries use inconsistent decimal formatting such as '1.06%' versus '1.9%'. Please standardize and complete the table.
  4. [Appendix C.2] The sensitivity analysis in Appendix C.2 is important and should be summarized in the main text, because it directly affects the reliability of the headline comparisons. As written, the appendix in effect acknowledges that the reported TPR@1%FPR values are fragile to hyperparameter choice, yet the main text presents them as stable findings.
  5. [Figure 1] The figure legend uses 'Avg-Case' while the text uses 'average-case'; please use a consistent terminology. Also, the caption claims that population average-case attacks [35] 'consistently underestimate' privacy leakage, but the paper only demonstrates this on a handful of methods and settings; consider softening the wording to 'underestimate in the settings studied here.'

Circularity Check

1 steps flagged · score 6.0 of 10

Efficacy measurement as implemented reduces to the U-LiRA Qu-vs-QR test, so the claimed dual-objective measurement is only partially realized; privacy measurements remain independent.

  1. other [Section 4.3.1 Eq. (2)-(3) and Section 4.3.2 Algorithm 1 lines 23-26]
    "θT (z) = ( θU(z)ifz∈D target ∩D train θR (z)ifz∈D target \D train (2) ... 23: f T ←Equation 2 24: O fT ←φ(f U(z)) ... 26: Ψ← p(O fT | ˆfunlearned) p(O fT | ˆfout)"

    Algorithm 1 line 24 always sets O_fT = φ(f_U(z)), never φ(f_T(z)). Eq. 2 defines θ_T(z) = θ_R(z) for z∉D_train, so the held-out branch is never executed. Consequently Ψ = p(θ_U|Qu)/p(θ_U|QR), which is exactly the U-LiRA-style comparison the paper says it supersedes (Section 4.3.2). The efficacy/retraining-gap results therefore do not measure the Game 3 test model; they are the prior Qu-vs-QR test under a different target-selection strategy.

full rationale

The privacy attack is a standard LiRA-style likelihood ratio built from shadow-model distributions (Eq. 1, Algorithm 1 line 25) and is self-contained; the vulnerable-sample target selection is an auditing choice, not a fitted input. No load-bearing self-citation or imported uniqueness theorem appears. The significant issue is the efficacy objective: the paper defines a test model θ_T(z) (Eq. 2) and an efficacy likelihood ratio Ψ (Eq. 3) under Game 3, but Algorithm 1 implements Ψ by querying f_U(z) for every target and comparing to Q_out. This makes the implemented efficacy metric identical to the U-LiRA Qu-vs-QR comparison the paper argues is inadequate, so the claimed new dual-objective measurement reduces, for efficacy, to a renamed prior test. The central privacy-leakage findings and the target-selection improvement are independent and empirically grounded; the paper is therefore partially circular rather than wholly so.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The framework's quantitative claims rest on several experimental design choices (shadow model count, vulnerability threshold, target set size, KDE bandwidth, baseline hyperparameters) and on domain assumptions about the sufficiency of the inference signal and the stability of vulnerability rankings. The test model is a new conceptual construct with no independent evidence.

free parameters (5)
  • number_of_shadow_models_N = 90
    Chosen for shadow model training in all experiments; affects stability of KDE likelihood ratios, especially at low false-positive rates. Not justified by a theory.
  • vulnerability_threshold_TPR = 0.01% FPR
    Threshold used to classify samples as highly memorized/vulnerable via LiRA; directly determines the target set and thus the reported attack success.
  • target_set_size = 600 (image), 500/400/1000 (text)
    Size of the target sample set in experiments; small sizes make TPR@1%FPR estimates noisy.
  • KDE_bandwidth = not reported
    Bandwidth for kernel density estimation of membership score distributions is not stated, yet it controls the smoothness of likelihood ratios and hence attack performance.
  • unlearning_baseline_hyperparameters = grid-searched per setting
    Hyperparameters for each unlearning baseline were tuned via grid search and the 'best unlearning instance' selected, which affects the measured attack success.
assumptions (5)
  • domain assumption The adversary knows the training and unlearning algorithms and can train shadow models from the same data distribution.
    Standard LiRA-style assumption; stated in Section 3 and Game 2. Needed to construct the unlearned, held-out, and retrained distributions.
  • domain assumption The inference function (loss or logit-scaled confidence) on a single query suffices to distinguish unlearned, held-out, and retrained behavior.
    Used throughout Algorithm 1; if the inference function lacks signal, the likelihood tests fail. Not formally justified.
  • domain assumption Kernel density estimates from finite shadow models provide accurate likelihood ratios, including at the 1% FPR tail.
    Step 2 of Algorithm 1; with 90 shadow models and per-sample scores, tail estimates are noisy. The appendix mentions no bandwidth or validation.
  • ad hoc to paper The test model defined in Equation 2 is a valid mechanism for measuring efficacy, meaning an oracle switching between the unlearned and retrained models yields a meaningful indistinguishability test.
    Introduced in Section 4.3.1; it is a hypothetical construct not grounded in an established evaluation framework.
  • domain assumption Per-sample vulnerability rankings computed on the original model are stable enough to define the target set for evaluating the unlearned model.
    Section 5.2 and Appendix A.3 identify vulnerable samples via LiRA on the pre-unlearning model; the paper does not test whether these remain the highest-risk samples after unlearning.
invented entities (1)
  • Test model
    purpose: A per-sample oracle that returns the unlearned model's output for unlearned samples and the retrained model's output for never-trained samples, enabling the efficacy likelihood ratio test.
    Defined in Equation 2; it is a conceptual device for evaluation, not a measurable object. Its validity is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective." pith.science (2026). https://pith.science/paper/BO7CPVTP

@misc{pith2026250613009,
  author       = {Pith},
  title        = {Pith review of: Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BO7CPVTP}},
  note         = {Machine review of arXiv:2506.13009}
}
read the original abstract

Machine unlearning focuses on efficiently removing specific data from trained models, addressing privacy and compliance concerns with reasonable costs. Although exact unlearning ensures complete data removal equivalent to retraining, it is impractical for large-scale models, leading to growing interest in inexact unlearning methods. However, the lack of formal guarantees in these methods necessitates the need for robust evaluation frameworks to assess their privacy and effectiveness. In this work, we first identify several key pitfalls of the existing unlearning evaluation frameworks, e.g., focusing on average-case evaluation or targeting random samples for evaluation, incomplete comparisons with the retraining baseline. Then, we propose RULI (Rectified Unlearning Evaluation Framework via Likelihood Inference), a novel framework to address critical gaps in the evaluation of inexact unlearning methods. RULI introduces a dual-objective attack to measure both unlearning efficacy and privacy risks at a per-sample granularity. Our findings reveal significant vulnerabilities in state-of-the-art unlearning methods, where RULI achieves higher attack success rates, exposing privacy risks underestimated by existing methods. Built on a game-based foundation and validated through empirical evaluations on both image and text data (spanning tasks from classification to generation), RULI provides a rigorous, scalable, and fine-grained methodology for evaluating unlearning techniques.

Figures

Figures reproduced from arXiv: 2506.13009 by the authors.

Figure 1
Figure 1. RULI (ours) separates privacy leakage and unlearn￾ing efficacy through targeted membership inference. Popula￾tion average-case attacks [35] consistently underestimate pri￾vacy leakage and fail to capture unlearning efficacy. This fig￾ure demonstrates some representative results, e.g., Scrub [44] on CIFAR-10 (ResNet-18) and TinyImageNet (Vision Trans￾former [47]) for image sample unlearning, and NPO [82] on WikiText-… view at source ↗
Figure 2
Figure 2. Overview of RULI shadow model training and unlearning to measure the privacy leakage and efficacy. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Two samples under inexact unlearning; A protected sample’s out and unlearned distributions are distinguishable (efficacy); vulnerable sample’s unlearned and held-out distri￾butions are distinguishable (privacy leakage). Canary Injection. By specifying the target set in RULI, we can focus on measuring privacy leakage (rather than efficacy), drawing inspiration from the well-known canary injection technique [14, 66] u… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Attack accuracy and TPR@1%FPR targeting vulner [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Comparing with retraining using RULI. All unlearning benchmarks are highly distinguishable from retraining. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: MIA for unlearning efficacy on different choices [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Privacy leakage results on WikiText-103 using a [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Unlearning efficacy on 400 target samples ( [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Logit-scaled confidences of two distributions held [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 70 canonical work pages

  1. [1]

    URL: https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=CELEX%3A32016R0679

    Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of nat- ural persons with regard to the processing of personal data and on the free movement of such data, and repealing Direc- tive 95/46/EC (General Data Protection Regulation), 2016. URL: https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=CELEX%3...

    2016

  2. [2]

    Deep learning with differential privacy

    Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMa- han, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. InCCS, pages 308–318, 2016

    2016

  3. [3]

    Evaluations of machine learning privacy defenses are misleading

    Michael Aerni, Jie Zhang, and Florian Tramèr. Evaluations of machine learning privacy defenses are misleading. InCCS, pages 1271–1284, 2024

    work page 2024

  4. [4]

    what do you want from the- ory alone?

    Meenatchi Sundaram Muthu Selva Annamalai, Georgi Ganev, and Emiliano De Cristofaro. "what do you want from the- ory alone?" experimenting with tight auditing of differentially private synthetic data generation. InUSENIX Security, 2024

    work page 2024

  5. [5]

    Task-agnostic privacy- preserving representation learning for federated learning against attribute inference attacks

    Caridad Arroyo Arevalo, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. Task-agnostic privacy- preserving representation learning for federated learning against attribute inference attacks. InAAAI, 2024

    work page 2024

  6. [6]

    Machine unlearning: Linear filtration for logit-based clas- sifiers.Machine Learning, 111(9):3203–3226, 2022

    Thomas Baumhauer, Pascal Schöttle, and Matthias Zeppelza- uer. Machine unlearning: Linear filtration for logit-based clas- sifiers.Machine Learning, 111(9):3203–3226, 2022

    work page 2022

  7. [7]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? InProceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623, 2021

    work page 2021

  8. [8]

    Pythia: A suite for analyzing large lan- guage models across training and scaling

    Stella Biderman, Hailey Schoelkopf, Quentin Gregory An- thony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Moham- mad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large lan- guage models across training and scaling. InICML, pages 2397–2430. PMLR, 2023

    work page 2023

Show all 82 references
  1. [9]

    Machine unlearning

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. InS&P, pages 141–159. IEEE, 2021

    2021

  2. [10]

    Language models are few-shot learners.NeurIPS, 33:1877–1901, 2020

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.NeurIPS, 33:1877–1901, 2020

    1901

  3. [11]

    California Consumer Privacy Act of 2018, 6 2018

    California State Legislature. California Consumer Privacy Act of 2018, 6 2018. Cal. Civ. Code § 1798.100 et seq. URL: https://leginfo.legislature.ca.gov/faces/ billTextClient.xhtml?bill_id=201720180AB375

    2018

  4. [12]

    Membership inference at- tacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, An- dreas Terzis, and Florian Tramer. Membership inference at- tacks from first principles. InS&P, pages 1897–1914, 2022

    1914

  5. [13]

    The privacy onion effect: Memorization is relative

    Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. InNeurIPS, 2022

    2022

  6. [14]

    The secret sharer: Evaluating and testing unin- tended memorization in neural networks

    Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unin- tended memorization in neural networks. InUSENIX Security, pages 267–284, 2019

    2019

  7. [15]

    Relaxloss: Defending membership inference attacks without losing utility

    Dingfan Chen, Ning Yu, and Mario Fritz. Relaxloss: Defending membership inference attacks without losing utility. InICLR, 2022

    2022

  8. [16]

    Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary

    Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep net- works via shifting the decision boundary. InCVPR, 2023

    2023

  9. [17]

    When machine unlearning jeopardizes privacy

    Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. When machine unlearning jeopardizes privacy. InCCS, pages 896–911, 2021

    2021

  10. [18]

    Stochastic gradient langevin unlearning.arXiv preprint arXiv:2403.17105, 2024

    Eli Chien, Haoyu Wang, Ziang Chen, and Pan Li. Stochastic gradient langevin unlearning.arXiv preprint arXiv:2403.17105, 2024

    2024 arXiv

  11. [19]

    Label-only membership inference attacks

    Christopher A Choquette-Choo, Florian Tramer, Nicholas Car- lini, and Nicolas Papernot. Label-only membership inference attacks. InICML, pages 1964–1974, 2021

    1964

  12. [20]

    Forget unlearning: Towards true data-deletion in machine learning

    Rishav Chourasia and Neil Shah. Forget unlearning: Towards true data-deletion in machine learning. InICML, 2023

    2023

  13. [21]

    Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In AAAI, 2023

    2023

  14. [22]

    Zero-shot machine unlearning.IEEE Transactions on Information Forensics and Security, 2023

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. Zero-shot machine unlearning.IEEE Transactions on Information Forensics and Security, 2023

    2023

  15. [23]

    Blind base- lines beat membership inference attacks for foundation models

    Debeshee Das, Jie Zhang, and Florian Tramèr. Blind base- lines beat membership inference attacks for foundation models. arXiv preprint arXiv:2406.16201, 2024

    2024 arXiv

  16. [24]

    Do membership inference attacks work on large language models? InCOLM, 2024

    Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Se- won Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? InCOLM, 2024

    2024

  17. [25]

    Chal- lenging forgets: Unveiling the worst-case forget sets in ma- chine unlearning

    Chongyu Fan, Jiancheng Liu, Alfred Hero, and Sijia Liu. Chal- lenging forgets: Unveiling the worst-case forget sets in ma- chine unlearning. InECCV, pages 278–297, 2025

    2025

  18. [26]

    Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation.arXiv preprint arXiv:2310.12508, 2023

    Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation.arXiv preprint arXiv:2310.12508, 2023

    arXiv

  19. [27]

    Harmonizing differential privacy mechanisms for federated learning: Boost- ing accuracy and convergence

    Shuya Feng, Meisam Mohammady, Hanbin Hong, Shenao Yan, Ashish Kundu, Binghui Wang, and Yuan Hong. Harmonizing differential privacy mechanisms for federated learning: Boost- ing accuracy and convergence. InCODASPY, 2025

    2025

  20. [28]

    DPI: ensuring strict differential privacy for infinite data streaming

    Shuya Feng, Meisam Mohammady, Han Wang, Xiaochen Li, Zhan Qin, and Yuan Hong. DPI: ensuring strict differential privacy for infinite data streaming. InS&P, 2024

    2024

  21. [29]

    Model inversion attacks that exploit confidence information and basic countermeasures

    Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. InCCS, pages 1322–1333, 2015

    2015

  22. [30]

    Property inference attacks on fully connected neural networks using permutation invariant representations

    Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. Property inference attacks on fully connected neural networks using permutation invariant representations. InCCS, pages 619–633, 2018

    2018

  23. [31]

    Deletion inference, recon- struction, and compliance in machine (un) learning

    Ji Gao, Sanjam Garg, Mohammad Mahmoody, and Prashant Nalini Vasudevan. Deletion inference, recon- struction, and compliance in machine (un) learning. InPETS, 2022

    2022

  24. [32]

    Making ai forget you: Data deletion in machine learning

    Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. Making ai forget you: Data deletion in machine learning. NeurIPS, 2019

    2019

  25. [33]

    Mixed-privacy forgetting in deep networks

    Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. Mixed-privacy forgetting in deep networks. InCVPR, pages 792–801, 2021

    2021

  26. [34]

    Eter- nal sunshine of the spotless net: Selective forgetting in deep networks

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eter- nal sunshine of the spotless net: Selective forgetting in deep networks. InCVPR, pages 9304–9312, 2020

    2020

  27. [35]

    Amnesiac machine learning

    Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. InAAAI, 2021

    2021

  28. [36]

    Certified data removal from machine learning models

    Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. Certified data removal from machine learning models. InICML, 2020

    2020

  29. [37]

    Adaptive machine un- learning.NeurIPS, 34:16319–16330, 2021

    Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Chris Waites. Adaptive machine un- learning.NeurIPS, 34:16319–16330, 2021

    2021

  30. [38]

    Inexact unlearning needs more careful evaluations to avoid a false sense of privacy

    Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. InSaTML, 2025

    2025

  31. [39]

    Learn what you want to unlearn: Unlearning inversion attacks against machine unlearning

    Hongsheng Hu, Shuo Wang, Tian Dong, and Minhui Xue. Learn what you want to unlearn: Unlearning inversion attacks against machine unlearning. InS&P, 2024

    2024

  32. [40]

    Defending membership inference attacks via privacy-aware sparsity tun- ing.arXiv preprint arXiv:2410.06814, 2024

    Qiang Hu, Hengxiang Zhang, and Hongxin Wei. Defending membership inference attacks via privacy-aware sparsity tun- ing.arXiv preprint arXiv:2410.06814, 2024

    2024 arXiv

  33. [41]

    Unlearn and burn: Adversarial machine unlearning requests destroy model accuracy

    Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, and Chiyuan Zhang. Unlearn and burn: Adversarial machine unlearning requests destroy model accuracy. InICLR, 2025

    2025

  34. [42]

    Measuring forgetting of memorized training examples

    Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ip- polito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al. Measuring forgetting of memorized training examples. InICLR, 2023

    2023

  35. [43]

    Knowledge unlearning for mitigating privacy risks in language models

    Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moon- tae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. In ACL, pages 14389–14408, Toronto, Canada, July 2023

    2023

  36. [44]

    Towards unbounded machine unlearning.Ad- vances in Neural Information Processing Systems, 36, 2024

    Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning.Ad- vances in Neural Information Processing Systems, 36, 2024

    2024

  37. [45]

    Continual learning and private unlearning

    Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. InConference on Lifelong Learning Agents, pages 243–254. PMLR, 2022

    2022

  38. [46]

    Model sparsity can simplify machine unlearning.NeurIPS, 36, 2024

    Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, Sijia Liu, et al. Model sparsity can simplify machine unlearning.NeurIPS, 36, 2024

    2024

  39. [47]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 10012–10022, 2021

    2021

  40. [48]

    Learn to forget: Machine unlearning via neuron masking.TDSC, 20(4):3194–3207, 2022

    Zhuo Ma, Yang Liu, Ximeng Liu, Jian Liu, Jianfeng Ma, and Kui Ren. Learn to forget: Machine unlearning via neuron masking.TDSC, 20(4):3194–3207, 2022

    2022

  41. [49]

    Membership infer- ence on word embedding and beyond.arXiv preprint arXiv:2106.11384, 2021

    Saeed Mahloujifar, Huseyin A Inan, Melissa Chase, Esha Ghosh, and Marcello Hasegawa. Membership infer- ence on word embedding and beyond.arXiv preprint arXiv:2106.11384, 2021

    2021 arXiv

  42. [50]

    New insights and perspectives on the natu- ral gradient method.Journal of Machine Learning Research, 21(146):1–76, 2020

    James Martens. New insights and perspectives on the natu- ral gradient method.Journal of Machine Learning Research, 21(146):1–76, 2020

    2020

  43. [51]

    Can LLMs keep a secret? testing privacy implications of language models via contextual integrity theory

    Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can LLMs keep a secret? testing privacy implications of language models via contextual integrity theory. InICLR, 2024

    2024

  44. [52]

    Tight auditing of differentially private machine learning

    Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Flo- rian Tramèr, Matthew Jagielski, Nicholas Carlini, and Andreas Terzis. Tight auditing of differentially private machine learning. InUSENIX Security, pages 1631–1648, 2023

    2023

  45. [53]

    Machine learning with membership privacy using adversarial regulariza- tion

    Milad Nasr, Reza Shokri, and Amir Houmansadr. Machine learning with membership privacy using adversarial regulariza- tion. InCCS, pages 634–646, 2018

    2018

  46. [54]

    Inf2Guard: An Information-Theoretic frame- work for learning Privacy-Preserving representations against inference attacks

    Sayedeh Leila Noorbakhsh, Binghui Zhang, Yuan Hong, and Binghui Wang. Inf2Guard: An Information-Theoretic frame- work for learning Privacy-Preserving representations against inference attacks. InUSENIX Security, 2024

    2024

  47. [55]

    Choquette-Choo, Milad Nasr, and Prateek Mittal

    Ashwinee Panda, Xinyu Tang, Christopher A. Choquette-Choo, Milad Nasr, and Prateek Mittal. Privacy auditing of large language models. InICLR, 2025

    2025

  48. [56]

    Pytorch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    2019

  49. [57]

    Language models are unsuper- vised multitask learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsuper- vised multitask learners. 2019

    2019

  50. [58]

    Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.Journal of machine learning research, 21(140):1–67, 2020

    2020

  51. [59]

    Sok: Let the privacy games begin! a unified treatment of data inference privacy in machine learning

    Ahmed Salem, Giovanni Cherubin, David Evans, Boris Köpf, Andrew Paverd, Anshuman Suri, Shruti Tople, and Santiago Zanella-Béguelin. Sok: Let the privacy games begin! a unified treatment of data inference privacy in machine learning. In S&P, 2023

    2023

  52. [60]

    Remember what you want to for- get: Algorithms for machine unlearning.NeurIPS, 34:18075– 18086, 2021

    Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. Remember what you want to for- get: Algorithms for machine unlearning.NeurIPS, 34:18075– 18086, 2021

    2021

  53. [61]

    Muse: Machine unlearning six-way evaluation for language models

    Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. InICLR, 2025

    2025

  54. [62]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. InS&P, pages 3–18. IEEE, 2017

    2017

  55. [63]

    Ununlearning: Un- learning is not sufficient for content regulation in advanced generative ai.arXiv preprint arXiv:2407.00106, 2024

    Ilia Shumailov, Jamie Hayes, Eleni Triantafillou, Guillermo Ortiz-Jimenez, Nicolas Papernot, Matthew Jagielski, Itay Yona, Heidi Howard, and Eugene Bagdasaryan. Ununlearning: Un- learning is not sufficient for content regulation in advanced generative ai.arXiv preprint arXiv:2...

    2024 arXiv

  56. [64]

    B. W. Silverman.Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986

    1986

  57. [65]

    Systematic evaluation of pri- vacy risks of machine learning models

    Liwei Song and Prateek Mittal. Systematic evaluation of pri- vacy risks of machine learning models. InUSENIX Security, 2021

    2021

  58. [66]

    Privacy auditing with one (1) training run.NeurIPS, 2024

    Thomas Steinke, Milad Nasr, and Matthew Jagielski. Privacy auditing with one (1) training run.NeurIPS, 2024

    2024

  59. [67]

    Lessons learned: defending against property infer- ence attacks.arXiv preprint arXiv:2205.08821, 2022

    Joshua Stock, Jens Wettlaufer, Daniel Demmler, and Hannes Federrath. Lessons learned: defending against property infer- ence attacks.arXiv preprint arXiv:2205.08821, 2022

    2022 arXiv

  60. [68]

    Mitigating membership inference attacks by self-distillation through a novel ensemble architecture

    Xinyu Tang, Saeed Mahloujifar, Liwei Song, Virat Shejwalkar, Milad Nasr, Amir Houmansadr, and Prateek Mittal. Mitigating membership inference attacks by self-distillation through a novel ensemble architecture. InUSENIX Security, 2022

    2022

  61. [69]

    Fast yet effective machine unlearning

    Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. Fast yet effective machine unlearning. IEEE Trans. Neural Netw. Learn. Syst, 2023

    2023

  62. [70]

    Unrolling sgd: Understanding factors influencing machine unlearning

    Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nico- las Papernot. Unrolling sgd: Understanding factors influencing machine unlearning. InEuroS&P, pages 303–319. IEEE, 2022

    2022

  63. [71]

    Gradients look alike: Sensitivity is often overestimated in DP-SGD

    Anvith Thudi, Hengrui Jia, Casey Meehan, Ilia Shumailov, and Nicolas Papernot. Gradients look alike: Sensitivity is often overestimated in DP-SGD. InUSENIX Security, 2024

    2024

  64. [72]

    On the necessity of auditable algorithmic definitions for machine unlearning

    Anvith Thudi, Hengrui Jia, Ilia Shumailov, and Nicolas Paper- not. On the necessity of auditable algorithmic definitions for machine unlearning. InUSENIX Security, 2022

    2022

  65. [73]

    Truth serum: Poisoning machine learning models to reveal their secrets

    Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their secrets. InCCS, pages 2779–2792, 2022

    2022

  66. [74]

    Are we making progress in unlearning? findings from the first neurips unlearning competition.arXiv:2406.09073, 2024

    Eleni Triantafillou, Peter Kairouz, Fabian Pedregosa, Jamie Hayes, Meghdad Kurmanji, Kairan Zhao, Vincent Dumoulin, Julio Jacques Junior, Ioannis Mitliagkas, Jun Wan, et al. Are we making progress in unlearning? findings from the first neurips unlearning competition.arXiv:2406...

    2024 arXiv

  67. [75]

    Privacy backdoors: En- hancing membership inference through poisoning pre-trained models

    Yuxin Wen, Leo Marchyok, Sanghyun Hong, Jonas Geiping, Tom Goldstein, and Nicholas Carlini. Privacy backdoors: En- hancing membership inference through poisoning pre-trained models. InNeurIPS, 2024

    2024

  68. [76]

    Rethinking evaluation methods for machine unlearning

    Leon Wichert and Sandipan Sikdar. Rethinking evaluation methods for machine unlearning. InEMNLP Findings, 2024

    2024

  69. [77]

    Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. Machine unlearning: A survey.ACM Comput. Surv., 56(1), aug 2023

    2023

  70. [78]

    Privacy risk in machine learning: Analyzing the connec- tion to overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connec- tion to overfitting. InCSF, pages 268–282. IEEE, 2018

    2018

  71. [79]

    Membership inference attacks and defenses in neural network pruning

    Xiaoyong Yuan and Lan Zhang. Membership inference attacks and defenses in neural network pruning. InUSENIX Security, pages 4561–4578, 2022

    2022

  72. [80]

    Low-cost high-power membership inference attacks

    Sajjad Zarifzadeh, Philippe Liu, and Reza Shokri. Low-cost high-power membership inference attacks. InICML, 2024

    2024

  73. [81]

    Towards certified unlearning for deep neural networks

    Binchi Zhang, Yushun Dong, Tianhao Wang, and Jundong Li. Towards certified unlearning for deep neural networks. In ICML, 2024

    2024

  74. [82]

    Negative pref- erence optimization: From catastrophic collapse to effective unlearning.arXiv preprint arXiv:2404.05868, 2024

    Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative pref- erence optimization: From catastrophic collapse to effective unlearning.arXiv preprint arXiv:2404.05868, 2024. Appendix A Further Implementation Details A.1 Training Details The training configurations for different...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.