Pith. sign in

REVIEW 4 major objections 4 minor 33 references

Auditing Approximate Machine Unlearning for Differentially Private Models

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Approximate machine unlearning can breach the differential-privacy guarantee for retained samples, not just fail to protect the removed ones.

desk verdict Useful attack and a timely question, but the central claim that approximate unlearning breaks DP for retained samples does not follow from the evidence as analyzed: Criterion 2 drops delta and uses an uncalibrated estimator. read the letter →

arxiv 2508.18671 v1 pith:KSSHDWSE submitted 2025-08-26 cs.LG cs.AI

Yuechun Gu , Jiajie He , Keke Chen This is my paper
classification cs.LGcs.AI
keywords machineunlearningdifferentialprivacyauditingmembershipinferenceattackLiRAonioneffectapproximateretainedsamples
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that approximate machine unlearning, which is supposed to erase specific data from a trained model, can inadvertently break the privacy promise of differentially private models for the data that is kept. The authors define two sample-level audit criteria: unlearned samples must become less vulnerable to membership inference, and retained samples must stay at or below the model's privacy budget. To make such audits practical, they introduce A-LiRA, an efficient membership-inference attack that estimates per-sample privacy risk at a fraction of the cost of existing attacks. Auditing three recent approximate unlearning methods on CIFAR-10 and CIFAR-100 differentially private models, they find that all three push some retained samples above the privacy budget, especially when the highest-risk samples are removed.

What carries the argument

The central mechanism is A-LiRA, an augmentation-based likelihood-ratio attack that measures per-sample membership-inference risk. For a target sample x, A-LiRA trains one shadow model on the full dataset (in) and one on the dataset without x (out), generates n augmented copies of x, applies a logit transform to the true-class confidence of each copy, fits Gaussian distributions to the in and out scores, and computes a likelihood ratio using the maximum augmented logit. It outputs a per-sample TPR/FPR value, which is then compared against the two audit criteria: Equation (1) requires unlearned samples' risk to drop after unlearning, and Equation (2) requires retained samples' risk to stay at

What would settle it

Audit a freshly trained DP model that has not undergone unlearning, using A-LiRA: if any retained sample is reported above epsilon, then Criterion 2 has false positives. Then repeat the unlearning experiments with full online-LiRA using 512 shadow models; if online-LiRA finds no retained sample above epsilon where A-LiRA did, the central claim that existing unlearning methods breach DP for retained samples would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that existing approximate machine unlearning algorithms may inadvertently compromise the privacy of retained samples in differentially private models. They reformulate successful unlearning as two criteria: for every unlearned sample, the membership-inference risk after unlearning should be lower than before unlearning; and for every retained sample, the risk should remain at or below the differential-privacy budget epsilon of the original model. To evaluate these criteria, they develop A-LiRA, an augmentation-based likelihood-ratio attack that trains one in-training and one out-training shadow model per target sample and uses data augmentations to estimate the t

Load-bearing premise

The audit's validity rests on A-LiRA's per-sample TPR/FPR being an unbiased and tight estimate of true differential-privacy risk; if the attack over-reports risk, then a retained sample exceeding epsilon in the figures is not necessarily a real privacy breach.

Editorial extensions

If this is right

  • Differentially private models that undergo approximate unlearning can no longer be certified at their original privacy budget, because the unlearning step itself is not privacy-preserving for retained samples.
  • Auditing machine unlearning must measure both sides of the operation: the forgotten samples and the kept samples, not just the removed ones.
  • A-LiRA provides a practical sample-level auditing tool that is close in accuracy to online-LiRA while cutting the time cost by roughly 88 percent, making routine audits feasible.
  • New unlearning algorithms are needed that are themselves differentially private, meaning they provably keep the retained samples within the privacy budget after the removal operation.
  • Removing the highest-risk samples is the most dangerous unlearning scenario, because it exposes retained samples to the largest privacy leak under all three tested methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A corollary the paper leaves implicit: deletion requests can themselves become a privacy attack vector, since an adversary who can induce removal of sensitive samples could weaken protection for everyone else in the training set.
  • The same two-sided audit could be applied to exact unlearning schemes and to large-model unlearning, though A-LiRA's shadow-model training cost would need scaling strategies.
  • Because the criteria treat empirical TPR/FPR as if it fully captures the (epsilon, delta)-DP guarantee, the reported failure rates may be conservative or optimistic depending on delta and on how well A-LiRA is calibrated; a calibration pass on untouched DP models would separate detector false positives from true breaches.
  • A-LiRA's Gaussian assumption on augmented logits could miss multimodal score distributions, so a nonparametric likelihood-ratio variant might yield a tighter and even more reliable audit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a holistic auditing framework for approximate machine unlearning applied to differentially private (DP) models. It defines two criteria: Criterion 1 requires unlearned samples' membership-inference risk to decrease relative to the original model, and Criterion 2 requires retained samples' per-sample TPR/FPR to remain no larger than the DP privacy budget epsilon. To make such auditing practical, the authors introduce A-LiRA, an augmentation-based likelihood-ratio attack that trains only one in-model and one out-model per target sample and uses augmented views to estimate the two distributions. Experiments on CIFAR-10/CIFAR-100 with DP-SGD and three approximate unlearning methods (SUNSHINE, SSD, SalUn) report that all three methods cause some retained samples to exceed the proposed epsilon bound, and that A-LiRA is competitive with online-LiRA at lower cost. The paper concludes that existing approximate unlearning can compromise retained-sample privacy and that differentially private unlearning algorithms are needed.

Significance. If the central claim were established, the paper would fill a real gap: most unlearning audits focus only on removed samples, while the retained-sample side is underexplored. The proposed A-LiRA is also a plausible efficient alternative to online-LiRA, and the authors release code, which supports reproducibility. However, the main empirical conclusion rests on an audit criterion that is not a valid test of an (epsilon,delta)-DP breach, and the paper includes an appendix figure that appears to show the audit itself producing false-positive violations on a retrained DP model. The conceptual direction is valuable, but the current evidence does not establish that existing unlearning methods break DP for retained samples. A corrected statistical framework and calibration experiments could make the claim defensible; thus the contribution is potentially significant but not yet supported.

major comments (4)
  1. [Section V-A, Eq. (2)] The criterion E(UMD,X,x) <= epsilon is presented as a consequence of DP, but the hypothesis-testing characterization of (epsilon,delta)-DP is TPR <= e^epsilon * FPR + delta, not ln(TPR/FPR) <= epsilon. DP-SGD is an approximate-DP mechanism with delta > 0, yet delta is never reported. At the per-sample FPR of about 1/250 = 0.004 used in the experiments, even delta = 1e-5 permits log(TPR/FPR) to exceed epsilon by roughly delta/(e^epsilon * FPR), a slack on the order of the failure rates reported in Figures 2 and 3. Therefore the observed failure rates are consistent with a correctly implemented DP model and do not establish a privacy breach. The paper must either use a valid bound that accounts for delta and the sampling noise of the audit, or state explicit assumptions under which the criterion is intended as a heuristic rather than a DP violation test.
  2. [Appendix B, Fig. 5] This figure is the closest the paper comes to a calibration check, and it undermines the central claim. It evaluates Criterion 2 after directly excluding the unlearned samples and retraining, i.e., on a model that satisfies DP by construction for the retained data. A-LiRA and online-LiRA still flag roughly 2-4% of retained samples as failing Criterion 2. These are false positives of the auditing procedure, not real DP violations. A valid audit must show a near-zero failure rate on the original DP model (or on a fully retrained DP model) at the chosen thresholds, ideally with confidence intervals or multiple-testing correction. Without such a baseline, the post-unlearning failure rates in Figures 2 and 3 cannot be attributed to unlearning. This is a load-bearing issue for the paper's main conclusion.
  3. [Section VI.A and Section III-D] The per-sample TPR/FPR estimates are extremely noisy and the experimental protocol does not report uncertainty. Each sample is used in roughly 250 member and 250 non-member models, so the smallest nonzero per-sample FPR is 1/250 = 0.004; at FPR=0.01, only about 2.5 non-member likelihood-ratio scores per sample determine the rate. No confidence intervals, standard errors, or multiple-testing corrections are reported for the failure rates in Figures 2-3. Moreover, A-LiRA's shadow distributions are trained for the original training algorithm (M(D) vs. M(D\x)), not for the unlearning transformation U(M(D),X). The attack therefore is not calibrated to the distribution of unlearned models. These issues do not by themselves disprove the paper's intuition, but they mean the quantitative failure rates should be interpreted as upper-bound-like estimates rather than measured violations.
  4. [Section V-A, Eq. (2) notation] The paper defines E(MD,x) as the ratio TPR_x/FPR_x, but Eq. (2) compares E to epsilon. A ratio and a privacy-loss value are not comparable unless E denotes the logarithm ln(TPR/FPR). The introduction and earlier text use ln(TPR/FPR) < epsilon, so the notation in Eq. (2) should be corrected to ln E(UMD,X,x) <= epsilon or the definition of E should be changed. This ambiguity directly affects how a reader interprets every failure-rate number in the paper.
minor comments (4)
  1. [Figure 3] The caption states 'epsilon = 4', while the text in Section VI.C says the second DP setting is 'epsilon = 6'. Please reconcile the caption with the text.
  2. [Abstract and references] Typo: 'pubished' should be 'published'. Also several references are duplicated (e.g., [3] and [29] appear twice); a cleanup would improve readability.
  3. [Appendix A] The non-DP upper bound E(UMD,X,x) <= max_x E(MD,x) + t2 is presented without any statistical justification or discussion of how t2 should be chosen. If retained as an exploratory metric, this should be stated explicitly; otherwise it risks being interpreted as a formal privacy guarantee.
  4. [Section VI.C] The paper reports that in DP models all unlearned samples become safer under Criterion 1 ('we don't show the results as a blank figure'). Presenting an empty or omitted figure is confusing; a sentence with summary statistics or a figure with a note would be more informative.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the audit is empirical and self-contained; self-citations are not load-bearing. (The (epsilon,delta)-DP sloppiness is a correctness issue, not a circularity.)

full rationale

No circular derivation found. The paper's central claim is an empirical audit: it defines privacy criteria (Eq. 1 for unlearned samples and Eq. 2 for retained samples), builds an attack A-LiRA, measures per-sample TPR/FPR, and reports failure rates. The criteria are operationalizations, not derived consequences of the attack; the failure rates could in principle be zero and are not forced by construction. The load-bearing theoretical premise that DP bounds attack TPR/FPR is cited to external work (Dwork et al., Jagielski et al., Tramer et al.) and is not a self-citation. The authors' prior work [17, 18] appears only as examples in a 'widely used' citation list and does not support any central inference. A-LiRA's parameters (normal means/variances, thresholds) are estimated from shadow models, not fitted to the reported failure rates. The most serious issue is that Criterion 2 drops the delta term of (epsilon,delta)-DP and treats an empirical A-LiRA estimate as a hard upper bound; however, that is a correctness/validity concern, not a circularity. Similarly, the paper states 'we observed that all unlearned samples become safer ... (we don't show the results as a blank figure)', an omitted result, but again not a circular step. No equation reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The audit criteria rest on the DP hypothesis-testing interpretation, A-LiRA's Gaussian assumption, and the non-standard membership labeling for unlearned samples. The free parameters are experimental thresholds and counts chosen by the authors; none are fit to make the central result true, but they affect failure-rate magnitudes.

free parameters (4)
  • t1 (Criterion 1 threshold) = 0
    Set to 0 in experiments to reflect the minimum requirement; user-defined.
  • t2 (non-DP Criterion 2 relaxation) = 0
    Set to 0 to keep the upper bound tight; user-defined.
  • Augmentation count n = 100
    Chosen by hand for A-LiRA; central to the attack's distribution estimation.
  • Shadow models for threshold = 30 in/out
    Selected to determine the FPR threshold; a different count could change per-sample TPR/FPR.
assumptions (4)
  • domain assumption (epsilon, delta)-DP implies TPR/FPR <= e^epsilon for any MIA on any sample
    This is used to derive Equation (2) in Section V.A, but the actual bound is TPR <= e^epsilon * FPR + delta, which permits TPR/FPR > e^epsilon when FPR is small. The assumption ignores delta and the FPR level.
  • domain assumption Augmented views are independent samples for estimating the in/out distributions
    A-LiRA treats 100 augmentations of one image as i.i.d. observations of the logit distribution, but augmentations are highly correlated, which may underestimate variance and overestimate attack power.
  • ad hoc to paper Ground truth membership for unlearned samples remains 'member' after unlearning
    The paper states in Section III-D and VI that true membership is kept unchanged to measure forgetting; this is not the standard MIA definition for a model that excludes the sample.
  • domain assumption The privacy onion effect (Carlini et al.) holds for DP models as well
    Used to interpret the increase in retained-sample risk; it was demonstrated for non-DP models and is extrapolated here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Auditing Approximate Machine Unlearning for Differentially Private Models." pith.science (2026). https://pith.science/paper/KSSHDWSE

@misc{pith2026250818671,
  author       = {Pith},
  title        = {Pith review of: Auditing Approximate Machine Unlearning for Differentially Private Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KSSHDWSE}},
  note         = {Machine review of arXiv:2508.18671}
}
read the original abstract

Approximate machine unlearning aims to remove the effect of specific data from trained models to ensure individuals' privacy. Existing methods focus on the removed records and assume the retained ones are unaffected. However, recent studies on the \emph{privacy onion effect} indicate this assumption might be incorrect. Especially when the model is differentially private, no study has explored whether the retained ones still meet the differential privacy (DP) criterion under existing machine unlearning methods. This paper takes a holistic approach to auditing both unlearned and retained samples' privacy risks after applying approximate unlearning algorithms. We propose the privacy criteria for unlearned and retained samples, respectively, based on the perspectives of DP and membership inference attacks (MIAs). To make the auditing process more practical, we also develop an efficient MIA, A-LiRA, utilizing data augmentation to reduce the cost of shadow model training. Our experimental findings indicate that existing approximate machine unlearning algorithms may inadvertently compromise the privacy of retained samples for differentially private models, and we need differentially private unlearning algorithms. For reproducibility, we have pubished our code: https://anonymous.4open.science/r/Auditing-machine-unlearning-CB10/README.md

Figures

Figures reproduced from arXiv: 2508.18671 by the authors.

Figure 1
Figure 1. Criterion 1: Approximate unlearning delivers less privacy protection to removed samples, compared to retraining. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Criterion 2: Approximate unlearning increases the privacy risk of some retained samples in differnetially private models [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Criterion 2: Approximate unlearning increases the privacy risk of some retained samples in differentially private models [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Criterion 2: Approximate unlearning breaches the [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Criterion 2: A-LiRA and online-LiRA detect more [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 31 canonical work pages

  1. [1]

    B., MIRONOV, I., T ALWAR, K., AND ZHANG , L

    A BADI , M., C HU, A., G OODFELLOW , I., M CMAHAN , H. B., MIRONOV, I., T ALWAR, K., AND ZHANG , L. Deep learning with differential privacy. InProceedings of the 2016 ACM SIGSAC conference on computer and communications security (2016), pp. 308–318

    work page 2016

  2. [2]

    Evaluations of machine learning privacy defenses are misleading

    A ERNI , M., Z HANG , J., AND TRAMÈR , F. Evaluations of machine learning privacy defenses are misleading. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (New York, NY , USA, 2024), CCS ’24, Association for Computing Machinery, pp. 1271–1284

    work page 2024

  3. [3]

    Evaluations of machine learning privacy defenses are misleading

    A ERNI , M., Z HANG , J., AND TRAMÈR , F. Evaluations of machine learning privacy defenses are misleading. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (2024), pp. 1271–1284

    work page 2024

  4. [4]

    A., JIA, H., T RAVERS , A., Z HANG , B., L IE, D., AND PAPERNOT , N

    B OURTOULE , L., C HANDRASEKARAN , V., CHOQUETTE -CHOO , C. A., JIA, H., T RAVERS , A., Z HANG , B., L IE, D., AND PAPERNOT , N. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) (2021), IEEE, pp. 141–159

    work page 2021

  5. [5]

    California consumer privacy act of 2018, 2018

    C ALIFORNIA STATE LEGISLATURE . California consumer privacy act of 2018, 2018. Cal. Civ. Code §§ 1798.100 et seq

    work page 2018

  6. [6]

    Towards making systems forget with machine unlearning

    C AO, Y., AND YANG , J. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy (2015), IEEE, pp. 463–480

    work page 2015

  7. [7]

    Membership inference attacks from first principles

    C ARLINI , N., C HIEN , S., N ASR , M., S ONG , S., T ERZIS , A., AND TRAMER , F. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP) (2022), IEEE, pp. 1897–1914

    work page 2022

  8. [8]

    The privacy onion effect: Memorization is relative

    C ARLINI , N., J AGIELSKI , M., Z HANG , C., PAPERNOT , N., T ERZIS , A., AND TRAMER , F. The privacy onion effect: Memorization is relative. Advances in Neural Information Processing Systems 35 (2022), 13263– 13276

    work page 2022

Show all 33 references
  1. [9]

    When machine unlearning jeopardizes privacy

    C HEN , M., Z HANG , Z., W ANG , T., B ACKES , M., H UMBERT , M., AND ZHANG , Y. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security (2021), pp. 896–911

    2021

  2. [10]

    Differential privacy: A survey of results

    D WORK , C. Differential privacy: A survey of results. In International conference on theory and applications of models of computation (2008), Springer, pp. 1–19

    2008

  3. [11]

    The algorithmic foundations of differential privacy

    D WORK , C., R OTH, A., ET AL . The algorithmic foundations of differential privacy. Foundations and Trends® in Theoretical Computer Science 9, 3–4 (2014), 211–407

    2014

  4. [12]

    Regulation (EU) 2016/679 of the European Parliament and of the Council, 2016

    E UROPEAN PARLIAMENT AND COUNCIL OF THE EUROPEAN UNION . Regulation (EU) 2016/679 of the European Parliament and of the Council, 2016. General Data Protection Regulation (GDPR), OJ L 119, 4 May 2016, pp. 1–88

    2016

  5. [13]

    Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation

    F AN, C., L IU, J., Z HANG , Y., WEI, D., W ONG , E., AND LIU, S. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In International Conference on Learning Representations (2024)

    2024

  6. [14]

    Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries

    F AROKHI , F., AND SANDBERG , H. Fisher information as a measure of privacy: Preserving privacy of households with smart meters using batteries. IEEE Transactions on Smart Grid 9 , 5 (2017), 4726–4734

    2017

  7. [15]

    Fast machine unlearning without retraining through selective synaptic dampening

    F OSTER , J., S CHOEPF , S., AND BRINTRUP , A. Fast machine unlearning without retraining through selective synaptic dampening. In Proceedings of the AAAI Conference on Artificial Intelligence (2024), vol. 38, pp. 12043–12051

    2024

  8. [16]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    G OLATKAR , A., A CHILLE , A., AND SOATTO, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020), pp. 9304–9312

    2020

  9. [17]

    Demo: Ft-privacyscore: Personal- ized privacy scoring service for machine learning participation

    G U, Y., H E, J., AND CHEN , K. Demo: Ft-privacyscore: Personal- ized privacy scoring service for machine learning participation. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (New York, NY , USA, 2024), CCS ’24, Association for Co...

    2024

  10. [18]

    Recps: Privacy risk scoring for recommender systems

    H E, J., G U, Y., AND CHEN , K. Recps: Privacy risk scoring for recommender systems. arXiv preprint arXiv:2507.18365 (2025)

    2025 arXiv

  11. [19]

    Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems 33 (2020), 22205–22216

    J AGIELSKI , M., U LLMAN , J., AND OPREA , A. Auditing differentially private machine learning: How private is private sgd? Advances in Neural Information Processing Systems 33 (2020), 22205–22216

    2020

  12. [20]

    The composition theorem for differential privacy

    K AIROUZ , P., O H, S., AND VISWANATH , P. The composition theorem for differential privacy. In International conference on machine learning (2015), PMLR, pp. 1376–1385

    2015

  13. [21]

    C., A FROZ , S., M ILLER , B., SHANKAR , V., B ACHWANI , R., J OSEPH , A

    K ANTCHELIAN , A., T SCHANTZ , M. C., A FROZ , S., M ILLER , B., SHANKAR , V., B ACHWANI , R., J OSEPH , A. D., AND TYGAR , J. D. Better malware ground truth: Techniques for weighting anti-virus vendor labels. In Proceedings of the 8th ACM Workshop on Artificial Intelli- gence...

    2015

  14. [22]

    Z., AND MALOOF , M

    K OLTER , J. Z., AND MALOOF , M. A. Learning to detect and classify malicious executables in the wild. Journal of Machine Learning Research 7, 12 (2006)

    2006

  15. [23]

    M., S ALMAN , H., AND M ˛ ADRY, A

    L ECLERC , G., I LYAS, A., E NGSTROM , L., P ARK , S. M., S ALMAN , H., AND M ˛ ADRY, A. Ffcv: Accelerating training by removing data bottlenecks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023), pp. 12011–12020

    2023

  16. [24]

    Membership inference attacks against language models via neighbourhood comparison

    M ATTERN , J., M IRESHGHALLAH , F., J IN, Z., S CHOELKOPF , B., SACHAN , M., AND BERG -KIRKPATRICK , T. Membership inference attacks against language models via neighbourhood comparison. In The 61st Annual Meeting Of The Association For Computational Linguistics (2023)

    2023

  17. [25]

    Tight auditing of differen- tially private machine learning

    N ASR , M., H AYES, J., S TEINKE , T., BALLE , B., T RAMÈR , F., JAGIEL - SKI , M., C ARLINI , N., AND TERZIS , A. Tight auditing of differen- tially private machine learning. In 32nd USENIX Security Symposium (USENIX Security 23) (2023), pp. 1631–1648

    2023

  18. [26]

    T., H UYNH , T

    N GUYEN , T. T., H UYNH , T. T., N GUYEN , P. L., L IEW, A. W.-C., Y IN, H., AND NGUYEN , Q. V. H. A survey of machine unlearning. arXiv preprint arXiv:2209.02299 (2022)

    2022 arXiv

  19. [27]

    Personal Information Protection and Elec- tronic Documents Act, 2000

    P ARLIAMENT OF CANADA . Personal Information Protection and Elec- tronic Documents Act, 2000. S.C. 2000, c. 5; Assented to 13 April 2000

    2000

  20. [28]

    Privacy auditing with one (1) training run

    S TEINKE , T., NASR , M., AND JAGIELSKI , M. Privacy auditing with one (1) training run. Advances in Neural Information Processing Systems 36 (2023), 49268–49280

    2023

  21. [29]

    Privacy auditing with one (1) training run

    S TEINKE , T., NASR , M., AND JAGIELSKI , M. Privacy auditing with one (1) training run. Advances in Neural Information Processing Systems 36 (2024)

    2024

  22. [30]

    Debugging differential privacy: A case study for privacy auditing

    T RAMER , F., T ERZIS , A., S TEINKE , T., S ONG , S., J AGIELSKI , M., AND CARLINI , N. Debugging differential privacy: A case study for privacy auditing. arXiv preprint arXiv:2202.12219 (2022)

    2022 arXiv

  23. [31]

    X U, H., Z HU, T., Z HANG , L., Z HOU , W., AND YU, P. S. Machine unlearning: A survey, 2023

    2023

  24. [32]

    Machine unlearning: Solutions and challenges

    X U, J., W U, Z., W ANG , C., AND JIA, X. Machine unlearning: Solutions and challenges. IEEE Transactions on Emerging Topics in Computational Intelligence (2024)

    2024

  25. [33]

    privacy onion effect

    Z ARIFZADEH , S., L IU, P., AND SHOKRI , R. Low-cost high-power membership inference attacks. In International Conference on Machine Learning (2024), PMLR, pp. 58244–58282. APPENDIX A: C RITERION 2 OF NON-DIFFERENTIALLY PRIVATE MODELS In this section, we introduce an alternati...

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.