Pith. sign in

REVIEW 4 major objections 6 minor 48 references

Mirror Mirror on the Wall, Have I Forgotten it All? A New Framework for Evaluating Machine Unlearning

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Machine unlearning methods fail to erase an adversary's ability to tell them apart from a retrained control model.

desk verdict The security-game definition and the empirical demonstration are worth taking seriously, but the impossibility theorems as stated do not survive contact with the proof details. read the letter →

arxiv 2505.08138 v1 pith:PGTMIBE4 submitted 2025-05-13 cs.LG cs.AIcs.CR

Brennon Brimhall , Philip Mathew , Neil Fendley , Yinzhi Cao , Matthew Green This is my paper · ORCID
classification cs.LGcs.AIcs.CR
keywords machineunlearningcomputationalindistinguishabilitymembershipinferencedifferentialprivacycertifiedremovalutilitycollapsesecuritygame
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a new standard for machine unlearning, called computational unlearning: an unlearning method succeeds only if no efficient adversary can distinguish its output from a control model retrained without the data to be forgotten. It reports experiments on ResNet-18 models trained on CIFAR-10 showing that four published unlearning methods fail this test under both a membership-inference-based score and a KL-divergence-based score. The paper also proves theoretical consequences: deterministic unlearning cannot achieve computational unlearning for entropic learning schemes, deterministic schemes that do achieve it must unlearn perfectly, and differential-privacy-based unlearning works only with negligible privacy parameters, which collapses utility to that of a randomly initialized model.

What carries the argument

The machinery is the security game itself, borrowed from cryptographic indistinguishability: two candidate models, one random bit, and an adversary who guesses the bit. The load-bearing objects are two distinguishers: MIAScore, which applies membership inference scores, and KLDScore, which measures KL divergence between the original model's outputs and the candidate model's outputs on perturbed forget-set inputs. The impossibility argument rests on the adversary being able to run the deterministic unlearning algorithm on the original model, while the entropic learning scheme rarely reproduces the same model twice.

What would settle it

Exhibit a learning scheme whose output distribution has Shannon entropy at least 1 bit but places probability $1-2^{-\lambda}$ on one particular model, pair it with a deterministic unlearner that outputs that likely model, and show an adversary cannot distinguish the result from the control model with non-negligible advantage; that would refute Theorem 18 as stated.

Watch

Extended reading notes

Core claim

The central claim is that unlearning should be defined as computational indistinguishability in a two-party game: a challenger hands the adversary an unlearned model and a mirror model (retrained on the retained data) in random order, and the adversary guesses which is which. A method achieves computational unlearning only if no probabilistic polynomial-time adversary can guess correctly except with negligible advantage. Empirically, all tested representative methods — bad teacher, amnesiac, selective synaptic dampening, and certified deep unlearning — are distinguishable in both white-box and black-box settings. Theoretically, the framework implies that entropic learning schemes require randomized unlearning, that deterministic schemes satisfying the definition must perform perfect unlearning, and that building unlearning from differential privacy forces utility down to that of a randomly initialized model.

Load-bearing premise

The impossibility proof for deterministic unlearning assumes the learning algorithm never outputs any single model with probability above 1/2, while the paper's written definition only requires Shannon entropy of at least 1 bit; these two conditions are not the same.

Editorial extensions

If this is right

  • Deterministic heuristic unlearning methods such as selective synaptic dampening and amnesiac unlearning can never satisfy the computational unlearning definition for entropic learning schemes.
  • Any deterministic learning scheme that satisfies the definition must implement perfect unlearning, meaning it outputs exactly the retrained model.
  • Differentially private unlearning constructions satisfy black-box computational unlearning only when the privacy parameters are negligible, at which point the unlearned model's utility equals that of a randomly initialized model.
  • Because black-box infeasibility implies white-box infeasibility, differential-privacy-based constructions cannot support the stronger white-box notion either.
  • The k-nearest neighbors algorithm admits a white-box computational unlearning algorithm simply by deleting the forgotten points, because the unlearned model is then identical to the control model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same game could be adapted to generative models by treating inference as sampling and measuring distinguishability on output distributions, though per-token scores would likely behave differently from the classification scores used here.
  • Beyond the paper: an adaptive adversary who queries the oracle repeatedly before choosing the forget set could shift the measured distinguishability; the current experiments use a fixed random forget set, so adaptive attacks are an untested extension.
  • Beyond the paper: the reported KLDScore intersection between σ = 0.001 and σ = 0.01 for certified deep unlearning suggests a parameter regime where the unlearned model may become harder to distinguish, and the paper leaves this regime open for future work.
  • Beyond the paper: the framework's strength depends heavily on the adversary knowing the original model; if that knowledge is withheld, certified removal with a sufficiently tight bound may remain a viable weaker standard, as the paper itself notes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a new formal notion of machine unlearning called computational unlearning, defined through a cryptographic indistinguishability game in which an adversary must tell apart a model produced by an unlearning method from a control model produced by retraining without the forget set. The game is given in white-box and black-box variants (Definitions 7 and 8). The authors construct two distinguishing scores, MIAScore and KLDScore, and report experiments on ResNet-18/CIFAR-10 with four unlearning methods, finding that an adversary distinguishes the unlearned model from the control with success rates above 60% at all tested forget-set sizes. The theoretical section claims (i) that no deterministic unlearning method can achieve computational unlearning for entropic learning schemes (Theorem 18), (ii) that any deterministic learning scheme satisfying computational unlearning must perfectly unlearn (Theorem 19), and (iii) that differentially private constructions collapse utility (Theorem 26 and Corollary 27).

Significance. The paper makes a useful conceptual contribution by moving the unlearning evaluation goal from score-based measures to a formal indistinguishability game, and the empirical study is a clear negative result for several representative heuristic and approximate unlearning methods. The use of KLDScore as a distinguisher is a nice complement to membership-inference-based scores. If the theoretical results were made correct, the framework would be a valuable step toward rigorous unlearning definitions. However, the current proofs of the central theorems contain load-bearing gaps and one false statement, so the theoretical conclusions are not established as written.

major comments (4)
  1. [Definition 14 and Theorem 18] Definition 14 defines an entropic learning scheme by Shannon entropy at least 1 bit, but the proof of Theorem 18 switches to "minimum entropy greater than 1 bit" and then uses p_max < 1/2. Shannon entropy at least 1 bit does not imply any such min-entropy bound: for example, a distribution with p(h0) = 1 - 1/λ and the remaining mass spread over 2^λ outcomes has Shannon entropy exceeding 1 bit while p_max tends to 1, not 1/2. Therefore the contradiction in the proof does not follow from the stated definition. The theorem may be repairable by changing the definition to a min-entropy condition, but as written the impossibility result for deterministic unlearning is not proven.
  2. [Theorem 19] Theorem 19 is false as stated. Let the learning scheme be deterministic and let unlearn be randomized, outputting the control model M_c with probability 1 - 2^{-λ} and some other model with probability 2^{-λ}. The adversary can compute M_c itself and compare the two challenge models to it; its advantage is 2^{-λ-1}, which is negligible, so Definitions 7 and 8 are satisfied. Yet unlearn is not perfect under Definition 16, since it does not always output the retrained model. This counterexample refutes the theorem and also invalidates Remark 20's dichotomy that a computational unlearning scheme must either be perfect or have both randomized learning and randomized unlearning.
  3. [Theorem 26 and Corollary 27] The proof of Theorem 26 is not a rigorous proof of an 'if and only if' statement. The two directions are asserted with informal statements about an adversary 'obtaining a query result with non-negligible privacy loss' after a polynomial number of queries, without bounding the adversary's advantage or relating the privacy-loss events to the distinguishing game. Additionally, the construction in which unlearn simply outputs the original model is not a forgetting operation in any intuitive sense, and the claim that it satisfies black-box computational unlearning needs a formal argument that the adversary's access to M_o in Definition 8 does not already trivially break indistinguishability. Furthermore, Corollary 27 uses the case D_f = D, while Definitions 7 and 8 require the adversary to select a proper subset D_f ⊂ D; if equality is excluded, the utility-collapse argument does not go through.
  4. [Lemma 25] Lemma 25 and its proof are too loose to support Theorem 26. The proof shows D_KL(P||Q) ≤ max_x L(x), but differential privacy only bounds the privacy loss with probability at least 1 - δ, not for every element, and the role of δ is absent from the derivation. A correct argument must account for the δ failure probability and then relate the resulting statistical divergence to the distinguishing advantage in the black-box game. As written, the lemma does not establish the 'negligible relative entropy' claim used in Theorem 26.
minor comments (6)
  1. [Definitions 7 and 8] The winning condition is written P(b'=b) < 1/2 + negl(λ); since the adversary can always succeed with probability at least 1/2, the inequality should be ≤ 1/2 + negl(λ) unless strictness is intentional and explained.
  2. [Remark 10] The displayed cost condition says cost(learn(D\D_f)) < cost(unlearn(M_o,D_f)), which is the opposite of the intended requirement that unlearning be cheaper than retraining. This should be cost(unlearn(M_o,D_f)) < cost(learn(D\D_f)) (or with ≤), otherwise the remark does not prevent the trivial init-based solution it is meant to exclude.
  3. [Section 2] There is a typo in the text 'known as the as the forget set' that should be corrected to 'known as the forget set.'
  4. [References] References [21] and [22] are the same He et al. paper and should be merged or differentiated.
  5. [Notation D_f ⊂ D] The notation D_f ⊂ D in Definitions 7 and 8 should be clarified as proper or non-strict subset; the proof of Corollary 27 uses D_f = D, which is only valid if non-strict subset is allowed.
  6. [Section 4.2] The phrase 'Beta distribution with the Jeffries prior' should be 'Jeffreys prior,' and it would help to state the actual interval construction used for the 95% confidence intervals.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the computational unlearning framework, empirical distinguishers, and theoretical consequences are self-contained, with only a non-load-bearing self-citation and independent proof-gap issues.

full rationale

The paper's central claims are derived from its own security-game definition rather than from fitted parameters or from conclusions assumed in the inputs. Computational unlearning (Definition 7/8) is stated as indistinguishability between an unlearned model and a control model; Theorems 18 and 19 then analyze deterministic/randomized learning and unlearning relative to that same game. These are direct logical consequences of the definition, not circular reductions. The empirical results use MIAScore and KLDScore as distinguishing tools; while some unlearning methods were designed to minimize MIA scores, the paper's finding is that the resulting scores are out of distribution relative to a control model, which is an empirical observation rather than a tautology. The only self-citation is to Cao and Yang [7], used as an example of accuracy-gap evaluation, and it is not load-bearing for the paper's claims. There are genuine correctness concerns: the proof of Theorem 18 invokes 'minimum entropy greater than 1 bit' while Definition 14 defines entropy as Shannon entropy, and the proof's assertion that p_max does not approach 1/2 is not generally true; Theorem 19's proof also contains a gap in the randomized case. These are mathematical proof issues, not circularity, because the conclusions do not reduce to the assumptions by construction. Accordingly, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The formal framework rests on standard cryptographic machinery plus the paper's own strong threat-model assumptions. The main load-bearing concerns are the entropy definition mismatch in Theorem 18 and the ambiguous D_f = D assumption in the DP utility-collapse result. No new physical or mathematical entities are introduced.

free parameters (3)
  • KLDScore noise variance = 0.1
    Hand-chosen Gaussian noise variance in the KLD distinguisher (Eq. 1). It affects the empirical attack's success rates but not the formal definition or theorems.
  • SSD dampening constant = 1
    Hyperparameter for the Selective Synaptic Dampening unlearning method, inherited from prior work; affects the empirical demonstration only.
  • SSD selection weighting = 100
    Hyperparameter for SSD, inherited from prior work; affects the empirical demonstration only.
assumptions (5)
  • standard math Standard cryptographic formalism: p.p.t. adversaries, negligible functions, semantic security games
    The computational unlearning game is modeled on IND-CPA from cryptography [4]; these are standard formal tools.
  • domain assumption Threat model: adversary is given M_o, learn, unlearn, D, D_f, cost, util
    This strong adversary model is explicit in Definitions 7 and 8 and flagged in Remark 9; all impossibility results depend on this access.
  • domain assumption Entropic learning schemes have output distributions with min-entropy greater than 1 bit
    Definition 14 uses Shannon entropy, but the proof of Theorem 18 relies on p_max <= 1/2. This mismatch is a load-bearing gap: Shannon entropy alone does not bound p_max.
  • domain assumption The forget set D_f may equal the full training set D
    Corollary 27's utility-collapse argument uses D_f = D. The text writes D_f subset D, which is ambiguous; if subset means proper subset, the corollary does not follow.
  • standard math Differential privacy properties: bounded privacy loss and post-processing immunity
    Used in Lemma 25 and Theorem 26; standard results from Dwork and Roth [13].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mirror Mirror on the Wall, Have I Forgotten it All? A New Framework for Evaluating Machine Unlearning." pith.science (2026). https://pith.science/paper/PGTMIBE4

@misc{pith2026250508138,
  author       = {Pith},
  title        = {Pith review of: Mirror Mirror on the Wall, Have I Forgotten it All? A New Framework for Evaluating Machine Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGTMIBE4}},
  note         = {Machine review of arXiv:2505.08138}
}
read the original abstract

Machine unlearning methods take a model trained on a dataset and a forget set, then attempt to produce a model as if it had only been trained on the examples not in the forget set. We empirically show that an adversary is able to distinguish between a mirror model (a control model produced by retraining without the data to forget) and a model produced by an unlearning method across representative unlearning methods from the literature. We build distinguishing algorithms based on evaluation scores in the literature (i.e. membership inference scores) and Kullback-Leibler divergence. We propose a strong formal definition for machine unlearning called computational unlearning. Computational unlearning is defined as the inability for an adversary to distinguish between a mirror model and a model produced by an unlearning method. If the adversary cannot guess better than random (except with negligible probability), then we say that an unlearning method achieves computational unlearning. Our computational unlearning definition provides theoretical structure to prove unlearning feasibility results. For example, our computational unlearning definition immediately implies that there are no deterministic computational unlearning methods for entropic learning algorithms. We also explore the relationship between differential privacy (DP)-based unlearning methods and computational unlearning, showing that DP-based approaches can satisfy computational unlearning at the cost of an extreme utility collapse. These results demonstrate that current methodology in the literature fundamentally falls short of achieving computational unlearning. We conclude by identifying several open questions for future work.

Figures

Figures reproduced from arXiv: 2505.08138 by the authors.

Figure 1
Figure 1. Overview of the security game for computational unlearning. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Forget set size against adversary success rate using [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Certified Deep Removal against KLDScore for different values of 𝜎. In our experiments we found that the adversary was able to distinguish using KLDScore with 100% accuracy for all choices of 𝜎. We found as 𝜎 increases the unlearned model’s KLDScore also increases (see [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Intuition of how KL divergence is able to distinguish between [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 26 canonical work pages

  1. [1]

    Deep Learning with Differential Privacy

    Martin Abadi et al. “Deep Learning with Differential Privacy”. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. CCS’16. ACM, 2016.doi: 10.1145/ 2976749.2978318. url: http://dx.doi.org/10.1145/2976749.2978318

    arXiv 2016

  2. [2]

    The Atlantic announces product and content partnership with OpenAI

    The Atlantic. The Atlantic announces product and content partnership with OpenAI. 2024.url: https: / / www . theatlantic . com / press - releases / archive / 2024 / 05 / atlantic - product - content - partnership-openai/678529/

    work page 2024

  3. [3]

    Nonlinear Programming

    D P Bertsekas. “Nonlinear Programming”. In:Journal of the Operational Research Society48.3 (1997), pp. 334–334. doi: 10.1057/palgrave.jors.2600425. eprint: https://doi.org/10.1057/palgrave. jors.2600425. url: https://doi.org/10.1057/palgrave.jors.2600425

    doi:10.1057/palgrave.jors.2600425 1997

  4. [4]

    Dan Boneh and Victor Shoup.A Graduate Course in Applied Cryptography. 2023. url: http://toc. cryptobook.us/book.pdf

    work page 2023

  5. [5]

    Machine Unlearning

    Lucas Bourtoule et al. Machine Unlearning. 2020. arXiv:1912.03817 [cs.CR]

    arXiv 2020

  6. [6]

    Brown et al.Language Models are Few-Shot Learners

    Tom B. Brown et al.Language Models are Few-Shot Learners. 2020. arXiv:2005.14165 [cs.CL]. url: https://arxiv.org/abs/2005.14165

    arXiv 2020

  7. [7]

    Towards Making Systems Forget with Machine Unlearning

    Yinzhi Cao and Junfeng Yang. “Towards Making Systems Forget with Machine Unlearning”. In:2015 IEEE Symposium on Security and Privacy. 2015, pp. 463–480.doi: 10.1109/SP.2015.35

    doi:10.1109/sp.2015.35 2015

  8. [8]

    Centers for Medicare and Medicaid Services.The Health Insurance Portability and Accountability Act of 1996 (HIPAA). 1996. url: http://www.cms.hhs.gov/hipaa/

    work page 1996

Show all 48 references
  1. [9]

    Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers

    Sungmin Cha et al. Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers. 2024. arXiv: 2301.11578 [cs.LG]. url: https://arxiv.org/abs/2301.11578

    2024 arXiv

  2. [10]

    Vikram S Chundawat et al.Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher. 2023. arXiv: 2205.08096 [cs.LG] . url: https://arxiv.org/abs/2205. 08096

    2023 arXiv

  3. [11]

    Now you can block OpenAI’s web crawler

    Emilia David. Now you can block OpenAI’s web crawler. 2023. url: https://www.theverge.com/ 2023/8/7/23823046/openai-data-scrape-block-ai

    2023

  4. [12]

    Vox Media and The Atlantic sign content deals with OpenAI

    Emilia David. Vox Media and The Atlantic sign content deals with OpenAI. 2024. url: https://www. theverge.com/2024/5/29/24167072/openai-content-copyright-vox-media-the-atlantic

    2024

  5. [13]

    The Algorithmic Foundations of Differential Privacy

    Cynthia Dwork and Aaron Roth. “The Algorithmic Foundations of Differential Privacy”. In:Found. Trends Theor. Comput. Sci.9.3–4(Aug.2014),pp.211–407. issn:1551-305X. doi: 10.1561/0400000042. url: https://doi.org/10.1561/0400000042

    2014 doi

  6. [14]

    European Parliament and Council of the European Union.Regulation (EU) 2016/679 of the European Parliament and of the Council. of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repeali...

    2016

  7. [15]

    Fast Machine Unlearning Without Retrain- ing Through Selective Synaptic Dampening

    Jack Foster, Stefan Schoepf, and Alexandra Brintrup. “Fast Machine Unlearning Without Retrain- ing Through Selective Synaptic Dampening”. In:ArXiv abs/2308.07707 (2023). url: https://api. semanticscholar.org/CorpusID:260900355

    2023 arXiv

  8. [16]

    Eternal Sunshine of the Spotless Net: Selec- tive Forgetting in Deep Networks

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. “Eternal Sunshine of the Spotless Net: Selec- tive Forgetting in Deep Networks”. In:2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2019), pp. 9301–9309. url: https://api.semanticscholar.org/Co...

    2019

  9. [17]

    Planting Undetectable Backdoors in Machine Learning Models

    Shafi Goldwasser et al. Planting Undetectable Backdoors in Machine Learning Models. 2022. arXiv: 2204.06974 [cs.LG]

    2022 arXiv

  10. [18]

    Laura Graves, Vineel Nagisetty, and Vijay Ganesh.Amnesiac Machine Learning. 2020. arXiv:2010. 10981 [cs.LG]. url: https://arxiv.org/abs/2010.10981

    2020 arXiv

  11. [19]

    Certified Data Removal from Machine Learning Models

    Chuan Guo et al. Certified Data Removal from Machine Learning Models. 2023. arXiv: 1911.03030 [cs.LG]. url: https://arxiv.org/abs/1911.03030

    2023 arXiv

  12. [20]

    AdaptiveMachineUnlearning

    VarunGuptaetal.“AdaptiveMachineUnlearning”.In: Advances in Neural Information Processing Sys- tems. Ed. by M. Ranzato et al. Vol. 34. Curran Associates, Inc., 2021, pp. 16319–16330.url: https:// proceedings.neurips.cc/paper_files/paper/2021/file/87f7ee4fdb57bdfd52179947211b7eb...

    2021

  13. [22]

    Deep Residual Learning for Image Recognition

    Kaiming He et al. Deep Residual Learning for Image Recognition. 2015. arXiv:1512.03385 [cs.CV]. url: https://arxiv.org/abs/1512.03385

    2015 arXiv

  14. [23]

    A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services

    Hongsheng Hu et al. A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Services. 2024. arXiv:2309.08230 [cs.CR]. url: https://arxiv.org/abs/2309.08230

    2024 arXiv

  15. [24]

    Hongsheng Hu et al.Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearning. 2024. arXiv:2404.03233 [cs.CR]. url: https://arxiv.org/abs/2404.03233

    2024 arXiv

  16. [25]

    Condé Nast Signs Deal With OpenAI

    Kate Knibbs. Condé Nast Signs Deal With OpenAI. 2024. url: https://www.wired.com/story/ conde-nast-openai-deal/

    2024

  17. [26]

    Deep Unlearning: Fast and Efficient Gradient- freeClassForgetting

    Sangamesh Kodge, Gobinda Saha, and Kaushik Roy. “Deep Unlearning: Fast and Efficient Gradient- freeClassForgetting”.In: Trans. Mach. Learn. Res.2024(2023). url: https://api.semanticscholar. org/CorpusID:271596242

    2023

  18. [27]

    Learning Multiple Layers of Features from Tiny Images

    Alex Krizhevsky. “Learning Multiple Layers of Features from Tiny Images”. In: Learning Multiple Layers of Features from Tiny Images. 2009. url: https://api.semanticscholar.org/CorpusID: 18268744

    2009

  19. [28]

    On Information and Sufficiency

    S. Kullback and R. A. Leibler. “On Information and Sufficiency”. In: The Annals of Mathematical Statistics 22.1 (1951), pp. 79–86.doi: 10.1214/aoms/1177729694. url: https://doi.org/10.1214/ aoms/1177729694

    1951

  20. [29]

    Amazon Is Investigating Perplexity Over Claims of Scraping Abuse.2024

    Dhruv Mehrotra and Andrew Couts. Amazon Is Investigating Perplexity Over Claims of Scraping Abuse.2024. url: https://www.wired.com/story/aws-perplexity-bot-scraping-investigation/

    2024

  21. [30]

    Deep Unlearning via Randomized Conditionally Independent Hessians

    Ronak R. Mehta et al. “Deep Unlearning via Randomized Conditionally Independent Hessians”. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2022), pp. 10412– 10421. url: https://api.semanticscholar.org/CorpusID:248227997

    2022

  22. [31]

    Thanh Tam Nguyen et al.A Survey of Machine Unlearning. 2022. arXiv:2209.02299 [cs.LG]

    2022 arXiv

  23. [32]

    A content and product partnership with The Atlantic

    OpenAI. A content and product partnership with The Atlantic. 2024. url: https : / / openai . com / index/enhancing-news-in-chatgpt-with-the-atlantic/

    2024

  24. [33]

    A Content and Product Partnership with Vox Media

    OpenAI. A Content and Product Partnership with Vox Media. 2023. url: https : / / openai . com / index/a-content-and-product-partnership-with-vox-media/ . 18

    2023

  25. [34]

    OpenAI partners with Condé Nast

    OpenAI. OpenAI partners with Condé Nast. 2024. url: https://openai.com/index/conde-nast/

    2024

  26. [35]

    Partnership with Axel Springer to deepen beneficial use of AI in journalism

    OpenAI. Partnership with Axel Springer to deepen beneficial use of AI in journalism. 2023. url: https://openai.com/index/axel-springer-partnership/

    2023

  27. [36]

    LAION-5B: An open large-scale dataset for training next generation image- text models

    ChristophSchuhmannetal. LAION-5B: An open large-scale dataset for training next generation image- text models. 2022. eprint:2210.08402

    2022 arXiv

  28. [37]

    Ayush Sekhari et al.Remember What You Want to Forget: Algorithms for Machine Unlearning. 2021. arXiv: 2103.03279 [cs.LG]

    2021 arXiv

  29. [38]

    Reza Shokri et al.Membership Inference Attacks against Machine Learning Models. 2017. arXiv:1610. 05820 [cs.CR]

    2017

  30. [39]

    Axel Springer and OpenAI partner to deepen beneficial use of AI in journalism

    Axel Springer. Axel Springer and OpenAI partner to deepen beneficial use of AI in journalism. 2023. url: https : / / www . axelspringer . com / en / ax - press - release / axel - springer - and - openai - partner-to-deepen-beneficial-use-of-ai-in-journalism

    2023

  31. [40]

    Fast Yet Effective Machine Unlearning

    Ayush K Tarun et al. “Fast Yet Effective Machine Unlearning”. In: IEEE Transactions on Neural Networks and Learning Systems35 (2021), pp. 13046–13055.url: https://api.semanticscholar. org/CorpusID:244270535

    2021

  32. [41]

    Identifying and Eliminating CSAM in Generative ML Training Data and Models

    David Thiel. Identifying and Eliminating CSAM in Generative ML Training Data and Models. 2023. url: https://doi.org/10.25740/kh752sm9123

    2023 doi

  33. [42]

    On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning

    Anvith Thudi et al. “On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning”. In: 31st USENIX Security Symposium (USENIX Security 22). Boston, MA: USENIX Association, Aug. 2022, pp. 4007–4022. isbn: 978-1-939133-31-1. url: https://www.usenix.org/conferenc...

    2022

  34. [43]

    LLaMA: Open and Efficient Foundation Language Models

    Hugo Touvron et al. LLaMA: Open and Efficient Foundation Language Models. 2023. arXiv: 2302. 13971

    2023

  35. [44]

    Machine Unlearning via Algorithmic Stability

    Enayat Ullah et al. Machine Unlearning via Algorithmic Stability. 2021. arXiv:2102.13179 [cs.LG]. url: https://arxiv.org/abs/2102.13179

    2021 arXiv

  36. [45]

    Digital Millennium Copyright Act of 1998 (DMCA)

    United States Congress. Digital Millennium Copyright Act of 1998 (DMCA). 1996. url: https:// www.govinfo.gov/content/pkg/PLAW-105publ304/pdf/PLAW-105publ304.pdf

    1998

  37. [46]

    Fair Credit Reporting Act of 1970 (FCRA).1970

    UnitedStatesCongress. Fair Credit Reporting Act of 1970 (FCRA).1970. url: https://www.govinfo. gov/content/pkg/STATUTE-84/pdf/STATUTE-84-Pg1114-2.pdf

    1970

  38. [47]

    Machine Unlearning of Features and Labels

    Alexander Warnecke et al. Machine Unlearning of Features and Labels. 2023. arXiv: 2108 . 11577 [cs.LG]. url: https://arxiv.org/abs/2108.11577

    2023 arXiv

  39. [48]

    Anthropic’s crawler is ignoring websites’ anti-AI scraping policies

    Jess Weatherbed. Anthropic’s crawler is ignoring websites’ anti-AI scraping policies. 2024.url: https: //www.theverge.com/2024/7/25/24205943/anthropic- ai- web- crawler- claudebot- ifixit- scraping-training-data

    2024

  40. [49]

    Binchi Zhang et al.Towards Certified Unlearning for Deep Neural Networks. 2024. arXiv:2408.00920 [cs.LG]. url: https://arxiv.org/abs/2408.00920. 19

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.