pith. sign in

arxiv: 2506.15115 · v3 · submitted 2025-06-18 · 💻 cs.LG

Towards Reliable Forgetting: A Survey on Machine Unlearning Verification

Lulu Xue , Shengshan Hu , Wei Lu , Yan Shen , Dongxu Li , Peijin Guo , Ziqi Zhou , Minghui Li
show 2 more authors
Yanjun Zhang Leo Yu Zhang
This is my paper

Pith reviewed 2026-05-19 09:32 UTC · model grok-4.3

classification 💻 cs.LG
keywords machine unlearningverificationsurveytaxonomybehavioral verificationparametric verificationprivacydata forgetting
0
0 comments X

The pith

Machine unlearning verification methods fall into behavioral or parametric categories based on evidence type.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This survey sets out to organize the various ways researchers check whether a machine learning model has successfully removed specific data from its training. It groups these verification techniques by whether they examine the model's observable outputs and performance or the changes inside its parameters. Such organization would matter because reliable checks are essential for meeting privacy rules and giving users control over trained models. With a shared structure, developers could better select and improve methods suited to their constraints.

Core claim

The paper claims that verification of machine unlearning success can be systematically divided into behavioral verification, which relies on evidence from model outputs or task performance, and parametric verification, which relies on evidence from alterations to model parameters or weights, and that this division allows analysis of each method's assumptions, strengths, and limitations.

What carries the argument

A taxonomy that sorts verification techniques by the type of evidence they use to confirm unlearning fidelity, separating them into behavioral and parametric groups.

If this is right

  • Methods can be compared and improved using a common set of categories and evaluation points.
  • Vulnerabilities in real-world use of verification become easier to spot and address.
  • Open problems are listed to focus future work on filling gaps in coverage and theory.
  • Unlearning techniques gain clearer criteria for proving they meet regulatory standards.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The taxonomy could be extended with hybrid methods that combine behavioral and parametric evidence for stronger checks.
  • Similar evidence-based groupings might help organize verification in adjacent areas such as model auditing or bias detection.
  • Applying the taxonomy to newly proposed unlearning algorithms would test whether it remains useful as the field grows.

Load-bearing premise

Existing verification methods can be comprehensively partitioned into behavioral and parametric categories without major overlaps or important approaches that fit neither.

What would settle it

Identification of even one verification method whose evidence comes from a source that is neither model behavior nor model parameters would show the taxonomy is incomplete.

Figures

Figures reproduced from arXiv: 2506.15115 by Dongxu Li, Leo Yu Zhang, Lulu Xue, Minghui Li, Peijin Guo, Shengshan Hu, Wei Lu, Yanjun Zhang, Yan Shen, Ziqi Zhou.

Figure 1
Figure 1. Figure 1: An overview of our work. Contributions. 1) We present the first structured survey on unlearning verification, categorizing existing meth￾ods into behavioral and parametric approaches based on their verification signals. This classification clarifies core assumptions, enables systematic comparison, and highlights key challenges in building reliable verification systems. 2) We define seven key evaluation dim… view at source ↗
Figure 2
Figure 2. Figure 2: A framework diagram of machine unlearning. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The process of machine unlearning verification. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A taxonomy of unlearning verification methods. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
read the original abstract

With growing demands for privacy protection, security, and legal compliance (e.g., GDPR), machine unlearning has emerged as a critical technique for ensuring the controllability and regulatory alignment of machine learning models. However, a fundamental challenge in this field lies in effectively verifying whether unlearning operations have been successfully and thoroughly executed. Despite a growing body of work on unlearning techniques, verification methodologies remain comparatively underexplored and often fragmented. Existing approaches lack a unified taxonomy and a systematic framework for evaluation. To bridge this gap, this paper presents the first structured survey of machine unlearning verification methods. We propose a taxonomy that organizes current techniques into two principal categories -- behavioral verification and parametric verification -- based on the type of evidence used to assess unlearning fidelity. We examine representative methods within each category, analyze their underlying assumptions, strengths, and limitations, and identify potential vulnerabilities in practical deployment. In closing, we articulate a set of open problems in current verification research, aiming to provide a foundation for developing more robust, efficient, and theoretically grounded unlearning verification mechanisms.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. This paper presents the first structured survey of machine unlearning verification methods. It proposes a taxonomy that organizes existing techniques into two principal categories—behavioral verification (relying on output or behavior evidence) and parametric verification (relying on internal parameter evidence)—examines representative methods in each category along with their assumptions, strengths, limitations, and vulnerabilities, and identifies open problems to guide future work on robust verification for privacy and compliance needs such as GDPR.

Significance. If the taxonomy accurately captures the literature without significant omissions or forced classifications, the survey would provide a valuable organizing framework for an underexplored area of machine unlearning. The explicit discussion of vulnerabilities and open problems could help steer development toward more reliable, efficient, and theoretically grounded verification mechanisms.

major comments (1)
  1. [Taxonomy section] Taxonomy section (as described in the abstract and central claim): the two-way partition into behavioral versus parametric verification does not specify explicit handling rules for hybrid methods (e.g., influence-function auditing or combined membership-inference-plus-gradient checks) or for purely formal proof-based approaches. Without a third category or clear assignment criteria, the taxonomy risks becoming an artificial constraint rather than a natural organization, directly affecting the survey's utility as a comprehensive foundation.
minor comments (1)
  1. [Abstract] Abstract: the claim that verification methodologies are 'comparatively underexplored and often fragmented' would be strengthened by citing at least two concrete examples of fragmentation or missing unified evaluation frameworks from the surveyed literature.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback on our survey. The comment on the taxonomy is well-taken and highlights an opportunity to improve clarity. We address it point by point below and will incorporate revisions accordingly.

read point-by-point responses

  1. Referee: [Taxonomy section] Taxonomy section (as described in the abstract and central claim): the two-way partition into behavioral versus parametric verification does not specify explicit handling rules for hybrid methods (e.g., influence-function auditing or combined membership-inference-plus-gradient checks) or for purely formal proof-based approaches. Without a third category or clear assignment criteria, the taxonomy risks becoming an artificial constraint rather than a natural organization, directly affecting the survey's utility as a comprehensive foundation.

    Authors: We appreciate this observation. The taxonomy is deliberately organized around the primary type of evidence used to assess unlearning (behavioral outputs versus internal parameters), as stated in the abstract and Section 3. To address potential ambiguity, the revised version will include an explicit subsection clarifying assignment rules: hybrid methods are classified according to their dominant evidence source, with cross-references to the secondary aspect (e.g., influence-function auditing is placed under parametric verification because it directly inspects parameter influence, while combined membership-inference-plus-gradient checks are discussed in behavioral verification with a note on their parametric component). Purely formal proof-based approaches, which remain sparse in the current literature, will be noted as aligning most closely with parametric verification when they certify parameter-level guarantees; we will add a short discussion of their current scarcity and potential as a future extension. These additions preserve the two-category structure while providing the requested assignment criteria, thereby strengthening rather than constraining the framework. revision: yes

Circularity Check

0 steps flagged

Survey taxonomy is an organizational proposal with no self-referential reductions

full rationale

This paper is a survey that reviews existing machine unlearning verification techniques from other authors and proposes a two-category taxonomy (behavioral vs. parametric) based on evidence type. No derivations, equations, predictions, or fitted quantities appear in the provided abstract or context. The central claim is a classification framework rather than a result derived from the authors' prior work or self-defined inputs. Self-citations, if present, are not load-bearing for any mathematical or predictive claim, satisfying the criteria for an independent survey contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a survey paper. No free parameters, mathematical axioms, or invented entities are introduced to support a technical claim.

pith-pipeline@v0.9.0 · 5740 in / 911 out tokens · 26435 ms · 2026-05-19T09:32:12.721561+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Transferable Physical-World Adversarial Patches Against Object Detection in Autonomous Driving

    cs.CV 2026-04 unverdicted novelty 6.0

    AdvAD produces physical-world adversarial patches with improved transferability to unseen object detectors by multi-model optimization, adaptive balancing, and physical variation robustness.

  2. Transferable Physical-World Adversarial Patches Against Pedestrian Detection Models

    cs.CV 2026-04 unverdicted novelty 6.0

    TriPatch generates transferable physical adversarial patches via multi-stage triplet loss, appearance consistency, and data augmentation to achieve higher attack success rates on pedestrian detectors than prior methods.

Reference graph

Works this paper leans on

137 extracted references · 137 canonical work pages · cited by 2 Pith papers · 5 internal anchors

  1. [1]

    General data protection regulation

    2018. General data protection regulation. Intouch 25 (2018), 1–5

    work page 2018

  2. [2]

    Ahmed Abusnaina, Yuhang Wu, Sunpreet Arora, Yizhen Wang, Fei Wang, Hao Yang, and David Mohaisen. 2021. Adversarial example detection using latent neighborhood graph. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV’21) . 7687–7696

    work page 2021

  3. [3]

    Nasser Aldaghri, Hessam Mahdavifar, and Ahmad Beirami. 2021. Coded machine unlearning. IEEE Access 9 (2021), 88137–88150

    work page 2021

  4. [4]

    Haonan An, Guang Hua, Zhiping Lin, and Yuguang Fang. 2024. Box-free model watermarks are prone to black-box removal attacks. arXiv preprint arXiv:2405.09863 (2024)

    work page arXiv 2024

  5. [5]

    Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. 2021. Recent advances in adversarial training for adversarial robustness. arXiv preprint arXiv:2102.01356 (2021)

    work page arXiv 2021

  6. [6]

    Alexander Becker and Thomas Liebig. 2022. Evaluating machine unlearning via epistemic uncertainty. arXiv preprint arXiv:2208.10836 (2022)

    work page arXiv 2022

  7. [7]

    Martin Bertran, Shuai Tang, Michael Kearns, Jamie H Morgenstern, Aaron Roth, and Steven Z Wu. 2024. Reconstruction attacks on machine unlearning: Simple models are vulnerable. In Proceedings of the 38th Annual Conference on Neural Information Processing Systems (NeurIPS’24) , Vol. 37. 104995–105016

    work page 2024

  8. [8]

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP‘21’). IEEE, 141–159

    work page 2021

  9. [9]

    Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv Nikankin, and Michal Irani. 2023. Deconstructing data reconstruction: Multiclass, weight decay and general losses. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems (NeurIPS’23) , Vol. 36. 51515–51535

    work page 2023

  10. [10]

    Yinzhi Cao and Junfeng Yang. 2015. Towards making systems forget with machine unlearning. In Proceedings of the 2015 IEEE symposium on Security and Privacy (SP’15). IEEE, 463–480

    work page 2015

  11. [11]

    Shuwen Chai and Jinghui Chen. 2022. One-shot neural backdoor erasing via adversarial weight masking. In Proceedings of the 36th Annual Conference on Neural Information Processing Systems (NeurIPS’22), Vol. 35. 22285–22299

    work page 2022

  12. [12]

    Aobo Chen, Yangyi Li, Chenxu Zhao, and Mengdi Huai. 2025. A survey of security and privacy issues of machine unlearning

    work page 2025

  13. [13]

    Huili Chen, Bita Darvish Rohani, and Farinaz Koushanfar. 2018. DeepMarks: A Digital Fingerprinting Framework for Deep Neural Networks. Cryptology ePrint Archive (2018)

    work page 2018

  14. [14]

    Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. 2023. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’23) . 7766–7775

    work page 2023

  15. [15]

    Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang. 2021. When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS’21) . 896–911

    work page 2021

  16. [16]

    Eli Chien, Chao Pan, and Olgica Milenkovic. 2022. Efficient model updates for approximate unlearning of graph-structured data. In The 11st International Conference on Learning Representations (ICLR’22). Manuscript submitted to ACM Towards Reliable Forgetting: A Survey on Machine Unlearning Verification 31

    work page 2022

  17. [17]

    Eli Chien, Haoyu Wang, Ziang Chen, and Pan Li. 2024. Certified Machine Unlearning via Noisy Stochastic Gradient Descent. In Proceedings of the 38th Neural Information Processing Systems (NeurIPS’24)

    work page 2024

  18. [18]

    Eli Chien, Haoyu Wang, Ziang Chen, and Pan Li. 2024. Langevin Unlearning: A New Perspective of Noisy Gradient Descent for Machine Unlearning. In Proceedings of the 38th Neural Information Processing Systems (NeurIPS’24)

    work page 2024

  19. [19]

    Somnath Basu Roy Chowdhury, Krzysztof Marcin Choromanski, Arijit Sehanobish, Kumar Avinava Dubey, and Snigdha Chaturvedi. 2025. Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning. InProceedings of the 13th International Conference on Learning Representations (ICLR’25)

    work page 2025

  20. [20]

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. 2023. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In Proceedings of the 2023 AAAI Conference on Artificial Intelligence (AAAI’23) , Vol. 37. 7210–7217

    work page 2023

  21. [21]

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. 2023. Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security 18 (2023), 2345–2354

    work page 2023

  22. [22]

    Sumanth Dathathri, Stephan Zheng, Tianwei Yin, Richard M Murray, and Yisong Yue. 2018. Detecting adversarial examples via neural fingerprinting. arXiv preprint arXiv:1803.03870 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  23. [23]

    Zonglin Di, Sixie Yu, Yevgeniy Vorobeychik, and Yang Liu. 2025. Adversarial Machine Unlearning. InProceedings of the 13th International Conference on Learning Representations (ICLR’25)

    work page 2025

  24. [24]

    Yushun Dong, Binchi Zhang, Zhenyu Lei, Na Zou, and Jundong Li. 2024. Idea: A flexible framework of certified unlearning for graph neural networks. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’24) . 621–630

    work page 2024

  25. [25]

    Yonatan Dukler, Benjamin Bowman, Alessandro Achille, Aditya Golatkar, Ashwin Swaminathan, and Stefano Soatto. 2023. Safe: Machine unlearning with shard graphs. In Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV’23) . 17108–17118

    work page 2023

  26. [26]

    Cynthia Dwork. 2006. Differential privacy. InProceedings of the 2006 International Colloquium on Automata, Languages, and Programming (ICALP’06). Springer, 1–12

    work page 2006

  27. [27]

    Thorsten Eisenhofer, Doreen Riepel, Varun Chandrasekaran, Esha Ghosh, Olga Ohrimenko, and Nicolas Papernot. 2025. Verifiable and provably secure machine unlearning. In Proceedings of the 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML’25) . IEEE, 479–496

    work page 2025

  28. [28]

    Chongyu Fan, Jiancheng Liu, Yihua Zhang, Eric Wong, Dennis Wei, and Sijia Liu. 2024. SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation. In Proceedings of the 12th International Conference on Learning Representations (ICLR’24)

    work page 2024

  29. [29]

    Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. 2020. Local model poisoning attacks to {Byzantine-Robust} federated learning. In Proceedings of the 29th USENIX Security Symposium (Security’20) . 1605–1622

    work page 2020

  30. [30]

    Vitaly Feldman and Chiyuan Zhang. 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. In Proceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS’20) , Vol. 33. 2881–2891

    work page 2020

  31. [31]

    Daniel L Felps, Amelia D Schwickerath, Joyce D Williams, Trung N Vuong, Alan Briggs, Matthew Hunt, Evan Sakmar, David D Saranchak, and Tyler Shumaker. 2020. Class clown: Data redaction in machine unlearning at enterprise scale. arXiv preprint arXiv:2012.04699 (2020)

    work page arXiv 2020

  32. [32]

    Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS’15) . 1322–1333

    work page 2015

  33. [33]

    Xiangshan Gao, Xingjun Ma, Jingyi Wang, Youcheng Sun, Bo Li, Shouling Ji, Peng Cheng, and Jiming Chen. 2024. Verifi: Towards verifiable federated unlearning. IEEE Transactions on Dependable and Secure Computing (2024)

    work page 2024

  34. [34]

    Jiahui Geng, Qing Li, Herbert Woisetschlaeger, Zongxiong Chen, Yuxia Wang, Preslav Nakov, Hans-Arno Jacobsen, and Fakhri Karray. 2025. A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models. arXiv preprint arXiv:2503.01854 (2025)

    work page arXiv 2025

  35. [35]

    Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. 2019. Making ai forget you: Data deletion in machine learning. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS’19) , Vol. 32

    work page 2019

  36. [36]

    Shashwat Goel, Ameya Prabhu, Amartya Sanyal, Ser-Nam Lim, Philip Torr, and Ponnurangam Kumaraguru. 2022. Towards adversarial evaluations for inexact machine unlearning. arXiv preprint arXiv:2201.06640 (2022)

    work page arXiv 2022

  37. [37]

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR’20) . 9304–9312

    work page 2020

  38. [38]

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Proceedings of the 2020 European Conference on Computer Vision (ECCV’20) . Springer, 383–398

    work page 2020

  39. [39]

    Aditya Golatkar, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2023. Training data protection with compositional diffusion models. arXiv preprint arXiv:2308.01937 (2023)

    work page arXiv 2023

  40. [40]

    Xueluan Gong, Ziyao Wang, Shuaike Li, Yanjiao Chen, and Qian Wang. 2023. A gan-based defense framework against model inversion attacks. IEEE Transactions on Information Forensics and Security 18 (2023), 4475–4487

    work page 2023

  41. [41]

    Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021. Knowledge distillation: A survey. International Journal of Computer Vision 129, 6 (2021), 1789–1819

    work page 2021

  42. [42]

    Laura Graves, Vineel Nagisetty, and Vijay Ganesh. 2021. Amnesiac machine learning. In Proceedings of the 2021 AAAI Conference on Artificial Intelligence (AAAI’21), Vol. 35. 11516–11524

    work page 2021

  43. [43]

    Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. 2020. Certified Data Removal from Machine Learning Models. In Proceedings of the 2020 International Conference on Machine Learning (ICML’20) . PMLR, 3832–3842. Manuscript submitted to ACM 32 Xue et al

    work page 2020

  44. [44]

    Shangwei Guo, Tianwei Zhang, Han Qiu, Yi Zeng, Tao Xiang, and Yang Liu. 2021. Fine-tuning is not enough: A simple yet effective watermark removal attack for DNN models. InProceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI’21). Springer, 3635–3641

    work page 2021

  45. [45]

    Yu Guo, Yu Zhao, Saihui Hou, Cong Wang, and Xiaohua Jia. 2023. Verifying in the dark: Verifiable machine unlearning by using invisible backdoor triggers. IEEE Transactions on Information Forensics and Security 19 (2023), 708–721

    work page 2023

  46. [46]

    Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. 2022. Reconstructing training data from trained neural networks. InProceedings of the 36th Annual Conference on Neural Information Processing Systems (NeurIPS’22) , Vol. 35. 22911–22924

    work page 2022

  47. [47]

    Mengde Han, Tianqing Zhu, Lefeng Zhang, Huan Huo, and Wanlei Zhou. 2025. Vertical Federated Unlearning via Backdoor Certification. IEEE Transactions on Services Computing (2025)

    work page 2025

  48. [48]

    Jamie Hayes, Ilia Shumailov, Eleni Triantafillou, Amr Khalifa, and Nicolas Papernot. 2024. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. arXiv preprint arXiv:2403.01218 (2024)

    work page arXiv 2024

  49. [49]

    Hongsheng Hu, Shuo Wang, Tian Dong, and Minhui Xue. 2024. Learn what you want to unlearn: Unlearning inversion attacks against machine unlearning. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (SP’24) . IEEE, 3257–3275

    work page 2024

  50. [50]

    Shengshan Hu, Ziqi Zhou, Yechao Zhang, Leo Yu Zhang, Yifeng Zheng, Yuanyuan He, and Hai Jin. 2022. Badhash: Invisible backdoor attacks against deep hashing with clean label. In Proceedings of the 30th ACM International Conference on Multimedia (ACM MM’22) . 678–686

    work page 2022

  51. [51]

    Thanh Trung Huynh, Trong Bang Nguyen, Thanh Toan Nguyen, Phi Le Nguyen, Hongzhi Yin, Quoc Viet Hung Nguyen, and Thanh Tam Nguyen

    work page

  52. [52]

    ACM Transactions on Information Systems 43, 2 (2025), 1–29

    Certified unlearning for federated recommendation. ACM Transactions on Information Systems 43, 2 (2025), 1–29

    work page 2025

  53. [53]

    Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. 2021. Approximate data deletion from machine learning models. In Proceedings of the 2021 International Conference on Artificial Intelligence and Statistics (AISTATS’21) . PMLR, 2008–2016

    work page 2021

  54. [54]

    Layan Jaman, Reem Alsharabi, and Passent M ElKafrawy. 2024. Machine Unlearning: An Overview of the Paradigm Shift in the Evolution of AI. In Proceedings of the 21st Learning and Technology Conference (L&T’24) . IEEE, 25–29

    work page 2024

  55. [55]

    Dongjae Jeon, Wonje Jeung, Taeheon Kim, Albert No, and Jonghyun Choi. 2024. An Information Theoretic Evaluation Metric For Strong Unlearning. arXiv preprint arXiv:2405.17878 (2024)

    work page arXiv 2024

  56. [56]

    Hengrui Jia, Mohammad Yaghini, Christopher A Choquette-Choo, Natalie Dullerud, Anvith Thudi, Varun Chandrasekaran, and Nicolas Papernot

    work page

  57. [57]

    In Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP’21)

    Proof-of-learning: Definitions and practice. In Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP’21) . IEEE, 1039–1056

    work page 2021

  58. [58]

    Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. 2019. Memguard: Defending against black-box membership inference attacks via adversarial examples. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS’19) . 259–274

    work page 2019

  59. [59]

    Xiaojun Jia, Yong Zhang, Baoyuan Wu, Ke Ma, Jue Wang, and Xiaochun Cao. 2022. Las-at: adversarial training with learnable attack strategy. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) . 13398–13408

    work page 2022

  60. [60]

    Ruinan Jin, Minghui Chen, Qiong Zhang, and Xiaoxiao Li. 2023. Forgettable Federated Linear Learning with Certified Data Unlearning. arXiv preprint arXiv:2306.02216 (2023)

    work page internal anchor Pith review arXiv 2023

  61. [61]

    Minsung Kim, Nakyeong Yang, and Kyomin Jung. 2025. Rethinking Post-Unlearning Behavior of Large Vision-Language Models. arXiv preprint arXiv:2506.02541 (2025)

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  62. [62]

    Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, and Dan Roth. 2022. Privacy adhering machine un-learning in nlp. arXiv preprint arXiv:2212.09573 (2022)

    work page arXiv 2022

  63. [63]

    Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. 2023. Towards unbounded machine unlearning. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems (NeurIPS’23) , Vol. 36. 1957–1987

    work page 2023

  64. [64]

    Chunxiao Li, Haipeng Jiang, Jiankang Chen, Yu Zhao, Shuxuan Fu, Fangming Jing, and Yu Guo. 2024. An overview of machine unlearning. High-Confidence Computing (2024), 100254

    work page 2024

  65. [65]

    Minghui Li, Jiangxiong Wang, Hao Zhang, Ziqi Zhou, Shengshan Hu, and Xiaobing Pei. 2024. Transferable Adversarial Facial Images for Privacy Protection. In Proceedings of the 32nd ACM International Conference on Multimedia (MM’24)

    work page 2024

  66. [66]

    Na Li, Chunyi Zhou, Yansong Gao, Hui Chen, Zhi Zhang, Boyu Kuang, and Anmin Fu. 2025. Machine unlearning: Taxonomy, metrics, applications, challenges, and prospects. IEEE Transactions on Neural Networks and Learning Systems (2025)

    work page 2025

  67. [67]

    Hengzhu Liu, Ping Xiong, Tianqing Zhu, and Philip S Yu. 2025. A survey on machine unlearning: Techniques and new emerged privacy risks. Journal of Information Security and Applications 90 (2025), 104010

    work page 2025

  68. [68]

    Jiaqi Liu, Jian Lou, Zhan Qin, and Kui Ren. 2023. Certified minimax unlearning with generalization rates and deletion capacity. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems (NeurIPS’23) , Vol. 36. 62821–62852

    work page 2023

  69. [69]

    Junxu Liu, Mingsheng Xue, Jian Lou, Xiaoyu Zhang, Li Xiong, and Zhan Qin. 2023. Muter: Machine unlearning on adversarially trained models. In Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision (ICCV’23) . 4892–4902

    work page 2023

  70. [70]

    Yang Liu, Mingyuan Fan, Cen Chen, Ximeng Liu, Zhuo Ma, Li Wang, and Jianfeng Ma. 2022. Backdoor defense with machine unlearning. In Proceedings of the 2022 IEEE conference on computer communications (INFOCOM’22) . IEEE, 280–289

    work page 2022

  71. [71]

    Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. 2022. The right to be forgotten in federated learning: An efficient realization with rapid retraining. In Proceedings of the 2022 IEEE Conference on Computer Communications (INFOCOM’22) . IEEE, 1749–1758

    work page 2022

  72. [72]

    Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024. Machine unlearning in generative ai: A survey. arXiv preprint arXiv:2407.20516 (2024). Manuscript submitted to ACM Towards Reliable Forgetting: A Survey on Machine Unlearning Verification 33

    work page arXiv 2024

  73. [73]

    Ziyao Liu, Yu Jiang, Jiyuan Shen, Minyi Peng, Kwok-Yan Lam, Xingliang Yuan, and Xiaoning Liu. 2024. A survey on federated unlearning: Challenges, methods, and future directions. Comput. Surveys 57, 1 (2024), 1–38

    work page 2024

  74. [74]

    Ziyao Liu, Huanyi Ye, Chen Chen, Yongsen Zheng, and Kwok-Yan Lam. 2025. Threats, attacks, and defenses in machine unlearning: A survey.IEEE Open Journal of the Computer Society (2025)

    work page 2025

  75. [75]

    Xinyang Lu, Xinyuan Niu, Gregory Kang Ruey Lau, Bui Thi Cam Nhung, Rachael Hwee Ling Sim, Fanyu Wen, Chuan-Sheng Foo, See-Kiong Ng, and Bryan Kian Hsiang Low. 2025. WaterDrum: Watermarking for Data-centric Unlearning Metric. arXiv preprint arXiv:2505.05064 (2025)

    work page arXiv 2025

  76. [76]

    Zhaobo Lu, Yilei Wang, Qingzhe Lv, Minghao Zhao, and Tiancai Liang. 2022. Fp 2-mia: A membership inference attack free of posterior probability in machine unlearning. In Proceedings of the 2022 International Conference on Provable Security (ICPS’22) . Springer, 167–175

    work page 2022

  77. [77]

    Ronak Mehta, Sourav Pal, Vikas Singh, and Sathya N Ravi. 2022. Deep unlearning via randomized conditionally independent hessians. InProceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) . 10422–10431

    work page 2022

  78. [78]

    Salvatore Mercuri, Raad Khraishi, Ramin Okhrati, Devesh Batra, Conor Hamill, Taha Ghasempour, and Andrew Nowlan. 2022. An introduction to machine unlearning. arXiv preprint arXiv:2209.00939 (2022)

    work page arXiv 2022

  79. [79]

    Siqiao Mu and Diego Klabjan. 2024. Rewind-to-delete: Certified machine unlearning for nonconvex functions. arXiv preprint arXiv:2409.09778 (2024)

    work page arXiv 2024

  80. [80]

    Quoc Phong Nguyen, Bryan Kian Hsiang Low, and Patrick Jaillet. 2020. Variational bayesian unlearning. InProceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS’20), Vol. 33. 17176–17186

    work page 2020

Showing first 80 references.