REVIEW 3 major objections 5 minor 44 references
How to Protect Models against Adversarial Unlearning?
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Fine-tuning on similar 'twin' examples restores accuracy lost in machine unlearning.
desk verdict The healing idea is genuinely new, but the paper's strongest adversarial experiment shows twin selection losing to random replacement, so the headline claim outruns the data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the spare set and the twin-replacement rule. Healing is defined in Section 4 through Algorithm 1 (General Spare Set) and the Twins strategy: before training, a set DS of spare elements is withheld; when an element z is requested for unlearning, the model picks the most similar element z* in DS under a metric d, uses it for a fine-tuning step only if d(z,z*) < δ, and then removes z* from the spare set. The Twins strategy precomputes a similar surrogate for each protected training element, possibly extending to triplets and quadruplets. The similarity metrics tested are raw-pixel L2 distance, feature-space cosine similarity, and Mahalanobis distance computed on model embeddings. The mechanism does the work of replacing removed knowledge with transferable properties of a near-duplicate real example rather than synthetic data.
What would settle it
A single experiment would settle the role of twin matching: request unlearning of the 25 training points that are farthest from every held-out spare example under all three metrics, heal with the closest available spares, and compare accuracy against the gold-standard retrained model. If the gap is no larger than when random training points are removed, the twin mechanism is not load-bearing and healing is just fine-tuning on remaining data.
Extended reading notes
Core claim
The central claim is that replacing deleted training examples with similar real examples is an effective protection against both spontaneous and adversarial unlearning. In Section 4 healing is defined as retraining the unlearned model on the retained data together with a surrogate z* drawn from a withheld spare set, where z* is the closest example under a chosen metric and is used only if the distance is below a threshold δ. Experiments with 25 forgotten images show healing recovers accuracy close to the gold-standard retrained model MGold: on MNIST the best healed model reached 99.03% against 98.87% gold, on CIFAR-10 91.41% against 90.97%, and on AFHQ 99.73% against 99.40%, after roughly half the original training epochs of fine-tuning. The paper also claims that model susceptibility to unlearning differs by backbone and adversary knowledge, and that exact unlearning via SISA degrades steadily as deletion fraction grows while naive retraining is robust.
Load-bearing premise
The method assumes that a stock of real examples similar enough to the data that may later be deleted exists and has been withheld from training, and that the chosen similarity metric identifies examples that can stand in for the removed ones during fine-tuning.
Editorial extensions
If this is right
- A model owner who withholds a spare set before training can answer any unlearning request by fine-tuning on the retained data plus a matched twin, typically closing the accuracy gap to retraining from scratch without paying the full retraining cost.
- The protection is not limited to sophisticated attacks: even random deletion under the exact SISA method steadily lowers accuracy as the deleted fraction grows, so healing is relevant for routine deletion requests too.
- Raw Fisher unlearning without healing collapses models to near single-class accuracy on MNIST, CIFAR-10, and AFHQ, so approximate unlearning pipelines that use it need a healing step to be usable.
- Longer healing fine-tuning of about half the original training epochs is consistently better than a single epoch, and adding matched twins often gives the best final accuracy, while twins give a faster initial correction after just one epoch.
- Healing works from both Fisher and Influence starting points, bringing best-case healed accuracy to within a few tenths of a point of the gold-standard model in the tested settings.
Reading between the lines
- The paper leaves implicit that healing deliberately keeps some functional trace of the removed data through its surrogates, so it is likely to satisfy a formal deletion request while falling short of strict definitions of forgetting; the paper itself notes healing is not unlearning in a forgetting-quality sense.
- A stress test the paper does not run: an adversary who can choose the forget set will pick points with no twin within the threshold, reducing healing to plain fine-tuning; whether plain fine-tuning already matches the gold model determines how much the twin mechanism contributes.
- The spare set could be shrunk by exploiting the memorization-versus-generalization distinction the paper cites as future work: models need fewer surrogates for items they have generalised from, so the minimal protective set may be much smaller than the training set.
- Group healing, replacing a deleted subset by a similar subset of surrogates, could reduce the required spare-set size for large deletion requests, a direction the paper lists but does not evaluate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies adversarial machine unlearning, in which a malicious party submits deletion requests intended to degrade a trained model's accuracy. It proposes "healing": after an unlearning operation, the model owner fine-tunes on the remaining training data plus a small set of real examples selected to be similar to the deleted examples, drawn from a withheld spare set. The authors introduce two selection strategies (General Spare Set and Twins Strategy) and several similarity metrics (pixel L2, feature cosine, Mahalanobis). Experiments on MNIST, CIFAR-10, and AFHQ with Fisher and Influence unlearning show that fine-tuning generally restores accuracy, often close to the gold-standard retrained model, but with notable exceptions in the strongest adversarial setting.
Significance. The question of how to mitigate performance loss from adversarial unlearning is timely and practically relevant. The paper contributes a broad experimental sweep across three datasets, several unlearning methods, multiple similarity metrics, and an explicit adversary taxonomy. It also honestly acknowledges that healing is not true unlearning. However, the central claim that healing with similar examples protects against adversary actions is not consistently supported by the reported experiments, particularly in the worst-25-logits Influence scenario, where random replacement outperforms twin-based healing. If the authors can provide multi-run statistics, add the missing baselines, and reframe the contribution as an empirical study of post-unlearning fine-tuning, the paper could become a useful reference; in its current form the overstatement is load-bearing.
major comments (3)
- [Section 5.3.2, Table 5] The strongest adversarial scenario contradicts the abstract's claim of protecting against adversary actions. In the MNIST worst-25-logits experiment with Influence-initiated unlearning, the initial accuracy is 11.71%, and after 5 healing epochs the best twin variant (T-RawMahal) reaches 87.73%, while the random-replacement baseline reaches 92.65% and the gold standard is 98.72%. Thus twin-based healing is worse than randomly chosen replacement samples precisely where the model is most degraded and protection is most needed. Because Table 5 is based on a single representative run, the reported 5-percentage-point gap may not be robust; please report means and standard deviations over repeated healing runs.
- [Section 5.3.2, Table 4] The benefit of twin-based healing over plain fine-tuning is not consistently demonstrated. RemainOnly (R) achieves the best accuracy in 4 of the 12 settings (including 2 of 6 N/2-epoch settings), and in Table 5 random replacement beats the best twin variant in the IF 5-epoch row. Because "Best Acc" is the maximum over five data configurations, the reported twin advantage could be an artifact of selection. A direct statistical comparison (e.g., paired tests across repeated runs) among RemainOnly, random replacement, and each twin metric is needed to establish that similarity-based selection, rather than additional fine-tuning data, drives recovery.
- [Sections 4 and 6] The method is explicitly not an unlearning method. The text states healing "is not unlearning or even a type of performance recovery" and "leads to a certain paradox" by re-introducing similar real examples, and Section 6 concedes it "is not essentially unlearned in the sense of metrics on the forgetting quality." The abstract's phrase "protecting model performance from these side effects" should be qualified to indicate that the method provides apparent unlearning without forgetting guarantees. Otherwise the contribution risks being circular: fine-tuning on surrogate examples similar to the removed ones should, by construction, restore accuracy on data resembling the removed instances.
minor comments (5)
- [Algorithm 1] Lines 2-3 contain a typographical error: the argmin is written as d(z', z*) but should be d(z, z'), and the condition line reads "if then d(z, z*) < delta."
- [Section 5.1] There are typos such as "Unleraning procedures" and "M' F isher"; the manuscript would benefit from a careful proofread.
- [Section 3] In the adversary taxonomy, the phrases "chosen trading set" and "known training set" are used with overlapping and confusing wording; please clarify the distinction between knowing the training set and being able to add elements to it.
- [Section 5.3.1] The five data setups are described as "(i) using the remaining primary training data combined with the method-based twins; (ii) using only the remaining primary training data," which lists only two of the five; please specify the four twin variants explicitly.
- [Section 5.3.2] The text around Table 5 states that "in most scenarios presented, employing twins ... provides better healing outcomes than randomly chosen samples," but Table 5 shows that random replacement is better in the IF 5-epoch row; please reconcile the statement with the table.
Circularity Check
No significant circularity: healing is an empirical intervention evaluated against baselines, not a derivation forced by its own definition.
full rationale
The paper's central contribution is an empirical intervention: after unlearning, fine-tune on remaining data plus real examples selected for similarity to the removed set. The claimed protective effect is not derived from the definition of the intervention; it is tested against gold-standard retraining and baselines including remain-only fine-tuning and random replacement. The similarity metrics (pixel L2, feature cosine, Mahalanobis) are computed from the data and model but do not encode the accuracy outcome, so there is no definitional or fitted-parameter equivalence between the input (similar-example selection) and the reported output (test accuracy). The authors explicitly state that healing is not unlearning and that it transfers properties of z back to the model, which is a limitation of the method, not a circular derivation. No load-bearing self-citations appear in the reference list; all cited methods (SISA, Fisher, Influence, LiSSA) are external prior work. The experiments include multiple baselines, and the conclusions are hedged as holding 'in selected settings.' The strongest counterexample (Table 5, where random replacement outperforms twin-based healing in the adversarial Influence setting) undermines the abstract's broad claim, but that is a correctness or robustness concern, not circularity. Accordingly, no circular step can be exhibited with the required specificity.
Assumptions & free parameters
free parameters (5)
- Reserve set size k =
unspecified in text
- Similarity threshold delta =
not reported
- Healing fine-tuning epochs =
1 or N/2 (5, 3, 2 depending on dataset)
- Fisher unlearning noise sigma =
3.5e-3 (MNIST), 3.5e-5 (CIFAR-10), 1e-3 (AFHQ)
- Influence LiSSA depth and scale =
e.g., 5000/1000 (MNIST), 1000/10000 (CIFAR-10), 100/100 (AFHQ)
assumptions (4)
- domain assumption Small distance in pixel or feature space implies substitutability after unlearning
- domain assumption Fine-tuning on remaining data plus twins does not violate the formal unlearning requirement
- standard math MGold, retraining from scratch on the remaining data, is the correct gold standard
- domain assumption The threat model taxonomy (blind, output-aware, parameter-aware adversary) covers realistic unlearning request streams
Cite this review
Pith. "Pith review of How to Protect Models against Adversarial Unlearning?." pith.science (2026). https://pith.science/paper/TNCS5FIC
@misc{pith2026250710886,
author = {Pith},
title = {Pith review of: How to Protect Models against Adversarial Unlearning?},
year = {2026},
howpublished = {\url{https://pith.science/paper/TNCS5FIC}},
note = {Machine review of arXiv:2507.10886}
}
read the original abstract
AI models need to be unlearned to fulfill the requirements of legal acts such as the AI Act or GDPR, and also because of the need to remove toxic content, debiasing, the impact of malicious instances, or changes in the data distribution structure in which a model works. Unfortunately, removing knowledge may cause undesirable side effects, such as a deterioration in model performance. In this paper, we investigate the problem of adversarial unlearning, where a malicious party intentionally sends unlearn requests to deteriorate the model's performance maximally. We show that this phenomenon and the adversary's capabilities depend on many factors, primarily on the backbone model itself and strategy/limitations in selecting data to be unlearned. The main result of this work is a new method of protecting model performance from these side effects, both in the case of unlearned behavior resulting from spontaneous processes and adversary actions.
Figures
Reference graph
Works this paper leans on
-
[13]
Z. Di, S. Yu, Y . V orobeychik, and Y . Liu. Adversarial machine unlearn- ing. In The Thirteenth International Conference on Learning Represen- tations, 2025. URL https://openreview.net/forum?id=swWF948IiC
work page 2025
-
[1]
N. Agarwal, B. Bullins, and E. Hazan. Second-order stochastic opti- mization for machine learning in linear time. J. Mach. Learn. Res. , 18: 116:1–116:40, 2016
work page 2016
-
[2]
L. Bourtoule, V . Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) , pages 141–159,
work page 2021
-
[3]
California consumer privacy act of 2018, 2018
California State Legislature. California consumer privacy act of 2018, 2018. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id= 201720180AB375
work page 2018
-
[4]
X. Cao, J. Jia, Z. Zhang, and N. Z. Gong. Fedrecover: Recovering from poisoning attacks in federated learning using historical information. In 44th IEEE Symposium on Security and Privacy, SP 2023, San Fran- cisco, CA, USA, May 21-25, 2023 , pages 1366–1383. IEEE, 2023
work page 2023
-
[5]
A. Chen, Y . Li, C. Zhao, and M. Huai. A survey of security and privacy issues of machine unlearning. AI Mag., 46(1), 2025
work page 2025
-
[6]
R. Chen, J. Yang, H. Xiong, J. Bai, T. Hu, J. Hao, Y . Feng, J. T. Zhou, J. Wu, and Z. Liu. Fast model debias with machine unlearning. In Proceedings of the 37th International Conference on Neural Informa- tion Processing Systems, NIPS ’23, Red Hook, NY , USA, 2024. Curran Associates Inc
work page 2024
-
[7]
Y . Choi, Y . Uh, J. Yoo, and J.-W. Ha. Stargan v2: Diverse image syn- thesis for multiple domains. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020
work page 2020
Show all 44 references
-
[8]
V . S. Chundawat, A. K. Tarun, M. Mandal, and M. S. Kankanhalli. Zero- shot machine unlearning. IEEE Trans. Inf. F orensics Secur ., 18:2345– 2354, 2023. doi: 10.1109/TIFS.2023.3265506. URL https://doi.org/10. 1109/TIFS.2023.3265506
2023
-
[9]
A. E. Cinà, K. Grosse, A. Demontis, B. Biggio, F. Roli, and M. Pelillo. Machine learning security against data poisoning: Are we there yet? Computer, 57(3):26–34, 2024. doi: 10.1109/MC.2023.3299572. URL https://doi.org/10.1109/MC.2023.3299572
2024
-
[10]
Council regulation (EU) no 269/2014, 2014
Council of European Union. Council regulation (EU) no 269/2014, 2014. http://eur-lex.europa.eu/legal-content/EN/TXT/?qid= 1416170084502&uri=CELEX:32014R0269
2014
-
[11]
Dhasade, Y
A. Dhasade, Y . Ding, S. Guo, A. Kermarrec, M. de V os, and L. Wu. Quickdrop: Efficient federated unlearning via synthetic data generation. In J. Cao, Z. Jin, V . Schiavoni, and J. Edinger, editors, Proceedings of the 25th International Middleware Conference, MIDDLEWARE 2024, ...
2024
-
[12]
J. Z. Di, J. Douglas, J. Acharya, G. Kamath, and A. Sekhari. Hidden poi- son: Machine unlearning enables camouflaged poisoning attacks. InAd- vances in Neural Information Processing Systems 36: Annual Confer- ence on Neural Information Processing Systems 2023, NeurIPS 2023, Ne...
2023
-
[14]
V . Feldman. Does learning require memorization? a short tale about a long tail. In K. Makarychev, Y . Makarychev, M. Tulsiani, G. Kamath, and J. Chuzhoy, editors, Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, STOC 2020, Chicago, IL, USA, June 22-...
2020
-
[15]
Golatkar, A
A. Golatkar, A. Achille, and S. Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In 2020 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 9301–9309. Computer Vision Foun- ...
2020
-
[16]
Golatkar, A
A. Golatkar, A. Achille, and S. Soatto. Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks . In 2020 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR) , pages 9301–9309, Los Alamitos, CA, USA, June 2020. IEEE Computer Soci- ety
2020
-
[17]
Golatkar, A
A. Golatkar, A. Achille, A. Ravichandran, M. Polito, and S. Soatto. Mixed-privacy forgetting in deep networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021 , pages 792–801. Computer Vision Foundation / IEEE,
2021
-
[18]
Graves, V
L. Graves, V . Nagisetty, and V . Ganesh. Amnesiac machine learning. CoRR, abs/2010.10981, 2020. URL https://arxiv.org/abs/2010.10981
2010 arXiv
-
[19]
doi: 10.1109/CVPR46437.2021.00085
2021
-
[20]
H. Hu, S. Wang, J. Chang, H. Zhong, R. Sun, S. Hao, H. Zhu, and M. Xue. A duty to forget, a right to be assured? exposing vulnerabilities in machine unlearning services, 2024. URL https://arxiv.org/abs/2309. 08230
2024
-
[21]
C. Guo, T. Goldstein, A. Hannun, and L. van der Maaten. Certified data removal from machine learning models, 2023. URL https://arxiv.org/ abs/1911.03030
2023 arXiv
-
[22]
Huang, D
Y . Huang, D. Liu, L. Chua, B. Ghazi, P. Kamath, R. Kumar, P. Manu- rangsi, M. Nasr, A. Sinha, and C. Zhang. Unlearn and burn: Adversar- ial machine unlearning requests destroy model accuracy, 2024. URL https://arxiv.org/abs/2410.09591
2024 arXiv
- [23]
-
[24]
P. W. Koh and P. Liang. Understanding black-box predictions via influ- ence functions, 2020. URL https://arxiv.org/abs/1703.04730
2020 arXiv
-
[25]
Jeong, S
H. Jeong, S. Ma, and A. Houmansadr. Sok: Challenges and opportu- nities in federated unlearning, 2024. URL https://arxiv.org/abs/2403. 02437
2024
-
[26]
Lapuschkin, S
S. Lapuschkin, S. Wäldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller. Unmasking Clever Hans predictors and assessing what machines really learn. Nature Communications, 10(1):1096, 2019
2019
-
[27]
Krizhevsky and G
A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009
2009
-
[28]
B. Ma, T. Zheng, H. Hu, D. Wang, S. Wang, Z. Ba, Z. Qin, and K. Ren. Releasing malevolence from benevolence: The menace of benign data on machine unlearning, 2024. URL https://arxiv.org/abs/2407.05112
2024 arXiv
-
[29]
Lecun, L
Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner. Gradient-based learn- ing applied to document recognition. Proceedings of the IEEE, 86(11): 2278–2324, 1998. doi: 10.1109/5.726791
1998 doi
-
[31]
Mahadevan and M
A. Mahadevan and M. Mathioudakis. Certifiable machine unlearning for linear models, 2021. URL https://arxiv.org/abs/2106.15093
2021 arXiv
-
[32]
J. Martens. New insights and perspectives on the natural gradient method, 2020. URL https://arxiv.org/abs/1412.1193
2020 arXiv
-
[33]
URL https://arxiv.org/abs/2109.08266
-
[34]
J. Martens. Deep learning via hessian-free optimization. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10, page 735–742, Madison, WI, USA, 2010. Omnipress. ISBN 9781605589077
2010
-
[35]
T. T. Nguyen. awesome-machine-unlearning. Online repos- itory. GitHub, 2025. Available: https://github.com/tamlhp/ awesome-machine-unlearning
2025
-
[36]
Martens and R
J. Martens and R. Grosse. Optimizing neural networks with kronecker- factored approximate curvature, 2020. URL https://arxiv.org/abs/1503. 05671
2020
-
[37]
Mercuri, R
S. Mercuri, R. Khraishi, R. Okhrati, D. Batra, C. Hamill, T. Ghasem- pour, and A. Nowlan. An introduction to machine unlearning, 2022. URL https://arxiv.org/abs/2209.00939
2022 arXiv
-
[38]
Shaik, X
T. Shaik, X. Tao, H. Xie, L. Li, X. Zhu, and Q. Li. Exploring the land- scape of machine unlearning: A comprehensive survey and taxonomy,
-
[39]
T. T. Nguyen, T. T. Huynh, P. L. Nguyen, A. W.-C. Liew, H. Yin, and Q. V . H. Nguyen. A survey of machine unlearning, 2022. URL https: //arxiv.org/abs/2209.02299
2022 arXiv
-
[40]
W. Qian, C. Zhao, W. Le, M. Ma, and M. Huai. Towards understanding and enhancing robustness of deep learning models against malicious unlearning attacks. In A. K. Singh, Y . Sun, L. Akoglu, D. Gunopulos, X. Yan, R. Kumar, F. Ozcan, and J. Ye, editors,Proceedings of the 29th AC...
2023
- [41]
-
[43]
Thudi, H
A. Thudi, H. Jia, I. Shumailov, and N. Papernot. On the necessity of auditable algorithmic definitions for machine unlearning. In K. R. B. Butler and K. Thomas, editors, 31st USENIX Security Symposium, USENIX Security 2022, Boston, MA, USA, August 10-12, 2022 , pages 4007–4022...
2022
-
[44]
Wallis and I
D. Wallis and I. Buvat. Clever hans effect found in a widely used brain tumour mri dataset. Medical Image Analysis , 77: 102368, 2022. ISSN 1361-8415. doi: https://doi.org/10.1016/j.media. 2022.102368. URL https://www.sciencedirect.com/science/article/pii/ S1361841522000214
2022
-
[2021]
doi: 10.1109/SP40001.2021.00019
2021
-
[2024]
URL https://arxiv.org/abs/2305.06360
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.