REVIEW 3 major objections 4 minor 1 cited by
On the importance of multiple training seeds for evaluating machine unlearning
T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Machine unlearning evaluation depends heavily on the training seed used to produce the starting model, and running the unlearning algorithm many times cannot fix that.
desk verdict Useful, reproducible demonstration that single training seeds can mislead unlearning evaluations, but the paper overclaims in the abstract and botches its variance-decomposition explanation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The paper's central device is a variance decomposition for an unlearning metric Z=h(S, S̃), where S is the training seed and S̃ the unlearning seed. The law of total variance splits var[Z] into var[E(Z|S)] + E[var(Z|S)], showing that single-training-seed evaluation estimates only the second (conditional) term, while missing the first term that captures training-seed sensitivity. For deterministic unlearning methods, var(Z|S)=0, so the entire variability lies in the first term.
What would settle it
For a given method and dataset, compute the distribution of the unlearning metric over many randomly chosen training seeds (e.g., 100). If this distribution is concentrated around the value obtained from a single fixed training seed for all methods — that is, if the between-seed variance term var[E(Z|S)] is near zero — then single-training-seed evaluation is not misleading and the paper's advice to always use multiple training seeds would be unnecessary.
Extended reading notes
Core claim
The central discovery is that unlearning metrics are not simply noisy functions of the unlearning algorithm's own randomness; they inherit variability from the training process. For deterministic methods like SSD and LFSSD, the metric is constant across unlearning seeds for a given trained model, so all apparent variation in a typical evaluation comes from the training seed. Because common practice fixes one training seed and varies only unlearning seeds, it estimates the conditional distribution of the metric given that seed, which can be far from the unconditional distribution a user would see in practice. The paper quantifies this through the law of total variance: the unconditional varia
Load-bearing premise
The variance decomposition and the conclusions depend on the 11 training seeds used in each protocol being a representative sample of the seed distribution; if those seeds happen to be unrepresentative, the estimated unconditional distribution could be as misleading as the conditional one.
Editorial extensions
If this is right
- Evaluations that report only a single training seed can rank unlearning methods incorrectly; a method that looks stable may simply be deterministic, and a method that looks variable may be suffering from one unlucky seed.
- Multiple unlearning seeds from one trained model give a false sense of robustness for deterministic methods, since those methods return the same result regardless of unlearning seed.
- The variance decomposition explains the counterintuitive case where a single training seed shows larger variability than multiple training seeds: the within-seed variance can exceed the average within-seed variance.
- The recommendation to use multiple training seeds applies not just to image classification but also to federated online learning to rank, where similar effects appear.
- The paper's guidance implies that studies should report the number of training seeds and unlearning seeds explicitly, and ideally show distributions across training seeds.
Reading between the lines
- The same seed-sensitivity argument likely extends to any approximate model-editing or pruning operation that is deterministic given the trained model, such as quantization or selective weight masking; their apparent reproducibility across runs may hide dependence on initialization.
- The variance decomposition suggests a practical protocol: run a small number of training seeds first to estimate the between-seed term; if it is negligible relative to within-seed variance, then a single training seed may be acceptable, but this must be verified empirically rather than assumed.
- A testable extension is to plot or report the 2-Wasserstein distance between single-seed and multi-seed distributions for each method, which the paper uses; this metric could serve as a standardised diagnostic in future benchmark suites.
- Because the paper uses 11 seeds, a more precise recommendation about how many training seeds are needed would require estimating the standard error of the variance estimates; that is an inference about sample size, not something the paper directly addresses.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that empirical evaluation of machine unlearning methods can be seriously misleading when only one training seed is used, even if multiple unlearning seeds are used. The authors demonstrate that deterministic unlearning methods such as SSD and LFSSD have zero variance across unlearning seeds for a fixed training seed, but large variance across training seeds. They support this with image-classification experiments on CIFAR variants, a federated online learning-to-rank experiment, and a variance-decomposition argument intended to show why increasing the number of unlearning seeds cannot generally compensate for a single training seed. The paper recommends that comparisons of unlearning methods report variability across multiple training seeds, echoing prior work by Cadet et al.
Significance. If the empirical findings hold, the paper addresses a practically important methodological flaw in the machine-unlearning evaluation literature. The demonstration that deterministic unlearning methods are particularly vulnerable to training-seed sensitivity is a useful and non-obvious observation, and the extension to federated learning-to-rank broadens the relevance beyond image classification. The paper is also commendable for releasing reproduction code based on publicly available implementations. The variance-decomposition framing is a principled way to explain the phenomenon, and the empirical results are visually clear. However, the paper's own theoretical support is misstated as written, and the abstract claims an LLM experiment that is not present in the manuscript.
major comments (3)
- [§2, Eq. (2)] The variance-decomposition argument as written is reversed. Eq. (2) defines VB = var[Z] = var[E(Z|S)] + E[VA(S)]. The sentence after Eq. (2) says 'This shows that B neglects the variability due to the training seed implied by the first term from (2)' — but B includes that term, while the common-practice quantity VA(s1) omits it. The next sentence, 'on average, we can expect B to underestimate the variability,' is also the opposite of the inequality implied by Eq. (2): since var[E(Z|S)] ≥ 0, VB ≥ E[VA(S)]. Thus B is, on average, at least as variable as A; a particular training seed can make VA(s1) larger than VB, which is the paper's valid counter-intuitive point, but the stated explanation needs correction before it can support contributions (2) and (3).
- [Abstract and §5] The abstract states that 'Further experiments on federated learning-to-rank, and large language models confirm that this issue extends beyond image classification.' However, the experimental section contains no large-language-model experiments; §5 explicitly lists 'unlearning in large language models' as future work. The FOLTR experiments alone support the claim that the issue extends beyond image classification, but the abstract overclaims by invoking LLMs. Remove the LLM claim from the abstract or provide the missing experiments.
- [§3 and §5] The empirical comparison is based on a single instance of protocol A (one training seed with J=11 unlearning seeds) and a single instance of protocol B (11 training seeds with J=1 unlearning seed), as acknowledged in §5. This is enough to show that there exist seed choices for which the common practice is highly non-representative, but it does not by itself establish the distributional claim that 'multiple unlearning seeds cannot generally compensate.' The latter relies on Eq. (2) and on the 11 seeds being representative of the seed distribution. The manuscript should state this caveat where the claim is introduced in §2–3, not only in the limitations section, and should clarify whether the central claim is an existence result or a distributional result.
minor comments (4)
- [§4] Typo: 'Iamge' should be 'Image' in the sentence describing unlearning metrics.
- [§2] In the bullet defining protocol A, the phrase 'given the single training-seed values 1' is unclear and appears to have a corrupted reference or equation numbering. Please rephrase.
- [§3.2] The FOLTR results are presented as mean trajectories with shaded standard deviations, but no numerical comparison or test statistic is given for the difference between one training seed and ten training seeds. A concise quantitative summary would strengthen the claim.
- [§3.1] Figure 1 panel (b) is described as 'CIFAR20 – sea (sub) class' in the caption, while the text refers to 'CIFAR-20 sub-class'. Please make the class naming consistent.
Circularity Check
No circularity: the variance decomposition is textbook, the experiments are independent, and the Section 2 sign error is a correctness issue, not a circular derivation.
full rationale
The paper's central claim is an empirical demonstration, not a derived prediction that coincides with its inputs. Algorithm 1 directly samples I×J unlearned models, and the comparison between common practice (I=1, J=11) and recommendation (I=11, J=1) is a measured contrast across real training runs on CIFAR, FOLTR, and LLM settings; no parameter is fitted to data and then renamed as a prediction. The variance decomposition in Eq. (2) is a standard identity and is not equivalent, by construction, to the empirical conclusion. The paper's citations to its own authors' earlier work [15,16] are contextual and not load-bearing, and the central recommendation echoes external work [2,27]. The limitations section candidly notes that only one instance of each evaluation approach is compared, which weakens generality but does not make the argument circular. One notable issue in Section 2 is a mathematical exposition error: the text says 'This shows that B neglects the variability due to the training seed implied by the first term from (2)', whereas Eq. (2) shows B includes var[E(Z|S)], and the next sentence 'on average, we can expect B to underestimate the variability' reverses the inequality since VB = E[VA(S)] + var[E(Z|S)] ≥ E[VA(S)]. This is a correctness concern, not circularity: the empirical figures stand independently of that paragraph, and no derivation reduces to its own target. Therefore the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption Training seed fully determines model initialization and training stochasticity.
- domain assumption For deterministic unlearning methods SSD and LFSSD, the unlearning output is a deterministic function of the trained model, so VA ≡ 0.
- standard math The variance decomposition var[Z] = var[E(Z|S)] + E[var(Z|S)] applies to the unlearning metric Z.
- domain assumption Empirical distributions estimated from 11 realizations are informative proxies for the underlying seed distributions.
Cite this review
Pith. "Pith review of On the importance of multiple training seeds for evaluating machine unlearning." pith.science (2026). https://pith.science/paper/2T62Q75O
@misc{pith2026251026714,
author = {Pith},
title = {Pith review of: On the importance of multiple training seeds for evaluating machine unlearning},
year = {2026},
howpublished = {\url{https://pith.science/paper/2T62Q75O}},
note = {Machine review of arXiv:2510.26714}
}
read the original abstract
Machine unlearning aims to remove the influence of certain data points from a trained model without costly retraining. Most practical unlearning algorithms are only approximate and their performance can only be assessed empirically. Common practice is to run unlearning algorithms multiple times independently (i.e., using multiple unlearning seeds) starting from the same trained model (i.e., using only a single training seed ). In image-classification experiments, this practice can give non-representative results as unlearning performance can be sensitive to the choice of training seed. This is particularly relevant for deterministic unlearning methods which always produce the same result when started from the same trained model. Further experiments on federated learning-to-rank, and large language models confirm that this issue extends beyond image classification. We also explain why increasing the number of unlearning seeds cannot generally compensate for the lack of multiple training seeds. Finally, we give guidance on how to select the number of training and unlearning seeds.
Figures
Forward citations
Cited by 1 Pith paper
-
SUPREME: A Multi-GPU Framework for Reproducible Image Unlearning Method Evaluation
SUPREME is a multi-GPU framework with registry-based extensibility that enables reproducible image unlearning evaluation, demonstrated on Pins Face Recognition with ResNet18 and ViT across ten seeds for full-class and...
Reference graph
Works this paper leans on
-
[2]
In: 2025 IEEE 10th European Symposium on Se- curity and Privacy (EuroS&P)
Cadet, X.F., Borovykh, A., Malekzadeh, M., Ahmadi-Abhari, S., Haddadi, H.: Deep Unlearn: Bench- marking Machine Unlearning for Image Classification . In: 2025 IEEE 10th European Symposium on Se- curity and Privacy (EuroS&P). pp. 939–962. IEEE Computer Society, Los Alamitos, CA, USA (Jul 2025). https://doi.org/10.1109/EuroSP63326.2025.00058
arXiv 2025
-
[27]
Triantafillou, E., Kairouz, P., Pedregosa, F., Hayes, J., Kurmanji, M., Zhao, K., Dumoulin, V ., Junior, J.J., Mitliagkas, I., Wan, J., Hosoya, L.S., Escalera, S., Dziugaite, G.K., Triantafillou, P., Guyon, I.: Are we making progress in unlearning? findings from the first neurips unlearning competition (2024), https://arxiv.org/ab s/2406.09073
arXiv 2024
-
[1]
In: 2021 IEEE Symposium on Security and Privacy (SP)
Bourtoule, L., Chandrasekaran, V ., Choquette-Choo, C.A., Jia, H., Travers, A., Zhang, B., Lie, D., Papernot, N.: Machine unlearning. In: 2021 IEEE Symposium on Security and Privacy (SP). pp. 141–159. IEEE (2021)
2021
-
[3]
In: 2015 IEEE Symposium on Security and Privacy
Cao, Y ., Yang, J.: Towards making systems forget with machine unlearning. In: 2015 IEEE Symposium on Security and Privacy. pp. 463–480 (2015). https://doi.org/10.1109/SP.2015.35
-
[4]
In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communica- tions Security
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., Zhang, Y .: When machine unlearning jeop- ardizes privacy. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communica- tions Security. p. 896–911. CCS ’21, Association for Computing Machinery, New York, NY , USA (2021). https://doi.org/10.1145/3460120.3484756
arXiv 2021
-
[5]
In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence
Chundawat, V .S., Tarun, A.K., Mandal, M., Kankanhalli, M.: Can bad teaching induce forgetting? Unlearning in deep networks using an incompetent teacher. In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence. AAAI Press (2023). https://doi.org/10.1609/aaai.v37i6.25879
-
[6]
IEEE Transactions on Information Forensics and Security18, 2345––2354 (2023)
Chundawat, V .S., Tarun, A.K., Mandal, M., Kankanhalli, M.: Zero-shot machine unlearning. IEEE Transactions on Information Forensics and Security18, 2345––2354 (2023). https://doi.org/10.1109/tifs.2023.3265506
arXiv 2023
-
[7]
In: Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence
Foster, J., Schoepf, S., Brintrup, A.: Fast machine unlearning without retraining through selective synaptic dampening. In: Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence. AAAI Press (2024). https://doi.org/10.1609/aaai.v38i11.29092
Show all 31 references
-
[8]
In: Tiny Papers @ ICLR (2024), https: //openreview.net/forum?id=bCPz7uqmmh
Foster, J., Schoepf, S., Brintrup, A.: Loss-free machine unlearning. In: Tiny Papers @ ICLR (2024), https: //openreview.net/forum?id=bCPz7uqmmh
2024
-
[9]
In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Golatkar, A., Achille, A., Ravichandran, A., Polito, M., Soatto, S.: Mixed-Privacy Forgetting in Deep Networks . In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 792–801. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2021). https://doi.or...
2021
-
[10]
In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Golatkar, A., Achille, A., Soatto, S.: Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks . In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9301–9309. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2020). https://...
2020
-
[11]
In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M
Golatkar, A., Achille, A., Soatto, S.: Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.M. (eds.) Computer Vision – ECCV 2020. pp. 383–398. Springer International Publ...
2020
-
[12]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Graves, L., Nagisetty, V ., Ganesh, V .: Amnesiac machine learning. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 11516–11524 (2021)
2021
-
[13]
Halimi, A., Kadhe, S., Rawat, A., Baracaldo, N.: Federated unlearning: How to efficiently erase a client in fl? (2023),https://arxiv.org/abs/2207.05521
2023 arXiv
-
[14]
In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2016). https://doi.org/10.1109/CVPR.2016.90
2016 doi
-
[15]
Hou, J., Finke, A., Cosma, G.: Neural corrective machine unranking (2025), https://arxiv.org/abs/2411.0 8562
2025
-
[16]
IEEE Transactions on Neural Networks and Learning Systems pp
Hou, J., Finke, A., Cosma, G.: Neural machine unranking. IEEE Transactions on Neural Networks and Learning Systems pp. 1–12 (2025). https://doi.org/10.1109/TNNLS.2025.3639808
2025
-
[17]
Krizhevsky, A.: Learning multiple layers of features from tiny images (2009), https://www.cs.toronto.edu /~kriz/cifar.html
2009
-
[18]
Lee, J., Yu, S., Jang, Y ., Woo, S.S., Jo, J.: Unlearning comparator: A visual analytics system for comparative evaluation of machine unlearning methods (2025),https://arxiv.org/abs/2508.12730
2025
-
[19]
In: 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS)
Liu, G., Ma, X., Yang, Y ., Wang, C., Liu, J.: Federaser: Enabling efficient client-level data removal from federated learning models. In: 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS). pp. 1–10 (2021). https://doi.org/10.1109/IWQOS52092.2021.9521274
2021
-
[20]
Computer Law & Security Review29(3), 229–235 (2013)
Mantelero, A.: The EU proposal for a general data protection regulation and the roots of the ‘right to be forgotten’. Computer Law & Security Review29(3), 229–235 (2013). https://doi.org/10.1016/j.clsr.2013.03.010 6 LANYON ET AL. (2025)
2013 doi
-
[21]
CoRRabs/1306.2597(2013), http://arxiv.org/abs/1306 .2597
Qin, T., Liu, T.: Introducing LETOR 4.0 datasets. CoRRabs/1306.2597(2013), http://arxiv.org/abs/1306 .2597
2013 arXiv
-
[22]
In: 2017 IEEE Symposium on Security and Privacy (SP)
Shokri, R., Stronati, M., Song, C., Shmatikov, V .: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP). pp. 3–18. IEEE Computer Society, Los Alamitos, CA, USA (May 2017). https://doi.org/10.1109/SP.2017.41
2017 doi
-
[23]
In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval
Tao, Y ., Wang, S., Yang, J., Zuccon, G.: Unlearning for federated online learning to rank: A reproducibility study. In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. p. 3377–3386. SIGIR ’25, Association for Co...
2025
-
[24]
IEEE Transactions on Neural Networks and Learning Systems35(9), 13046–13055 (2024)
Tarun, A.K., Chundawat, V .S., Mandal, M., Kankanhalli, M.: Fast yet effective machine un- learning. IEEE Transactions on Neural Networks and Learning Systems35(9), 13046–13055 (2024). https://doi.org/10.1109/TNNLS.2023.3266233
2024
-
[25]
In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J
Tarun, A.K., Chundawat, V .S., Mandal, M., Kankanhalli, M.: Deep regression unlearning. In: Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., Scarlett, J. (eds.) Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Learning...
2023
-
[26]
In: 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P)
Thudi, A., Deza, G., Chandrasekaran, V ., Papernot, N.: Unrolling SGD: Understanding Factors Influencing Machine Unlearning . In: 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P). pp. 303–319. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2022). https://d...
2022
-
[28]
In: Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval
Wang, S., Liu, B., Zhuang, S., Zuccon, G.: Effective and privacy-preserving federated online learning to rank. In: Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval. p. 3–12. ICTIR ’21, Association for Computing Machinery, New York, ...
2021
-
[29]
In: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining
Yuan, W., Yin, H., Wu, F., Zhang, S., He, T., Wang, H.: Federated unlearning for on-device recommendation. In: Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. p. 393–401. WSDM ’23, Association for Computing Machinery, New York, NY , USA...
2023
-
[30]
IEEE Transactions on Information Forensics and Security18, 4732–4746 (2023)
Zhang, L., Zhu, T., Zhang, H., Xiong, P., Zhou, W.: Fedrecovery: Differentially private machine unlearning for federated learning frameworks. IEEE Transactions on Information Forensics and Security18, 4732–4746 (2023). https://doi.org/10.1109/TIFS.2023.3297905
2023
-
[31]
In: Proceedings of the 30th ACM International Conference on Multimedia
Zhang, P.F., Bai, G., Huang, Z., Xu, X.S.: Machine unlearning for image retrieval: A generative scrubbing approach. In: Proceedings of the 30th ACM International Conference on Multimedia. p. 237–245. MM ’22, Association for Computing Machinery, New York, NY , USA (2022). https...
2022
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.