REVIEW 2 major objections 4 minor 1 cited by
Federated Unlearning with Gradient Descent and Conflict Mitigation
T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Federated unlearning that erases a client's data in a few rounds without collapsing model utility, by moving orthogonally to the other clients' gradients and projecting post-training updates so the model does not revert.
desk verdict Solid empirical paper with a real reverting problem and good ablations, but the convergence proofs are not valid; referee should ask for repair or reframing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are the Unlearning Cross-Entropy loss and two constrained-update rules. The UCE loss $L_{\mathrm{UCE}}=-\sum_c y_{o,c}\log(1-p_{o,c}/2)$ turns unlearning into a bounded minimization problem, so the target client can use ordinary gradient descent instead of explosive gradient ascent. The orthogonal steepest descent direction $d^t$ is the solution of Problem (5): it keeps $G d^t=0$ against all remaining clients' gradients while maximizing cosine similarity to the target client's unlearning direction, with its norm fixed to $\|g_u^t\|$. The post-training projection $g'^t_i=g^t_i-\frac{g^t_i\cdot g^t_a}{\|g^t_a\|^2}g^t_a$ removes the component of each remaining client's gradient that points back toward the original model $\omega_0$, so utility recovery happens without re-approaching the forgotten client's optimum. These mechanisms together are what the paper argues let FedOSD unlearn fast, preserve retained-client accuracy, and keep the erasure in place.
What would settle it
Run FedOSD's post-training on a convex quadratic federated task with a known optimum; if the projected-gradient path converges to a point with a strictly larger objective value than retraining on the remaining clients, the proof's substitution of projected gradients for true gradients is false.
Extended reading notes
Core claim
FedOSD's central claim is that federated unlearning can be made stable by treating it as constrained gradient descent rather than gradient ascent. The target client minimizes a modified loss $L_{\mathrm{UCE}}=-\sum_c y_{o,c}\log(1-p_{o,c}/2)$, which has a lower bound and therefore avoids the gradient explosion of CE-based gradient ascent. At each round the server solves a constrained search for the update direction $d^t$ that is as close as possible to the target client's unlearning direction $-g_u^t$ while satisfying $G d^t=0$ for the matrix of remaining clients' gradients, so unlearning proceeds without conflicting with the clients that stay. In post-training, each remaining client's gradient is projected onto the normal plane of $g_a=\nabla_{\omega^t}\frac{1}{2}\|\omega^t-\omega_0\|^2$ whenever its inner product with $g_a$ is positive, which prevents the model from moving back toward the original model and re-learning the erased client. The paper reports that this yields zero attack success rate after unlearning and, after post-training, retained-client accuracy close to the pre-unlearning model without the ASR rebound that marks model-reverting in other methods.
Load-bearing premise
The post-training convergence proof relies on the assumption that using each remaining client's gradient with the component toward the original model removed is still a valid gradient of the training objective, so utility recovery is guaranteed only if that replacement does not change the optimum.
Editorial extensions
If this is right
- If FedOSD is right, a server can honor a client deletion request within about 100 unlearning rounds plus a short post-training phase, instead of retraining from scratch.
- The target client's attack success rate can be driven to zero in non-IID and IID settings while retained-client accuracy stays within a few points of the original model, including with 10, 20, or 50 clients.
- The projection step prevents the model-reverting failure that causes baseline methods to regain the erased client's information during post-training.
- Because sample unlearning can be reframed as unlearning a virtual client owning that sample set, the same procedure provides a path to sample-level erasure.
- The method avoids the extra storage of historical-update baselines and avoids the hand-tuned radius parameter of projection-based gradient ascent.
Reading between the lines
- Beyond the paper: the orthogonal unlearning direction exists because the number of remaining clients is far smaller than the parameter dimension; as the client population grows toward the model dimension, the feasible subspace shrinks and the closest-to-target direction becomes more constrained, which could be tested by scaling experiments.
- Beyond the paper: the fixed constant 2 in the UCE loss is calibrated to keep the target client's gradient norm below the remaining clients'; an adaptive variant that rescales this constant from local gradient statistics is a natural extension the paper does not explore.
- Beyond the paper: the post-training projection only blocks movement toward the original checkpoint $\omega_0$; projecting against the span of the unlearned client's historical gradients would be a stronger forgetfulness check, but the paper does not analyze that variant.
- Beyond the paper: the privacy discussion treats plaintext gradients as safe for large batches, but the projection residual $g_i^t-g'^t_i$ could still encode information about the direction to the original model, so a differential-privacy or encryption analysis of the projected updates is an open question.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FedOSD, a client-level federated unlearning algorithm. It replaces gradient ascent with a bounded Unlearning Cross-Entropy (UCE) loss, computes an update direction that is orthogonal to the remaining clients' gradients while closest to the negative gradient of the target client, and applies a gradient projection in the post-training stage to prevent the model from reverting to the pre-unlearning model ω0. The authors provide convergence analyses in Appendix A.2 and evaluate on MNIST, FMNIST, CIFAR-10/100 under IID and non-IID partitions, reporting attack success rate (ASR) and retained accuracy (R-Acc).
Significance. If the empirical results hold, FedOSD is a practical contribution: it achieves near-zero ASR with better retained accuracy than gradient-ascent baselines across many scenarios, and the post-training projection mitigates the reverting phenomenon observed in baselines. The paper releases code, provides runtime measurements, and includes ablations (M1–M6) that support the design choices. However, the theoretical convergence guarantees in Appendix A.2 are not established, so the claim that post-training recovers model utility with a guarantee should be treated as heuristic unless the proofs are repaired.
major comments (2)
- [Appendix A.2, Convergence in the Unlearning Stage] Equation (28) asserts that for any t1 and t2, \tilde{L}_u(\omega_{t1}) \ge \tilde{L}_u(\omega_{t2}) + (-d_t)\cdot(\omega_{t1}-\omega_{t2}) because the angle between -g_t^u and d_t is smaller than 90 degrees. This inequality does not follow from the angle condition, and it is not a valid descent lemma; the subsequent derivation of Eq. (35) and the limit in Eq. (36) therefore do not prove convergence of the unlearning stage.
- [Appendix A.2, Convergence in the Post-training Stage] The proof claims E(A2)=0 for the term A2 = 2\eta\langle\omega_t - \omega^* - \eta\bar{g}, \bar{g}-\bar{g}'\rangle in Eq. (44). However, the projected gradient g'_i is a deterministic, biased function of the true gradient g_i, so \bar{g}' - \bar{g} generally has nonzero expectation. The proof never bounds E\langle\omega_t - \omega^* - \eta\bar{g}, \bar{g}-\bar{g}'\rangle, and Lemma 2's bound (56) does not vanish at the retained optimum. The projected dynamics can have fixed points where \bar{g}'=0 but \bar{g}\neq 0 (for instance, when every true gradient is parallel to \omega_t - \omega_0), so Lemmas 1–3 do not imply the claimed limit (71) to the objective (38). The model-utility recovery in post-training is therefore an empirical heuristic rather than a proven guarantee.
minor comments (4)
- [Appendix A.2 and Tables 1, 6–10] There are typos: 'FedOD' in Eq. (37) should be 'FedOSD', and the signal 'r' is described as 'ignifies' instead of 'signifies' in multiple table captions.
- [Appendix B.2 and Section 4.1] The naming of data partitions is inconsistent: the main text uses Pat-20 and Pat-50, while the appendix tables use Pat-2 and Pat-5. Please unify the terminology to avoid confusion.
- [Section 3.1 and Appendix B.2] The constant '2' in the UCE loss is introduced as a heuristic to bound the target client's gradient norm, and the M6 ablation shows that removing it causes instability. The claim that FedOSD introduces 'no extra hyper-parameters' is therefore somewhat overstated, since this constant is a free design parameter even if it is not tuned.
- [Equation (6)] The SVD notation in Eq. (6) and Eq. (22) uses G G^T = V\Sigma U^T, which is the reverse of the standard SVD convention A=U\Sigma V^T; clarifying the convention would improve readability.
Circularity Check
No significant circularity; the unlearning loss and projection steps are constructed heuristics evaluated against an external backdoor metric, and self-citations are not load-bearing.
full rationale
FedOSD's central components are designed objectives rather than fitted predictions. The UCE loss is explicitly constructed to drive p_o,c toward 0 (Eq. 3) and is then assessed with an independently defined backdoor ASR, so the reported unlearning evaluation does not reduce to the loss definition. The orthogonal steepest descent direction is the closed-form solution of a constrained optimization (Eq. 6) and is not fitted to test outcomes; the ablation M4 against random orthogonal directions gives this construction independent empirical content. The post-training projection (Eq. 7) is a heuristic that enforces g'_i · g_a = 0 by construction, but the paper's convergence claim to the retained FL objective (Problem 38) is supported by a proof that substitutes projected gradients for true gradients; while that proof is mathematically questionable as a guarantee, the flaw is one of correctness and rigor, not circularity, because the conclusion is not assumed in the premises. The hand-set constant '2' in the UCE loss is a hyperparameter choice, not a quantity fitted to the evaluation metric, so it does not make the reported ASR/R-Acc results true by construction. Self-citations to the authors' prior FedMDFG/FedLF work appear only as background for gradient-conflict definitions and SGD conventions, and they are not load-bearing for the central unlearning claims.
Assumptions & free parameters
free parameters (2)
- UCE loss constant '2' =
2
- Learning rate eta per method =
one of {0.005, 0.025, 0.001, 0.0005}
assumptions (4)
- domain assumption Local objectives are L-smooth and mu-strongly convex
- ad hoc to paper Projected gradient g'_i remains a valid descent direction for the true federated objective
- domain assumption Backdoor attack success rate is a valid measure of unlearning
- domain assumption The nullspace of remaining clients' gradients is large enough to contain useful unlearning directions
Cite this review
Pith. "Pith review of Federated Unlearning with Gradient Descent and Conflict Mitigation." pith.science (2026). https://pith.science/paper/R32YG2YG
@misc{pith2026241220200,
author = {Pith},
title = {Pith review of: Federated Unlearning with Gradient Descent and Conflict Mitigation},
year = {2026},
howpublished = {\url{https://pith.science/paper/R32YG2YG}},
note = {Machine review of arXiv:2412.20200}
}
read the original abstract
Federated Learning (FL) has received much attention in recent years. However, although clients are not required to share their data in FL, the global model itself can implicitly remember clients' local data. Therefore, it's necessary to effectively remove the target client's data from the FL global model to ease the risk of privacy leakage and implement ``the right to be forgotten". Federated Unlearning (FU) has been considered a promising way to remove data without full retraining. But the model utility easily suffers significant reduction during unlearning due to the gradient conflicts. Furthermore, when conducting the post-training to recover the model utility, the model is prone to move back and revert what has already been unlearned. To address these issues, we propose Federated Unlearning with Orthogonal Steepest Descent (FedOSD). We first design an unlearning Cross-Entropy loss to overcome the convergence issue of the gradient ascent. A steepest descent direction for unlearning is then calculated in the condition of being non-conflicting with other clients' gradients and closest to the target client's gradient. This benefits to efficiently unlearn and mitigate the model utility reduction. After unlearning, we recover the model utility by maintaining the achievement of unlearning. Finally, extensive experiments in several FL scenarios verify that FedOSD outperforms the SOTA FU algorithms in terms of unlearning and model utility.
Figures
Figures from the paper (6 more)
Forward citations
Cited by 1 Pith paper
-
MMFGU: Multimodal Federated Graph Unlearning
A target-carrier decoupling pipeline with probe-based residual repair and prototype-guided cross-client purge gives the strongest reported utility–unlearning trade-off for multimodal federated graph unlearning.
Reference graph
Works this paper leans on
-
[1]
A.; Jia, H.; Travers, A.; Zhang, B.; Lie, D.; and Papernot, N
Bourtoule, L.; Chandrasekaran, V.; Choquette-Choo, C. A.; Jia, H.; Travers, A.; Zhang, B.; Lie, D.; and Papernot, N. 2021. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), 141--159. IEEE
work page 2021
-
[2]
Brock, A.; De, S.; and Smith, S. L. 2021. Characterizing signal propagation to close the performance gap in unnormalized ResNets. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net
work page 2021
-
[3]
De, K.; and Pedersen, M. 2021. Impact of colour on robustness of deep neural networks. In Proceedings of the IEEE/CVF international conference on computer vision, 21--30
work page 2021
-
[4]
Fliege, J.; and Svaiter, B. F. 2000. Steepest descent methods for multicriteria optimization. Mathematical Methods of Operations Research, 51(3): 479--494
work page 2000
-
[5]
Halimi, A.; Kadhe, S.; Rawat, A.; and Baracaldo, N. 2022. Federated unlearning: How to efficiently erase a client in fl? arXiv preprint arXiv:2207.05521
arXiv 2022
-
[6]
Harding, E. L.; Vanto, J. J.; Clark, R.; Hannah Ji, L.; and Ainsworth, S. C. 2019. Understanding the scope and impact of the california consumer privacy act of 2018. Journal of Data Protection & Privacy, 2(3): 234--253
work page 2019
-
[7]
Krizhevsky, A.; and Hinton, G. 2009. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4)
2009
-
[8]
LeCun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278--2324
1998
Show all 28 references
-
[9]
Li, G.; Shen, L.; Sun, Y.; Hu, Y.; Hu, H.; and Tao, D. 2023. Subspace based federated unlearning. arXiv preprint arXiv:2302.12448
2023 arXiv
-
[10]
K.; Talwalkar, A.; and Smith, V
Li, T.; Sahu, A. K.; Talwalkar, A.; and Smith, V. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3): 50--60
2020
-
[11]
Liu, G.; Ma, X.; Yang, Y.; Wang, C.; and Liu, J. 2021. Federaser: Enabling efficient client-level data removal from federated learning models. In 2021 IEEE/ACM 29th international symposium on quality of service, 1--10. IEEE
2021
-
[12]
Liu, Z.; Jiang, Y.; Shen, J.; Peng, M.; Lam, K.-Y.; and Yuan, X. 2023. A survey on federated unlearning: Challenges, methods, and future directions. arXiv preprint arXiv:2310.20448
2023 arXiv
-
[13]
McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273--1282. PMLR
2017
-
[14]
Pan, Z.; Li, C.; Yu, F.; Wang, S.; Wang, H.; Tang, X.; and Zhao, J. 2024. FedLF: Layer-Wise Fair Federated Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 14527--14535
2024
-
[15]
Pan, Z.; Wang, S.; Li, C.; Wang, H.; Tang, X.; and Zhao, J. 2023. Fedmdfg: Federated learning with multi-gradient descent and fair guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 9364--9371
2023
-
[16]
E.; Perescu-Popescu, L.; and Mastorakis, N
Popescu, M.-C.; Balas, V. E.; Perescu-Popescu, L.; and Mastorakis, N. 2009. Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems, 8(7): 579--588
2009
-
[17]
Saha, G.; Garg, I.; and Roy, K. 2021. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762
2021 arXiv
-
[18]
Su, N.; and Li, B. 2023. Asynchronous federated unlearning. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications, 1--10. IEEE
2023
-
[19]
Voigt, P.; and Von Bussche, A. 2017. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10(3152676): 10--5555
2017
-
[20]
Wang, Z.; Fan, X.; Qi, J.; Wen, C.; Wang, C.; and Yu, R. 2021. Federated Learning with Fair Averaging. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 1615--1623. IJCAI Organization
2021
-
[21]
Wu, C.; Zhu, S.; and Mitra, P. 2022. Federated unlearning with knowledge distillation. arXiv preprint arXiv:2201.09441
2022 arXiv
-
[22]
Wu, L.; Guo, S.; Wang, J.; Hong, Z.; Zhang, J.; and Ding, Y. 2022. Federated unlearning: Guarantee the right of clients to forget. IEEE Network, 36(5): 129--135
2022
-
[23]
Xiao, H.; Rasul, K.; and Vollgraf, R. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747
2017 arXiv
-
[24]
Yang, J.; and Zhao, Y. 2023. A survey of federated unlearning: A taxonomy, challenges and future directions. arXiv preprint arXiv:2310.19218
2023 arXiv
-
[25]
V.; and Yin, H
Ye, G.; Chen, T.; Hung Nguyen, Q. V.; and Yin, H. 2024. Heterogeneous decentralised machine unlearning with seed model distillation. CAAI Transactions on Intelligence Technology
2024
-
[26]
Yu, B.; Mao, W.; Lv, Y.; Zhang, C.; and Xie, Y. 2022. A survey on federated learning in data mining. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1): e1443
2022
-
[27]
Zhang, L.; Zhu, T.; Zhang, H.; Xiong, P.; and Zhou, W. 2023. Fedrecovery: Differentially private machine unlearning for federated learning frameworks. IEEE Transactions on Information Forensics and Security
2023
-
[28]
Zhao, Y.; Wang, P.; Qi, H.; Huang, J.; Wei, Z.; and Zhang, Q. 2023. Federated unlearning with momentum degradation. IEEE Internet of Things Journal
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.