Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

Federated Unlearning with Gradient Descent and Conflict Mitigation

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Federated unlearning that erases a client's data in a few rounds without collapsing model utility, by moving orthogonally to the other clients' gradients and projecting post-training updates so the model does not revert.

desk verdict Solid empirical paper with a real reverting problem and good ablations, but the convergence proofs are not valid; referee should ask for repair or reframing. read the letter →

arxiv 2412.20200 v1 pith:R32YG2YG submitted 2024-12-28 cs.LG cs.AIcs.CRcs.DC

classification cs.LGcs.AIcs.CRcs.DC
keywords federatedunlearningrighttobeforgottengradientconflictorthogonalsteepestdescentprojectionmodelutilitybackdoorattacksuccessrateclient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a client's contribution can be removed from a trained federated model in a few communication rounds without full retraining and without the usual collapse in model utility. It proposes FedOSD, which replaces gradient ascent with gradient descent on an Unlearning Cross-Entropy loss, chooses the update direction closest to the target client's gradient while orthogonal to the remaining clients' gradients, and adds a gradient projection in post-training so the model does not drift back to the pre-unlearning state. The reported experiments on MNIST, FMNIST, CIFAR-10, and CIFAR-100 show the attack success rate falling to zero while retained-client accuracy stays near the original model's level, whereas competing methods either destroy utility or regain the erased client's information. If the method works as claimed, it offers a practical route to the right to be forgotten in federated learning without paying the cost of retraining from scratch.

What carries the argument

The load-bearing objects are the Unlearning Cross-Entropy loss and two constrained-update rules. The UCE loss $L_{\mathrm{UCE}}=-\sum_c y_{o,c}\log(1-p_{o,c}/2)$ turns unlearning into a bounded minimization problem, so the target client can use ordinary gradient descent instead of explosive gradient ascent. The orthogonal steepest descent direction $d^t$ is the solution of Problem (5): it keeps $G d^t=0$ against all remaining clients' gradients while maximizing cosine similarity to the target client's unlearning direction, with its norm fixed to $\|g_u^t\|$. The post-training projection $g'^t_i=g^t_i-\frac{g^t_i\cdot g^t_a}{\|g^t_a\|^2}g^t_a$ removes the component of each remaining client's gradient that points back toward the original model $\omega_0$, so utility recovery happens without re-approaching the forgotten client's optimum. These mechanisms together are what the paper argues let FedOSD unlearn fast, preserve retained-client accuracy, and keep the erasure in place.

What would settle it

Run FedOSD's post-training on a convex quadratic federated task with a known optimum; if the projected-gradient path converges to a point with a strictly larger objective value than retraining on the remaining clients, the proof's substitution of projected gradients for true gradients is false.

Watch

Extended reading notes

Core claim

FedOSD's central claim is that federated unlearning can be made stable by treating it as constrained gradient descent rather than gradient ascent. The target client minimizes a modified loss $L_{\mathrm{UCE}}=-\sum_c y_{o,c}\log(1-p_{o,c}/2)$, which has a lower bound and therefore avoids the gradient explosion of CE-based gradient ascent. At each round the server solves a constrained search for the update direction $d^t$ that is as close as possible to the target client's unlearning direction $-g_u^t$ while satisfying $G d^t=0$ for the matrix of remaining clients' gradients, so unlearning proceeds without conflicting with the clients that stay. In post-training, each remaining client's gradient is projected onto the normal plane of $g_a=\nabla_{\omega^t}\frac{1}{2}\|\omega^t-\omega_0\|^2$ whenever its inner product with $g_a$ is positive, which prevents the model from moving back toward the original model and re-learning the erased client. The paper reports that this yields zero attack success rate after unlearning and, after post-training, retained-client accuracy close to the pre-unlearning model without the ASR rebound that marks model-reverting in other methods.

Load-bearing premise

The post-training convergence proof relies on the assumption that using each remaining client's gradient with the component toward the original model removed is still a valid gradient of the training objective, so utility recovery is guaranteed only if that replacement does not change the optimum.

Editorial extensions

If this is right

  • If FedOSD is right, a server can honor a client deletion request within about 100 unlearning rounds plus a short post-training phase, instead of retraining from scratch.
  • The target client's attack success rate can be driven to zero in non-IID and IID settings while retained-client accuracy stays within a few points of the original model, including with 10, 20, or 50 clients.
  • The projection step prevents the model-reverting failure that causes baseline methods to regain the erased client's information during post-training.
  • Because sample unlearning can be reframed as unlearning a virtual client owning that sample set, the same procedure provides a path to sample-level erasure.
  • The method avoids the extra storage of historical-update baselines and avoids the hand-tuned radius parameter of projection-based gradient ascent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the orthogonal unlearning direction exists because the number of remaining clients is far smaller than the parameter dimension; as the client population grows toward the model dimension, the feasible subspace shrinks and the closest-to-target direction becomes more constrained, which could be tested by scaling experiments.
  • Beyond the paper: the fixed constant 2 in the UCE loss is calibrated to keep the target client's gradient norm below the remaining clients'; an adaptive variant that rescales this constant from local gradient statistics is a natural extension the paper does not explore.
  • Beyond the paper: the post-training projection only blocks movement toward the original checkpoint $\omega_0$; projecting against the span of the unlearned client's historical gradients would be a stronger forgetfulness check, but the paper does not analyze that variant.
  • Beyond the paper: the privacy discussion treats plaintext gradients as safe for large batches, but the projection residual $g_i^t-g'^t_i$ could still encode information about the direction to the original model, so a differential-privacy or encryption analysis of the projected updates is an open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes FedOSD, a client-level federated unlearning algorithm. It replaces gradient ascent with a bounded Unlearning Cross-Entropy (UCE) loss, computes an update direction that is orthogonal to the remaining clients' gradients while closest to the negative gradient of the target client, and applies a gradient projection in the post-training stage to prevent the model from reverting to the pre-unlearning model ω0. The authors provide convergence analyses in Appendix A.2 and evaluate on MNIST, FMNIST, CIFAR-10/100 under IID and non-IID partitions, reporting attack success rate (ASR) and retained accuracy (R-Acc).

Significance. If the empirical results hold, FedOSD is a practical contribution: it achieves near-zero ASR with better retained accuracy than gradient-ascent baselines across many scenarios, and the post-training projection mitigates the reverting phenomenon observed in baselines. The paper releases code, provides runtime measurements, and includes ablations (M1–M6) that support the design choices. However, the theoretical convergence guarantees in Appendix A.2 are not established, so the claim that post-training recovers model utility with a guarantee should be treated as heuristic unless the proofs are repaired.

major comments (2)
  1. [Appendix A.2, Convergence in the Unlearning Stage] Equation (28) asserts that for any t1 and t2, \tilde{L}_u(\omega_{t1}) \ge \tilde{L}_u(\omega_{t2}) + (-d_t)\cdot(\omega_{t1}-\omega_{t2}) because the angle between -g_t^u and d_t is smaller than 90 degrees. This inequality does not follow from the angle condition, and it is not a valid descent lemma; the subsequent derivation of Eq. (35) and the limit in Eq. (36) therefore do not prove convergence of the unlearning stage.
  2. [Appendix A.2, Convergence in the Post-training Stage] The proof claims E(A2)=0 for the term A2 = 2\eta\langle\omega_t - \omega^* - \eta\bar{g}, \bar{g}-\bar{g}'\rangle in Eq. (44). However, the projected gradient g'_i is a deterministic, biased function of the true gradient g_i, so \bar{g}' - \bar{g} generally has nonzero expectation. The proof never bounds E\langle\omega_t - \omega^* - \eta\bar{g}, \bar{g}-\bar{g}'\rangle, and Lemma 2's bound (56) does not vanish at the retained optimum. The projected dynamics can have fixed points where \bar{g}'=0 but \bar{g}\neq 0 (for instance, when every true gradient is parallel to \omega_t - \omega_0), so Lemmas 1–3 do not imply the claimed limit (71) to the objective (38). The model-utility recovery in post-training is therefore an empirical heuristic rather than a proven guarantee.
minor comments (4)
  1. [Appendix A.2 and Tables 1, 6–10] There are typos: 'FedOD' in Eq. (37) should be 'FedOSD', and the signal 'r' is described as 'ignifies' instead of 'signifies' in multiple table captions.
  2. [Appendix B.2 and Section 4.1] The naming of data partitions is inconsistent: the main text uses Pat-20 and Pat-50, while the appendix tables use Pat-2 and Pat-5. Please unify the terminology to avoid confusion.
  3. [Section 3.1 and Appendix B.2] The constant '2' in the UCE loss is introduced as a heuristic to bound the target client's gradient norm, and the M6 ablation shows that removing it causes instability. The claim that FedOSD introduces 'no extra hyper-parameters' is therefore somewhat overstated, since this constant is a free design parameter even if it is not tuned.
  4. [Equation (6)] The SVD notation in Eq. (6) and Eq. (22) uses G G^T = V\Sigma U^T, which is the reverse of the standard SVD convention A=U\Sigma V^T; clarifying the convention would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the unlearning loss and projection steps are constructed heuristics evaluated against an external backdoor metric, and self-citations are not load-bearing.

full rationale

FedOSD's central components are designed objectives rather than fitted predictions. The UCE loss is explicitly constructed to drive p_o,c toward 0 (Eq. 3) and is then assessed with an independently defined backdoor ASR, so the reported unlearning evaluation does not reduce to the loss definition. The orthogonal steepest descent direction is the closed-form solution of a constrained optimization (Eq. 6) and is not fitted to test outcomes; the ablation M4 against random orthogonal directions gives this construction independent empirical content. The post-training projection (Eq. 7) is a heuristic that enforces g'_i · g_a = 0 by construction, but the paper's convergence claim to the retained FL objective (Problem 38) is supported by a proof that substitutes projected gradients for true gradients; while that proof is mathematically questionable as a guarantee, the flaw is one of correctness and rigor, not circularity, because the conclusion is not assumed in the premises. The hand-set constant '2' in the UCE loss is a hyperparameter choice, not a quantity fitted to the evaluation metric, so it does not make the reported ASR/R-Acc results true by construction. Self-citations to the authors' prior FedMDFG/FedLF work appear only as background for gradient-conflict definitions and SGD conventions, and they are not load-bearing for the central unlearning claims.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The algorithm introduces no new physical or conceptual entities. It relies on standard optimization assumptions for convergence, a hand-set constant in the unlearning loss, a learning-rate search, and the proxy assumption that backdoor ASR measures forgetting. The central contribution is therefore empirical rather than axiomatic.

free parameters (2)
  • UCE loss constant '2' = 2
    Hand-chosen in Eq (3) to keep the target client's gradient norm from exceeding remaining clients' gradient norms; only an ablation without division by 2 is shown, no sweep over other constants.
  • Learning rate eta per method = one of {0.005, 0.025, 0.001, 0.0005}
    The best learning rate is selected for each algorithm during comparison, which is standard but means reported results depend on per-method tuning.
assumptions (4)
  • domain assumption Local objectives are L-smooth and mu-strongly convex
    Used in Appendix A.2 Assumptions 1-2 for the post-training convergence proof; deep neural network losses in the experiments are not strongly convex, so the proof does not strictly apply to the evaluated models.
  • ad hoc to paper Projected gradient g'_i remains a valid descent direction for the true federated objective
    The post-training projection in Eq (26) removes a component of g_i, so the aggregated update is no longer the true gradient of Problem (38); convergence to the true optimum is assumed rather than shown.
  • domain assumption Backdoor attack success rate is a valid measure of unlearning
    Section 4.1 uses low ASR on poisoned target-client samples as the effectiveness metric, but ASR is a behavioral proxy and not a membership-inference or certified-forgetting guarantee.
  • domain assumption The nullspace of remaining clients' gradients is large enough to contain useful unlearning directions
    Section 3.2 relies on rank(G) <= m-1 << D so that many orthogonal directions exist; this holds for overparameterized models but may fail in low-dimensional regimes where the target gradient lies almost entirely in the span of remaining gradients.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Federated Unlearning with Gradient Descent and Conflict Mitigation." pith.science (2026). https://pith.science/paper/R32YG2YG

@misc{pith2026241220200,
  author       = {Pith},
  title        = {Pith review of: Federated Unlearning with Gradient Descent and Conflict Mitigation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R32YG2YG}},
  note         = {Machine review of arXiv:2412.20200}
}
read the original abstract

Federated Learning (FL) has received much attention in recent years. However, although clients are not required to share their data in FL, the global model itself can implicitly remember clients' local data. Therefore, it's necessary to effectively remove the target client's data from the FL global model to ease the risk of privacy leakage and implement ``the right to be forgotten". Federated Unlearning (FU) has been considered a promising way to remove data without full retraining. But the model utility easily suffers significant reduction during unlearning due to the gradient conflicts. Furthermore, when conducting the post-training to recover the model utility, the model is prone to move back and revert what has already been unlearned. To address these issues, we propose Federated Unlearning with Orthogonal Steepest Descent (FedOSD). We first design an unlearning Cross-Entropy loss to overcome the convergence issue of the gradient ascent. A steepest descent direction for unlearning is then calculated in the condition of being non-conflicting with other clients' gradients and closest to the target client's gradient. This benefits to efficiently unlearn and mitigate the model utility reduction. After unlearning, we recover the model utility by maintaining the achievement of unlearning. Finally, extensive experiments in several FL scenarios verify that FedOSD outperforms the SOTA FU algorithms in terms of unlearning and model utility.

Figures

Figures reproduced from arXiv: 2412.20200 by the authors.

Figure 1
Figure 1. A demo of three clients. g1, g2, g3 represent the gradient of clients. d denotes the update direction for un￾learning client 3, which is conflicting with g1 and g2, i.e., g1 · d < 0 and g2 · d < 0. dF edOSD represents the direction obtained by FedOSD, which doesn’t conflict with g1 and g2. trained model (Zhang et al. 2023), especially when clients opt to withdraw from FL. This is known as the right to be for￾gotten … view at source ↗
Figure 2
Figure 2. The FedOSD framework comprises two main stages: (b) the unlearning stage and (c) the post-training stage. Subfigure (a) depicts the previous FL training pro￾cedure before the client requests for unlearning, where the obtained model is denoted as ω 0 and serves as the original model for unlearning. the target data that needed to be unlearned, they are not suit￾able for sample unlearning. Since in the sample unlearnin… view at source ↗
Figure 3
Figure 3. A comparison between (a) Cross-Entropy and (b) [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: A demo depicting the model reverting issue in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The ASR, the mean R-Acc, and the distance away from [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The best, the average, and the worst R-Acc across [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: A demo of adding backdoor triggers on the data sample of (a) MNIST, (b) FMNIST, (c) CIFAR-10, and (d) CIFAR [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: The ASR, mean R-Acc, and the distance away from [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: The ASR, the mean R-Acc, and the distance away from [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MMFGU: Multimodal Federated Graph Unlearning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A target-carrier decoupling pipeline with probe-based residual repair and prototype-guided cross-client purge gives the strongest reported utility–unlearning trade-off for multimodal federated graph unlearning.

Reference graph

Works this paper leans on

28 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [1]

    A.; Jia, H.; Travers, A.; Zhang, B.; Lie, D.; and Papernot, N

    Bourtoule, L.; Chandrasekaran, V.; Choquette-Choo, C. A.; Jia, H.; Travers, A.; Zhang, B.; Lie, D.; and Papernot, N. 2021. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), 141--159. IEEE

  2. [2]

    Brock, A.; De, S.; and Smith, S. L. 2021. Characterizing signal propagation to close the performance gap in unnormalized ResNets. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net

  3. [3]

    De, K.; and Pedersen, M. 2021. Impact of colour on robustness of deep neural networks. In Proceedings of the IEEE/CVF international conference on computer vision, 21--30

  4. [4]

    Fliege, J.; and Svaiter, B. F. 2000. Steepest descent methods for multicriteria optimization. Mathematical Methods of Operations Research, 51(3): 479--494

  5. [5]

    Halimi, A.; Kadhe, S.; Rawat, A.; and Baracaldo, N. 2022. Federated unlearning: How to efficiently erase a client in fl? arXiv preprint arXiv:2207.05521

  6. [6]

    L.; Vanto, J

    Harding, E. L.; Vanto, J. J.; Clark, R.; Hannah Ji, L.; and Ainsworth, S. C. 2019. Understanding the scope and impact of the california consumer privacy act of 2018. Journal of Data Protection & Privacy, 2(3): 234--253

  7. [7]

    Krizhevsky, A.; and Hinton, G. 2009. Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4)

  8. [8]

    LeCun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278--2324

Show all 28 references
  1. [9]

    Li, G.; Shen, L.; Sun, Y.; Hu, Y.; Hu, H.; and Tao, D. 2023. Subspace based federated unlearning. arXiv preprint arXiv:2302.12448

  2. [10]

    K.; Talwalkar, A.; and Smith, V

    Li, T.; Sahu, A. K.; Talwalkar, A.; and Smith, V. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3): 50--60

  3. [11]

    Liu, G.; Ma, X.; Yang, Y.; Wang, C.; and Liu, J. 2021. Federaser: Enabling efficient client-level data removal from federated learning models. In 2021 IEEE/ACM 29th international symposium on quality of service, 1--10. IEEE

  4. [12]

    Liu, Z.; Jiang, Y.; Shen, J.; Peng, M.; Lam, K.-Y.; and Yuan, X. 2023. A survey on federated unlearning: Challenges, methods, and future directions. arXiv preprint arXiv:2310.20448

  5. [13]

    McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273--1282. PMLR

  6. [14]

    Pan, Z.; Li, C.; Yu, F.; Wang, S.; Wang, H.; Tang, X.; and Zhao, J. 2024. FedLF: Layer-Wise Fair Federated Learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 14527--14535

  7. [15]

    Pan, Z.; Wang, S.; Li, C.; Wang, H.; Tang, X.; and Zhao, J. 2023. Fedmdfg: Federated learning with multi-gradient descent and fair guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 9364--9371

  8. [16]

    E.; Perescu-Popescu, L.; and Mastorakis, N

    Popescu, M.-C.; Balas, V. E.; Perescu-Popescu, L.; and Mastorakis, N. 2009. Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems, 8(7): 579--588

  9. [17]

    Saha, G.; Garg, I.; and Roy, K. 2021. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762

  10. [18]

    Su, N.; and Li, B. 2023. Asynchronous federated unlearning. In IEEE INFOCOM 2023-IEEE Conference on Computer Communications, 1--10. IEEE

  11. [19]

    Voigt, P.; and Von Bussche, A. 2017. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing, 10(3152676): 10--5555

  12. [20]

    Wang, Z.; Fan, X.; Qi, J.; Wen, C.; Wang, C.; and Yu, R. 2021. Federated Learning with Fair Averaging. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 1615--1623. IJCAI Organization

  13. [21]

    Wu, C.; Zhu, S.; and Mitra, P. 2022. Federated unlearning with knowledge distillation. arXiv preprint arXiv:2201.09441

  14. [22]

    Wu, L.; Guo, S.; Wang, J.; Hong, Z.; Zhang, J.; and Ding, Y. 2022. Federated unlearning: Guarantee the right of clients to forget. IEEE Network, 36(5): 129--135

  15. [23]

    Xiao, H.; Rasul, K.; and Vollgraf, R. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  16. [24]

    Yang, J.; and Zhao, Y. 2023. A survey of federated unlearning: A taxonomy, challenges and future directions. arXiv preprint arXiv:2310.19218

  17. [25]

    V.; and Yin, H

    Ye, G.; Chen, T.; Hung Nguyen, Q. V.; and Yin, H. 2024. Heterogeneous decentralised machine unlearning with seed model distillation. CAAI Transactions on Intelligence Technology

  18. [26]

    Yu, B.; Mao, W.; Lv, Y.; Zhang, C.; and Xie, Y. 2022. A survey on federated learning in data mining. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12(1): e1443

  19. [27]

    Zhang, L.; Zhu, T.; Zhang, H.; Xiong, P.; and Zhou, W. 2023. Fedrecovery: Differentially private machine unlearning for federated learning frameworks. IEEE Transactions on Information Forensics and Security

  20. [28]

    Zhao, Y.; Wang, P.; Qi, H.; Huang, J.; Wei, Z.; and Zhang, Q. 2023. Federated unlearning with momentum degradation. IEEE Internet of Things Journal

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.