{"id":"870a8ebb-4fb1-4b2c-be4b-42aaef44b007","arxiv_id":"2412.20200","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"FedOSD uses a bounded unlearning loss, an orthogonal steepest descent direction, and gradient projection in post-training to unlearn a federated learning client with little utility loss.","lead":"This paper presents FedOSD, a federated unlearning algorithm that removes a client's data from a shared model while limiting accuracy loss. It reports strong results over prior methods on image benchmarks and provides code.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Post-training convergence proof replaces true gradients with projected ones; the projected dynamics need not minimize the retained clients' objective, so the utility-recovery guarantee in Appendix A.2 is unsupported.","rationale":"The central empirical claim (FedOSD outperforms SOTA on ASR and R-Acc) is supported by broad experiments, ablations, and a public code link. The weakest point is the theoretical guarantee for the post-training stage, exactly as the reader identified. The proof's invalid substitution of projected gradients for true gradients matters because the paper explicitly claims convergence to the retained clients' federated objective, and that claim is what turns utility recovery from a heuristic into a guarantee. This does not overturn the empirical comparison, but it does mean the paper currently overstates its theoretical support. I also noticed a secondary slip: the μ in Eq. (25) makes ||d_t|| = ||g_u||^2 rather than ||g_u|| as claimed, which further suggests the appendix derivations need careful revision. Conditional acceptance remains appropriate: the algorithmic idea and experiments are valuable, but the convergence proofs should be corrected or explicitly softened before the guarantees are taken at face value.","tokens_in":34908,"tokens_out":9121,"duration_ms":88534,"concrete_test":"On a two-client quadratic instance, take L_i(ω) = (1/2)||ω - v_i||^2, choose ω0 so that the retained optimum ω* = (v_1 + v_2)/2 satisfies ω* ≠ ω0, and simulate Algorithm 1's post-training with exact gradients for T = 10^4 steps with decaying η. Record ||ω_T - ω*|| and the angle between the aggregate true gradient and ω_T - ω0. If ||ω_T - ω*|| does not go to 0 (e.g., it stalls on a sphere centered at ω0 with the aggregate gradient parallel to ω_T - ω0), the Appendix A.2 convergence claim is false. If it does converge to ω*, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix A.2's post-training proof (Eqs. 26-27, Lemma 1-3, Eq. 71) assumes the projected gradients g'_i can stand in for the true local gradients g_i when proving convergence to Problem (38). This is not a harmless substitution: the projection plane in Eq. (26) is normal to a_t = ω_t - ω_0, so it depends on the current iterate. The aggregate update g' is not the gradient of L(ω) = Σ (1/|S|) L_i(ω), nor of any fixed potential. A nonzero fixed point of the projected dynamics can occur where every true gradient is parallel to ω_t - ω_0; such a point need not be a stationary point of L. The proof only bounds ||g'_i - g_i|| (Theorem 1) and never shows this bias vanishes at the retained optimum; Lemmas 1-3 therefore do not imply the stated limit (Eq. 71). Consequently, the claim that post-training 'converges to the local optimum of Problem (38)', and hence recovers model utility with a guarantee, is not established. The empirical R-Acc results may still be good, but the utility-recovery guarantee is a heuristic rather than a proven property.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FedOSD, a client-level federated unlearning algorithm. It replaces gradient ascent with a bounded Unlearning Cross-Entropy (UCE) loss, computes an update direction that is orthogonal to the remaining clients' gradients while closest to the negative gradient of the target client, and applies a gradient projection in the post-training stage to prevent the model from reverting to the pre-unlearning model ω0. The authors provide convergence analyses in Appendix A.2 and evaluate on MNIST, FMNIST, CIFAR-10/100 under IID and non-IID partitions, reporting attack success rate (ASR) and retained accuracy (R-Acc).","tokens_in":35143,"tokens_out":8479,"duration_ms":77764,"significance":"If the empirical results hold, FedOSD is a practical contribution: it achieves near-zero ASR with better retained accuracy than gradient-ascent baselines across many scenarios, and the post-training projection mitigates the reverting phenomenon observed in baselines. The paper releases code, provides runtime measurements, and includes ablations (M1–M6) that support the design choices. However, the theoretical convergence guarantees in Appendix A.2 are not established, so the claim that post-training recovers model utility with a guarantee should be treated as heuristic unless the proofs are repaired.","major_comments":[{"comment":"Equation (28) asserts that for any t1 and t2, \\tilde{L}_u(\\omega_{t1}) \\ge \\tilde{L}_u(\\omega_{t2}) + (-d_t)\\cdot(\\omega_{t1}-\\omega_{t2}) because the angle between -g_t^u and d_t is smaller than 90 degrees. This inequality does not follow from the angle condition, and it is not a valid descent lemma; the subsequent derivation of Eq. (35) and the limit in Eq. (36) therefore do not prove convergence of the unlearning stage.","section":"Appendix A.2, Convergence in the Unlearning Stage"},{"comment":"The proof claims E(A2)=0 for the term A2 = 2\\eta\\langle\\omega_t - \\omega^* - \\eta\\bar{g}, \\bar{g}-\\bar{g}'\\rangle in Eq. (44). However, the projected gradient g'_i is a deterministic, biased function of the true gradient g_i, so \\bar{g}' - \\bar{g} generally has nonzero expectation. The proof never bounds E\\langle\\omega_t - \\omega^* - \\eta\\bar{g}, \\bar{g}-\\bar{g}'\\rangle, and Lemma 2's bound (56) does not vanish at the retained optimum. The projected dynamics can have fixed points where \\bar{g}'=0 but \\bar{g}\\neq 0 (for instance, when every true gradient is parallel to \\omega_t - \\omega_0), so Lemmas 1–3 do not imply the claimed limit (71) to the objective (38). The model-utility recovery in post-training is therefore an empirical heuristic rather than a proven guarantee.","section":"Appendix A.2, Convergence in the Post-training Stage"}],"minor_comments":[{"comment":"There are typos: 'FedOD' in Eq. (37) should be 'FedOSD', and the signal 'r' is described as 'ignifies' instead of 'signifies' in multiple table captions.","section":"Appendix A.2 and Tables 1, 6–10"},{"comment":"The naming of data partitions is inconsistent: the main text uses Pat-20 and Pat-50, while the appendix tables use Pat-2 and Pat-5. Please unify the terminology to avoid confusion.","section":"Appendix B.2 and Section 4.1"},{"comment":"The constant '2' in the UCE loss is introduced as a heuristic to bound the target client's gradient norm, and the M6 ablation shows that removing it causes instability. The claim that FedOSD introduces 'no extra hyper-parameters' is therefore somewhat overstated, since this constant is a free design parameter even if it is not tuned.","section":"Section 3.1 and Appendix B.2"},{"comment":"The SVD notation in Eq. (6) and Eq. (22) uses G G^T = V\\Sigma U^T, which is the reverse of the standard SVD convention A=U\\Sigma V^T; clarifying the convention would improve readability.","section":"Equation (6)"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is the main strength of the paper; however, the theoretical appendix contains invalid steps. If the authors can repair the convergence proofs or replace them with a statement that post-training utility recovery is empirically motivated, the paper may be suitable for publication. The current version overclaims the guarantee."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead FedOSD. The empirical core is stronger than the theory around it. The paper is worth taking seriously: it identifies a real failure mode (post-unlearning reverting), proposes a bounded unlearning loss and an orthogonal steepest descent direction that avoids retained clients' gradients, and validates against several FU baselines across MNIST/FMNIST/CIFAR-10/100 and non-IID partitions. The ablations M1-M5 are useful, the runtime is reported, and code is provided. If the unlearning-as-backdoor-removal framing is accepted, the results are convincing: FedOSD drives ASR to near zero while retaining much more utility than GA-based baselines, and the post-training projection keeps ASR low.\n\nThe soft spots are in the theory, and they are not cosmetic. The post-training convergence proof in Appendix A.2 claims convergence to the true federated objective (38), but the update uses projected gradients g'_i, not true gradients. The projection plane depends on omega_t - omega_0, so the aggregate update is not the gradient of any fixed potential. Lemmas 1-3 bound norms but never show the projection bias vanishes at the retained optimum. The proof also seems to assume E[A2]=0 without justification. So the utility-recovery guarantee is not established; the empirical R-Acc recovery is evidence, not a theorem. The unlearning-stage proof has an invalid step too: Eq. (28) does not follow from the angle condition or from convexity, and it is used as a descent inequality. The claim that FedOSD converges to the local optimum of Problem (4) is therefore not proven.\n\nThe unlearning metric is another soft spot: backdoor ASR is a useful proxy, but it is not a formal measure of forgetting. A privacy-oriented reader will want membership-inference or other removal tests before treating the method as satisfying deletion requests. That is a limitation, not a fatal flaw, given how the FU literature evaluates.\n\nNone of this disproves the central algorithmic claim. The experiments are extensive, the baselines are handled carefully, and the reverting problem is genuinely identified. The paper deserves a serious referee. I would recommend accepting it conditionally if the authors either fix the convergence proofs or explicitly reframe the convergence material as heuristic motivation, and if they add a sentence acknowledging that the unlearning success is measured by backdoor removal rather than formal privacy. It would be a useful methodological contribution to the FU subfield.\n\nFor a reading group, I would bring it: the gap between claimed guarantees and actual proof is instructive, and the empirical comparison is a good reference.","headline":"Solid empirical paper with a real reverting problem and good ablations, but the convergence proofs are not valid; referee should ask for repair or reframing.","tokens_in":35651,"tokens_out":2501,"would_cite":true,"duration_ms":25589,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated unlearning that erases a client's data in a few rounds without collapsing model utility, by moving orthogonally to the other clients' gradients and projecting post-training updates so the model does not revert.","keywords":["federated unlearning","right to be forgotten","gradient conflict","orthogonal steepest descent","gradient projection","model utility","backdoor attack success rate","client unlearning"],"falsifier":"Run FedOSD's post-training on a convex quadratic federated task with a known optimum; if the projected-gradient path converges to a point with a strictly larger objective value than retraining on the remaining clients, the proof's substitution of projected gradients for true gradients is false.","tokens_in":34709,"feed_emoji":"🧹","tokens_out":9234,"duration_ms":86310,"temperature":0.7,"pith_summary":"This paper tries to establish that a client's contribution can be removed from a trained federated model in a few communication rounds without full retraining and without the usual collapse in model utility. It proposes FedOSD, which replaces gradient ascent with gradient descent on an Unlearning Cross-Entropy loss, chooses the update direction closest to the target client's gradient while orthogonal to the remaining clients' gradients, and adds a gradient projection in post-training so the model does not drift back to the pre-unlearning state. The reported experiments on MNIST, FMNIST, CIFAR-10, and CIFAR-100 show the attack success rate falling to zero while retained-client accuracy stays near the original model's level, whereas competing methods either destroy utility or regain the erased client's information. If the method works as claimed, it offers a practical route to the right to be forgotten in federated learning without paying the cost of retraining from scratch.","feed_headline":"Erase a client's data without retraining — and keep it erased","feed_subtitle":"FedOSD unlearns along directions that avoid other clients' gradients and blocks post-training reverting.","key_machinery":"The load-bearing objects are the Unlearning Cross-Entropy loss and two constrained-update rules. The UCE loss $L_{\\mathrm{UCE}}=-\\sum_c y_{o,c}\\log(1-p_{o,c}/2)$ turns unlearning into a bounded minimization problem, so the target client can use ordinary gradient descent instead of explosive gradient ascent. The orthogonal steepest descent direction $d^t$ is the solution of Problem (5): it keeps $G d^t=0$ against all remaining clients' gradients while maximizing cosine similarity to the target client's unlearning direction, with its norm fixed to $\\|g_u^t\\|$. The post-training projection $g'^t_i=g^t_i-\\frac{g^t_i\\cdot g^t_a}{\\|g^t_a\\|^2}g^t_a$ removes the component of each remaining client's gradient that points back toward the original model $\\omega_0$, so utility recovery happens without re-approaching the forgotten client's optimum. These mechanisms together are what the paper argues let FedOSD unlearn fast, preserve retained-client accuracy, and keep the erasure in place.","core_discovery":"FedOSD's central claim is that federated unlearning can be made stable by treating it as constrained gradient descent rather than gradient ascent. The target client minimizes a modified loss $L_{\\mathrm{UCE}}=-\\sum_c y_{o,c}\\log(1-p_{o,c}/2)$, which has a lower bound and therefore avoids the gradient explosion of CE-based gradient ascent. At each round the server solves a constrained search for the update direction $d^t$ that is as close as possible to the target client's unlearning direction $-g_u^t$ while satisfying $G d^t=0$ for the matrix of remaining clients' gradients, so unlearning proceeds without conflicting with the clients that stay. In post-training, each remaining client's gradient is projected onto the normal plane of $g_a=\\nabla_{\\omega^t}\\frac{1}{2}\\|\\omega^t-\\omega_0\\|^2$ whenever its inner product with $g_a$ is positive, which prevents the model from moving back toward the original model and re-learning the erased client. The paper reports that this yields zero attack success rate after unlearning and, after post-training, retained-client accuracy close to the pre-unlearning model without the ASR rebound that marks model-reverting in other methods.","pith_inferences":["Beyond the paper: the orthogonal unlearning direction exists because the number of remaining clients is far smaller than the parameter dimension; as the client population grows toward the model dimension, the feasible subspace shrinks and the closest-to-target direction becomes more constrained, which could be tested by scaling experiments.","Beyond the paper: the fixed constant 2 in the UCE loss is calibrated to keep the target client's gradient norm below the remaining clients'; an adaptive variant that rescales this constant from local gradient statistics is a natural extension the paper does not explore.","Beyond the paper: the post-training projection only blocks movement toward the original checkpoint $\\omega_0$; projecting against the span of the unlearned client's historical gradients would be a stronger forgetfulness check, but the paper does not analyze that variant.","Beyond the paper: the privacy discussion treats plaintext gradients as safe for large batches, but the projection residual $g_i^t-g'^t_i$ could still encode information about the direction to the original model, so a differential-privacy or encryption analysis of the projected updates is an open question."],"forward_implications":["If FedOSD is right, a server can honor a client deletion request within about 100 unlearning rounds plus a short post-training phase, instead of retraining from scratch.","The target client's attack success rate can be driven to zero in non-IID and IID settings while retained-client accuracy stays within a few points of the original model, including with 10, 20, or 50 clients.","The projection step prevents the model-reverting failure that causes baseline methods to regain the erased client's information during post-training.","Because sample unlearning can be reframed as unlearning a virtual client owning that sample set, the same procedure provides a path to sample-level erasure.","The method avoids the extra storage of historical-update baselines and avoids the hand-tuned radius parameter of projection-based gradient ascent."],"supporting_citations":[{"why":"Defines the FedAvg pretraining procedure that produces the global model $\\omega_0$ used as the starting point for unlearning.","marker":"McMahan et al. 2017"},{"why":"Establishes the gradient-ascent formulation and L2-ball projection that FedOSD targets as its main GA-based comparison.","marker":"Halimi et al. 2022"},{"why":"Provides EWCSGA, the regularization-based gradient-ascent baseline, and the observation that plain gradient ascent destroys model utility.","marker":"Wu et al. 2022"},{"why":"Supplies FedEraser, the historical-update accelerated retraining baseline whose utility preservation FedOSD must match.","marker":"Liu et al. 2021"},{"why":"Supplies MoDe and the backdoor-trigger protocol used to measure unlearning as attack success rate.","marker":"Zhao et al. 2023"},{"why":"Supplies FedRecovery, the historical-gradient calibration baseline, and the experimental protocol for local SGD with one local epoch.","marker":"Zhang et al. 2023"},{"why":"Provides SFU, the subspace-projection unlearning method whose impracticality motivates FedOSD's gradient-space orthogonal direction.","marker":"Li et al. 2023"},{"why":"Introduces the gradient-conflict notion that justifies the orthogonality constraint on the unlearning direction.","marker":"Wang et al. 2021"}],"fun_headline_variants":["Unlearn client data without retraining or utility loss","Steepest descent for stable federated unlearning","Avoid gradient conflicts while erasing a client's data","Block reverting after client data removal in FL","Orthogonal steepest descent: unlearn without conflicts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The post-training convergence proof relies on the assumption that using each remaining client's gradient with the component toward the original model removed is still a valid gradient of the training objective, so utility recovery is guaranteed only if that replacement does not change the optimum.","fun_headline_variants_meta":{"raw":{"variants":["Unlearn client data without retraining or utility loss","Steepest descent for stable federated unlearning","Avoid gradient conflicts while erasing a client's data","Block reverting after client data removal in FL","Orthogonal steepest descent: unlearn without conflicts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000884,"raw_usage":{"total_tokens":3873,"prompt_tokens":1055,"completion_tokens":2818,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":671,"completion_tokens_details":{"reasoning_tokens":2757}},"tokens_in":671,"tokens_out":2818,"duration_ms":21747,"temperature":1.0,"reasoning_tokens":2757,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:27:19.219861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FedOSD's post-training on a convex quadratic federated task with a known optimum; if the projected-gradient path converges to a point with a strictly larger objective value than retraining on the remaining clients, the proof's substitution of projected gradients for true gradients is false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies FedEraser, the historical-update accelerated retraining baseline whose utility preservation FedOSD must match."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies MoDe and the backdoor-trigger protocol used to measure unlearning as attack success rate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies FedRecovery, the historical-gradient calibration baseline, and the experimental protocol for local SGD with one local epoch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the gradient-conflict notion that justifies the orthogonality constraint on the unlearning direction."}],"review_version":1}