pith. sign in

arxiv: 2605.20341 · v1 · pith:RW2J74PZnew · submitted 2026-05-19 · 💻 cs.LG · cs.AI· cs.CR· cs.PF

Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions

Ali Mahdavi , Azadeh Zamanifar , Amirfarhad Farhadi , Omid Kashefi This is my paper

Pith reviewed 2026-05-21 08:19 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.CRcs.PF
keywords federated learningmachine unlearninginfluence functionsKrylov subspacecausal weightingdata deletionprivacy
0
0 comments X

The pith

HF-KCU approximates influence functions via Krylov subspaces to unlearn client data in federated learning with causal updates only to affected clients.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Federated learning must comply with data deletion for privacy but full retraining is too slow. The paper proposes HF-KCU which approximates the effect of removing a client's data using conjugate gradient in a Krylov subspace, lowering cost from cubic to linear in dimension. Causal weighting restricts parameter changes to clients that actually held the deleted data, avoiding unnecessary updates elsewhere. Experiments on image datasets show it matches retraining accuracy and privacy metrics while being nearly 48 times faster. Convergence analysis ties the error to the square root of the Hessian condition number.

Core claim

The central claim is that influence reversal for unlearning can be achieved exactly or approximately by solving a reduced system in the Krylov subspace generated by conjugate gradient iterations, augmented by causal weighting that localizes updates based on data possession, all while tolerating bounded adversarial changes to the Hessian and gradients.

What carries the argument

Krylov subspace conjugate gradient approximation of the inverse-Hessian influence function, combined with causal weighting to ensure only clients with the deleted data receive updates.

If this is right

  • Compliance with right-to-be-forgotten regulations becomes feasible in large-scale federated deployments without repeated full retraining.
  • Model quality for clients without the deleted data remains unchanged, preventing degradation in asynchronous settings.
  • Privacy is restored as membership inference attacks on the forget set perform no better than chance, matching the retrained model.
  • Each update is traceable to the influence of the specific deleted data, offering interpretability.
  • The approach scales to transformer architectures like ViT-Lite on partitioned datasets such as CIFAR-10.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the bounded perturbation assumption holds broadly, HF-KCU could extend to non-federated distributed optimization problems.
  • Further reducing the subspace dimension k might trade accuracy for even higher speedups in very large models.
  • Applying the method to sequential deletions could test cumulative effects on model stability over time.

Load-bearing premise

The approach assumes that adversarial perturbations to the Hessian and gradient stay bounded, which allows the Krylov approximation and causal weighting to deliver stable and localized updates.

What would settle it

A test where the membership inference success rate on the forget set rises above 0.5 or where test accuracy falls more than 0.6% below the retrained baseline on CIFAR-10 with Dirichlet alpha 0.5 partitioning would disprove the performance and privacy claims.

Figures

Figures reproduced from arXiv: 2605.20341 by Ali Mahdavi, Amirfarhad Farhadi, Azadeh Zamanifar, Omid Kashefi.

Figure 1
Figure 1. Figure 1: Performance comparison of HF-KCU against baselines. (a) Pareto frontier showing speedup [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
read the original abstract

Federated learning systems must support data deletion requests to comply with privacy regulations, yet retraining from scratch after each deletion is computationally prohibitive. We present HF-KCU, a method that removes a client's contribution by approximating the influence function through conjugate gradient iterations in Krylov subspaces, reducing complexity from O(d^3) to O(kd) where k<<d.A causal weighting mechanism ensures that only clients holding the deleted data receive parameter updates, preventing spurious changes to unaffected clients. Our method is designed to handle bounded adversarial perturbations to the Hessian and gradient, providing graceful degradation under realistic threat models. We validate HF-KCU across convolutional (ResNet-18, SimpleCNN) and transformer (ViT-Lite) architectures on CIFAR-10, MNIST, and Fashion-MNIST. On CIFAR-10 under Dirichlet (alpha=0.5) partitioning, HF-KCU achieves 47.75 times speedup over retraining while maintaining test accuracy within 0.60% of the rational baseline(71.16 vs 71.76 %). Membership inference attacks on the forget set yield success rates of 0.499 matching the retrained model and confirming effective privacy restoration. We provide convergence guarantees showing that the Krylov approximation error decreases as O((k ^1/2-1)/(k^1/2+1)) where k is the Hessian condition number. The causal weighting mechanism ensures surgical updates, where only clients holding deleted data are modified, preserving model quality for unaffected participants and avoiding the instability of gradient-based approaches in asynchronous federated settings. This design provides interpretability as each update is directly traceable to the influence of the deleted data. The method's efficiency and precision make it suitable for production federated systems where deletion requests arrive asynchronously and computational budgets are constrained.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 1 minor

Summary. The manuscript presents HF-KCU for unlearning client data contributions in federated learning. It approximates influence functions via conjugate gradient iterations restricted to Krylov subspaces of dimension k, reducing per-update cost from O(d^3) to O(kd). A causal weighting scheme restricts parameter changes to clients that hold the deleted data. The approach is claimed to tolerate bounded adversarial perturbations to the Hessian and gradient. Experiments on ResNet-18, SimpleCNN and ViT-Lite across CIFAR-10, MNIST and Fashion-MNIST report a 47.75× speedup on Dirichlet-partitioned CIFAR-10 while keeping test accuracy within 0.60 % of the retrained baseline and producing membership-inference success rates of 0.499 on the forget set.

Significance. If the approximation error can be rigorously bounded and the causal weighting shown to be stable under asynchrony, the method would supply a practical, interpretable mechanism for regulatory-compliant deletion in federated systems. The reported speedups and privacy metrics are quantitatively concrete, yet the absence of a correct iteration-dependent convergence rate leaves the claimed graceful degradation and accuracy retention without theoretical support.

major comments (1)
  1. [Abstract] Abstract, convergence-guarantee paragraph: the stated rate O((k^{1/2}-1)/(k^{1/2}+1)) re-uses the symbol k already defined as Krylov dimension and omits any dependence on iteration count t. Standard CG analysis bounds the A-norm error by a quantity of the form [(√κ-1)/(√κ+1)]^{2t} where κ is the condition number; the given scalar expression is therefore non-informative. Because the 47.75× speedup and 0.60 % accuracy gap both rest on the Krylov approximation being sufficiently accurate, this misstatement is load-bearing for the central efficiency and privacy claims.
minor comments (1)
  1. [Abstract] Abstract: 'rational baseline(71.16 vs 71.76 %)' lacks a space and the term 'rational baseline' is undefined; it appears to denote the fully retrained model.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and for highlighting the notation and completeness issues in the abstract's convergence statement. We agree this requires correction and will revise the manuscript to use consistent notation and include the iteration dependence, thereby strengthening the link between the theoretical bound and the reported empirical results.

read point-by-point responses

  1. Referee: [Abstract] Abstract, convergence-guarantee paragraph: the stated rate O((k^{1/2}-1)/(k^{1/2}+1)) re-uses the symbol k already defined as Krylov dimension and omits any dependence on iteration count t. Standard CG analysis bounds the A-norm error by a quantity of the form [(√κ-1)/(√κ+1)]^{2t} where κ is the condition number; the given scalar expression is therefore non-informative. Because the 47.75× speedup and 0.60 % accuracy gap both rest on the Krylov approximation being sufficiently accurate, this misstatement is load-bearing for the central efficiency and privacy claims.

    Authors: We acknowledge the error in notation and the incomplete rate expression. The symbol k was inadvertently reused; it should be κ for the Hessian condition number. The stated scalar is the per-iteration contraction factor but omits the exponent 2t. The correct bound is O([(√κ−1)/(√κ+1)]^{2t}). In the body of the paper we derive this rate for the Krylov-subspace CG approximation under bounded perturbations (Theorem 3.2). We will revise the abstract to employ κ, include the full iteration-dependent form, and add an explicit cross-reference to the theorem. We will also insert a short remark explaining that, for the chosen Krylov dimension k, the bound guarantees the approximation error remains small enough to preserve the observed accuracy gap and privacy metrics, thereby providing the missing theoretical support for graceful degradation. revision: yes

Circularity Check

0 steps flagged

No circularity: derivation relies on standard Krylov approximation and external influence-function literature

full rationale

The paper's core claim is an efficient approximation of influence functions via conjugate-gradient iterations in Krylov subspaces, combined with a causal weighting scheme that restricts updates to clients holding the deleted data. Both the O(kd) complexity reduction and the causal mechanism are presented as direct algorithmic consequences of the chosen subspace method and the definition of influence reversal; they are not obtained by fitting parameters to the target accuracy or MIA metrics and then relabeling those fits as predictions. The reported 47.75× speedup and 0.60 % accuracy gap are empirical outcomes measured against retraining, not quantities that appear inside the derivation itself. The convergence statement, although mathematically imprecise in the abstract, is offered as an external guarantee rather than a self-referential definition that forces the experimental numbers. No self-citation chain is load-bearing for the central result, and the method remains falsifiable against the retrained baseline. Consequently the derivation chain does not collapse to its inputs by construction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claims rest on the assumption of bounded adversarial perturbations to Hessian and gradient, plus the choice of Krylov dimension k much smaller than model dimension d. No explicit free parameters are named, but the subspace size k functions as a tunable approximation parameter.

free parameters (1)
  • k (Krylov subspace dimension)
    Selected such that k << d to achieve O(kd) complexity; its value directly controls the approximation error bound stated in the abstract.
axioms (1)
  • domain assumption Hessian and gradient perturbations remain bounded under realistic threat models
    Invoked to guarantee graceful degradation and stability of the causal weighting updates.

pith-pipeline@v0.9.0 · 5881 in / 1449 out tokens · 48698 ms · 2026-05-21T08:19:52.268463+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages · 1 internal anchor

  1. [1]

    Artificial Intelligence and Statistics , pages =

    Brendan McMahan and Eider Moore and Daniel Ramage and Seth Hampson and Blaise Aguera y Arcas , title =. Artificial Intelligence and Statistics , pages =. 2017 , organization =

    work page 2017

  2. [2]

    Brendan McMahan and Brendan Avent and Aur

    Peter Kairouz and H. Brendan McMahan and Brendan Avent and Aur. Advances and open problems in federated learning , journal =

    work page

  3. [3]

    Proceedings of Machine Learning and Systems , volume =

    Tian Li and Anit Kumar Sahu and Manzil Zaheer and Maziar Sanjabi and Ameet Talwalkar and Virginia Smith , title =. Proceedings of Machine Learning and Systems , volume =

    work page

  4. [4]

    ACM Computing Surveys , year =

    Ziyao Liu and Yu Jiang and Jiyuan Shen and Minyi Peng and Kwok-Yan Lam and Xingliang Yuan and Xiaoning Liu , title =. ACM Computing Surveys , year =

    work page

  5. [5]

    2015 IEEE Symposium on Security and Privacy , pages =

    Yinzhi Cao and Junfeng Yang , title =. 2015 IEEE Symposium on Security and Privacy , pages =. 2015 , organization =

    work page 2015

  6. [6]

    Choquette-Choo and Hengrui Jia and Adelin Travers and Baiwu Zhang and David Lie and Nicolas Papernot , title =

    Lucas Bourtoule and Varun Chandrasekaran and Christopher A. Choquette-Choo and Hengrui Jia and Adelin Travers and Baiwu Zhang and David Lie and Nicolas Papernot , title =. 2021 IEEE Symposium on Security and Privacy (SP) , pages =. 2021 , organization =

    work page 2021

  7. [7]

    International Conference on Machine Learning , pages =

    Chuan Guo and Tom Goldstein and Awni Hannun and Laurens Van Der Maaten , title =. International Conference on Machine Learning , pages =. 2020 , organization =

    work page 2020

  8. [8]

    Zou , title =

    Antonio Ginart and Melody Guan and Gregory Valiant and James Y. Zou , title =. Advances in Neural Information Processing Systems , volume =

    work page

  9. [9]

    2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS) , pages =

    Gaoyang Liu and Xiaoqiang Ma and Yang Yang and Chen Wang and Jiangchuan Liu , title =. 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS) , pages =. 2021 , organization =

    work page 2021

  10. [10]

    2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS) , pages =

    Yue Wu and Shuaicheng Zhao and Wenchao Li and Jie Liu and Haibing Guan , title =. 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS) , pages =. 2022 , organization =

    work page 2022

  11. [11]

    Federated unlearning: How to effi- ciently erase a client in fl? arXiv preprint arXiv:2207.05521,

    Anisa Halimi and Swanand Kadhe and Ambrish Rawat and Nathalie Baracaldo , title =. arXiv preprint arXiv:2207.05521 , year =

    work page arXiv

  12. [12]

    International Conference on Machine Learning , pages =

    Pang Wei Koh and Percy Liang , title =. International Conference on Machine Learning , pages =. 2017 , organization =

    work page 2017

  13. [13]

    Dennis Cook and Sanford Weisberg , title =

    R. Dennis Cook and Sanford Weisberg , title =. Technometrics , volume =

    work page

  14. [14]

    Jorge Nocedal and Stephen Wright , title =

    work page

  15. [15]

    Proceedings of the 27th International Conference on Machine Learning , pages =

    James Martens , title =. Proceedings of the 27th International Conference on Machine Learning , pages =

    work page

  16. [16]

    Schraudolph , title =

    Nicol N. Schraudolph , title =. Neural Computation , volume =

    work page

  17. [17]

    Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms

    Han Xiao and Kashif Rasul and Roland Vollgraf , title =. arXiv preprint arXiv:1708.07747 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [18]

    Gradient-based learning applied to document recognition , journal =

    Yann LeCun and L. Gradient-based learning applied to document recognition , journal =

    work page

  19. [19]

    IEEE Access , volume =

    Hong Xi Tae and Chee Seng Chan , title =. IEEE Access , volume =. 2025 , doi =

    work page 2025

  20. [20]

    Pearlmutter , title =

    Barak A. Pearlmutter , title =. Neural Computation , volume =

    work page

  21. [21]

    2017 IEEE Symposium on Security and Privacy , pages =

    Reza Shokri and Marco Stronati and Congzheng Song and Vitaly Shmatikov , title =. 2017 IEEE Symposium on Security and Privacy , pages =. 2017 , organization =

    work page 2017

  22. [22]

    Alex Krizhevsky and Geoffrey Hinton , title =

    work page

  23. [23]

    International Conference on Computer Communications (INFOCOM) , pages =

    Yong-Xin Liu and Yi-Xing Liu and Zheng-Yang Liu and Yang Liu , title =. International Conference on Computer Communications (INFOCOM) , pages =. 2021 , organization =

    work page 2021

  24. [24]

    Shewchuk , title =

    Jonathan R. Shewchuk , title =

    work page

  25. [25]

    Yousef Saad , title =

    work page

  26. [26]

    Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation) , journal =

    work page 2016

  27. [27]

    California Consumer Privacy Act (CCPA) of 2018 , journal =

    work page 2018

  28. [28]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Aditya Golatkar and Alessandro Achille and Stefano Soatto , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    work page

  29. [29]

    Chundawat and Murari Mandal and Mohan Kankanhalli , title =

    Ayush Tarun and Vikram S. Chundawat and Murari Mandal and Mohan Kankanhalli , title =. IEEE Transactions on Neural Networks and Learning Systems , volume =

    work page

  30. [30]

    Deep ensembles: A loss landscape perspective.arXiv preprint arXiv:1912.02757,

    Stanislav Fort and Huiyi Hu and Balaji Lakshminarayanan , title =. arXiv preprint arXiv:1912.02757 , year =

    work page arXiv 1912

  31. [31]

    International Conference on Learning Representations (ICLR) , year =

    Rahma Entezari and Hanie Sedghi and Olga Saukh and Behnam Neyshabur , title =. International Conference on Learning Representations (ICLR) , year =

    work page

  32. [32]

    Ainsworth and Jonathan Hayase and Siddhartha Srinivasan , title =

    Samuel K. Ainsworth and Jonathan Hayase and Siddhartha Srinivasan , title =. International Conference on Learning Representations (ICLR) , year =

    work page