Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions

Ali Mahdavi; Amirfarhad Farhadi; Azadeh Zamanifar; Omid Kashefi

REVIEW 2 major objections 2 minor 1 cited by

HF-KCU approximates the influence function via conjugate gradients in Krylov subspaces to reverse a client's contribution in federated learning without full retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 18:06 UTC pith:RW2J74PZ

load-bearing objection HF-KCU pairs Krylov influence approximation with causal weighting for federated unlearning but rests on a bounded-perturbation assumption that lacks boundary testing. the 2 major comments →

arxiv 2605.20341 v2 pith:RW2J74PZ submitted 2026-05-19 cs.LG cs.AIcs.CRcs.PF

Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions

Ali Mahdavi , Azadeh Zamanifar , Amirfarhad Farhadi , Omid Kashefi This is my paper

classification cs.LG cs.AIcs.CRcs.PF

keywords federated learningmachine unlearninginfluence functionsKrylov subspace methodscausal weightingadversarial robustnessdata deletionprivacy preservation

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents HF-KCU as a way to handle data deletion requests in federated learning by approximating influence reversal efficiently. Retraining from scratch after each deletion is too slow for practical systems that must comply with privacy rules. The method reduces computation from cubic to linear in a small parameter k using Krylov methods and ensures updates only reach clients with the deleted data through causal weighting. It also tolerates bounded adversarial changes to the model parameters. Experiments show this delivers nearly the same accuracy and privacy as a full retrain but much quicker across several neural network architectures.

Core claim

By approximating the inverse Hessian action through conjugate gradient iterations restricted to Krylov subspaces and combining it with a causal weighting scheme that isolates updates to clients possessing the deleted data, HF-KCU achieves influence reversal at reduced cost O(kd) while providing convergence guarantees of order O((sqrt(k)-1)/(sqrt(k)+1)) and graceful degradation under bounded Hessian and gradient perturbations.

What carries the argument

The Krylov-subspace conjugate-gradient approximation to the influence function, augmented by causal weighting that restricts parameter changes to clients holding the deleted data.

Load-bearing premise

Hessian and gradient perturbations stay bounded under the adversarial threat model so that the Krylov approximation and causal weighting still produce valid influence reversal.

What would settle it

Running HF-KCU on the CIFAR-10 Dirichlet partition and measuring a membership inference attack success rate on the forget set well above 0.5 would show that privacy restoration did not occur.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Data deletion in federated systems becomes computationally feasible without retraining the entire model from scratch.
Only clients that contributed the deleted data receive updates, leaving others unaffected.
Membership inference attack success on the forgotten data returns to random guessing levels matching a retrained model.
The approximation error is bounded and decreases with the number of iterations according to the given rate.
Performance holds across convolutional and transformer models on standard image datasets under non-IID partitioning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the bounded perturbation assumption holds in practice, the approach could scale to very large models where full influence computation is impossible.
Similar Krylov techniques might apply to unlearning in centralized training or other collaborative optimization settings beyond federated learning.
The causal weighting provides a form of interpretability that could help audit which data points affect which clients.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

HF-KCU pairs Krylov influence approximation with causal weighting for federated unlearning but rests on a bounded-perturbation assumption that lacks boundary testing.

read the letter

The paper introduces HF-KCU as a way to handle client data deletion in federated learning without full retraining. It approximates influence functions via conjugate gradient in Krylov subspaces and adds a causal weighting step so only clients with the deleted data get updated. The reported result is a 47.75x speedup on CIFAR-10 with test accuracy within 0.6% of retraining and membership inference attack rates matching the retrained model.

What stands out as new is the specific combination of the Krylov method with the causal weighting for selective, asynchronous updates. The efficiency claim from O(d^3) to O(kd) and the convergence rate statement follow from standard CG analysis, and the approach is applied across ResNet, SimpleCNN, and ViT-Lite on three datasets.

The soft spots are in the central assumption and the evidence. The method is stated to handle bounded adversarial perturbations to the Hessian and gradient, yet the Dirichlet experiments do not appear to include explicit checks at the boundary of that bound. If perturbations exceed it, the approximation error grows and the accuracy or privacy numbers can slip without the claimed graceful behavior. The abstract also gives no error bars, full protocol, or derivation steps, so the 47.75x and 0.499 MIA figures are hard to judge for robustness. The convergence notation re-uses k for both subspace dimension and condition number, which is a minor but avoidable slip.

This is for people working on federated unlearning and privacy-compliant collaborative training. A reader who already knows influence functions and CG methods will get the most from the technical choices. The problem is practical and the method has enough grounding to merit referee time, even with the gaps.

I would send it to peer review and ask for stress tests on the perturbation bound plus expanded experimental details.

Referee Report

2 major / 2 minor

Summary. The manuscript presents HF-KCU, a method for unlearning client contributions in federated learning by approximating the influence function via conjugate gradient iterations in Krylov subspaces (reducing complexity from O(d^3) to O(kd)), combined with a causal weighting mechanism to ensure surgical updates only to clients holding the deleted data. It claims to handle bounded adversarial perturbations to the Hessian and gradients, provides convergence guarantees of the form O((√κ-1)/(√κ+1)), and reports empirical results on CIFAR-10, MNIST, and Fashion-MNIST (ResNet-18, SimpleCNN, ViT-Lite) showing 47.75x speedup over retraining, test accuracy within 0.60% of baseline (71.16% vs 71.76%), and MIA success rate of 0.499 matching the retrained model under Dirichlet partitioning.

Significance. If the bounded-perturbation assumption and Krylov approximation hold, the work provides a scalable, interpretable alternative to full retraining for asynchronous data-deletion requests in federated systems, with potential production impact due to its efficiency, privacy restoration, and avoidance of instability in gradient-based unlearning.

major comments (2)

[Abstract] Abstract: The convergence guarantee is stated both as the standard CG rate O((√κ-1)/(√κ+1)) and as O((k^{1/2}-1)/(k^{1/2}+1)) 'where k is the Hessian condition number', while k is also used for the Krylov subspace dimension (k<<d). This notation clash directly affects the clarity and verifiability of the central theoretical claim.
[Abstract] Abstract: The headline empirical claims (47.75x speedup, accuracy within 0.60%, MIA=0.499) rest on the assumption that 'Hessian and gradient perturbations remain bounded under the stated adversarial threat model' allowing graceful degradation. The Dirichlet (alpha=0.5) experiments provide no explicit stress tests at the boundary of this boundedness regime, which is load-bearing for the influence-reversal validity and privacy metrics.

minor comments (2)

[Abstract] Abstract: Typo 'rational baseline(71.16 vs 71.76 %)' – missing space after 'baseline'.
[Abstract] Abstract: The rate expression contains inconsistent spacing and exponent notation: 'O((k ^1/2-1)/(k^1/2+1))'.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful review and constructive feedback on the notation and empirical validation. We address each major comment below.

read point-by-point responses

Referee: [Abstract] Abstract: The convergence guarantee is stated both as the standard CG rate O((√κ-1)/(√κ+1)) and as O((k^{1/2}-1)/(k^{1/2}+1)) 'where k is the Hessian condition number', while k is also used for the Krylov subspace dimension (k<<d). This notation clash directly affects the clarity and verifiability of the central theoretical claim.

Authors: We agree there is a notation clash. The intended convergence rate is the standard CG rate O((√κ-1)/(√κ+1)) with κ denoting the Hessian condition number. The symbol k is reserved for the Krylov subspace dimension (k ≪ d). The erroneous phrase 'where k is the Hessian condition number' will be corrected to use κ. This is a purely notational issue that does not change the underlying analysis or claims. revision: yes
Referee: [Abstract] Abstract: The headline empirical claims (47.75x speedup, accuracy within 0.60%, MIA=0.499) rest on the assumption that 'Hessian and gradient perturbations remain bounded under the stated adversarial threat model' allowing graceful degradation. The Dirichlet (alpha=0.5) experiments provide no explicit stress tests at the boundary of this boundedness regime, which is load-bearing for the influence-reversal validity and privacy metrics.

Authors: The bounded-perturbation assumption is part of the stated threat model and enables the theoretical graceful-degradation guarantee. The reported experiments use standard Dirichlet (α=0.5) partitioning without explicit adversarial perturbations, and under these conditions the method matches retraining on accuracy and MIA metrics. We acknowledge that boundary stress tests are absent. We will add a clarifying sentence in the abstract and experimental discussion stating that the empirical results are obtained under nominal conditions while the bounded-perturbation property remains a theoretical guarantee. revision: partial

Circularity Check

0 steps flagged

No circularity; standard CG rate and empirical validation are independent of method inputs

full rationale

The abstract and reader's summary present HF-KCU as an approximation of influence functions via Krylov CG plus causal weighting, with the convergence rate O((√κ-1)/(√κ+1)) stated as a known CG property rather than derived or fitted within the paper. Accuracy/MIA numbers are reported as post-hoc validation on CIFAR-10 under Dirichlet partitioning, not as inputs that define the method. No self-citations, self-definitional equations, or fitted-input-as-prediction patterns appear. The bounded-perturbation assumption is an explicit modeling choice, not a reduction of the result to itself. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review prevents identification of specific free parameters, axioms, or invented entities; none are explicitly named in the provided text.

pith-pipeline@v0.9.1-grok · 5881 in / 1060 out tokens · 30205 ms · 2026-06-30T18:06:10.177059+00:00 · methodology

0 comments

read the original abstract

Federated learning systems must support data deletion requests to comply with privacy regulations, yet retraining from scratch after each deletion is computationally prohibitive. We present HF-KCU, a method that removes a client's contribution by approximating the influence function through conjugate gradient iterations in Krylov subspaces, reducing complexity from O(d^3) to O(kd) where k<<d.A causal weighting mechanism ensures that only clients holding the deleted data receive parameter updates, preventing spurious changes to unaffected clients. Our method is designed to handle bounded adversarial perturbations to the Hessian and gradient, providing graceful degradation under realistic threat models. We validate HF-KCU across convolutional (ResNet-18, SimpleCNN) and transformer (ViT-Lite) architectures on CIFAR-10, MNIST, and Fashion-MNIST. On CIFAR-10 under Dirichlet (alpha=0.5) partitioning, HF-KCU achieves 47.75 times speedup over retraining while maintaining test accuracy within 0.60% of the rational baseline(71.16 vs 71.76 %). Membership inference attacks on the forget set yield success rates of 0.499 matching the retrained model and confirming effective privacy restoration. We provide convergence guarantees showing that the Krylov approximation error decreases as O((k ^1/2-1)/(k^1/2+1)) where k is the Hessian condition number. The causal weighting mechanism ensures surgical updates, where only clients holding deleted data are modified, preserving model quality for unaffected participants and avoiding the instability of gradient-based approaches in asynchronous federated settings. This design provides interpretability as each update is directly traceable to the influence of the deleted data. The method's efficiency and precision make it suitable for production federated systems where deletion requests arrive asynchronously and computational budgets are constrained.

Figures

Figures reproduced from arXiv: 2605.20341 by Ali Mahdavi, Amirfarhad Farhadi, Azadeh Zamanifar, Omid Kashefi.

Review history (2 revisions) →

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Spectral-LSH: Sub-Quadratic Prompt Compression via Krylov-Projected Locality-Sensitive Hashing
cs.AI 2026-06 conditional novelty 6.0

At compression ratios above 8x, grouping tokens via a Krylov-projected LSH of an implicit attention kernel preserves LLM output quality far better than block averaging, at a large preprocessing latency cost.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · cited by 1 Pith paper

[1]

We measure KLoutput =E x∼Dtest DKL pretrain(· |x)∥p unlearn(· |x)

Output KL divergence:For classification tasks, let pretrain(· |x) and punlearn(· |x) denote the softmax output distributions of the retrained and unlearned models on input x. We measure KLoutput =E x∼Dtest DKL pretrain(· |x)∥p unlearn(· |x) . 2.Logit MSE:For regression or when logits are more interpretable, we measure MSElogit =E x∼Dtest ∥zretrain(x)−z un...

work page
[2]

These metrics are invariant to permutation symmetry and basin geometry, focusing instead on whether the unlearned model behaves like the retrained model on held-out data

Test accuracy gap:The absolute difference in test accuracy between retrained and unlearned models,|Acc retrain −Acc unlearn|. These metrics are invariant to permutation symmetry and basin geometry, focusing instead on whether the unlearned model behaves like the retrained model on held-out data. Empirical results on functional metrics.Table 9 reports thes...

work page

[1] [1]

We measure KLoutput =E x∼Dtest DKL pretrain(· |x)∥p unlearn(· |x)

Output KL divergence:For classification tasks, let pretrain(· |x) and punlearn(· |x) denote the softmax output distributions of the retrained and unlearned models on input x. We measure KLoutput =E x∼Dtest DKL pretrain(· |x)∥p unlearn(· |x) . 2.Logit MSE:For regression or when logits are more interpretable, we measure MSElogit =E x∼Dtest ∥zretrain(x)−z un...

work page

[2] [2]

These metrics are invariant to permutation symmetry and basin geometry, focusing instead on whether the unlearned model behaves like the retrained model on held-out data

Test accuracy gap:The absolute difference in test accuracy between retrained and unlearned models,|Acc retrain −Acc unlearn|. These metrics are invariant to permutation symmetry and basin geometry, focusing instead on whether the unlearned model behaves like the retrained model on held-out data. Empirical results on functional metrics.Table 9 reports thes...

work page