REVIEW 4 major objections 4 minor 20 references
Label Inference Attacks against Federated Unlearning
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read An attacker can infer the labels of forgotten data from parameter changes left by federated unlearning, reaching 100% success on class- and client-level unlearning under IID data.
desk verdict Federated unlearning leaks label information, and this paper is the first to show it, but the key reconstruction step needs proof before the 100% ASR numbers are taken at face value. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a gradient-label mapping built on parameter-change differencing. The attacker computes $\Delta_{\mathrm{local}}$ and $\Delta_{\mathrm{global}}$ from pre/post-unlearning models, estimates the effective learning rate as the averaged norm ratio $\eta_{\mathrm{approx}} = \frac{1}{T}\sum_t \|\Delta_{\mathrm{local}}\| / \|\Delta_{\mathrm{global}}\|$, and recovers an approximate gradient difference $\hat{\Delta}_g \approx (\Delta_{\mathrm{global}} - \Delta_{\mathrm{local}})/\eta_{\mathrm{approx}}$ (up to aggregation weight). This reconstructed gradient is then compared, per label, with the average gradient variation in the output layer; the label with largest accumulat
What would settle it
Reproduce ULIA on FedEraser, Rapid Retrain, and SGA-EWC while switching from full-batch single-step updates to mini-batch SGD with several local epochs and an adaptive optimizer, then measure ASR and the cosine similarity between the reconstructed gradient $\hat{\Delta}_g$ from Eq. (9) and the true gradient of the forgotten samples; if either metric collapses, the linear reconstruction chain is the load-bearing assumption. A simpler check is removing access to earlier-round local/global models so the learning-rate estimate cannot be formed and seeing whether ASR falls.
Extended reading notes
Core claim
The paper's central discovery is that the difference between the global model's parameter change $\Delta_{\mathrm{global}}$ and the target client's local parameter change $\Delta_{\mathrm{local}}$ during unlearning is proportional to the gradient contribution of the forgotten data: $\Delta_{\mathrm{global}} - \Delta_{\mathrm{local}} \approx \eta \omega_k g_{\mathrm{forget}}$, where $\eta$ is the learning rate, $\omega_k$ the client's aggregation weight, and $g_{\mathrm{forget}}$ the forgotten samples' gradient. An attacker who can observe pre- and post-unlearning models can therefore reconstruct an approximate gradient difference, provided the learning rate is known or estimated from past lo
Load-bearing premise
The attack assumes the measured parameter change after unlearning is just the forgotten samples' gradient contribution scaled by a known learning rate and aggregation weight, with no noise from batching, multiple local update steps, or optimizer dynamics.
Editorial extensions
If this is right
- Observing model parameters before and after an unlearning request is enough to mount a label inference attack; no access to client data is required beyond the parameter history.
- Larger unlearning requests leak more: class-level and client-level unlearning give 100% ASR on IID data, while sample-level unlearning loses accuracy as the number of forgotten label categories grows.
- Small forget requests are not safe: with only 1% of a client's data forgotten, ASR still ranges from 93% down to 62.3% across settings.
- Not knowing how many labels were forgotten costs only a few points of ASR, so the attack does not depend on strong prior knowledge.
- Non-IID data reduces but does not eliminate the leak, with per-setting ASRs roughly 0.6 to 0.96 in the reported tables.
Reading between the lines
- The reconstruction chain assumes a clean deterministic gradient contribution; testing ULIA under mini-batch SGD, multiple local epochs, or adaptive optimizers would show how much of the reported ASR survives real training dynamics.
- A natural defense is to restrict access to pre/post-unlearning model snapshots or to add calibrated noise to parameter differences, because the linear model implies that differentially private updates would directly reduce the label signal.
- The same gradient-label mapping could be turned around to infer other attributes of forgotten data or to test whether an unlearning request was actually honored, since a residual gradient signal would indicate incomplete removal.
- Because the learning-rate estimator only needs past local and global parameter norms, ULIA-style attacks may transfer to other gradient-leakage settings with known aggregation rules.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ULIA, a label inference attack against federated unlearning (FU). The attack assumes that the difference between pre- and post-unlearning global and target-client model parameters can be used to reconstruct the gradient contribution of the forgotten data. It estimates a learning rate from norm ratios, derives a gradient difference, and maps output-layer gradient magnitudes to label predictions using a significance threshold λ. The authors evaluate ULIA on MNIST and CIFAR-10 under three FU methods (FedEraser, Rapid Retrain, SGA-EWC) and three unlearning levels (sample, class, client), reporting near-100% Attack Success Rate (ASR) for class- and client-level unlearning and degraded but still high ASR for sample-level unlearning, including under non-IID data.
Significance. If validated, the result would constitute a meaningful new privacy threat: federated unlearning is intended to erase data influence, and ULIA claims to recover labels from the parameter differences induced by unlearning. The underlying signal—cross-entropy gradients carry label-specific information in the output layer—is standard and plausible, and the paper is among the first to frame this specifically for FU. However, the paper's contribution is not yet established: the central reconstruction equations are asserted without a rigorous derivation, no baselines or ablations are reported, and the significance threshold is fixed without calibration or sensitivity analysis. The high reported ASRs are plausible but could arise from simpler cues. The paper would be a useful contribution after these validation gaps are addressed.
major comments (4)
- [Equation (8)-(9), Section 3] Equation (8) asserts that δ_global - δ_local equals a term proportional to the forgotten data's gradient contribution. This reduction from Eq. (7) is valid only when all non-target clients have zero local parameter change. The paper's own definitions state that class-level unlearning removes class f from every client, and client-level unlearning removes one client and retrains with the remaining clients. In those cases, δ_global - δ_local,K contains a sum over other clients' local changes, which is not generally zero and is not proportional to client K's forgotten gradient. Thus Eq. (9)'s reconstruction chain is not justified for two of the three claimed unlearning levels. Please derive the exact expression separately for each FU level or restrict the claim to sample-level unlearning.
- [Tables 1-3, Section 5] No baseline or ablation is reported. All tables report only ULIA's ASR. To support the claim that high ASR comes from the reconstructed gradient G_f, the paper needs comparisons against (i) random label guessing adjusted for class prior, (ii) attacks using δ_local alone, (iii) attacks using δ_global alone, and (iv) simple non-gradient heuristics such as the per-output-unit magnitude of parameter change. Without these controls, the 100% ASR for class- and client-level unlearning could be explained by trivial parameter-shift cues rather than by the gradient-label mapping mechanism central to the paper.
- [Section 4 Experimental Settings; Section 5.3] The significance threshold λ is set to 0.9, and no sensitivity analysis or calibration procedure is presented. Section 5.3 states that the impact of the significance threshold is studied, but no results or table for λ are provided. Since the ASR in Eq. (13) depends on thresholding the gradient-label score, the reported values are conditional on a free parameter chosen on the evaluation settings. Please report ASR as a function of λ, calibrate λ on a held-out split, or otherwise demonstrate that the results are not an artifact of threshold tuning.
- [Equation (6)] The learning-rate estimate η_approx is asserted without derivation or error analysis. The ratio of norms of local and global parameter changes, averaged over R rounds, is not in general equal to the true learning rate under mini-batch SGD (batch size 64, Section 4), multiple local epochs, and non-IID data; the bias depends on gradient noise, correlation, and aggregation weights. Moreover, Eq. (9) divides by this scalar while the proportionality constant in Eq. (8) also involves the unknown aggregation weight w_K. Please provide the distributional assumptions, an error bound, or treat the overall scale as a free parameter and show that the attack is insensitive to it.
minor comments (4)
- [Equation (10)] The notation in Eq. (10) is unclear: \(\bar{g}_l = (1/|L|) \sum_{l\in L} G_f l\) mixes a summation index with a vector quantity. Please define G_f and l unambiguously.
- [Table 3] The table is labeled 'Impact of the Non-IID Data Distribution,' but the text does not describe how non-IID data was generated (e.g., Dirichlet concentration parameter, number of clients, or degree of heterogeneity). Without this, the non-IID results cannot be reproduced or interpreted.
- [Equation (13)] The ASR is defined as Intersection over Union over label sets. The interpretation of this metric depends on whether the attacker knows the number of forgotten label categories. Please clarify the relation between the metric and a standard label-recovery accuracy, and report the corresponding random-guessing baseline.
- [General presentation] Several equations are malformed in the submitted text (e.g., Eq. (1) is missing the learning-rate multiplication, Eq. (6) has ambiguous summation bounds). A careful proofread of the mathematical notation is needed.
Circularity Check
No significant circularity: ULIA's label inference rests on an external gradient-label identity and empirical evaluation.
full rationale
The derivation chain is not circular. ULIA observes pre/post-unlearning parameter changes (Eqs. 4-5), estimates a learning rate from those changes (Eq. 6), reconstructs a gradient contribution via Eq. (9) from the update rules (Eqs. 1-2, 7-8), and infers labels using the standard cross-entropy gradient-label relation (Eq. 10). The gradient-label mapping is an external property of the loss function, not fitted to the forgotten labels. The paper does not fit a parameter to the labels it then predicts, and it does not invoke a load-bearing self-citation or an imported uniqueness theorem. The significance threshold is a hyperparameter of the attack, and its sensitivity is studied; even if one questioned its calibration, that would be a correctness/robustness concern, not circularity. The reconstruction formula is algebraically derived from the model update equations, and its assumptions (full-batch SGD, known weights) are threat-model limitations, not circular reductions.
Assumptions & free parameters
free parameters (2)
- lambda (significance threshold) =
not stated in extracted text
- R (number of rounds for learning-rate averaging) =
not stated
assumptions (5)
- standard math Output-layer gradients under cross-entropy loss encode one-hot label information (gradient is softmax probability minus indicator of true label).
- domain assumption Client updates are a single full-batch SGD step with fixed learning rate, as in Equation (1).
- domain assumption Post-unlearning models approximate retraining on the remaining data, so parameter changes before and after unlearning isolate the forgotten data's gradient contribution.
- domain assumption The attacker can observe both pre- and post-unlearning local model parameters of the target client and the global model parameters, and knows aggregation weights and the number of forgotten label categories (or sets the threshold to compensate).
- ad hoc to paper The significance threshold lambda can be set to separate true labels from noise, and its choice is part of the attack design.
Cite this review
Pith. "Pith review of Label Inference Attacks against Federated Unlearning." pith.science (2026). https://pith.science/paper/XTEV33Y2
@misc{pith2026250806789,
author = {Pith},
title = {Pith review of: Label Inference Attacks against Federated Unlearning},
year = {2026},
howpublished = {\url{https://pith.science/paper/XTEV33Y2}},
note = {Machine review of arXiv:2508.06789}
}
read the original abstract
Federated Unlearning (FU) has emerged as a promising solution to respond to the right to be forgotten of clients, by allowing clients to erase their data from global models without compromising model performance. Unfortunately, researchers find that the parameter variations of models induced by FU expose clients' data information, enabling attackers to infer the label of unlearning data, while label inference attacks against FU remain unexplored. In this paper, we introduce and analyze a new privacy threat against FU and propose a novel label inference attack, ULIA, which can infer unlearning data labels across three FU levels. To address the unique challenges of inferring labels via the models variations, we design a gradient-label mapping mechanism in ULIA that establishes a relationship between gradient variations and unlearning labels, enabling inferring labels on accumulated model variations. We evaluate ULIA on both IID and non-IID settings. Experimental results show that in the IID setting, ULIA achieves a 100% Attack Success Rate (ASR) under both class-level and client-level unlearning. Even when only 1% of a user's local data is forgotten, ULIA still attains an ASR ranging from 93% to 62.3%.
Reference graph
Works this paper leans on
-
[2]
����� ������� [16], utilizes gradient and curvature information to identify more optimal descent directions, facilitating efficient retraining. 3)������� [27], proposes an efficient FU framework using reverse Stochastic Gradient As- cent (SGA) and Elastic Weight Consolidation (EWC) to quickly adjust model parameters and eliminate the influence of specific...
-
[15]
Liu, J., Lyu, X.: Clustering label inference attack against practical split learning. arXiv e-prints pp. arXiv–2203 (2022)
work page 2022
-
[16]
In: IEEE INFOCOM 2022- IEEE Conference on Computer Communications
Liu, Y., Xu, L., Yuan, X., Wang, C., Li, B.: The right to be forgotten in federated learning: An efficient realization with rapid retraining. In: IEEE INFOCOM 2022- IEEE Conference on Computer Communications. pp. 1749–1758. IEEE (2022)
work page 2022
-
[17]
Liu, Z., Jiang, Y., Shen, J., Peng, M., Lam, K.Y., Yuan, X., Liu, X.: A survey on federatedunlearning:Challenges,methods,andfuturedirections.ACMComputing Surveys 57(1), 1–38 (2024)
work page 2024
-
[18]
In: Federated learning: privacy and incentive, pp
Long, G., Tan, Y., Jiang, J., Zhang, C.: Federated learning for open banking. In: Federated learning: privacy and incentive, pp. 240–254. Springer (2020)
work page 2020
-
[19]
Future Generation Computer Systems 115, 619–640 (2021)
Mothukuri, V., Parizi, R.M., Pouriyeh, S., Huang, Y., Dehghantanha, A., Srivas- tava, G.: A survey on security and privacy of federated learning. Future Generation Computer Systems 115, 619–640 (2021)
work page 2021
-
[20]
IEEE Transactions on Dependable and Secure Computing20(5), 3653–3668 (2022)
Qiu, P., Zhang, X., Ji, S., Du, T., Pu, Y., Zhou, J., Wang, T.: Your labels are selling you out: Relation leaks in vertical federated learning. IEEE Transactions on Dependable and Secure Computing20(5), 3653–3668 (2022)
work page 2022
-
[21]
IEEE Transactions on Neural Networks and Learning Systems (2024)
Shaik, T., Tao, X., Xie, H., Li, L., Zhu, X., Li, Q.: Exploring the landscape of machine unlearning: A comprehensive survey and taxonomy. IEEE Transactions on Neural Networks and Learning Systems (2024)
work page 2024
Show all 20 references
-
[22]
In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management
Sheng, X., Bao, W., Ge, L.: Robust federated unlearning. In: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. pp. 2034–2044 (2024)
-
[23]
Su,N.,Li,B.:Asynchronousfederatedunlearning.In:IEEEINFOCOM2023-IEEE Conference on Computer Communications. pp. 1–10. IEEE (2023)
2023
-
[24]
A Practical Guide, 1st Ed., Cham: Springer International Publishing10(3152676), 10–5555 (2017)
Voigt, P., Von dem Bussche, A.: The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing10(3152676), 10–5555 (2017)
2017
-
[25]
In: Proceedings of the ACM Web Conference 2022
Wang, J., Guo, S., Xie, X., Qi, H.: Federated unlearning via class-discriminative pruning. In: Proceedings of the ACM Web Conference 2022. pp. 622–632 (2022)
2022
-
[26]
ACM Transactions on Internet Technology24(2), 1–22 (2024)
Wang, Z., Gao, X., Wang, C., Cheng, P., Chen, J.: Efficient vertical federated unlearning via fast retraining. ACM Transactions on Internet Technology24(2), 1–22 (2024)
2024
-
[27]
IEEE Network36(5), 129–135 (2022)
Wu, L., Guo, S., Wang, J., Hong, Z., Zhang, J., Ding, Y.: Federated unlearning: Guarantee the right of clients to forget. IEEE Network36(5), 129–135 (2022)
2022
-
[28]
In: Proceedings of the sixteenth ACM international conference on web search and data mining
Yuan, W., Yin, H., Wu, F., Zhang, S., He, T., Wang, H.: Federated unlearning for on-device recommendation. In: Proceedings of the sixteenth ACM international conference on web search and data mining. pp. 393–401 (2023)
2023
-
[29]
IEEE Transactions on Information Forensics and Security (2023)
Zhang, L., Zhu, T., Zhang, H., Xiong, P., Zhou, W.: Fedrecovery: Differentially private machine unlearning for federated learning frameworks. IEEE Transactions on Information Forensics and Security (2023)
2023
-
[30]
In: International Conference on Machine Learning for Cyber Security
Zhang, X., Zhou, X., Chen, K.: Data leakage with label reconstruction in dis- tributed learning environments. In: International Conference on Machine Learning for Cyber Security. pp. 185–197. Springer (2022)
2022
-
[31]
arXiv preprint arXiv:2001.02610 (2020)
Zhao, B., Mopuri, K.R., Bilen, H.: idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610 (2020)
2001 arXiv
-
[32]
IEEE Internet of Things Journal (2023)
Zhao, Y., Wang, P., Qi, H., Huang, J., Wei, Z., Zhang, Q.: Federated unlearning with momentum degradation. IEEE Internet of Things Journal (2023)
2023
-
[33]
In: Proceedings of the AAAI conference on artificial intelligence
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-iou loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence. vol. 34, pp. 12993–13000 (2020)
2020
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.