REVIEW 3 major objections 5 minor 36 references
Label Leakage in Federated Inertial-based Human Activity Recognition
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Federated training leaks activity labels past 90 percent accuracy
desk verdict First real benchmark of label leakage attacks on federated HAR; the core leak finding holds, but the causal claims about class count and imbalance rest on a confounded cross-dataset comparison and the abstract oversells trained-model accuracy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the attacks is the linear structure of the final-layer bias gradients: for a batch of size $N$ with $\lambda_i$ samples of class $i$, the bias gradient component for class $i$ is approximately $-\lambda_i/N$ in an untrained model, so each bias component encodes the per-class label count. Bias-based attacks (EBI, LLBG, iLRG) estimate this per-sample gradient impact $m$ and iteratively reconstruct the label multiset by adding labels whose bias-gradient components remain minimal until the batch size $N$ is reached. Weight-based attacks (LLG, LLG*) use a similar sum-of-weight-gradient sign property but are weaker. The paper's analysis of sampling strategies and its ClassAcc metric are the tools that expose when this mechanism is most dangerous.
What would settle it
Run the same five attacks against clients that use a class-weighted cross-entropy loss, or that locally vary the batch size per update while the server assumes a fixed $N$, and measure ClassAcc; if the reported above-90% reconstruction collapses, the leakage is an artifact of the assumed calibration rather than an inherent property of federated HAR gradients.
Extended reading notes
Core claim
The central claim is that label leakage in federated HAR is severe and systematically underestimated by metrics that favor majority classes. The authors show that bias-based attacks—EBI, LLBG, and iLRG—which read the bias-gradient components of the final classification layer, reconstruct per-batch label counts with high accuracy; under sequential sampling these attacks keep ClassAcc above 90% on the WEAR dataset even for a trained model. The discovery is that HAR's structural properties, especially the dominant NULL class and temporally ordered sliding windows that make sequential batches label-exclusive, turn a known computer-vision vulnerability into a critical privacy risk for wearable and stream-based applications. The paper also introduces ClassAcc, a class-averaged accuracy metric, to counter the misleadingly optimistic picture given by majority-biased LeAcc and LnAcc.
Load-bearing premise
The load-bearing premise is that the honest-but-curious server knows the exact local batch size $N$ and the full model architecture; if clients could hide or vary $N$, or used class-weighted losses, the per-class gradient impact estimates that all bias-based attacks rely on would no longer be calibrated and the reported accuracies would likely drop.
Editorial extensions
If this is right
- If the paper is right, any stream-based federated HAR deployment that sends updates after short recording sessions is exposing the user's current activity to the aggregating server.
- Sequential or temporally ordered sampling should be treated as a privacy hazard, because label-exclusive batches make bias-gradient reconstruction nearly perfect.
- LeAcc and LnAcc metrics are insufficient for HAR; class-balanced ClassAcc should be reported in any privacy evaluation on imbalanced sensor data.
- Common LDP defenses selectively applied to the last layer do not provide reliable protection; the combination of clipping and noise reduces precision but class presence often remains inferable.
- Training without bias terms in the classification layer is a suggested structural mitigation.
Reading between the lines
- If the paper's assumptions are relaxed, a client-side protocol that uses hidden, randomized local batch sizes could substantially blunt bias-based attacks without altering the model; the paper explicitly excludes this case but does not test it.
- The finding that label-exclusive batches drive leakage implies that client-side buffering that enforces class diversity before an update is sent could be a more practical privacy control than differential privacy tuning, and a direct test would measure ClassAcc under a diversity-constrained sampling policy.
- Since bias-based attacks outperform weight-based ones, the paper's recommendation to drop bias terms could be evaluated as a standalone defense; an ablation on the two architectures would quantify how much leakage is actually removed.
- The iLRG attack's volatility under LDP and multi-step averaging suggests a hard-to-model interaction between linear-equation reconstruction and gradient noise; studying that interaction may yield better adaptive defenses.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a benchmark study of five gradient-based label leakage attacks (LLG, LLG*, EBI, iLRG, LLBG) applied to federated inertial-based human activity recognition (HAR). The evaluation covers two HAR datasets (WEAR, Wetlab) and two architectures (DeepConvLSTM, TinyHAR), under shuffled, sequential, and balanced batch sampling, for trained and untrained models, with single- and multi-step local updates, and under local differential privacy defenses (gradient noise, clipping, and their combination). The authors propose a new class-averaged accuracy metric, ClassAcc, to mitigate majority-class bias in existing metrics. The main reported findings are that bias-based attacks are highly effective, sequential sampling is particularly leaky due to label-exclusive batches, and combined clipping plus noise provides only partial protection. The paper claims that the number of activity classes, class imbalance, and sampling strategy are critical factors determining label leakage, and it offers practical recommendations for privacy-aware federated HAR deployment.
Significance. If its central claims hold, this is a timely and useful empirical contribution: it transfers label leakage attacks from computer vision to HAR, demonstrates their practical effectiveness on real sensor data, and provides a new evaluation metric (ClassAcc) that is better suited to imbalanced activity datasets. The public release of code is a clear strength for reproducibility. However, the paper's most prominent quantitative claim—that reconstruction accuracies exceed 90% on two benchmark datasets even for trained models—is not supported by the reported numbers, and the causal attribution of leakage differences to the number of classes and class imbalance is confounded by the two-dataset comparison. These issues weaken the abstract and the deductive force of the conclusions, though the core experimental effort and the qualitative findings about sequential sampling remain valuable.
major comments (3)
- [Abstract; Section 4, Table 1] The abstract states that reconstruction accuracies reach 'well-above 90% on two benchmark datasets, even for trained models.' Table 1 does not support this: for trained models, the best bias-based ClassAcc is 80.71% on WEAR under sequential sampling and 70.76% on Wetlab under sequential sampling; none of the trained-model entries in the table reach 90%. Even untrained Wetlab bias-based performance is 81.67%. Please either correct the abstract to reflect the actual ranges (e.g., 'well above 90% on WEAR for untrained models under sequential sampling, and above 70% for trained models') or provide per-attack, per-architecture results that justify the stronger claim.
- [Section 4 (Table 1) and Section 5 (Limitations)] The paper identifies 'the number of activity classes' and 'class imbalance' as critical factors influencing label leakage, but the only evidence is the contrast between WEAR (18 classes) and Wetlab (9 classes). These datasets differ in sensor placement, activity semantics, recording protocol, class distribution, temporal autocorrelation, and model accuracy, so the observed leakage difference is confounded. No within-dataset manipulation of class count or imbalance is performed. The limitations section does not acknowledge this confound. To make the causal claim defensible, the authors should either add controlled experiments (e.g., subsampling classes or rebalancing within a single dataset) or weaken the claim to an observed association rather than a demonstrated causal factor.
- [Section 4, 'Single-step gradient updates'] The text states that 'under sequential sampling, EBI and LLBG maintain ClassAcc levels above 90% on WEAR, even with trained models,' but Table 1 reports the bias-based average ClassAcc for WEAR trained sequential as 80.71%. If the claim refers to specific attacks or architectures, it should be made explicit and supported with the corresponding numbers; otherwise the text and table are inconsistent. This inconsistency is similar to the abstract overstatement and should be corrected in the same revision.
minor comments (5)
- [Section 3.4, Equation (4)] Equation (4) uses vertical bars around a difference of boolean expressions without defining the absolute value or indicator function; please use explicit indicator notation, e.g., |I(p_gt_j > 0) - I(p_j > 0)|, for clarity.
- [Section 3.4] The definition of ClassAcc is informal ('average of the per-class LnAcc computed across the complete dataset of a client'). A precise formula would help readers understand the difference from LnAcc, especially because the paper argues this metric is a contribution.
- [Section 5, Limitations] The statement that applying clipping and noise to the last-layer gradients 'does not impair the prediction accuracy of trained models' is made without showing results in the paper. Please add a small table or a reference to the code repository output to substantiate this claim.
- [Section 6] The word 'proofed' should be 'proved' or 'shown' ('LDP measures ... proofed to be partially effective').
- [Figure 3 caption] The caption says 'We report the average LnAcc difference between single step and multi-step gradient updates,' but the text in Section 4 mentions both ClassAcc and LnAcc differences. Please clarify which metric is actually plotted, or plot both consistently.
Circularity Check
No circularity: the label leakage results are empirical benchmarks of independent attack methods, not derivations that assume their conclusions.
full rationale
The paper's central claims are empirical evaluations of existing gradient-based label leakage attacks (LLG, LLG*, EBI, iLRG, LLBG) developed in independent prior work by Wainakh et al., Gat et al., and Ma et al. The headline result, that EBI and LLBG maintain ClassAcc above 90% on WEAR under sequential sampling, is obtained by applying these attacks to batch gradients and measuring reconstruction accuracy; no equation or parameter is fitted to the leakage outcome and then renamed as a prediction. The proposed ClassAcc metric is defined independently of any attack and is used only to re-score reconstructed label histograms. The stated assumptions that the server knows the batch size N and operates white-box are explicit threat model assumptions, not circular inputs that enforce the measured leakage. Self-citations are present—the WEAR dataset [1] and the training setup from [2] are the authors' own prior work—but they serve as benchmark data and training configuration, not as load-bearing premises that entail the leakage results. The claim that the number of activity classes and class imbalance are critical factors is weakened by the confounded two-dataset comparison (WEAR vs. Wetlab differ in many ways besides class count and imbalance), but that is an experimental-design limitation and a correctness concern, not circularity. No step in the paper's derivation chain reduces by construction to its own inputs.
Assumptions & free parameters
free parameters (6)
- LDP noise standard deviation sigma =
0.1
- LDP gradient clipping threshold rho =
0.1 L2 norm
- Batch size for single-step and shuffled experiments =
100 sliding windows
- Training length for trained models =
100 epochs on other participants
- Sliding window parameters =
1 second windows with 50% overlap
- Number of local update steps in multi-step experiments =
2 and 5
assumptions (4)
- domain assumption The server knows the exact local batch size N for each update and has full white-box model knowledge.
- domain assumption Clients send correct gradients computed on valid data with an unweighted cross-entropy loss.
- standard math Final-layer gradients encode per-class label presence and counts through approximately linear per-sample contributions.
- domain assumption The two datasets are representative of real-world FL-HAR deployments.
Cite this review
Pith. "Pith review of Label Leakage in Federated Inertial-based Human Activity Recognition." pith.science (2026). https://pith.science/paper/WJCXZSES
@misc{pith2026250520924,
author = {Pith},
title = {Pith review of: Label Leakage in Federated Inertial-based Human Activity Recognition},
year = {2026},
howpublished = {\url{https://pith.science/paper/WJCXZSES}},
note = {Machine review of arXiv:2505.20924}
}
read the original abstract
While prior work has shown that Federated Learning updates can leak sensitive information, label reconstruction attacks, which aim to recover input labels from shared gradients, have not yet been examined in the context of Human Activity Recognition (HAR). Given the sensitive nature of activity labels, this study evaluates the effectiveness of state-of-the-art gradient-based label leakage attacks on HAR benchmark datasets. Our findings show that the number of activity classes, sampling strategy, and class imbalance are critical factors influencing the extent of label leakage, with reconstruction accuracies reaching well-above 90% on two benchmark datasets, even for trained models. Moreover, we find that Local Differential Privacy techniques such as gradient noise and clipping offer only limited protection, as certain attacks still reliably infer both majority and minority class labels. We conclude by offering practical recommendations for the privacy-aware deployment of federated HAR systems and identify open challenges for future research. Code to reproduce our experiments is publicly available via github.com/mariusbock/leakage_har.
Figures
Reference graph
Works this paper leans on
-
[1]
Marius Bock, Hilde Kuehne, Kristof Van Laerhoven, and Michael Moeller. 2024. WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recog- nition. Proceedings of the ACM on Interactive, Mobile, Wearableand Ubiquitous Technologies 8, 4 (2024), 1–21. doi:10.1145/3699776
doi:10.1145/3699776 2024
-
[2]
Marius Bock, Michael Moeller, and Kristof Van Laerhoven. 2024. Temporal Action Localization for Inertial-based Human Activity Recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 4 (2024). doi:10.1145/3699770
-
[3]
Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at Scale: System Design. doi:10.48550/ARXIV.1902.01046
-
[4]
Kongyang Chen, Dongping Zhang, Sijia Guan, Bing Mi, Jiaxing Shen, and Guo- qing Wang. 2024. Private Data Leakage in Federated Human Activity Recognition for Wearable Healthcare Devices. arXiv:2405.10979 [cs] Label Leakage in Federated Inertial-based Human Activity Recognition
arXiv 2024
-
[5]
Roth, Aoxiao Zhong, Ahmed Harouni, Amilcare Gentili, Anas Z
Ittai Dayan, Holger R. Roth, Aoxiao Zhong, Ahmed Harouni, Amilcare Gentili, Anas Z. Abidin, Andrew Liu, Anthony Beardsworth Costa, Bradford J. Wood, Chien-Sung Tsai, Chih-Hung Wang, et al. 2021. Federated Learning for Predicting Clinical Outcomes in Patients with COVID-19. Nature Medicine 27, 10 (2021), 1735–1743. doi:10.1038/s41591-021-01506-3
-
[6]
Sannara Ek, Francois Portet, Philippe Lalanda, and German Vega. 2021. A Feder- ated Learning Aggregation Algorithm for Pervasive Computing: Evaluation and Comparison. In 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, Kassel, Germany, 1–10. doi:10.1109/ PERCOM50583.2021.9439129
-
[7]
Fatima Elhattab, Sara Bouchenak, and Cédric Boscher. 2023. PASTEL: Privacy- Preserving Federated Learning in Edge Computing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 4 (2023), 1–29. doi:10.1145/3633808
-
[8]
Nadav Gat and Mahmood Sharif. 2024. Harmful Bias: A General Label-Leakage Attack on Federated Learning from Bias Gradients. In Proceedings of the 2024 Workshop on Artificial Intelligence and Security. ACM, Salt Lake City UT USA, 31–41. doi:10.1145/3689932.3694768
arXiv 2024
Show all 36 references
-
[9]
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting Gradients - How Easy Is It to Break Privacy in Federated Learning?. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 16937–16947
2020
-
[10]
Raouf Kerkouche, Gergely Acs, and Mario Fritz. 2023. Client-Specific Property Inference against Secure Aggregation in Federated Learning. In Proceedings of the 22nd Workshop on Privacy in the Electronic Society. ACM, Copenhagen Denmark, 45–60. doi:10.1145/3603216.3624964
2023
- [11]
-
[12]
Youpeng Li, Xuyu Wang, and Lingling An. 2023. Hierarchical Clustering-based Personalized Federated Learning for Robust and Fair Human Activity Recogni- tion. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 1 (2023), 1–38. doi:10.1145/3580795
2023 doi
-
[13]
Songfeng Liu, Jinyan Wang, Wenliang Zhang, Guangxi Key Lab of Multi-source Information Mining and Security, Guangxi Normal University, Guilin, China, and College of Computer Science and Engineering, Guangxi Normal University, Guilin, China. 2021. Federated Personalized Random ...
2021 doi
-
[14]
Kailang Ma, Yu Sun, Jian Cui, Dawei Li, Zhenyu Guan, and Jianwei Liu. 2023. Instance-Wise Batch Label Restoration via Gradients in Federated Learning. In The Eleventh International Conference on Learning Representations
2023
-
[15]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-Efficient Learning of Deep Networks From Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. PMLR, 1273–1282
2017
-
[16]
Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. 2018. Learning Differentially Private Recurrent Language Models. In International Conference on Learning Representations
2018
-
[17]
Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. MDPI Sensors 16, 1 (2016). doi:10.3390/s16010115
2016 doi
-
[18]
Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Mo- riai. 2017. Privacy-Preserving Deep Learning: Revisited and Enhanced. In Applications and Techniques in Information Security, Lynn Batten, Dong Seong Kim, Xuyun Zhang, and Gang Li (Eds.). Vol. 719. Sprin...
2017 doi
-
[19]
Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai
-
[20]
Riccardo Presotto, Gabriele Civitarese, and Claudio Bettini. 2022. Prelimi- nary Results on Sensitive Data Leakage in Federated Human Activity Recog- nition. In 2022 IEEE International Conference on Pervasive Computing and Communications Workshopsand Other Affiliated Events (P...
2022
-
[21]
Debaditya Roy, Ahmed Lekssays, Sarunas Girdzijauskas, Barbara Carminati, and Elena Ferrari. 2023. Private, Fair and Secure Collaborative Learning Frame- work for Human Activity Recognition. In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and ...
2023
- [22]
-
[23]
Scholl, Matthias Wille, and Kristof Van Laerhoven
Philipp M. Scholl, Matthias Wille, and Kristof Van Laerhoven. 2015. Wearables in the Wet Lab: A Laboratory System for Capturing and Guiding Experiments. In ACM International Joint Conference on Pervasive and Ubiquitous Computing. doi:10.1145/2750858.2807547
2015
-
[24]
Konstantin Sozinov, Vladimir Vlassov, and Sarunas Girdzijauskas
-
[25]
Aidmar Wainakh, Till Mussig, Tim Grube, and Max Muhlhauser. 2021. Label Leakage from Gradients in Distributed Machine Learning. In 2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC). IEEE, Las Vegas, NV, USA, 1–4. doi:10.1109/CCNC49032.2021.9369498
2021
-
[26]
Human Activity Recognition Using Federated Learning. In 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IU...
2018
-
[27]
Zhiwen Xiao, Xin Xu, Huanlai Xing, Fuhong Song, Xinhan Wang, and Bowen Zhao. 2021. A Federated Learning System with Enhanced Feature Extraction for Human Activity Recognition. Knowledge-Based Systems 229 (2021), 107338. doi:10.1016/j.knosys.2021.107338
2021
-
[28]
Aidmar Wainakh, Fabrizio Ventola, Till Müßig, Jens Keim, Carlos Garcia Cordero, Ephraim Zimmer, Tim Grube, Kristian Kersting, and Max Mühlhäuser. 2022. User-Level Label Leakage from Gradients in Federated Learning. Proceedings on Privacy Enhancing Technologies 2022, 2 (2022), ...
2022 doi
-
[29]
Alvarez, Jan Kautz, and Pavlo Molchanov
Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, and Pavlo Molchanov. 2021. See through Gradients: Image Batch Recovery via GradInversion. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Nashville, TN, USA, 16332–16341. do...
2021
- [30]
-
[31]
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. iDLG: Improved Deep Leakage from Gradients. arXiv:2001.02610 [cs]
2020 arXiv
-
[32]
Hongzheng Yu, Zekai Chen, Xiao Zhang, Xu Chen, Fuzhen Zhuang, Hui Xiong, and Xiuzhen Cheng. 2023. FedHAR: Semi-Supervised Online Learning for Per- sonalized Federated Human Activity Recognition. IEEE Transactions on Mobile Computing 22, 6 (2023), 3318–3332. doi:10.1109/TMC.202...
2023
-
[33]
Blaschko
Junyi Zhu and Matthew B. Blaschko. 2021. R-{GAP}: Recursive Gradient Attack on Privacy. In International Conference on Learning Representations
2021
-
[34]
Yexu Zhou, Haibin Zhao, Yiran Huang, Till Riedel, Michael Hefenbrock, and Michael Beigl. 2022. TinyHAR: A Lightweight Deep Learning Model De- signed for Human Activity Recognition. In ACM International Symposium on Wearable Computers. doi:10.1145/3544794.3558467
2022
-
[36]
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep Leakage from Gradients. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., Red Hook, NY, USA
2019
-
[2018]
IEEE Transactions on Information Forensics and Security 13, 5 (2018), 1333–1345
Privacy-Preserving Deep Learning via Additively Homomorphic Encryp- tion. IEEE Transactions on Information Forensics and Security 13, 5 (2018), 1333–1345. doi:10.1109/TIFS.2017.2787987
2018
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.