REVIEW 3 major objections 4 minor 34 references
A Federated Random Forest Solution for Secure Distributed Machine Learning
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims a federated Random Forest can match centralized accuracy within about 9% while keeping patient data in place.
desk verdict An honest but modest engineering report on a PySyft-based federated random forest whose central privacy claim does not survive close reading; the accuracy comparisons are also too thin to support the numeric margins. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is weighted tree sampling for forest aggregation. After each silo trains a local Random Forest, the coordinator selects $k_i = \lfloor w'_i \times N_i \rfloor$ trees from silo $i$, where $w'_i$ is the silo's normalized weight and $N_i$ is its local forest size, then assembles the selected trees into a single global forest. Uniform sampling serves as the default when no weights are given, and the scheme is linear in the total number of trees. The paper also uses warm-start incremental rounds, in which each silo adds new trees to an existing forest, allowing the global model to refine over multiple federated rounds. The secure remote-computation layer is what keeps local data on-site while training code is sent to it.
What would settle it
Run a membership-inference or model-inversion attack against the serialized forest returned by a silo: if a specific patient record can be distinguished as having been in the training set, or its feature values recovered, from the tree splits alone, the privacy guarantee in the paper's central claim fails even if the accuracy numbers hold.
Extended reading notes
Core claim
The central claim is that federated Random Forest training can preserve most of the predictive performance of a centralized model while allowing each participating institution to keep raw data local. Experiments on two healthcare datasets show accuracy degradation of 0.26–8.17% and 2.23–9.05% depending on the number of silos. The paper attributes the stability to a weighted aggregation protocol in which the global forest contains the same total number of trees as a single client model, sampled from client forests proportionally to client weights. It also reports that one federated configuration slightly exceeded its centralized baseline, and that relative degradation patterns were consistent across the two datasets. The privacy guarantee, however, is asserted through the use of a secure remote-computation layer rather than measured; no differential privacy, secure aggregation, or threat-model analysis is presented.
Load-bearing premise
The privacy half of the central claim rests on the assumption that exchanging serialized decision trees between sites and the coordinator cannot be used to infer anything meaningful about the local training data; the paper states this but does not test it with a threat model, differential privacy, or secure aggregation.
Editorial extensions
If this is right
- Healthcare institutions can collaboratively train a Random Forest without moving raw records, keeping final-model accuracy within roughly 9% of a centralized model when the number of participating sites is moderate.
- Weighted aggregation lets larger or more representative datasets exert more influence on the global forest, providing a practical way to handle statistical heterogeneity across sites.
- Incremental warm-start rounds allow new silos to join mid-training and let the global model improve over time without retraining from scratch.
- Excessive fragmentation degrades performance by up to roughly 9 percentage points, setting a practical limit on how finely data can be split while preserving accuracy.
- Local evaluation on each silo's own data gives sites a way to check how well the federated model generalizes to their specific population, supporting transparency in clinical contexts.
Reading between the lines
- The paper's privacy claim is stronger than its evidence: because no differential privacy or secure aggregation is applied, an outside observer with access to the serialized trees could attempt membership inference, and the paper's own text lists this as future work.
- The weighted sampling rule could be extended to weight silos by estimated data quality or label balance rather than volume alone, which would give a direct test of whether the observed accuracy ceiling can be pushed lower on skewed datasets.
- The reported degradation pattern suggests a design guideline the authors do not state explicitly: federations beyond roughly five partitions should expect accuracy loss in the 2–9% range, so deployments should weigh fragmentation costs against privacy benefits.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents Fed-RF, a PySyft-based federated learning framework for Random Forest classifiers. The proposed protocol trains local forests at each data silo, serializes and transmits them to a central coordinator, and aggregates trees by weighted sampling; it also supports incremental warm-start rounds and local evaluation. Experiments on two healthcare datasets report accuracy within 0.26-8.17% and 2.23-9.05% of a centralized baseline, and the paper claims to satisfy 'stringent privacy requirements' by leveraging PySyft. The authors release the implementation as an open-source package.
Significance. If the accuracy results and privacy claims were rigorously established, the work would fill a real gap in tree-based federated learning and provide a useful open-source tool. The authors deserve credit for releasing the implementation, making the aggregation procedure explicit in Algorithm 1, and evaluating on two real healthcare datasets. However, the privacy guarantee is asserted rather than demonstrated, and the empirical evidence lacks the repeated trials, error bars, and statistical tests needed to support the stated accuracy margins. The contribution is therefore promising but not yet substantiated as written.
major comments (3)
- [Section III.A and III.C; Conclusion] The central claim that the method 'satisf[ies] stringent privacy requirements' is not supported by the described protocol. Section III.C states that locally trained forests are 'serialized and sent back to the coordinator' with no encryption, secure aggregation, or noise addition; Section III.A asserts that PySyft's remote-execution architecture mitigates model inversion and membership inference, but exchanging plaintext decision trees does not provide such protection, as leaf thresholds, split features, and sample counts can leak information about training records. The conclusion's future-work sentence on differential privacy confirms that no such protection is currently included. Since privacy is half of the paper's stated contribution, this is a load-bearing gap that must be addressed by either adding a concrete privacy mechanism or substantially weakening the privacy claims.
- [Section IV.B, Tables I and II] The experimental evaluation does not support the reported accuracy margins. Each configuration appears to be a single run with no error bars, cross-validation, or significance tests; differences such as 0.26% (Table I, 3 Silos) and 2.23% (Table II, 3 Silos) are likely within sampling noise. Additionally, the centralized baseline is trained on the entire dataset while the federated silos train on the 80% training partition, giving the baseline an advantage and making the reported 'accuracy deviation' a biased measure. Repeated runs with variance reporting, or paired statistical tests on the same test partition, are needed before the competitive-accuracy claim can be evaluated.
- [Algorithm 1, lines 4-6] The weighted sampling procedure as written does not guarantee that the global forest has the same total number of trees as a client model. Because k_i = floor(w'_i * N_i), rounding down can leave the sum of k_i smaller than N_i, and no renormalization or resampling step is described. Since Section III.C claims the global forest is 'composed of a mixture from all clients' with the same total number of trees, this ambiguity should be clarified and the implementation checked against the intended guarantee.
minor comments (4)
- [Section IV.C and Discussion] The text says N varied from 1 to 10, but Tables I and II report only 3, 5, and 10 silos; the Discussion's N=2 result (accuracy 0.7260) for the diabetic retinopathy dataset is not shown in any table, so the reader cannot verify the claimed peak above baseline.
- [Introduction, Abstract] There are several typographical issues: 'an novel' in the Introduction, 'fed rf' in the abstract's repository URL, and 'I NTRODUCTION' in the section heading; these should be corrected.
- [References] Reference [1] (Silva and Oliveira, 'BIT. UA at IDPP') appears unrelated to the present paper and is not cited in the body; please remove it or replace it with a directly relevant citation.
- [Section III.A] The sentence 'mitigating risks such as model inversion or membership inference attacks, this is a key feature that will enable our tool to provide privacy and confidentiality to data' is a run-on and should be split; moreover, the phrase 'will enable' is inconsistent with the paper's claim that privacy is already provided.
Circularity Check
No circularity: accuracy results are empirical measurements with externally sourced hyperparameters; the privacy gap is an unsupported assertion, not a circular derivation.
full rationale
The paper contains no derivation chain that could be circular. The central accuracy claims are empirical measurements: centralized Random Forest baselines are trained and federated variants are evaluated on held-out test data, with results reported in Tables I and II. The 'maximum 9% margin' is an arithmetic summary of those tables, not a fitted target. Hyperparameters, notably the 4100 estimators for the AIDS dataset, are taken from external references [6] and [32] rather than tuned to force the reported outcome. The weighted aggregation rule in Algorithm 1 is an explicit sampling procedure (ki = floor(w'_i x N_i) trees selected from each forest), not an inverse-engineered fit to the accuracies. There is no claimed prediction that reduces by construction to an input, no parameter fitted to a subset of data and then renamed a prediction, and no load-bearing uniqueness theorem or self-citation: the only overlapping-author reference, [1], is not cited as evidence for any central claim. The N=2 silos result in Section V is a post-hoc observation that the authors themselves attribute to favorable data splits or initialization, so it is a reporting artifact rather than a circular prediction. The privacy assertion is unsupported—the protocol serializes and sends trees via PySyft with no differential privacy, secure aggregation, or threat model, and the conclusion defers differential privacy to future work—but that is a correctness and validity gap in the security claim, not circularity, because the claim is not derived from its own inputs by definition. Accordingly, no circular steps are identified and the score is 0.
Assumptions & free parameters
free parameters (4)
- train/test split ratio 20/80 =
20% test, 80% train
- n_estimators per dataset =
4100 for Dataset1, 100 for Dataset2
- single test silo as evaluation partition =
one 20% silo per configuration
- equal client weights =
1/N for N silos
assumptions (3)
- domain assumption Exchanging serialized Random Forest trees between silos does not leak sensitive information about local training data.
- domain assumption The centralized baseline was trained on the full dataset, including the partition used as the federated test set.
- domain assumption Warm-starting scikit-learn Random Forests across silos preserves model quality in incremental learning.
Cite this review
Pith. "Pith review of A Federated Random Forest Solution for Secure Distributed Machine Learning." pith.science (2026). https://pith.science/paper/PRXXUZWB
@misc{pith2026250508085,
author = {Pith},
title = {Pith review of: A Federated Random Forest Solution for Secure Distributed Machine Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/PRXXUZWB}},
note = {Machine review of arXiv:2505.08085}
}
read the original abstract
Privacy and regulatory barriers often hinder centralized machine learning solutions, particularly in sectors like healthcare where data cannot be freely shared. Federated learning has emerged as a powerful paradigm to address these concerns; however, existing frameworks primarily support gradient-based models, leaving a gap for more interpretable, tree-based approaches. This paper introduces a federated learning framework for Random Forest classifiers that preserves data privacy and provides robust performance in distributed settings. By leveraging PySyft for secure, privacy-aware computation, our method enables multiple institutions to collaboratively train Random Forest models on locally stored data without exposing sensitive information. The framework supports weighted model averaging to account for varying data distributions, incremental learning to progressively refine models, and local evaluation to assess performance across heterogeneous datasets. Experiments on two real-world healthcare benchmarks demonstrate that the federated approach maintains competitive predictive accuracy - within a maximum 9\% margin of centralized methods - while satisfying stringent privacy requirements. These findings underscore the viability of tree-based federated learning for scenarios where data cannot be centralized due to regulatory, competitive, or technical constraints. The proposed solution addresses a notable gap in existing federated learning libraries, offering an adaptable tool for secure distributed machine learning tasks that demand both transparency and reliable performance. The tool is available at https://github.com/ieeta-pt/fed_rf.
Figures
Reference graph
Works this paper leans on
-
[1]
Silva, J, Oliveira, J, “BIT. UA at IDPP: predictive analytics on ALS disease progression using sensor data with machine learning,” 2024
work page 2024
-
[2]
Federated machine learning: Concept and applications,
Yang, Qiang, Liu, Yang, Chen, Tianjian, Tong, Yongxin, “Federated machine learning: Concept and applications,” in ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10(2), pp. 1–19, 2019
work page 2019
-
[3]
Communication-Efficient Learning of Deep Networks from Decentralized Data,
McMahan, Brendan, Moore, Eider, Ramage, Daniel, Hampson, Seth, Arcas, Blaise Aguera y, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in Proceedings of the 20th Inter- national Conference on Artificial Intelligence and Statistics, vol. 54, pp. 1273–1282, 2017
work page 2017
-
[4]
Breiman, Leo, “Random forests,” in Machine learning, vol. 45, pp. 5–32, 2001
work page 2001
-
[5]
Federated learning to comply with data protection regulations,
Chalamala, Srinivasa Rao, Kummari, Naveen Kumar, Singh, Ajeet Ku- mar, Saibewar, Aditya, Chalavadi, Krishna Mohan, “Federated learning to comply with data protection regulations,” in CSI Transactions on ICT, vol. 10(1), pp. 47–60, 2022
work page 2022
-
[6]
A Ma- chine Learning Approach to Aids Clinical Trials Group (Actg) Study,
Malyala, Lakshminarayana Rao, Thiyagarajan, Sneha, others, “A Ma- chine Learning Approach to Aids Clinical Trials Group (Actg) Study,” 2024, doi: 10.21203/rs.3.rs-5247011/v1
-
[7]
Fed- erated learning: Challenges, methods, and future directions,
Li, Tian, Sahu, Anit Kumar, Talwalkar, Ameet, Smith, Virginia, “Fed- erated learning: Challenges, methods, and future directions,” in IEEE signal processing magazine, vol. 37(3), pp. 50–60, 2020
work page 2020
-
[8]
A generic framework for privacy preserving deep learning,
Ryffel, Theo, Trask, Andrew, Dahl, Morten, Wagner, Bobby, Man- cuso, Jason, Rueckert, Daniel, Passerat-Palmbach, Jonathan, “A generic framework for privacy preserving deep learning,” in arXiv preprint arXiv:1811.04017, 2018
arXiv 2018
Show all 34 references
-
[9]
Towards federated learning at scale: System design,
Bonawitz, Keith, Eichner, Hubert, Grieskamp, Wolfgang, Huba, Dzmitry, Ingerman, Alex, Ivanov, Vladimir, Kiddon, Chloe, Kone ˇcn`y, Jakub, Mazzocchi, Stefano, McMahan, Brendan, others, “Towards federated learning at scale: System design,” in Proceedings of machine learning and ...
2019
-
[10]
The future of digital health with federated learning. NPJ Digital Medicine, 3, 119,
Rieke, Nicola, Hancox, Jonny, Li, Wenqi, Milletar `ı, Fausto, Roth, Holger R, Albarqouni, Shadi, Bakas, Spyridon, Galtier, Mathieu N, Landman, Bennett A, Maier-Hein, Klaus, others, “The future of digital health with federated learning. NPJ Digital Medicine, 3, 119,” 2020
2020
-
[11]
Federated learning: Opportunities and chal- lenges,
Mammen, Priyanka Mary, “Federated learning: Opportunities and chal- lenges,” in arXiv preprint arXiv:2101.05428, 2021
2021 arXiv
-
[12]
Federated personal- ized random forest for human activity recognition,
Liu, Songfeng, Wang, Jinyan, Zhang, Wenliang, “Federated personal- ized random forest for human activity recognition,” in Mathematical Biosciences and Engineering, vol. 19(1), pp. 953–971, 2022
2022
-
[13]
Transfer learning via random forests: A one-shot federated approach,
Xiang, Pengcheng, Zhou, Ling, Tang, Lu, “Transfer learning via random forests: A one-shot federated approach,” in Computational Statistics & Data Analysis, vol. 197, pp. 107975, 2024
2024
-
[14]
COMMUTE: communication- efficient transfer learning for multi-site risk prediction,
Gu, Tian, Lee, Phil H, Duan, Rui, “COMMUTE: communication- efficient transfer learning for multi-site risk prediction,” in Journal of biomedical informatics, vol. 137, pp. 104243, 2023
2023
-
[15]
Differentially private secure multi-party computation for federated learning in financial applications,
Byrd, David, Polychroniadou, Antigoni, “Differentially private secure multi-party computation for federated learning in financial applications,” in Proceedings of the first ACM international conference on AI in finance, pp. 1–9, 2020
2020
-
[16]
Can a trusted environment provide security?,
Kaufman, Lori M, “Can a trusted environment provide security?,” in IEEE Security & Privacy, vol. 8(1), pp. 50–52, 2010
2010
-
[17]
Trusted execution environment: What it is, and what it is not,
Sabt, Mohamed, Achemlal, Mohammed, Bouabdallah, Abdelmadjid, “Trusted execution environment: What it is, and what it is not,” in 2015 IEEE Trustcom/BigDataSE/Ispa, vol. 1, pp. 57–64, 2015
2015
-
[18]
Xgboost: A scalable tree boosting system,
Chen, Tianqi, Guestrin, Carlos, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794, 2016
2016
-
[19]
A unified approach to interpreting model predictions,
Lundberg, Scott M, Lee, Su-In, “A unified approach to interpreting model predictions,” in Advances in neural information processing sys- tems, vol. 30, 2017
2017
-
[20]
” Why should i trust you?
Ribeiro, Marco Tulio, Singh, Sameer, Guestrin, Carlos, “” Why should i trust you?” Explaining the predictions of any classifier,” in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135–1144, 2016
2016
-
[21]
Why do tree- based models still outperform deep learning on typical tabular data?,
Grinsztajn, L ´eo, Oyallon, Edouard, Varoquaux, Ga ¨el, “Why do tree- based models still outperform deep learning on typical tabular data?,” in Advances in neural information processing systems, vol. 35, pp. 507– 520, 2022
2022
-
[22]
Federated learning: Strategies for improving communication efficiency,
Kone ˇcn`y, Jakub, McMahan, H Brendan, Yu, Felix X, Richt ´arik, Peter, Suresh, Ananda Theertha, Bacon, Dave, “Federated learning: Strategies for improving communication efficiency,” in arXiv preprint arXiv:1610.05492, 2016
2016 arXiv
-
[23]
A survey on security and privacy of federated learning,
Mothukuri, Viraaji, Parizi, Reza M, Pouriyeh, Seyedamin, Huang, Yan, Dehghantanha, Ali, Srivastava, Gautam, “A survey on security and privacy of federated learning,” in Future Generation Computer Systems, vol. 115, pp. 619–640, 2021
2021
-
[24]
Open-source federated learning frameworks for IoT: A comparative review and analysis,
Kholod, Ivan, Yanaki, Evgeny, Fomichev, Dmitry, Shalugin, Evgeniy, Novikova, Evgenia, Filippov, Evgeny, Nordlund, Mats, “Open-source federated learning frameworks for IoT: A comparative review and analysis,” in Sensors, vol. 21(1), pp. 167, 2020
2020
-
[25]
OpenFL: An open-source framework for Federated Learning,
Reina, G Anthony, Gruzdev, Alexey, Foley, Patrick, Perepelkina, Olga, Sharma, Mansi, Davidyuk, Igor, Trushkin, Ilya, Radionov, Maksim, Mokrov, Aleksandr, Agapov, Dmitry, others, “OpenFL: An open-source framework for Federated Learning,” in arXiv preprint arXiv:2105.06413, 2021
2021 arXiv
-
[26]
Federated random forests can improve local per- formance of predictive models for various healthcare applications,
Hauschild, Anne-Christin, Lemanczyk, Marta, Matschinske, Julian, Frisch, Tobias, Zolotareva, Olga, Holzinger, Andreas, Baumbach, Jan, Heider, Dominik, “Federated random forests can improve local per- formance of predictive models for various healthcare applications,” in Bioinf...
2022
-
[27]
Neural natural language pro- cessing for unstructured data in electronic health records: a review,
Li, Irene, Pan, Jessica, Goldwasser, Jeremy, Verma, Neha, Wong, Wai Pan, Nuzumlalı, Muhammed Yavuz, Rosand, Benjamin, Li, Yixin, Zhang, Matthew, Chang, David, others, “Neural natural language pro- cessing for unstructured data in electronic health records: a review,” in Comput...
2022
-
[28]
Towards reverse- engineering black-box neural networks,
Oh, Seong Joon, Schiele, Bernt, Fritz, Mario, “Towards reverse- engineering black-box neural networks,” in Explainable AI: interpreting, explaining and visualizing deep learning, pp. 121–144, 2019
2019
-
[29]
Flower: A friendly federated learning research framework,
Beutel, Daniel J, Topal, Taner, Mathur, Akhil, Qiu, Xinchi, Fernandez- Marques, Javier, Gao, Yan, Sani, Lorenzo, Li, Kwing Hei, Parcol- let, Titouan, de Gusm ˜ao, Pedro Porto Buarque, others, “Flower: A friendly federated learning research framework,” in arXiv preprint arXiv:2...
2007 arXiv
-
[30]
Fate: An industrial grade platform for collaborative learning with data protection,
Liu, Yang, Fan, Tao, Chen, Tianjian, Xu, Qian, Yang, Qiang, “Fate: An industrial grade platform for collaborative learning with data protection,” in Journal of Machine Learning Research, vol. 22(226), pp. 1–6, 2021
2021
-
[31]
Tackling system and statistical heterogeneity for federated learning with adaptive client sampling,
Luo, Bing, Xiao, Wenli, Wang, Shiqiang, Huang, Jianwei, Tassiulas, Leandros, “Tackling system and statistical heterogeneity for federated learning with adaptive client sampling,” in IEEE INFOCOM 2022-IEEE conference on computer communications, pp. 1739–1748, 2022
2022
-
[32]
An effective inte- grated machine learning approach for detecting diabetic retinopathy,
Pragathi, Penikalapati, Nagaraja Rao, Agastyaraju, “An effective inte- grated machine learning approach for detecting diabetic retinopathy,” in Open Computer Science, vol. 12(1), pp. 83–91, 2022
2022
-
[33]
FedTree: A Fast, Effective, and Secure Tree- based Federated Learning System,
Li, Qinbin, Cai, Yanzheng, Han, Yuxuan, Yung, Ching Man, Fu, Tianyuan, He, Bingsheng, “FedTree: A Fast, Effective, and Secure Tree- based Federated Learning System,” 2022
2022
-
[34]
Nvidia flare: Federated learning from simulation to real-world,
Roth, Holger R, Cheng, Yan, Wen, Yuhong, Yang, Isaac, Xu, Ziyue, Hsieh, Yuan-Ting, Kersten, Kristopher, Harouni, Ahmed, Zhao, Can, Lu, Kevin, others, “Nvidia flare: Federated learning from simulation to real-world,” in arXiv preprint arXiv:2210.13291, 2022
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.