REVIEW 3 major objections 4 minor 28 references
Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT
T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read A hybrid server-edge federated learning framework achieves up to 99.2% intrusion detection accuracy with up to 42% less memory and 75% less training time.
desk verdict The reported 99.2% accuracy is a test-set leakage artifact — pre-training on NSL-Test and evaluating on the same data — but the resource-efficiency measurements are real and the paper deserves a corrective review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the split of training labor between server-side pre-training and client-side layer-wise fine-tuning. A server first trains the entire CNN-MLP model on proxy data; clients then freeze all CNN feature-extraction layers and update only a small number of fully connected layers in the MLP head using their private local data. Because the frozen layers produce no new gradients or activations on the client, memory and computation are dominated by the head, which is what yields the reported 42% memory reduction and 75% training-time reduction. The number of fine-tuned layers, one versus three, acts as a knob controlling the accuracy-versus-resource trade-off.
What would settle it
Run the same FedFT pipeline but with server pre-training restricted to a proxy dataset that is disjoint from the evaluation set, then measure accuracy on the untouched NSL-Test split. If accuracy stays near 99.2%, the framework's accuracy claim generalizes; if it drops sharply, the headline accuracy is an artifact of pre-training on the test set.
Extended reading notes
Core claim
The central discovery is that a hybrid server-edge federated learning framework, named FedFT, makes on-device intrusion-detection training lightweight without giving up accuracy. The server pre-trains a CNN-plus-MLP model on proxy data and then distributes it; clients freeze the CNN module and fine-tune only the MLP classification head on their local data, sending the updated head parameters back for federated averaging. This keeps the expensive feature extractor shared and frozen, so each client only allocates memory for the head's parameters, activations, and gradients. The authors report that FedFT-3, which fine-tunes three fully connected layers, reaches 99.2% accuracy with 20% lower memory use and 5% shorter execution time than a centralized baseline, while FedFT-1, which fine-tunes one layer, reaches 94.3% accuracy with 42% lower memory use and 75% shorter execution time. Scalability runs with four, six, and eight clients show accuracy between 93.9% and 94.8%, a drop of less than one percentage point.
Load-bearing premise
The server pre-trains the model on the same NSL-Test data that is later used to measure accuracy, so the reported accuracy assumes it is acceptable for the evaluation set to be seen during pre-training.
Editorial extensions
If this is right
- The same framework could let vehicles with very different hardware join the same federated training round, since each vehicle only needs to run the small classification head update.
- Operators can tune the number of fine-tuned layers per vehicle, trading a few accuracy points for large resource savings in the most constrained devices.
- The design keeps the global model up to date across many clients because only the head parameters are exchanged, and the scalability runs suggest a small accuracy drop as fleet size grows.
- Applying this to the NSL-KDD benchmark positions the approach for network-based attack detection in V2X traffic, not just the laboratory dataset.
Reading between the lines
- If the server were pre-trained on a proxy set actually disjoint from the test set, the 99.2% accuracy figure would likely fall, because the current setup lets the server see the evaluation data during pre-training.
- The eight-client maximum in the scalability study is far below real connected-vehicle fleet sizes, so the 'minimal degradation' claim has only been demonstrated for small groups, not for deployment-scale networks.
- The layer-freezing recipe is transportable: any IoT sensing task with a suitable server-side proxy dataset could adopt the same server-pre-train, client-fine-tune-head pattern.
- The paper assumes IID data across clients; real fleets generate non-IID data, so the resource and accuracy trade-offs under data heterogeneity remain open questions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FedFT, a hybrid server-edge federated learning framework for intrusion detection in connected and autonomous vehicle networks. The server pre-trains an IDS model on proxy data, then clients fine-tune only the classification head on local data. Experiments on NSL-KDD claim competitive accuracy (up to 99.2%) with reduced memory usage (up to 42%) and training time (up to 75%) compared to centralized and FL-IDS baselines, plus scalability to 8 clients.
Significance. If the results were sound, the framework would be a useful engineering contribution for resource-constrained FL deployments: it provides a concrete recipe (server pre-training plus client-side fine-tuning of the classifier head) and reports detailed resource measurements from real Raspberry Pi 4 clients. The scalability analysis is a reasonable first step. However, the central accuracy claim is undermined by a test-set leakage problem in the experimental protocol, and the abstract's efficiency numbers are inconsistent with the reported table. As published, the paper does not provide trustworthy evidence for its main claims.
major comments (3)
- [IV-A] The evaluation protocol leaks the test set into pre-training. Section IV-A states, "we utilize the entire testing data as proxy data for pre-training on the server," and the only testing data in the paper is NSL-Test. The same NSL-Test set is then used for evaluating the final model (the paper reports no other held-out set). The global model therefore sees the features and labels of the evaluation data before fine-tuning, rendering the reported 99.2% accuracy and the comparison with baselines in Table III invalid as estimates of generalization. A correct evaluation must hold out a separate test set that is not touched during either server pre-training or client fine-tuning, or partition NSL-Train into client and test portions.
- [Abstract and Table III] The abstract claims memory usage reductions "up to 42%" and training time decreases "up to 75%". Table III, however, shows FedFT-1 reducing memory from 8.1 MB to 4.3 MB, which is a 47% reduction relative to the centralized baseline (or 49% relative to FL-IDS), and execution time from 1339 s to 564 s, a 58% reduction. No configuration in Table III achieves a 75% training time reduction. The abstract's numbers are not supported by the presented data and should be corrected.
- [IV-A and VI] Section IV-A explicitly states "We assume that the dataset is independently and identically distributed (IID)." This assumption is particularly strong for CAV networks, where traffic patterns, attack types, and device conditions are naturally heterogeneous. The paper nevertheless claims the framework is suitable for large-scale CAV networks. This mismatch is acknowledged only in the future-work sentence about "data heterogeneity." For the paper's central deployment claim to be credible, the experiments need to include at least one non-IID partition, or the claims must be scaled back accordingly.
minor comments (4)
- [IV-B] The paper states each experiment is repeated three times and averaged, but no standard deviations, variance, or confidence intervals are reported. Adding these would strengthen the reliability of the resource and accuracy comparisons.
- [V-B] The scalability analysis in Table IV reports only accuracy, loss, memory, and time, but does not report the accuracy of the centralized or FL-IDS baselines at the same client counts, nor does it describe whether the "virtual Flower client instances" run on the same Raspberry Pis or on the server. This makes the scalability claim hard to interpret.
- [III-B] The classification is binary (normal versus attack), yet NSL-KDD has four attack classes. The paper should justify why a binary decision is appropriate for the CAV intrusion detection use case, especially since the dataset description in Section IV-A lists four attack categories.
- [V-A] The paper compares FedFT against only one FL baseline ([28]). Given the large literature on communication-efficient and resource-aware federated learning, a broader comparison (or a justification for the single baseline) would strengthen the resource-efficiency claims.
Circularity Check
Server pre-trains on NSL-Test and accuracy is measured on that same set; reported 99.2% is a test-set-leakage artifact, not evidence of competitive performance.
-
fitted input called prediction
[Section IV-A (Dataset) and Section V-A / Table III]
"For evaluation, the dataset is split as follows: we utilize the entire testing data as proxy data for pre-training on the server. Among participating devices, we simulate having different data from multiple clients by splitting the original NSL-KDD Train dataset into equal multiple partitions... FedFT-3 achieves a competitive accuracy of 99.2%, comparable to the centralized framework (99.8%)."
The reported accuracy (99.2% for FedFT-3) is presented as a prediction of generalization, but the evaluation set is the same NSL-Test data that the server used for pre-training. Since the server-side model has already fit the features and labels of the test distribution during pre-training, the subsequent accuracy measurement is not an independent estimate of performance on unseen data. The paper describes no other held-out evaluation set, so the central accuracy claim reduces to a test-set-leakage artifact: the 'prediction' is statistically forced by the fact that the evaluation input was also a training input.
full rationale
The central circularity in this paper is experimental-design-based rather than equation-level: the server pre-trains on the entire NSL-Test set and the reported accuracy is computed on that same set, so the headline accuracy claim is not an honest generalization result. By the prompt's fitted-input-called-prediction pattern, this is a real circular step because the prediction is statistically forced by the training/evaluation overlap. The resource-efficiency claims (memory reduction up to 42%, execution time reduction up to 75%) are measured on physical Raspberry Pi devices and are independent of this leakage, so the engineering contribution retains some independent content. The comparison baseline FL-IDS [28] is a self-citation by two of the authors, but it is used only as a baseline, not as load-bearing justification for the framework's design or claimed uniqueness, so it does not independently raise the score. No other derivation in the paper reduces to its inputs by construction. The overall score is therefore 6 rather than 8 or 10 because the paper's core efficiency claims remain empirically grounded, but the central accuracy claim is partially circular due to test-set leakage.
Assumptions & free parameters
free parameters (8)
- batch size =
32
- local epochs per round =
5
- learning rate =
0.01
- momentum =
0.9
- number of fine-tuned layers =
1 or 3 (FedFT-1, FedFT-3)
- PCA components =
unspecified
- pre-training epochs =
unspecified
- communication rounds =
unspecified
assumptions (4)
- domain assumption Client data partitions are independent and identically distributed (IID)
- domain assumption NSL-KDD is a valid proxy for CAV network traffic
- standard math FedAvg aggregation converges for partial fine-tuning
- domain assumption Memory and execution time measurements are accurate
Cite this review
Pith. "Pith review of Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT." pith.science (2026). https://pith.science/paper/GNUY4253
@misc{pith2026250206099,
author = {Pith},
title = {Pith review of: Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT},
year = {2026},
howpublished = {\url{https://pith.science/paper/GNUY4253}},
note = {Machine review of arXiv:2502.06099}
}
read the original abstract
The rapid advancement of machine learning (ML) and on-device computing has revolutionized various industries, including transportation, through the development of Connected and Autonomous Vehicles (CAVs) and Intelligent Transportation Systems (ITS). These technologies improve traffic management and vehicle safety, but also introduce significant security and privacy concerns, such as cyberattacks and data breaches. Traditional Intrusion Detection Systems (IDS) are increasingly inadequate in detecting modern threats, leading to the adoption of ML-based IDS solutions. Federated Learning (FL) has emerged as a promising method for enabling the decentralized training of IDS models on distributed edge devices without sharing sensitive data. However, deploying FL-based IDS in CAV networks poses unique challenges, including limited computational and memory resources on edge devices, competing demands from critical applications such as navigation and safety systems, and the need to scale across diverse hardware and connectivity conditions. To address these issues, we propose a hybrid server-edge FL framework that offloads pre-training to a central server while enabling lightweight fine-tuning on edge devices. This approach reduces memory usage by up to 42%, decreases training times by up to 75%, and achieves competitive IDS accuracy of up to 99.2%. Scalability analyses further demonstrates minimal performance degradation as the number of clients increase, highlighting the framework's feasibility for CAV networks and other IoT applications.
Figures
Reference graph
Works this paper leans on
-
[28]
M. Bhavsar, Y . Bekele, K. Roy, J. Kelly, and D. Limbrick, “Fl- ids: Federated learning-based intrusion detection system using edge devices for transportation iot,” IEEE Access, 2024
work page 2024
-
[1]
T. Ahmad, R. Madonski, D. Zhang, C. Huang, and A. Mujeeb, “Data-driven probabilistic machine learning in sustainable smart en- ergy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm,” Renew- able and Sustainable Energy Reviews , vol. 160, p. 112128, 2022
work page 2022
-
[2]
A survey of recent advances in edge-computing-powered artificial intelligence of things,
Z. Chang, S. Liu, X. Xiong, Z. Cai, and G. Tu, “A survey of recent advances in edge-computing-powered artificial intelligence of things,” IEEE Internet of Things Journal , vol. 8, no. 18, pp. 13 849–13 875, 2021
work page 2021
-
[3]
Integrating connected vehicles in internet of things ecosystems: Challenges and solutions,
S. K. Datta, R. P. F. Da Costa, J. Härri, and C. Bonnet, “Integrating connected vehicles in internet of things ecosystems: Challenges and solutions,” in 2016 IEEE 17th international symposium on a world of wireless, mobile and multimedia networks (WoWMoM) . IEEE, 2016, pp. 1–6
work page 2016
-
[4]
Research advances and chal- lenges of autonomous and connected ground vehicles,
A. Eskandarian, C. Wu, and C. Sun, “Research advances and chal- lenges of autonomous and connected ground vehicles,” IEEE Trans- actions on Intelligent Transportation Systems , vol. 22, no. 2, pp. 683– 711, 2019
work page 2019
-
[5]
V2x access technologies: Regulation, research, and remaining challenges,
Z. MacHardy, A. Khan, K. Obana, and S. Iwashina, “V2x access technologies: Regulation, research, and remaining challenges,” IEEE Communications Surveys & Tutorials , vol. 20, no. 3, pp. 1858–1877, 2018
work page 2018
-
[6]
Security and privacy issues in autonomous vehicles: A layer-based survey,
M. Hataba, A. Sherif, M. Mahmoud, M. Abdallah, and W. Alasmary, “Security and privacy issues in autonomous vehicles: A layer-based survey,” IEEE Open Journal of the Communications Society , vol. 3, pp. 811–829, 2022
work page 2022
-
[7]
Communication-efficient learning of deep networks from decentral- ized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentral- ized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282
2017
Show all 28 references
-
[8]
Federated learning for connected and automated vehicles: A survey of existing approaches and challenges,
V . P. Chellapandi, L. Yuan, C. G. Brinton, S. H. ˙Zak, and Z. Wang, “Federated learning for connected and automated vehicles: A survey of existing approaches and challenges,” IEEE Transactions on Intelligent Vehicles, 2023
2023
-
[9]
Hyperparameter tuning for federated learning– systems and practices,
S. Zawad and F. Yan, “Hyperparameter tuning for federated learning– systems and practices,” in Federated Learning. Elsevier, 2024, pp. 219–235
2024
-
[10]
Deep transfer learning based intrusion detection system for electric vehicular networks,
S. T. Mehedi, A. Anwar, Z. Rahman, and K. Ahmed, “Deep transfer learning based intrusion detection system for electric vehicular networks,” Sensors, vol. 21, no. 14, 2021. [Online]. Available: https://www.mdpi.com/1424-8220/21/14/4736
2021
-
[11]
Deep learning- based intrusion detection systems: a systematic review,
J. Lansky, S. Ali, M. Mohammadi, M. K. Majeed, S. H. T. Karim, S. Rashidi, M. Hosseinzadeh, and A. M. Rahmani, “Deep learning- based intrusion detection systems: a systematic review,” IEEE Access, vol. 9, pp. 101 574–101 599, 2021
2021
-
[12]
Review of intrusion detection systems based on deep learning techniques: coher- ent taxonomy, challenges, motivations, recommendations, substantial analysis and future directions,
A. Aleesa, B. Zaidan, A. Zaidan, and N. M. Sahar, “Review of intrusion detection systems based on deep learning techniques: coher- ent taxonomy, challenges, motivations, recommendations, substantial analysis and future directions,” Neural Computing and Applications , vol. 32, ...
2020
-
[13]
One-shot federated learning,
N. Guha, A. Talwalkar, and V . Smith, “One-shot federated learning,” arXiv preprint arXiv:1902.11175 , 2019
1902 arXiv
-
[14]
A survey of federated learning for connected and automated vehicles,
V . P. Chellapandi, L. Yuan, S. H. ˙Zak, and Z. Wang, “A survey of federated learning for connected and automated vehicles,” in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2023, pp. 2485–2492
2023
-
[15]
Privacy-preserving backdoor attacks mitigation in federated learning using functional encryption,
F. Olagunju, I. Adom, and M. N. Mahmoud, “Privacy-preserving backdoor attacks mitigation in federated learning using functional encryption,” in SoutheastCon 2024. IEEE, 2024, pp. 531–539
2024
-
[16]
Deepfed: Federated deep learning for intrusion detection in industrial cyber–physical systems,
B. Li, Y . Wu, J. Song, R. Lu, T. Li, and L. Zhao, “Deepfed: Federated deep learning for intrusion detection in industrial cyber–physical systems,” IEEE Transactions on Industrial Informatics , vol. 17, no. 8, pp. 5615–5624, 2020
2020
-
[17]
Dïot: A federated self-learning anomaly detection system for iot,
T. D. Nguyen, S. Marchal, M. Miettinen, H. Fereidooni, N. Asokan, and A.-R. Sadeghi, “Dïot: A federated self-learning anomaly detection system for iot,” in 2019 IEEE 39th International conference on distributed computing systems (ICDCS) . IEEE, 2019, pp. 756–767
2019
-
[18]
Towards federated learning at scale: Syste m design,
K. Bonawitz, “Towards federated learning at scale: Syste m design,” arXiv preprint arXiv:1902.01046 , 2019
1902 arXiv
-
[19]
Once-for-all: Train one network and specialize it for efficient deployment,
H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, “Once-for-all: Train one network and specialize it for efficient deployment,” arXiv preprint arXiv:1908.09791, 2019
1908 arXiv
-
[20]
Resource-efficient machine learning in 2 kb ram for the internet of things,
A. Kumar, S. Goyal, and M. Varma, “Resource-efficient machine learning in 2 kb ram for the internet of things,” in International conference on machine learning . PMLR, 2017, pp. 1935–1944
2017
-
[21]
Novel deep learning-enabled lstm autoencoder architec- ture for discovering anomalous events from intelligent transportation systems,
J. Ashraf, A. D. Bakhshi, N. Moustafa, H. Khurshid, A. Javed, and A. Beheshti, “Novel deep learning-enabled lstm autoencoder architec- ture for discovering anomalous events from intelligent transportation systems,” IEEE Transactions on Intelligent Transportation Systems , vol....
2020
-
[22]
A review of deep transfer learning and recent advancements,
M. Iman, H. R. Arabnia, and K. Rasheed, “A review of deep transfer learning and recent advancements,” Technologies, vol. 11, no. 2, p. 40, 2023
2023
-
[23]
Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,
H.-C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers, “Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,” IEEE transactions on medical imaging, vol. 35, n...
2016
-
[24]
Tinytl: Reduce memory, not parameters for efficient on-device learning,
H. Cai, C. Gan, L. Zhu, and S. Han, “Tinytl: Reduce memory, not parameters for efficient on-device learning,” Advances in Neural Information Processing Systems , vol. 33, pp. 11 285–11 297, 2020
2020
-
[25]
Flower: A friendly federated learning research framework,
D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y . Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão et al. , “Flower: A friendly federated learning research framework,” arXiv preprint arXiv:2007.14390, 2020
2007 arXiv
-
[26]
A study on nsl-kdd dataset for intrusion detection system based on classification algorithms,
L. Dhanabal and S. Shantharajah, “A study on nsl-kdd dataset for intrusion detection system based on classification algorithms,” Inter- national journal of advanced research in computer and communication engineering, vol. 4, no. 6, pp. 446–452, 2015
2015
-
[27]
A detailed analysis of the kdd cup 99 data set,
M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the kdd cup 99 data set,” in 2009 IEEE symposium on computational intelligence for security and defense applications. Ieee, 2009, pp. 1–6
2009
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.