Pith. sign in

REVIEW 3 major objections 4 minor 28 references

Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Pith's one-line read A hybrid server-edge federated learning framework achieves up to 99.2% intrusion detection accuracy with up to 42% less memory and 75% less training time.

desk verdict The reported 99.2% accuracy is a test-set leakage artifact — pre-training on NSL-Test and evaluating on the same data — but the resource-efficiency measurements are real and the paper deserves a corrective review. read the letter →

arxiv 2502.06099 v1 pith:GNUY4253 submitted 2025-02-10 cs.LG

classification cs.LG
keywords FederatedLearningIntrusionDetectionConnectedandAutonomousVehiclesTransferFine-TuningResourceConstraintsScalabilityNSL-KDD
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the resource bottleneck of federated intrusion detection on connected vehicles can be broken by splitting the work between a central server and edge devices. The server pre-trains a shared model on a proxy dataset, and each client freezes the feature-extraction layers and fine-tunes only a small classification head on its local data. On the NSL-KDD benchmark the authors measure up to 99.2% accuracy, up to 42% lower training-memory use, and up to 75% shorter training time than the baselines they compare against. They also report that accuracy degrades by less than one percent as the number of clients grows from four to eight. If these results are correct, the framework offers a practical way to run intrusion detection training on vehicles with tight memory and compute budgets.

What carries the argument

The load-bearing mechanism is the split of training labor between server-side pre-training and client-side layer-wise fine-tuning. A server first trains the entire CNN-MLP model on proxy data; clients then freeze all CNN feature-extraction layers and update only a small number of fully connected layers in the MLP head using their private local data. Because the frozen layers produce no new gradients or activations on the client, memory and computation are dominated by the head, which is what yields the reported 42% memory reduction and 75% training-time reduction. The number of fine-tuned layers, one versus three, acts as a knob controlling the accuracy-versus-resource trade-off.

What would settle it

Run the same FedFT pipeline but with server pre-training restricted to a proxy dataset that is disjoint from the evaluation set, then measure accuracy on the untouched NSL-Test split. If accuracy stays near 99.2%, the framework's accuracy claim generalizes; if it drops sharply, the headline accuracy is an artifact of pre-training on the test set.

Watch

Extended reading notes

Core claim

The central discovery is that a hybrid server-edge federated learning framework, named FedFT, makes on-device intrusion-detection training lightweight without giving up accuracy. The server pre-trains a CNN-plus-MLP model on proxy data and then distributes it; clients freeze the CNN module and fine-tune only the MLP classification head on their local data, sending the updated head parameters back for federated averaging. This keeps the expensive feature extractor shared and frozen, so each client only allocates memory for the head's parameters, activations, and gradients. The authors report that FedFT-3, which fine-tunes three fully connected layers, reaches 99.2% accuracy with 20% lower memory use and 5% shorter execution time than a centralized baseline, while FedFT-1, which fine-tunes one layer, reaches 94.3% accuracy with 42% lower memory use and 75% shorter execution time. Scalability runs with four, six, and eight clients show accuracy between 93.9% and 94.8%, a drop of less than one percentage point.

Load-bearing premise

The server pre-trains the model on the same NSL-Test data that is later used to measure accuracy, so the reported accuracy assumes it is acceptable for the evaluation set to be seen during pre-training.

Editorial extensions

If this is right

  • The same framework could let vehicles with very different hardware join the same federated training round, since each vehicle only needs to run the small classification head update.
  • Operators can tune the number of fine-tuned layers per vehicle, trading a few accuracy points for large resource savings in the most constrained devices.
  • The design keeps the global model up to date across many clients because only the head parameters are exchanged, and the scalability runs suggest a small accuracy drop as fleet size grows.
  • Applying this to the NSL-KDD benchmark positions the approach for network-based attack detection in V2X traffic, not just the laboratory dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the server were pre-trained on a proxy set actually disjoint from the test set, the 99.2% accuracy figure would likely fall, because the current setup lets the server see the evaluation data during pre-training.
  • The eight-client maximum in the scalability study is far below real connected-vehicle fleet sizes, so the 'minimal degradation' claim has only been demonstrated for small groups, not for deployment-scale networks.
  • The layer-freezing recipe is transportable: any IoT sensing task with a suitable server-side proxy dataset could adopt the same server-pre-train, client-fine-tune-head pattern.
  • The paper assumes IID data across clients; real fleets generate non-IID data, so the resource and accuracy trade-offs under data heterogeneity remain open questions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FedFT, a hybrid server-edge federated learning framework for intrusion detection in connected and autonomous vehicle networks. The server pre-trains an IDS model on proxy data, then clients fine-tune only the classification head on local data. Experiments on NSL-KDD claim competitive accuracy (up to 99.2%) with reduced memory usage (up to 42%) and training time (up to 75%) compared to centralized and FL-IDS baselines, plus scalability to 8 clients.

Significance. If the results were sound, the framework would be a useful engineering contribution for resource-constrained FL deployments: it provides a concrete recipe (server pre-training plus client-side fine-tuning of the classifier head) and reports detailed resource measurements from real Raspberry Pi 4 clients. The scalability analysis is a reasonable first step. However, the central accuracy claim is undermined by a test-set leakage problem in the experimental protocol, and the abstract's efficiency numbers are inconsistent with the reported table. As published, the paper does not provide trustworthy evidence for its main claims.

major comments (3)
  1. [IV-A] The evaluation protocol leaks the test set into pre-training. Section IV-A states, "we utilize the entire testing data as proxy data for pre-training on the server," and the only testing data in the paper is NSL-Test. The same NSL-Test set is then used for evaluating the final model (the paper reports no other held-out set). The global model therefore sees the features and labels of the evaluation data before fine-tuning, rendering the reported 99.2% accuracy and the comparison with baselines in Table III invalid as estimates of generalization. A correct evaluation must hold out a separate test set that is not touched during either server pre-training or client fine-tuning, or partition NSL-Train into client and test portions.
  2. [Abstract and Table III] The abstract claims memory usage reductions "up to 42%" and training time decreases "up to 75%". Table III, however, shows FedFT-1 reducing memory from 8.1 MB to 4.3 MB, which is a 47% reduction relative to the centralized baseline (or 49% relative to FL-IDS), and execution time from 1339 s to 564 s, a 58% reduction. No configuration in Table III achieves a 75% training time reduction. The abstract's numbers are not supported by the presented data and should be corrected.
  3. [IV-A and VI] Section IV-A explicitly states "We assume that the dataset is independently and identically distributed (IID)." This assumption is particularly strong for CAV networks, where traffic patterns, attack types, and device conditions are naturally heterogeneous. The paper nevertheless claims the framework is suitable for large-scale CAV networks. This mismatch is acknowledged only in the future-work sentence about "data heterogeneity." For the paper's central deployment claim to be credible, the experiments need to include at least one non-IID partition, or the claims must be scaled back accordingly.
minor comments (4)
  1. [IV-B] The paper states each experiment is repeated three times and averaged, but no standard deviations, variance, or confidence intervals are reported. Adding these would strengthen the reliability of the resource and accuracy comparisons.
  2. [V-B] The scalability analysis in Table IV reports only accuracy, loss, memory, and time, but does not report the accuracy of the centralized or FL-IDS baselines at the same client counts, nor does it describe whether the "virtual Flower client instances" run on the same Raspberry Pis or on the server. This makes the scalability claim hard to interpret.
  3. [III-B] The classification is binary (normal versus attack), yet NSL-KDD has four attack classes. The paper should justify why a binary decision is appropriate for the CAV intrusion detection use case, especially since the dataset description in Section IV-A lists four attack categories.
  4. [V-A] The paper compares FedFT against only one FL baseline ([28]). Given the large literature on communication-efficient and resource-aware federated learning, a broader comparison (or a justification for the single baseline) would strengthen the resource-efficiency claims.

Circularity Check

1 steps flagged · score 6.0 of 10

Server pre-trains on NSL-Test and accuracy is measured on that same set; reported 99.2% is a test-set-leakage artifact, not evidence of competitive performance.

  1. fitted input called prediction [Section IV-A (Dataset) and Section V-A / Table III]
    "For evaluation, the dataset is split as follows: we utilize the entire testing data as proxy data for pre-training on the server. Among participating devices, we simulate having different data from multiple clients by splitting the original NSL-KDD Train dataset into equal multiple partitions... FedFT-3 achieves a competitive accuracy of 99.2%, comparable to the centralized framework (99.8%)."

    The reported accuracy (99.2% for FedFT-3) is presented as a prediction of generalization, but the evaluation set is the same NSL-Test data that the server used for pre-training. Since the server-side model has already fit the features and labels of the test distribution during pre-training, the subsequent accuracy measurement is not an independent estimate of performance on unseen data. The paper describes no other held-out evaluation set, so the central accuracy claim reduces to a test-set-leakage artifact: the 'prediction' is statistically forced by the fact that the evaluation input was also a training input.

full rationale

The central circularity in this paper is experimental-design-based rather than equation-level: the server pre-trains on the entire NSL-Test set and the reported accuracy is computed on that same set, so the headline accuracy claim is not an honest generalization result. By the prompt's fitted-input-called-prediction pattern, this is a real circular step because the prediction is statistically forced by the training/evaluation overlap. The resource-efficiency claims (memory reduction up to 42%, execution time reduction up to 75%) are measured on physical Raspberry Pi devices and are independent of this leakage, so the engineering contribution retains some independent content. The comparison baseline FL-IDS [28] is a self-citation by two of the authors, but it is used only as a baseline, not as load-bearing justification for the framework's design or claimed uniqueness, so it does not independently raise the score. No other derivation in the paper reduces to its inputs by construction. The overall score is therefore 6 rather than 8 or 10 because the paper's core efficiency claims remain empirically grounded, but the central accuracy claim is partially circular due to test-set leakage.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The central efficiency claims depend on several hand-chosen hyperparameters (batch size, epochs, learning rate, momentum, fine-tuned layers) and on assumptions that the data is IID and that NSL-KDD represents CAV traffic. The architecture details and measurement protocol are not fully specified, adding to the burden on the reader.

free parameters (8)
  • batch size = 32
    Chosen empirically as the best trade-off between resource consumption and accuracy (Section IV-C).
  • local epochs per round = 5
    Selected from 5, 10, and 15; higher values increased computational cost without significant accuracy gains (Section IV-C).
  • learning rate = 0.01
    Empirically chosen for SGD (Section IV-C).
  • momentum = 0.9
    Empirically chosen for SGD (Section IV-C).
  • number of fine-tuned layers = 1 or 3 (FedFT-1, FedFT-3)
    Configurations compared in Table III; not optimized.
  • PCA components = unspecified
    Feature extraction using PCA on each client (Section IV-A); number of components not given.
  • pre-training epochs = unspecified
    Server pre-trains for a specified number of epochs (Section III-A); not reported.
  • communication rounds = unspecified
    Number of federated rounds not reported.
assumptions (4)
  • domain assumption Client data partitions are independent and identically distributed (IID)
    Stated in Section IV-A: "We assume that the dataset is independently and identically distributed (IID)." This is unrealistic for CAV fleets where traffic patterns and attack distributions vary across vehicles.
  • domain assumption NSL-KDD is a valid proxy for CAV network traffic
    Section IV-A notes the dataset is not explicitly designed for CAV architecture but asserts DoS attacks are applicable to V2X communications; no further validation.
  • standard math FedAvg aggregation converges for partial fine-tuning
    The framework relies on the standard Federated Averaging algorithm [7] for server aggregation; convergence is assumed.
  • domain assumption Memory and execution time measurements are accurate
    The paper reports memory usage and execution time without a detailed measurement protocol, so the numbers are taken at face value.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT." pith.science (2026). https://pith.science/paper/GNUY4253

@misc{pith2026250206099,
  author       = {Pith},
  title        = {Pith review of: Fine-Tuning Federated Learning-Based Intrusion Detection Systems for Transportation IoT},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GNUY4253}},
  note         = {Machine review of arXiv:2502.06099}
}
read the original abstract

The rapid advancement of machine learning (ML) and on-device computing has revolutionized various industries, including transportation, through the development of Connected and Autonomous Vehicles (CAVs) and Intelligent Transportation Systems (ITS). These technologies improve traffic management and vehicle safety, but also introduce significant security and privacy concerns, such as cyberattacks and data breaches. Traditional Intrusion Detection Systems (IDS) are increasingly inadequate in detecting modern threats, leading to the adoption of ML-based IDS solutions. Federated Learning (FL) has emerged as a promising method for enabling the decentralized training of IDS models on distributed edge devices without sharing sensitive data. However, deploying FL-based IDS in CAV networks poses unique challenges, including limited computational and memory resources on edge devices, competing demands from critical applications such as navigation and safety systems, and the need to scale across diverse hardware and connectivity conditions. To address these issues, we propose a hybrid server-edge FL framework that offloads pre-training to a central server while enabling lightweight fine-tuning on edge devices. This approach reduces memory usage by up to 42%, decreases training times by up to 75%, and achieves competitive IDS accuracy of up to 99.2%. Scalability analyses further demonstrates minimal performance degradation as the number of clients increase, highlighting the framework's feasibility for CAV networks and other IoT applications.

Figures

Figures reproduced from arXiv: 2502.06099 by the authors.

Figure 1
Figure 1. Federated Learning Architecture for vehicular net [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The FedFT framework: Server-side pre-training on [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. IDS model architecture: Pre-trained CNN Module [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Deployment setup showing a Flower server co [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 22 canonical work pages

  1. [28]

    Fl- ids: Federated learning-based intrusion detection system using edge devices for transportation iot,

    M. Bhavsar, Y . Bekele, K. Roy, J. Kelly, and D. Limbrick, “Fl- ids: Federated learning-based intrusion detection system using edge devices for transportation iot,” IEEE Access, 2024

  2. [1]

    Ahmad, R

    T. Ahmad, R. Madonski, D. Zhang, C. Huang, and A. Mujeeb, “Data-driven probabilistic machine learning in sustainable smart en- ergy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm,” Renew- able and Sustainable Energy Reviews , vol. 160, p. 112128, 2022

  3. [2]

    A survey of recent advances in edge-computing-powered artificial intelligence of things,

    Z. Chang, S. Liu, X. Xiong, Z. Cai, and G. Tu, “A survey of recent advances in edge-computing-powered artificial intelligence of things,” IEEE Internet of Things Journal , vol. 8, no. 18, pp. 13 849–13 875, 2021

  4. [3]

    Integrating connected vehicles in internet of things ecosystems: Challenges and solutions,

    S. K. Datta, R. P. F. Da Costa, J. Härri, and C. Bonnet, “Integrating connected vehicles in internet of things ecosystems: Challenges and solutions,” in 2016 IEEE 17th international symposium on a world of wireless, mobile and multimedia networks (WoWMoM) . IEEE, 2016, pp. 1–6

  5. [4]

    Research advances and chal- lenges of autonomous and connected ground vehicles,

    A. Eskandarian, C. Wu, and C. Sun, “Research advances and chal- lenges of autonomous and connected ground vehicles,” IEEE Trans- actions on Intelligent Transportation Systems , vol. 22, no. 2, pp. 683– 711, 2019

  6. [5]

    V2x access technologies: Regulation, research, and remaining challenges,

    Z. MacHardy, A. Khan, K. Obana, and S. Iwashina, “V2x access technologies: Regulation, research, and remaining challenges,” IEEE Communications Surveys & Tutorials , vol. 20, no. 3, pp. 1858–1877, 2018

  7. [6]

    Security and privacy issues in autonomous vehicles: A layer-based survey,

    M. Hataba, A. Sherif, M. Mahmoud, M. Abdallah, and W. Alasmary, “Security and privacy issues in autonomous vehicles: A layer-based survey,” IEEE Open Journal of the Communications Society , vol. 3, pp. 811–829, 2022

  8. [7]

    Communication-efficient learning of deep networks from decentral- ized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentral- ized data,” in Artificial intelligence and statistics . PMLR, 2017, pp. 1273–1282

Show all 28 references
  1. [8]

    Federated learning for connected and automated vehicles: A survey of existing approaches and challenges,

    V . P. Chellapandi, L. Yuan, C. G. Brinton, S. H. ˙Zak, and Z. Wang, “Federated learning for connected and automated vehicles: A survey of existing approaches and challenges,” IEEE Transactions on Intelligent Vehicles, 2023

  2. [9]

    Hyperparameter tuning for federated learning– systems and practices,

    S. Zawad and F. Yan, “Hyperparameter tuning for federated learning– systems and practices,” in Federated Learning. Elsevier, 2024, pp. 219–235

  3. [10]

    Deep transfer learning based intrusion detection system for electric vehicular networks,

    S. T. Mehedi, A. Anwar, Z. Rahman, and K. Ahmed, “Deep transfer learning based intrusion detection system for electric vehicular networks,” Sensors, vol. 21, no. 14, 2021. [Online]. Available: https://www.mdpi.com/1424-8220/21/14/4736

  4. [11]

    Deep learning- based intrusion detection systems: a systematic review,

    J. Lansky, S. Ali, M. Mohammadi, M. K. Majeed, S. H. T. Karim, S. Rashidi, M. Hosseinzadeh, and A. M. Rahmani, “Deep learning- based intrusion detection systems: a systematic review,” IEEE Access, vol. 9, pp. 101 574–101 599, 2021

  5. [12]

    Review of intrusion detection systems based on deep learning techniques: coher- ent taxonomy, challenges, motivations, recommendations, substantial analysis and future directions,

    A. Aleesa, B. Zaidan, A. Zaidan, and N. M. Sahar, “Review of intrusion detection systems based on deep learning techniques: coher- ent taxonomy, challenges, motivations, recommendations, substantial analysis and future directions,” Neural Computing and Applications , vol. 32, ...

  6. [13]

    One-shot federated learning,

    N. Guha, A. Talwalkar, and V . Smith, “One-shot federated learning,” arXiv preprint arXiv:1902.11175 , 2019

  7. [14]

    A survey of federated learning for connected and automated vehicles,

    V . P. Chellapandi, L. Yuan, S. H. ˙Zak, and Z. Wang, “A survey of federated learning for connected and automated vehicles,” in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2023, pp. 2485–2492

  8. [15]

    Privacy-preserving backdoor attacks mitigation in federated learning using functional encryption,

    F. Olagunju, I. Adom, and M. N. Mahmoud, “Privacy-preserving backdoor attacks mitigation in federated learning using functional encryption,” in SoutheastCon 2024. IEEE, 2024, pp. 531–539

  9. [16]

    Deepfed: Federated deep learning for intrusion detection in industrial cyber–physical systems,

    B. Li, Y . Wu, J. Song, R. Lu, T. Li, and L. Zhao, “Deepfed: Federated deep learning for intrusion detection in industrial cyber–physical systems,” IEEE Transactions on Industrial Informatics , vol. 17, no. 8, pp. 5615–5624, 2020

  10. [17]

    Dïot: A federated self-learning anomaly detection system for iot,

    T. D. Nguyen, S. Marchal, M. Miettinen, H. Fereidooni, N. Asokan, and A.-R. Sadeghi, “Dïot: A federated self-learning anomaly detection system for iot,” in 2019 IEEE 39th International conference on distributed computing systems (ICDCS) . IEEE, 2019, pp. 756–767

  11. [18]

    Towards federated learning at scale: Syste m design,

    K. Bonawitz, “Towards federated learning at scale: Syste m design,” arXiv preprint arXiv:1902.01046 , 2019

  12. [19]

    Once-for-all: Train one network and specialize it for efficient deployment,

    H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, “Once-for-all: Train one network and specialize it for efficient deployment,” arXiv preprint arXiv:1908.09791, 2019

  13. [20]

    Resource-efficient machine learning in 2 kb ram for the internet of things,

    A. Kumar, S. Goyal, and M. Varma, “Resource-efficient machine learning in 2 kb ram for the internet of things,” in International conference on machine learning . PMLR, 2017, pp. 1935–1944

  14. [21]

    Novel deep learning-enabled lstm autoencoder architec- ture for discovering anomalous events from intelligent transportation systems,

    J. Ashraf, A. D. Bakhshi, N. Moustafa, H. Khurshid, A. Javed, and A. Beheshti, “Novel deep learning-enabled lstm autoencoder architec- ture for discovering anomalous events from intelligent transportation systems,” IEEE Transactions on Intelligent Transportation Systems , vol....

  15. [22]

    A review of deep transfer learning and recent advancements,

    M. Iman, H. R. Arabnia, and K. Rasheed, “A review of deep transfer learning and recent advancements,” Technologies, vol. 11, no. 2, p. 40, 2023

  16. [23]

    Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,

    H.-C. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and R. M. Summers, “Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning,” IEEE transactions on medical imaging, vol. 35, n...

  17. [24]

    Tinytl: Reduce memory, not parameters for efficient on-device learning,

    H. Cai, C. Gan, L. Zhu, and S. Han, “Tinytl: Reduce memory, not parameters for efficient on-device learning,” Advances in Neural Information Processing Systems , vol. 33, pp. 11 285–11 297, 2020

  18. [25]

    Flower: A friendly federated learning research framework,

    D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y . Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão et al. , “Flower: A friendly federated learning research framework,” arXiv preprint arXiv:2007.14390, 2020

  19. [26]

    A study on nsl-kdd dataset for intrusion detection system based on classification algorithms,

    L. Dhanabal and S. Shantharajah, “A study on nsl-kdd dataset for intrusion detection system based on classification algorithms,” Inter- national journal of advanced research in computer and communication engineering, vol. 4, no. 6, pp. 446–452, 2015

  20. [27]

    A detailed analysis of the kdd cup 99 data set,

    M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the kdd cup 99 data set,” in 2009 IEEE symposium on computational intelligence for security and defense applications. Ieee, 2009, pp. 1–6

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.