Pith. sign in

REVIEW 4 major objections 6 minor 32 references

Evaluating the Impact of Privacy-Preserving Federated Learning on CAN Intrusion Detection

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A federated version of the CANdito LSTM autoencoder IDS, trained with FedAvg across 50 vehicles, detects CAN bus attacks nearly as well as its centralized counterpart while keeping raw data local.

desk verdict Useful engineering validation of FL for LSTM-autoencoder CAN intrusion detection, with real communication overhead numbers; the claimed non-i.i.d. setting is under-specified and probably i.i.d., which weakens the feasibility generalization. read the letter →

arxiv 2506.04978 v1 pith:OZSVZGXF submitted 2025-06-05 cs.CR

Gabriele Digregorio , Elisabetta Cainazzo , Stefano Longari , Michele Carminati , Stefano Zanero This is my paper
classification cs.CR
keywords ControllerAreaNetworkFederatedLearningIntrusionDetectionLSTMautoencoderVehicularsecurityPrivacy-preservingmachine5GcommunicationCANbus
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Federated learning could make privacy-preserving intrusion detection on vehicle CAN buses practical, and this paper builds the case by turning an existing LSTM autoencoder detector into a federated one. The authors train the detector across simulated fleets of 5 to 50 vehicles using the FedAvg aggregation rule, then compare attack detection and communication cost against the same detector trained centrally on identical data. The federated model shows good detection of injection, masquerade, and drop attacks, with a modest performance gap behind the centralized version and, on some metrics, a lower median false positive rate. In exchange for that small gap, raw CAN traffic never leaves the vehicle. The paper concludes the added communication load is sustainable on current 5G links, making this a viable privacy-preserving alternative to central data collection.

What carries the argument

The load-bearing pieces are the LSTM autoencoder and the FedAvg loop that connects vehicles to a global server. The autoencoder learns normal CAN traffic by reconstructing windows of 40 consecutive packets per signal; at run time it flags an attack when the reconstruction error exceeds a threshold. In the federated setting, each vehicle trains only on its local session data, publishes its weight update to an MQTT broker, and the server averages the updates with FedAvg before sending the global model back. Two design choices carry the argument: the decision to compute the detection threshold on a single vehicle rather than per-vehicle thresholds, and a decentralized early-stopping rule based on the average validation loss across vehicles.

What would settle it

Train the same federated CANdito on the full set of CAN identifiers in the ReCAN dataset, or on a public CAN dataset from a different vehicle make, and compare detection rates against a centralized model trained on the same data; if the federated model's detection rate drops well below the centralized baseline on the new IDs, the paper's general feasibility conclusion would be disproved.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that FedAvg, the simplest federated aggregation strategy, is sufficient to train the CANdito LSTM autoencoder for CAN bus intrusion detection across up to 50 vehicles with only a slight loss in detection quality. The measured gap appears in the detection rate, while the federated model keeps a low false positive rate across the selected IDs, and it converges despite non-i.i.d. training data drawn from different driving sessions. The paper also quantifies the communication cost: each round ships a model update of roughly 373 kilobytes, and for the most decentralized 50-vehicle setting the total data exchanged per vehicle is tens to hundreds of MiB higher than a centralized upload. Given measured MQTT-over-5G latencies, the authors argue this overhead is acceptable and will only become easier with 6G. The takeaway is a feasibility result: privacy-preserving federated training does not require a fundamentally different detection algorithm or an impractical network.

Load-bearing premise

The evaluation uses only 13 CAN message identifiers taken from one vehicle's traffic and selected because earlier systems used them; if those identifiers do not represent attack-relevant signals on other vehicles or other datasets, the feasibility claim does not generalize.

Editorial extensions

If this is right

  • A vehicle manufacturer could deploy CAN intrusion detection without collecting raw fleet telemetry, since each vehicle contributes only weight updates during training.
  • The communication measurements bound the cost of federated training: roughly 373 KB per update and tens to hundreds of MiB per vehicle over the whole training run, within reach of 5G links.
  • The federated model's convergence on non-i.i.d. driving sessions means a fleet with different drivers and routes can still train a shared detector.
  • The same FedAvg-based training loop can be applied to other payload-based CAN detectors that use autoencoders, not only CANdito.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step the paper does not take is compression or quantization of the updates, since the overhead is dominated by round count and payload size; that could cut the communication cost substantially.
  • The single-vehicle threshold computation creates a single point of failure: if that vehicle is compromised, the whole fleet's detection threshold could be manipulated; a fault-tolerant or secure aggregation of thresholds would address this.
  • The paper reports that increasing the number of vehicles delays convergence, so very large fleets may need more rounds or a hierarchical aggregation scheme to keep communication manageable.
  • The privacy benefit is bounded by what can be inferred from model updates; without secure aggregation, an honest-but-curious server could still extract information from the gradients.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper proposes a federated version of CANdito, an LSTM-autoencoder-based intrusion detection system for the CAN bus, using FedAvg and FedProx with an MQTT-over-5G communication infrastructure. It evaluates detection rate and false positive rate on a subset of 13 CAN IDs from the ReCAN C-1 dataset, comparing a 50-vehicle federated training setup against centralized training of the same algorithm, and it measures the resulting communication overhead. The main claims are that federated CANdito achieves good but slightly lower detection performance than its centralized counterpart, and that the increased communication cost is sustainable given the capabilities of 5G.

Significance. If the claims hold, the paper provides a useful feasibility result: an existing payload-based CAN IDS can be trained in a federated manner with modest detection loss, addressing privacy concerns in vehicular networks. The strengths include the use of a real-world CAN dataset, a direct comparison against centralized training of the same algorithm, and actual latency measurements over a 5G network with a remote MQTT broker. The main weaknesses are the absence of quantitative performance tables, an under-specified data partitioning scheme, and a generalization claim based on a single vehicle's traffic. These issues are fixable and do not invalidate the direction of the work.

major comments (4)
  1. [§IV-A, §IV-B] The claimed non-i.i.d. federated setting is not demonstrated. Section IV-A states that the presence of data from different driving sessions makes the setting non-i.i.d., but Section IV-B and Table I only say that the training data was distributed across V vehicles in equal-sized partitions. No description is given of whether each vehicle receives a random shard of the combined sessions or a disjoint subset of sessions. If the partition is random, the clients are effectively i.i.d., which would make the FedProx comparison moot and would undermine the real-world feasibility conclusion. Please specify the exact vehicle-to-session assignment for each V and show how heterogeneity is measured.
  2. [§IV-B, Figure 3] The central comparison between federated and centralized CANdito is only presented as boxplots. No numeric DR/FPR values, standard deviations, or per-attack breakdowns are reported for the 13 IDs. The statement that the federated model is 'slightly lower' in performance than the centralized counterpart is therefore not quantitatively supported. Please provide a table with mean/median DR and FPR for each CAN ID and each attack type (injection, masquerade, drop), along with the number of runs and variance.
  3. [§IV-B, FedProx] The conclusion that FedProx provides no improvement over FedAvg is not supported by the evidence presented. The validation results for FedProx are explicitly omitted ('omitted due to space constraints'), and the text only states that increasing µ slows convergence. Since this conclusion is used to justify focusing on FedAvg for the rest of the evaluation, at least a summary table of final validation DR/FPR for all µ values should be included.
  4. [§IV-A] The representativeness of the 13 selected CAN IDs is not justified, and all experiments use a single vehicle's CAN traffic. The paper's conclusion that FL is 'well-suited for real-world scenarios' goes beyond this evidence. Please either add a justification for the selected IDs, test on additional vehicles or datasets, or soften the generalization claims.
minor comments (6)
  1. [§IV-B, Figure 2] Only CAN ID 192 is shown for FedAvg convergence; please justify that this ID is representative or show results for several IDs.
  2. [§III-A] The comparison of threshold computation configurations is not reported; the final choice of a single designated vehicle raises trust questions that should be addressed.
  3. [§IV-B] The early-stopping patience and δ settings are stated, but the 'validation tests (omitted for brevity)' are not summarized; please report at least the range tested and the final values.
  4. [§IV-B, Table II] The labels 'Subscriber' and 'Publisher' in Table II are ambiguous; clarify which side (vehicle vs. Global Server) each latency corresponds to.
  5. [References] Several references have formatting errors, e.g., 'V eh. Commun.' and 'F orensics'; please fix.
  6. [§II-C] The phrase 'shares summarized or aggregated data with the remote server' is imprecise; in FL, vehicles share model updates, not summarized data.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the federated-versus-centralized CANdito comparison is an empirical evaluation with independent content.

full rationale

This paper is an empirical study, not a mathematical derivation, and its central claim—that a federated version of the CANdito LSTM-autoencoder IDS shows slightly lower but still robust detection performance than the centralized version, at sustainable communication cost—is established by direct measurement on held-out data. The comparison is internally meaningful because both variants are trained on the same dataset and evaluated on the same injected-attack test set. The self-referential elements are experimental scaffolding rather than load-bearing premises: CANdito [1] is used as the base algorithm whose federated variant is being evaluated; the ReCAN dataset [20] is the benchmark; the 13 selected CAN IDs are chosen based on previous work [1,16]; and CANtack [1,16] is used to generate attacks. None of these choices defines the outcome in terms of the outcome: the FL detection rate, false positive rate, convergence behavior, and communication overhead are all measured, not derived from the choice of baseline. The paper also reports a negative result (FedProx's proximal term provides no improvement), which would be difficult to force by construction. The skeptical concern that the non-i.i.d. claim may not be fully supported by the data-partition description is a threat to external validity or experimental realism, not circularity: even if the federated split were effectively i.i.d., the central FL-versus-CL comparison would still be a genuine empirical result, just less representative of true heterogeneity. No fitted parameter is renamed as a prediction, no unique-solution theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation in a way that makes the conclusion equivalent to its assumptions. Under the rule that self-citation becomes circularity only when the load-bearing argument reduces to the citation itself, no such reduction is present here.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper is an empirical feasibility study. Its main claims rest on standard federated learning assumptions, the representativeness of the 13 selected CAN IDs, and several hyperparameters chosen via validation runs whose details are omitted. No new theoretical entities, forces, or conserved quantities are introduced, and no derivation is attempted.

free parameters (4)
  • number of vehicles V = 50
    Chosen as the most decentralized scenario for the main performance comparison; detection results depend on this value.
  • local epochs E = 1
    Selected for the final comparison as the most decentralized setting; convergence speed varies with E.
  • early stopping patience = 10 rounds
    Selected based on validation tests that are omitted from the paper.
  • early stopping delta = 3% of loss value
    Selected based on validation tests that are omitted from the paper; affects the number of rounds and thus communication overhead.
assumptions (4)
  • domain assumption CAN traffic is representable as a sequence of 40 consecutive packets per ID, following CANdito.
    Inherited from the CANdito paper [1]; central to the autoencoder input representation and not re-validated here.
  • domain assumption Signal definitions obtained via READ are coherent across all vehicles participating in federated learning.
    The paper explicitly states this coherence is required (Section III-A) but does not evaluate scenarios where vehicles have different signal layouts.
  • domain assumption The 13 selected CAN IDs are sufficient to judge overall IDS feasibility.
    The choice is based on prior CANdito and CANova papers, not on a comprehensive analysis of all ReCAN IDs; generalization to other IDs is untested.
  • domain assumption Averaging local validation MSE losses across vehicles is a reliable proxy for centralized early stopping.
    The federated early stopping strategy assumes this average reflects global convergence; validation details are omitted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating the Impact of Privacy-Preserving Federated Learning on CAN Intrusion Detection." pith.science (2026). https://pith.science/paper/OZSVZGXF

@misc{pith2026250604978,
  author       = {Pith},
  title        = {Pith review of: Evaluating the Impact of Privacy-Preserving Federated Learning on CAN Intrusion Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OZSVZGXF}},
  note         = {Machine review of arXiv:2506.04978}
}
read the original abstract

The challenges derived from the data-intensive nature of machine learning in conjunction with technologies that enable novel paradigms such as V2X and the potential offered by 5G communication, allow and justify the deployment of Federated Learning (FL) solutions in the vehicular intrusion detection domain. In this paper, we investigate the effects of integrating FL strategies into the machine learning-based intrusion detection process for on-board vehicular networks. Accordingly, we propose a FL implementation of a state-of-the-art Intrusion Detection System (IDS) for Controller Area Network (CAN), based on LSTM autoencoders. We thoroughly evaluate its detection efficiency and communication overhead, comparing it to a centralized version of the same algorithm, thereby presenting it as a feasible solution.

Figures

Figures reproduced from arXiv: 2506.04978 by the authors.

Figure 1
Figure 1. Overview of our system. From the left, the intrusion detection process shows the detection steps of CANdito [1], which - in the training phase - feed [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. FedAvg Detection Rate convergence results for different levels of [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Average Detection Rate on all attacks for each CAN ID and average False Positive Rate on all attacks and all CAN IDs for centralized (CL) and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 29 canonical work pages

  1. [1]

    Candito: Improving payload-based detection of attacks on controller area networks,

    S. Longari, C. A. Pozzoli, A. Nichelini, M. Carminati, and S. Zanero, “Candito: Improving payload-based detection of attacks on controller area networks,” inCyber Security, Cryptology, and Machine Learning - 7th International Symposium, CSCML 2023, Be’er Sheva, Israel, June 29-30, 2023, Proceedings(S. Dolev, E. Gudes, and P. Paillier, eds.), vol. 13914 of...

    work page 2023

  2. [2]

    Specification, “Bosch,”Robert Bosch GmbH, Postfach, vol

    C. Specification, “Bosch,”Robert Bosch GmbH, Postfach, vol. 50, p. 15, 1991

    work page 1991

  3. [3]

    Source identification using signal character- istics in controller area networks,

    P. Murvay and B. Groza, “Source identification using signal character- istics in controller area networks,”IEEE Signal Process. Lett., vol. 21, no. 4, 2014

    work page 2014

  4. [4]

    Parrot, a software-only anti-spoofing defense system for the can bus,

    T. Dagan and A. Wool, “Parrot, a software-only anti-spoofing defense system for the can bus,”ESCAR EUROPE, vol. 34, 2016

    work page 2016

  5. [5]

    Copycan: An error- handling protocol based intrusion detection system for controller area network,

    S. Longari, M. Penco, M. Carminati, and S. Zanero, “Copycan: An error- handling protocol based intrusion detection system for controller area network,” inProceedings of the ACM Workshop on Cyber-Physical Sys- tems Security & Privacy, CPS-SPC@CCS 2019, London, UK, November 11, 2019(L. Cavallaro, J. Kinder, and T. Holz, eds.), pp. 39–50, ACM, 2019

    work page 2019

  6. [6]

    Frequency-based anomaly detection for the automotive CAN bus,

    A. Taylor, N. Japkowicz, and S. Leblanc, “Frequency-based anomaly detection for the automotive CAN bus,” 2015

    work page 2015

  7. [7]

    Automotive intrusion detection based on constant CAN message frequencies across vehicle driving modes,

    C. Young, H. Olufowobi, G. Bloom, and J. Zambreno, “Automotive intrusion detection based on constant CAN message frequencies across vehicle driving modes,” 2019

    work page 2019

  8. [8]

    Graph-based intrusion detection system for controller area networks,

    R. Islam, R. U. D. Refat, S. M. Yerram, and H. Malik, “Graph-based intrusion detection system for controller area networks,”IEEE Trans. Intell. Transp. Syst., vol. 23, no. 3, pp. 1727–1736, 2022

    work page 2022

Show all 32 references
  1. [9]

    Cannolo: An anomaly detection system based on LSTM autoencoders for controller area network,

    S. Longari, D. H. N. Valcarcel, M. Zago, M. Carminati, and S. Zanero, “Cannolo: An anomaly detection system based on LSTM autoencoders for controller area network,”IEEE Trans. Netw. Serv. Manag., vol. 18, no. 2, pp. 1913–1924, 2021

    1913

  2. [10]

    In-vehicle network intrusion detection using deep convolutional neural network,

    H. M. Song, J. Woo, and H. K. Kim, “In-vehicle network intrusion detection using deep convolutional neural network,”V eh. Commun., vol. 21, 2020

    2020

  3. [11]

    Canet: An unsupervised intrusion detection system for high dimensional CAN bus data,

    M. Hanselmann, T. Strauss, K. Dormann, and H. Ulmer, “Canet: An unsupervised intrusion detection system for high dimensional CAN bus data,”IEEE Access, vol. 8, pp. 58194–58205, 2020

    2020

  4. [12]

    Intrusion detection in the automotive domain: A comprehensive review,

    B. Lampe and W. Meng, “Intrusion detection in the automotive domain: A comprehensive review,”IEEE Commun. Surv. Tutorials, vol. 25, no. 4, pp. 2356–2426, 2023

    2023

  5. [13]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” vol. 54, 2017

    2017

  6. [14]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” 2020

    2020

  7. [15]

    READ: reverse engineering of automotive data frames,

    M. Marchetti and D. Stabili, “READ: reverse engineering of automotive data frames,”IEEE Trans. Inf. F orensics Secur ., vol. 14, no. 4, 2019

    2019

  8. [16]

    Canova: A hybrid intrusion detection framework based on automatic signal classification for CAN,

    A. Nichelini, C. A. Pozzoli, S. Longari, M. Carminati, and S. Zanero, “Canova: A hybrid intrusion detection framework based on automatic signal classification for CAN,”Comput. Secur ., vol. 128, p. 103166, 2023

    2023

  9. [17]

    Eclipse mosquitto: An open source mqtt broker,

    Eclipse Mosquitto, “Eclipse mosquitto: An open source mqtt broker,”

  10. [18]

    MQTT Version 5.0

    “MQTT Version 5.0.” OASIS Standard, March 2019. Latest version available at: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html

    2019

  11. [19]

    LEAF: A benchmark for federated settings,

    S. Caldas, P. Wu, T. Li, J. Kone ˇcn´y, H. B. McMahan, V . Smith, and A. Talwalkar, “LEAF: A benchmark for federated settings,”CoRR, vol. abs/1812.01097, 2018

    2018 arXiv

  12. [20]

    Recan data - reverse engineering of controller area networks,

    Z. Mattia, L. Stefano, T. Andrea, C. Michele, G. P. Manuel, M. P. Gregorio, and S. Zanero, “Recan data - reverse engineering of controller area networks,” 2020

    2020

  13. [21]

    Practical secure aggregation for privacy-preserving machine learning,

    K. A. Bonawitz, V . Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” inProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS...

    2017

  14. [22]

    Privacy- preserving patient similarity learning in a federated environment: De- velopment and analysis,

    J. Lee, J. Sun, F. Wang, S. Wang, C.-H. Jun, and X. Jiang, “Privacy- preserving patient similarity learning in a federated environment: De- velopment and analysis,”JMIR Med Inform, vol. 6, Apr 2018

    2018

  15. [23]

    Federated learning of predictive models from federated electronic health records,

    T. S. Brisimi, R. Chen, T. Mela, A. Olshevsky, I. C. Paschalidis, and W. Shi, “Federated learning of predictive models from federated electronic health records,”Int. J. Medical Informatics, vol. 112, 2018

    2018

  16. [24]

    Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records,

    L. Huang, A. L. Shea, H. Qian, A. Masurkar, H. Deng, and D. Liu, “Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records,”J. Biomed. Informatics, vol. 99, 2019

    2019

  17. [25]

    Federated learning for the internet of things: Applications, challenges, and opportunities,

    T. Zhang, L. Gao, C. He, M. Zhang, B. Krishnamachari, and A. S. Avestimehr, “Federated learning for the internet of things: Applications, challenges, and opportunities,”IEEE Internet Things Mag., vol. 5, no. 1, 2022

    2022

  18. [26]

    A survey on federated learning for resource-constrained iot devices,

    A. Imteaj, U. Thakker, S. Wang, J. Li, and M. H. Amini, “A survey on federated learning for resource-constrained iot devices,”IEEE Internet of Things Journal, vol. 9, no. 1, 2022

    2022

  19. [27]

    Federated optimization: Distributed machine learning for on-device intelligence,

    J. Kone ˇcn´y, H. B. McMahan, D. Ramage, and P. Richt ´arik, “Federated optimization: Distributed machine learning for on-device intelligence,” CoRR, vol. abs/1610.02527, 2016

    2016 arXiv

  20. [28]

    QSGD: communication-efficient SGD via gradient quantization and encoding,

    D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. V ojnovic, “QSGD: communication-efficient SGD via gradient quantization and encoding,” inAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017,...

    2017

  21. [29]

    Federated learning in vehicular networks,

    A. M. Elbir, B. Soner, S. Coleri, D. G ¨und¨uz, and M. Bennis, “Federated learning in vehicular networks,” inIEEE International Mediterranean Conference on Communications and Networking, MeditCom 2022, Athens, Greece, September 5-8, 2022, IEEE, 2022

    2022

  22. [30]

    Imagefed: Practical privacy preserving intrusion detection system for in-vehicle can bus protocol,

    H. Taslimasa, S. Dadkhah, E. Carlos Pinto Neto, P. Xiong, S. Iqbal, S. Ray, and A. A. Ghorbani, “Imagefed: Practical privacy preserving intrusion detection system for in-vehicle can bus protocol,” in2023 IEEE 9th Intl Conference on Big Data Security on Cloud (BigDataSecurity),...

    2023

  23. [31]

    Federated graph neural network for fast anomaly detection in controller area networks,

    H. Zhang, K. Zeng, and S. Lin, “Federated graph neural network for fast anomaly detection in controller area networks,”IEEE Trans. Inf. F orensics Secur ., vol. 18, 2023

    2023

  24. [2024]

    Accessed on 17 January 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.