{"id":"942ae2a8-f0f9-42e9-9e36-66cdffedbf24","arxiv_id":"2507.21640","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"GUARD-CAN represents CAN message windows as graphs and uses an AE-GCN encoder with GRU sequence classification to detect four attack types, reporting 0.9702 accuracy and 0.9930 AUC at a 50-message window size.","lead":"GUARD-CAN turns short windows of car network (CAN) messages into graphs, then uses a graph convolution network and a gated recurrent unit to flag attacks. The paper reports high detection accuracy for flooding, fuzzing, replay, and spoofing on a public car hacking dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported AUC may be inflated by split leakage: the paper never shows that GCN pretraining data and the 6:2:2 anomaly-detection splits are temporally disjoint or that sequences do not cross split boundaries.","rationale":"The reader's weakest assumption is that the 6:2:2 split and the GCN pretraining set are temporally disjoint, so that no message used for training appears in test windows. I agree that this is the load-bearing premise, and I add one concrete mechanism the reader also implied: the overlapping sequence construction in Section 4.4 can cross split boundaries even if the packet-level split is clean, unless sequences are formed within each split. This makes the leakage risk concrete and testable. The paper's strengths are real: it uses a public dataset, reports a complete hyperparameter grid, gives an entropy-based window-size argument, and describes an architecture that does not require arbitration IDs or DBC. Those strengths do not, however, substitute for a documented split protocol. The reported AUC values are plausible for this dataset, but without split indices or a strict temporal re-run they do not yet establish that the model detects four attack types on unseen traffic. I do not see an internal inconsistency in the method; the vulnerability is entirely in the evaluation protocol. The reader's CONDITIONAL verdict is therefore appropriate and should remain unchanged, with the condition being a leakage-free, chronologically split replication.","tokens_in":8274,"tokens_out":5468,"duration_ms":74448,"concrete_test":"Ask the authors to release the exact split indices or timestamp boundaries for the GCN pretraining set and for the 6:2:2 split, and verify that (a) no timestamp in the GCN pretraining set appears in any validation or test window, and (b) no sequence used for validation or test shares a graph embedding with a training sequence. Then re-run the best configuration (window 50, sequence 50) under a strict chronological split: GCN pretrained only on the earliest normal segment, then the anomaly-detection data split into first 60% train, next 20% validation, last 20% test, with sequences formed entirely inside each split. Report per-attack AUC for flooding, fuzzing, replay, and spoofing. If the window-50/sequence-50 AUC remains near 0.993 under this protocol, the leakage concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, Section 5.3, is that GUARD-CAN reaches 0.9702 accuracy and 0.9930 AUC at window size 50 and sequence length 50 on the Car Hacking Challenge 2020 dataset. For this to be evidence of a general arbitration-ID-independent detector, the test windows must contain no messages used to train the GCN autoencoder and no messages whose context was seen by the GRU. Sections 4.3 and 5.1 do not establish this. The dataset consists of long normal segments followed by attack bursts; if the 6:2:2 split is applied after shuffling packets or windows, repeated flooding/replay payloads appear in both training and test data. Because the representation is a bag-of-binarized bytes aggregated over a path graph, memorizing frequent byte patterns can trivially inflate AUC. A second, separable leak is the sliding-window sequence construction: Section 4.4 forms overlapping sequences, but the paper never states that sequences are generated separately within train/validation/test or that windows do not straddle split boundaries. A test sequence overlapping a training window would feed the GRU embeddings it was trained on. The separate GCN pretraining set of 179,346 normal records is not shown to be timestamp-disjoint from the 1,799,046 normal records used for anomaly detection, and no split indices or timestamp ranges are provided. The concern is not that the architecture is unsound; it is that the reported numbers may reflect split leakage rather than generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"GUARD-CAN is a CAN-bus anomaly detection framework that works directly from DLC and payload bytes without arbitration IDs or DBC files. The method: (1) pads and splits payload bytes into eight columns; (2) builds non-overlapping packet windows and represents each window as a timestamp-ordered path graph; (3) trains an overcomplete autoencoder plus three-layer GCN on normal graphs to produce 32-dimensional graph embeddings via global mean pooling; (4) forms overlapping sequences of embeddings and feeds them to a two-layer GRU with binary cross-entropy loss; and (5) scores anomalies at sequence level and at window level by mean/max aggregation of overlapping sequence scores. Experiments on the Car Hacking Challenge 2020 dataset report a best sequence-level accuracy of 0.9702 and AUC of 0.9930 at window size 50 and sequence length 50, with additional results for other window sizes and sequence lengths, and an entropy analysis intended to justify window size selection.","tokens_in":8552,"tokens_out":7943,"duration_ms":87882,"significance":"The architecture is genuinely distinctive: it is arbitration-ID-independent, avoids DBC parsing, combines graph-structural and recurrent-temporal modeling, and evaluates detection at two granularities. The use of an external dataset and explicit descriptions of the GCN/AE/GRU configuration are strengths. However, the empirical support is incomplete: no temporal-separation protocol is described, no baseline or variance estimate is reported, and the headline number is a selected maximum over 75 configurations. If the split-leakage issues are resolved and the method is compared with at least one prior CAN IDS, the paper would make a useful contribution to intrusion detection for in-vehicle networks.","major_comments":[{"comment":"The experimental protocol does not establish temporal disjointness between the data used to train GCN and the train/validation/test splits for the GRU. Section 5.1 only says the anomaly-detection dataset was split into training, validation, and test sets in a 6:2:2 ratio, and Section 4.3 says GCN uses only normal data; the paper does not state whether the split is random or by contiguous time ranges, whether the 179,346 normal records in Table 1 are disjoint from the 1,799,046 normal records in the anomaly-detection set, or whether windows and sequences are generated separately within each split. If the split is random or if the overlapping sequences from Section 4.4 cross split boundaries, the reported AUC can be inflated by repeated normal byte patterns and by fragments of the same attack burst appearing in both training and test sets. Please provide an explicit split description (e.g., timestamp ranges or indices) and rerun the evaluation with strict temporal separation, generating windows and sequences independently within train, validation, and test.","section":"Section 5.1, Section 4.3, Section 4.4"},{"comment":"The headline result is selected as the best among five window sizes, five sequence lengths, and three scoring variants, yet no validation-based selection rule, no run-to-run variance, and no comparison with existing CAN IDSs are reported. Because AUC values from a single split can vary substantially with random initialization and because the 75-configuration search makes the reported maximum optimistic, the absolute numbers do not by themselves support the claim that GUARD-CAN detects attacks 'effectively.' At minimum, report mean and standard deviation over multiple training seeds for the chosen configuration on a fixed test set, and include at least one baseline method (e.g., a frequency/statistical detector or a published deep learning IDS) evaluated on the same chronological split.","section":"Section 5.3, Table 2"},{"comment":"The entropy analysis is described in terms of how evenly arbitration IDs appear within a window ('when the IDs appear more evenly...'), but the model deliberately does not use arbitration IDs as features. Please state exactly what quantity the entropy is computed over; if it is ID entropy, the analysis is not a valid justification for the window sizes chosen for a model whose node features are DLC and payload bytes. If it is byte-level or graph-level entropy, the text and figure description should be corrected accordingly.","section":"Section 5.2, Figure 2"}],"minor_comments":[{"comment":"The binarization step is underspecified. Section 4.1 says byte values are normalized to [0,1], while Section 5.3 refers to 'binarized byte values (integer values of 0 or 1)'. Please define the exact mapping used to obtain the 9-dimensional node features, because the comparison between Tables 2 and 3 depends on it.","section":"Section 4.3, Section 4.1"},{"comment":"The sequence indexing is imprecise: for sequence length 3, the last sequence should start at h_{m-2}, and the subscript n is not defined. Rewrite the formula with an explicit range such as i=0,...,m-L.","section":"Section 4.4"},{"comment":"The x-axis is labeled only 50, 100, 150, but the experiment uses sequence lengths 30, 50, 100, 120, 150; add all tick values or clarify the plot.","section":"Figure 3"},{"comment":"The paper should explain the provenance and relationship of the 179,346-record GCN training set to the 1,799,046-record anomaly-detection normal set (e.g., different sessions, different files, or a subset). Currently this distinction is unclear.","section":"Table 1"},{"comment":"There are several typos and formatting issues, including 'T able 1' and 'T able 2' captions, 'GUARD-' spacing in the abstract, and inconsistent use of 'binarized' vs. 'normalized' byte features. These should be cleaned up.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"The split-leakage issue is the key gate. If the authors can demonstrate strict temporal separation and add a baseline comparison, the paper could become acceptable; the architecture is credible and the self-citation pattern is not problematic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is genuinely useful. GUARD-CAN encodes fixed-size windows of CAN traffic as chain graphs over payload bytes and DLC, embeds them with an AE-pretrained GCN, then feeds sequences of embeddings to a GRU. That combination is not in the cited literature, and dropping arbitration IDs means no DBC file is needed. If it generalizes, that's a real practical win for automotive security monitoring.\n\nThe evaluation is where I part ways with the authors' confidence. The headline number (0.9702 accuracy, 0.9930 AUC) is selected from 5 window sizes × 5 sequence lengths × 3 scoring variants, with no model-selection protocol and no variance estimate across splits. Worse, there is no baseline comparison at all, so it's hard to say whether the GRU over graph embeddings beats a plain MLP on the same features or even a simple frequency-based detector. The reader's stress-test about split leakage is also fair: the paper never says the GCN pretraining set (179,346 normal records) is disjoint from the anomaly-detection set (1,799,046 normal records), nor whether windows and sequences are constructed separately within train/validation/test. On a dataset of long normal segments followed by attack bursts, overlapping windows can easily leak context into test sequences. These are not fatal design flaws—the architecture is coherent and the losses are standard—but the empirical claims are weaker than the abstract implies.\n\nCredit where due: the entropy analysis for window sizing is a nice touch, the two-level detection (sequence/window) is a thoughtful evaluation angle, and the paper is transparent about its hyperparameters. The related work is fine, a bit heavy on the authors' own group but not egregiously so. No circular derivation.\n\nThis paper is for people working on CAN intrusion detection, especially those who want a DBC-free detector. It deserves a serious referee, but the referee should demand a proper evaluation protocol: temporal split documentation, multiple runs with variance, baselines, and ideally a cross-dataset check. I would conditionally accept after major revision.","headline":"Useful architecture, weak evaluation: the GCN+GRU combination is novel, but the reported AUC is likely inflated by unspecified split procedures and hyperparameter selection.","tokens_in":9091,"tokens_out":2015,"would_cite":false,"duration_ms":24452,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph-plus-recurrent model spots four types of CAN bus attacks with 0.993 AUC, using only packet payload and length.","keywords":["Controller Area Network","CAN bus intrusion detection","Graph Convolutional Network","Gated Recurrent Unit","anomaly detection","in-vehicle network security","Car Hacking Challenge dataset","graph embedding"],"falsifier":"Re-run the experiment with a strictly temporal split (for example, first 60% of each attack session for training, last 20% for test, forbidding any window that straddles the boundary) and compare AUC; if the 0.993 AUC drops substantially, the original result relied on temporal leakage.","tokens_in":8024,"feed_emoji":"🚗","tokens_out":4049,"duration_ms":40635,"temperature":0.7,"pith_summary":"This paper tries to show that a CAN bus anomaly detector can work without the database of arbitration IDs (DBC) that most intrusion detection systems treat as essential, and without hand-built features. It does this by splitting raw CAN traffic into fixed-size windows, turning each window into an order-preserving graph whose node features are only the packet's Data Length Code and eight payload bytes, and learning graph embeddings with an overcomplete autoencoder paired with a Graph Convolutional Network. The embeddings are fed in sliding-window sequences to a Gated Recurrent Unit, which labels both sequences and individual windows as normal or anomalous. On the Car Hacking Challenge 2020 dataset the model reports 0.9702 accuracy and 0.9930 AUC at a window size of 50 and sequence length of 50, and the authors argue from Shannon-entropy analysis that small windows give the graph enough structure without diluting the signal. If true, the contribution is a practically deployable in-vehicle intrusion detection system that attaches to raw CAN traffic without private vehicle specifications.","feed_headline":"Graph+GRU detector hits 0.993 AUC on CAN attacks with no DBC","feed_subtitle":"It needs only packet length and payload bytes to spot flooding, fuzzing, replay, and spoofing attacks.","key_machinery":"The load-bearing mechanism is the ordered graph built from each CAN message window, combined with an overcomplete autoencoder and GCN that produce a 32-dimensional embedding per window, and a GRU that reads these embeddings as sequences. The graph edges preserve message arrival order so GCN can capture temporal structure without explicit timing features; the GRU then captures dependencies across windows. The overcomplete AE (9-dimensional input expanded to 16 dimensions) stabilizes GCN training by reconstructing the original node features.","core_discovery":"The central claim is that time-aware and structure-aware anomaly detection for CAN can be combined in a single pipeline that uses only the DLC and Data field of each packet. GUARD-CAN constructs one graph per fixed-size window by connecting consecutive packets in timestamp order; node features are the normalized DLC and the binarized payload bytes. An overcomplete autoencoder pre-processes these features, a three-layer GCN produces a 32-dimensional graph embedding via global mean pooling, and a two-layer GRU over sliding windows of embeddings outputs anomaly probabilities. The paper reports the best results at window size 50 and sequence length 50, with sequence-level detection achieving 0.9702 accuracy and 0.9930 AUC, and finds that these degrade as window size grows. The paper also claims that binarizing payload bytes outperforms normalized float bytes, because dense value distributions inflate the autoencoder's reconstruction error.","pith_inferences":["The paper does not state whether the 6:2:2 split is temporal; if training windows overlap test windows in time, the AUC partly reflects memorized normal patterns, so reproducing the experiment with a strictly temporal split is the natural next check.","Because a window is labeled anomalous if any packet in it is an attack, sparse attacks create mostly-normal positive windows; the model may be learning attack co-occurrence across a window rather than per-packet maliciousness.","The Shannon-entropy argument could be turned into a design rule: choose the smallest window size whose entropy growth rate has plateaued, which here points to 50; this is testable on other CAN datasets.","Applying the same graph-plus-GRU pipeline to other in-vehicle protocols (CAN-FD, LIN) or to synthetically replayed real-vehicle traces would test whether the DBC-free design transfers beyond the Challenge dataset."],"forward_implications":["An intrusion detection system for CAN can be deployed without a DBC, since only DLC and payload bytes are used; this removes the need for manufacturer-specific arbitration ID maps.","Window size around 50 and sequence length around 50 give the best accuracy and AUC, so a lightweight temporal context is sufficient for these attack types.","Binarized payload bytes are preferable to normalized floats, suggesting the detector keys on the presence or absence of byte patterns rather than their magnitudes.","Both sequence-level and window-level (mean and max) outputs are usable as detection decisions, giving the operator a choice of granularity."],"supporting_citations":[{"why":"Supplies the Car Hacking Challenge 2020 dataset used for all training and evaluation, including the four attack types.","marker":"[5]"},{"why":"The IEEE Dataport release of the same Car Hacking Challenge dataset, the reproducibility source for the data.","marker":"[13]"},{"why":"Defines the GCN convolution used in the encoder, the core graph-learning machinery.","marker":"[6]"},{"why":"The GCN-based CAN IDS baseline that the paper contrasts with, the one that uses arbitration IDs and lacks temporal sequence modeling.","marker":"[12]"},{"why":"The deep CNN CAN IDS baseline that motivates the claim of high performance without complex feature engineering.","marker":"[9]"}],"fun_headline_variants":["GUARD-CAN: Graph+GRU catches CAN attacks at 0.993 AUC","Graph-aware autoencoder + GRU beats four CAN attacks","Only DLC and payload bytes: GUARD-CAN detects CAN attacks","Windowed graph GRU achieves 0.993 AUC on CAN anomaly"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model's reported performance rests on the assumption that the 6:2:2 split is temporally disjoint, so no message used to train the GCN or GRU appears in the windows being tested.","fun_headline_variants_meta":{"raw":{"variants":["GUARD-CAN: Graph+GRU catches CAN attacks at 0.993 AUC","Graph-aware autoencoder + GRU beats four CAN attacks","Only DLC and payload bytes: GUARD-CAN detects CAN attacks","Windowed graph GRU achieves 0.993 AUC on CAN anomaly"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":3130,"prompt_tokens":953,"completion_tokens":2177,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":2097}},"tokens_in":569,"tokens_out":2177,"duration_ms":17891,"temperature":1.0,"reasoning_tokens":2097,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:31:36.942598+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the experiment with a strictly temporal split (for example, first 60% of each attack session for training, last 20% for test, forbidding any window that straddles the boundary) and compare AUC; if the 0.993 AUC drops substantially, the original result relied on temporal leakage.","supporting_citations":[{"cited_title":"Car hacking and defense competition on in-vehicle network","cited_arxiv_id":null,"evidence_quote":"Supplies the Car Hacking Challenge 2020 dataset used for all training and evaluation, including the four attack types."},{"cited_title":"Car hacking: Attack & defense challenge 2020 dataset.https: //dx.doi.org/10.21227/qvr7-n418","cited_arxiv_id":null,"evidence_quote":"The IEEE Dataport release of the same Car Hacking Challenge dataset, the reproducibility source for the data."},{"cited_title":"GCNIDS: Graph Convolutional Network-Based Intrusion Detection System for CAN Bus","cited_arxiv_id":"2309.10173","evidence_quote":"The GCN-based CAN IDS baseline that the paper contrasts with, the one that uses arbitration IDs and lacks temporal sequence modeling."},{"cited_title":"In-vehicle network intrusion detection using deep convolutional neural network","cited_arxiv_id":null,"evidence_quote":"The deep CNN CAN IDS baseline that motivates the claim of high performance without complex feature engineering."}],"review_version":1}