{"id":"c395be93-c609-4caf-91b5-67efbd0a4f62","arxiv_id":"2508.09060","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"A federated CNN with a two-step preprocessing stage and Block-Based Smart Aggregation achieves superior transferability and local detection rates for network intrusion detection.","lead":"Network intrusion detection systems need to catch attacks they have never seen before. This paper proposes a federated deep learning system that shares knowledge across devices to improve that transfer, and it reports strong results with code available online.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Transferability metric may be the optimized objective itself; abstract gives no evidence that evaluation uses held-out attack types, and the corrupted full text prevents checking.","rationale":"The reader's verdict is UNVERDICTED, and I see no basis to change that. The full text is unreadable mojibake, so no internal consistency check on equations or experiments is possible. The most load-bearing scientific concern is the one the reader identified: the abstract's 'transferability relationships' are simultaneously the optimization target and the evidence for generalization. If the metric is defined on the same data used to train the preprocessing and aggregation, then superior numbers on that metric do not establish detection of unknown attacks. This is a concrete, testable concern. I agree with the reader's weakest assumption. The proposed test—leave-one-attack-class-out evaluation using the provided code—would settle whether the concern lands. Since the evidence is currently inaccessible, the appropriate verdict remains UNVERDICTED. No ad hominem is intended; the issue is the relationship between the optimization objective and the evaluation target.","tokens_in":8110,"tokens_out":3057,"duration_ms":36862,"concrete_test":"Recover the readable full text (or download the GitHub repository ghosh64/tabfidsv2), locate the evaluation protocol, and check whether transferability is computed on attack types/classes excluded from both training and BBSA aggregation. Concretely, run the provided preprocessing and BBSA on CICIDS2017/UNSW-NB15 with a leave-one-attack-class-out split; if the reported detection rates on the held-out class are near random, the claimed transferability is in-sample. Alternatively, compare the 'transferability relationships' metric against a simple held-out attack-class F1 score.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the system reliably detects unknown attacks. The abstract states that the method 'maximize[s] the number of transferability relationships' via two-step preprocessing and BBSA, and reports 'superior transferability performance.' The load-bearing assumption is that maximizing these transferability relationships is a faithful surrogate for detecting attacks not seen in training. If 'transferability relationships' are merely the optimization target—e.g., measured as agreement across clients or datasets on the same known attack classes—then the reported transferability may be an in-sample artifact of fitting that target, not evidence of zero-day generalization. No held-out attack-class evaluation protocol is described in the abstract. The provided full text is corrupted Unicode, so the methods and evaluation sections cannot be inspected; the GitHub repository is the only accessible route for verification. The concern is not that the authors are wrong, but that the paper does not define 'transferability relationships' or show that the reported metric corresponds to detection of novel attack types.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a federated network intrusion detection system based on a CNN with two-stage data preprocessing and a Block-Based Smart Aggregation (BBSA) algorithm. The stated goal is to improve detection of unknown attacks by maximizing the number of 'transferability relationships.' The abstract claims superior transferability performance, strong local detection rates, and generalization across datasets and backbone architectures, with code available on GitHub. However, the body of the manuscript as provided is corrupted (mojibake) and unreadable, and the abstract itself contains no quantitative results, experimental protocol, or definitions of the key metrics. Thus the central claims cannot currently be verified from the manuscript.","tokens_in":8344,"tokens_out":3814,"duration_ms":47344,"significance":"If substantiated, the idea of explicitly encouraging transferability in a federated intrusion detection system would be a useful contribution to network security, particularly for zero-day/unknown attack detection. The authors' decision to release code is a positive step toward reproducibility. Nevertheless, the significance cannot be assessed from the present submission: the abstract states results without numbers, the evaluation methodology is undefined, and the full text is not legible. The potential circularity in using the number of 'transferability relationships' as both the optimization target and the evaluation criterion is a serious concern that must be addressed.","major_comments":[{"comment":"The central claim of 'superior transferability performance' is stated without any quantitative support: no detection rates, no baselines, no standard deviations, and no dataset names are reported. The abstract also does not define the transferability metric. As written, the main claim is unfalsifiable.","section":"Abstract"},{"comment":"These algorithms are explicitly designed to 'maximize the number of transferability relationships.' If the evaluation counts the same relationships, then 'superior transferability' may be a direct consequence of the optimization objective rather than an independent property of unknown-attack generalization. The paper must specify whether evaluation is performed on held-out attack types/relationships and must report a metric separate from the training objective.","section":"Methods: two-step pre-processing and BBSA (described in the Abstract)"},{"comment":"The body of the manuscript is unreadable in the submitted copy (corrupted Unicode/mojibake). Consequently, the experimental setup, data preprocessing, federated partitioning, baseline comparisons, and aggregation details cannot be verified. A GitHub repository is not a substitute for a self-contained paper; the manuscript itself must present all necessary methodology and results.","section":"Full text (all sections after the Abstract)"},{"comment":"The assertion that the method is 'generalizable, exhibiting transferability potential across datasets and even with different backbones' is not accompanied by any experiment description, table, or statistical test. At minimum, the paper must specify the datasets and backbone architectures used and report per-experiment results, including detection rates and transferability metrics.","section":"Abstract, generalization claim"}],"minor_comments":[{"comment":"The phrase 'deployed in a distributed setup across devices connected to a network' is vague; specify the federated topology, number of clients, and communication protocol.","section":"Abstract"},{"comment":"The GitHub repository URL should be formatted as a clickable reference with a version/commit number; otherwise the code cannot be uniquely identified for reproducibility.","section":"General"},{"comment":"The paper would benefit from a formal list of contributions and a clear evaluation protocol (datasets, train/test splits, held-out attack classes, baseline methods, and evaluation metrics).","section":"General"}],"recommendation":"reject","confidential_remarks":"The submitted full text is corrupted to the point of being unreadable; the editor may wish to check the source file. The most substantive issue is the possible circularity: if 'transferability relationships' are both the optimized objective and the evaluation metric, the reported superiority is not evidence of unknown-attack detection. The current manuscript cannot be accepted, but the authors could resubmit after rewriting with a readable text and a rigorous held-out evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a federated IDS paper whose abstract claims superior transferability via a CNN plus two-step preprocessing and a Block-Based Smart Aggregation algorithm. The stated aim—using known attacks to detect unknown ones—is practical, and the authors ship a GitHub link, which counts for something. But the abstract alone reports no numbers and no evaluation protocol, and the full text we received is mojibake, so nobody can actually check the methods or results. That limits everything below.\n\nWhat looks genuinely new: the specific combination, and the explicit attempt to maximize the number of transferability relationships rather than just local accuracy. That is a reasonable design goal for federated IDS, where clients see different attack distributions. The cross-dataset and cross-backbone claims, if supported, would matter for the subfield.\n\nThe soft spot, and it's a real one, is the circularity risk. \"Transferability relationships\" are both the optimization target and the claimed outcome. If the evaluation measures the same relationships the algorithm maximizes—for example, agreement on known classes across clients—then \"superior transferability\" could be an artifact of fitting the target, not evidence of zero-day generalization. The abstract doesn't say whether evaluation uses held-out attack types or held-out relationships. That is exactly the experiment that would make the claim convincing. I want to be clear: I am not saying the authors are wrong. There's only no evidence in front of us, and the corrupted text means we can't fill that gap.\n\nThe code is a positive. It makes the work independently checkable, which is more than many papers can say.\n\nWho is it for: people working on federated learning for intrusion detection, and anyone designing transfer or zero-day evaluation protocols. I would send it to a serious referee because the topic is real, the code is public, and the circularity concern is resolvable with a clear look at the evaluation section. But the referee needs to demand the held-out attack-type protocol and actual numbers before this becomes citable.\n\nFor us: not something I'd cite yet. Maybe worth a reading group discussion about what counts as transferability evaluation.","headline":"Plausible federated IDS design with a real circularity risk, but the unreadable full text means the central transferability claim is unverifiable from this submission.","tokens_in":8763,"tokens_out":1780,"would_cite":false,"duration_ms":18857,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that maximizing transferability relationships during preprocessing and federated aggregation lets a CNN-based intrusion detector recognize unknown attacks while keeping local detection rates high.","keywords":["intrusion detection","federated learning","transferability","unknown attacks","convolutional neural network","block-based aggregation","network security","data preprocessing"],"falsifier":"Train the system on a standard IDS dataset with one entire attack class withheld, meaning no samples from that class appear during training or validation, then measure detection on that withheld class. If the transferability-aware preprocessing and BBSA aggregation do not beat a plain federated CNN with naive averaging on the withheld class, the central claim is falsified. The withheld class must not influence any design choice in preprocessing or aggregation.","tokens_in":8055,"feed_emoji":"🛡️","tokens_out":9406,"duration_ms":97605,"temperature":0.7,"pith_summary":"Network intrusion detection must handle attacks that no one has labeled yet, and doing this across many connected devices means no single site holds all traffic. This paper claims that a federated—distributed, update-sharing—CNN-based detector can learn from known attacks in a way that transfers to unknown ones, by increasing the number of 'transferability relationships' the training data exposes. The proposed method combines a two-step preprocessing stage with a Block-Based Smart Aggregation (BBSA) algorithm, and reports that this pairing improves detection of unseen attacks without sacrificing each device's detection of local known attacks. If the claim holds, intrusion detection models could be trained across real network deployments and then recognize new attack families without retraining on centralized data, and the same recipe would carry across datasets and backbone architectures.","feed_headline":"Transferability relationships let federated IDS spot unknown attacks","feed_subtitle":"Preprocessing plus block-wise aggregation carries knowledge from known to unseen attacks without centralizing data.","key_machinery":"The load-bearing object is the transferability relationship—a regularity in feature space that lets a pattern learned on one attack type also separate an unseen attack type. The two-step preprocessing maximizes how many such relationships the training data exposes; BBSA then combines local model blocks selectively, preserving blocks that carry these relationships and limiting the influence of blocks fitted to local quirks. Together they turn a standard CNN for tabular network flows into a detector whose useful knowledge is not tied to the specific attack labels it saw.","core_discovery":"The central claim is that transferability can be engineered rather than hoped for. The paper defines transferability relationships as regularities between known and unknown attack classes that a model can exploit, then builds two mechanisms to maximize how many such relationships appear in training. The first is a two-step data preprocessing stage that reorganizes or selects features so these relationships are more visible. The second is BBSA, which aggregates local models block by block rather than wholesale, keeping the parts of each client model that preserve transferability relationships while limiting the influence of parts that only fit local quirks. In the paper's telling, this is why","pith_inferences":["If the transferability-relationship count is the active ingredient, then detection of unseen attacks should track that count even when the CNN is swapped for a simpler classifier; a reader could test this by varying only the preprocessing step.","The block-wise aggregation idea is not specific to intrusion detection—it could apply to any federated task with skewed class distributions, though the paper does not test that.","The method's reach is bounded by the flow-feature representation: attacks that avoid changing those statistics, such as encrypted or low-volume stealth traffic, may not share transferability relationships with known classes.","Because the paper reports transferability across datasets and backbones, the implication left unstated is that the preprocessing and aggregation, rather than the CNN itself, do most of the generalizing; an ablation replacing the CNN with a logistic regression would make this explicit."],"forward_implications":["A federated IDS built this way should flag attack families that were absent from all clients' training labels.","Clients keep strong local detection, so adding transferability does not force a trade-off with known-attack protection.","The pipeline is portable: reported transferability holds when moving to another dataset and when swapping in a different backbone model.","Raw network traffic can stay on each device; only model updates are shared, which is the privacy motivation for federated intrusion detection.","The two-step preprocessing and BBSA are modular additions to existing federated training, so they can be combined with other local IDS models."],"supporting_citations":[],"fun_headline_variants":["Engineered transferability helps federated IDS catch unknown attacks","Block-wise aggregation boosts federated IDS for unseen threats","Smart preprocessing maximizes transferability in federated IDS","Federated IDS learns to generalize from known to unknown attacks","Boosting transferability relationships for federated intrusion detection"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The pipeline assumes that the number of transferability relationships exposed by preprocessing is a faithful measure of—and a reliable lever for—a model's ability to detect attack types that never appear in training.","fun_headline_variants_meta":{"raw":{"variants":["Engineered transferability helps federated IDS catch unknown attacks","Block-wise aggregation boosts federated IDS for unseen threats","Smart preprocessing maximizes transferability in federated IDS","Federated IDS learns to generalize from known to unknown attacks","Boosting transferability relationships for federated intrusion detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1099,"prompt_tokens":669,"completion_tokens":430,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":413,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":413,"tokens_out":430,"duration_ms":4654,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:13:19.139246+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the system on a standard IDS dataset with one entire attack class withheld, meaning no samples from that class appear during training or validation, then measure detection on that withheld class. If the transferability-aware preprocessing and BBSA aggregation do not beat a plain federated CNN with naive averaging on the withheld class, the central claim is falsified. The withheld class must not influence any design choice in preprocessing or aggregation.","supporting_citations":[],"review_version":1}