{"id":"436a8e47-cfc4-488b-add5-2db2660cc64a","arxiv_id":"2504.17059","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The authors introduce an enriched NSL-KDD dataset that appends synthetic IP addresses and network centrality metrics, and they evaluate it with IBM Auto AI for classification.","lead":"This paper creates a version of the NSL-KDD cyberattack dataset with added synthetic IP addresses and graph centrality features for teaching graph-based security analysis. A generalist might read it to see whether graph features on a synthetic network add practical value to an outdated intrusion detection benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The appended graph features are derived from random IP addresses and an arbitrary edge probability of 0.2, so the claim that centrality measures improve classification is unsupported; a permutation test against the original NSL-KDD features would settle it.","rationale":"The reader's weakest assumption identifies exactly the arbitrary random graph construction in Section V.D, and the same concern is the most load-bearing one for the paper's central claim. The dataset artifact may still have educational value as a teaching tool, and the GitHub code is a positive step toward reproducibility, but the validation reported in Figures 6-7 is not sufficient to show that the graph features add predictive signal. A permutation-style comparison is a concrete, low-cost check that would settle the issue: if graph features confer no held-out improvement over the original features and over permuted graph features, the headline claim about Auto AI performance must be revised to a claim about the original NSL-KDD features alone. This does not change the reader's CONDITIONAL verdict, since the paper can be repaired with a proper benchmark and a clearer description of the graph construction; it does mean the classification performance claim should not be accepted as stated until that check is run.","tokens_in":5851,"tokens_out":4224,"duration_ms":41311,"concrete_test":"Run a stratified 5-fold cross-validation on the enriched dataset comparing three inputs: (a) the original 41 NSL-KDD features, (b) the original features plus the appended graph features as generated, and (c) the original features plus a row-wise permuted copy of the graph features (breaking any link between graph features and labels). If (b) does not clearly outperform (a) and (c) on held-out AUC/F1, the claim that centrality measures contribute signal is falsified. The same script should also record the Auto AI configuration and random seed so the result is reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section V.D constructs the graph from randomly generated IPv4 addresses with edge probability 0.2, then appends the resulting IPs and centrality values to NSL-KDD, which originally has no source/destination IP fields. Because node identities are random and edges are independent Bernoulli draws, the graph features are, conditional on the original 41 features, random noise relative to the attack labels. The paper's conclusion that 'centrality measures provide meaningful contributions to model performance' (Section V.D) therefore does not follow from the reported Auto AI experiment: Figures 6-7 show no baseline, no train/test split, and no quantitative metric, so 'no errors' is consistent with pure-noise features being selected by SelectKBest and with the classifier relying entirely on the original NSL-KDD attributes. This is the load-bearing weak point because both the classification claim and the educational value of the enrichment depend on the graph features carrying real structure; an arbitrary random graph cannot establish that.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes the construction of an enriched version of the NSL-KDD dataset for cybersecurity education. The enrichment appends synthetic source and destination IP addresses, centrality measures (degree, betweenness, closeness, PageRank), and community labels, where the underlying graph is generated with NetworkX using random IPv4 addresses and an edge probability of 0.2. The authors then apply IBM Auto AI to the enriched dataset and report 'high prediction performance' with 'no errors,' and they conclude that graph-based features contribute meaningfully to attack classification. The study follows a Design Science Research methodology and makes the enrichment code available on GitHub.","tokens_in":6061,"tokens_out":2224,"duration_ms":23271,"significance":"If the central claim were adequately supported, the enriched dataset could be a useful pedagogical resource for teaching graph-based intrusion detection. The paper has clear strengths: it identifies a real gap in educational datasets, provides a concrete enrichment pipeline, and releases the code on GitHub. However, the validation is currently not sufficient to establish that the graph features carry genuine signal. The classification experiment lacks a baseline, a held-out test split, and quantitative metrics, and the graph itself is generated from random IP addresses with an arbitrary edge probability, making the appended features, conditional on the original 41 NSL-KDD features, statistically independent of the attack labels. The educational value of the dataset is plausible but is not demonstrated by the reported evaluation.","major_comments":[{"comment":"The claim that 'centrality measures provide meaningful contributions to model performance' is not supported by the Auto AI experiment because no baseline is reported, no train/test split is described, and no quantitative metric (accuracy, F1, precision, recall, or AUC) is given. A result of 'no errors' is consistent with overfitting to the training data or with the classifier relying entirely on the original NSL-KDD features. The authors should compare the original 41-feature NSL-KDD set against the enriched set using identical cross-validation or a held-out test split and report standard classification metrics.","section":"Section V.D, Figures 6-7"},{"comment":"The graph features are derived from randomly generated IPv4 addresses with an edge generation probability of 0.2, and no random seed or complete generation procedure is provided, so the experiment is not reproducible. More importantly, because NSL-KDD does not contain real source and destination IP fields, the appended node identities and edge structure are unrelated to the actual network flows behind the dataset; conditional on the original 41 features, the centrality and community features are random noise relative to the attack labels. To establish that graph-based features carry real signal, the authors should use a dataset with genuine network structure (e.g., flow data with real IPs) or run a permutation test in which the graph features are randomly shuffled and the classification performance is compared with the unshuffled version.","section":"Section V.D, graph construction"},{"comment":"There is an internal inconsistency in the privacy discussion: Section V.C describes a pseudonymization technique that replaces 'each authentic IP address' with a pseudonym, but Section V.D and the description of NSL-KDD state that the dataset does not include source and destination IP features. Since the IPs are synthesized rather than pseudonymized from real values, the privacy-protection framing is misleading and should be corrected. The authors should clarify exactly which fields were generated, which were replaced, and how reproducibility is ensured (including the random seed and the exact NetworkX call).","section":"Sections V.C and V.D"},{"comment":"The statement that 'out degree centrality and betweenness centrality ranked within the top 50 features' is reported without showing the feature-selection results, the number of features evaluated, the selection criterion, or the variance of the ranking across random graph instantiations. Because the graph is random, the ranking could change substantially with different seeds. The authors should report the full feature ranking or at least the stability of the ranking over multiple random seeds, and should provide a statistical test of whether the graph features add predictive value beyond the original NSL-KDD attributes.","section":"Section V.D, SelectKBest"}],"minor_comments":[{"comment":"There are several typographical errors and informal expressions that should be corrected, including 'bench mark', 'Opens Systesm Interconnection', 'modeloud', and 'jupyter-lab' in Section V.D.","section":"Throughout"},{"comment":"The sentence beginning 'Next we developed an auto AI modeloud solution [19]' is unclear; it should state which IBM Auto AI service or tool was used, how the data were uploaded, and what the exact experimental configuration was.","section":"Section V.D"},{"comment":"The figures showing degree centrality histogram, directed graph view, communities view, and bipartite view would be more informative if they included axis labels, a legend, and a description of the node and edge counts; currently they are difficult to interpret.","section":"Figures 1-4"},{"comment":"The claim that NSL-KDD is 'widely adopted' is supported by GitHub and Kaggle counts, but no citation is given for these counts; consider adding a reference or a link to the data source.","section":"Section V.B"}],"recommendation":"major_revision","confidential_remarks":"The paper's stated contribution is educational, but the evaluation is framed as a classification-accuracy claim. The authors should either substantially strengthen the empirical validation (baseline, held-out split, quantitative metrics, permutation test, reproducibility) or reframe the contribution as a demonstration artifact without claiming predictive improvement. The current evidence does not support the 'no errors' and 'meaningful contributions' statements. The scope of the journal may also warrant a more explicit discussion of how the enriched dataset has been or will be used in actual courses."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the artifact—a version of NSL-KDD with appended IPs and centrality features—is a reasonable idea for a teaching dataset, but the paper's classification performance claim is not backed by the reported experiment. The graph features are built from randomly generated IPv4 addresses with an arbitrary 0.2 edge probability, so they are, as far as the paper describes, random noise relative to the attack labels. The 'no errors' result from IBM Auto AI means little without a baseline, a held-out test set, or a numeric metric.\n\nWhat is genuinely new and useful: a public GitHub repository with an enriched NSL-KDD dataset that lets students compute degree, betweenness, and closeness centralities, run Louvain, and inspect communities. That fills a small gap in teaching material for graph-based cybersecurity. The literature review is adequate and correctly identifies that most graph-based intrusion detection work uses datasets that already contain network topology, while NSL-KDD does not.\n\nWhere it falls down: the evaluation section. The Auto AI experiment appears to report only screenshots; there is no description of train/test splitting, no comparison against the original NSL-KDD features, and no error metric beyond 'no errors.' Because the synthetic IPs are assigned to records randomly, the centrality values are independent of the class label conditional on the original 41 features. Any selected centrality feature is likely selected by chance. The paper says SelectKBest ranked out-degree and betweenness in the top 50, but without a permutation test that is meaningless. There is also no code or seed given to reproduce the graph generation, and the privacy discussion about pseudonymizing IPs is confusing because NSL-KDD has no source or destination IP fields; there is nothing to pseudonymize.\n\nThe educational angle survives, barely: students can use the dataset to learn how to apply graph algorithms, even if the underlying graph is synthetic. But the paper would need to state honestly that the graph is a random construction for teaching, not a model of the actual traffic in NSL-KDD.\n\nRecommendation: send it to peer review, but the referee should require a proper experimental comparison (original vs enriched features, fixed split, multiple runs with a seeded graph) and a rewrite of the claims. Without that, the performance section should be removed. The dataset may be a useful teaching aid, but as written the evidence for 'centrality measures provide meaningful contributions' does not exist.","headline":"The dataset idea is worth a referee, but the performance claim is unsupported because the graph features are built from random IPs and no baseline or held-out evaluation is shown.","tokens_in":6501,"tokens_out":3544,"would_cite":false,"duration_ms":33611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that enriching the widely used NSL-KDD intrusion-detection dataset with graph-derived features such as centrality and community labels yields a hands-on educational resource and high-accuracy attack classification in an…","keywords":["cybersecurity education","graph theory","NSL-KDD","data enrichment","centrality measures","community detection","intrusion detection","Auto AI"],"falsifier":"Re-run the released enrichment code and compare the same Auto AI pipeline on the original NSL-KDD features versus the enriched features using a strict train/test split; if a shuffled version of the graph features preserves the reported accuracy, or if the enriched set fails to beat the original on held-out classes, the claim that the graph-derived features support high-performance attack classification is refuted.","tokens_in":5656,"feed_emoji":"🛡️","tokens_out":8676,"duration_ms":74960,"temperature":0.7,"pith_summary":"The paper tries to establish that a standard intrusion-detection benchmark can be turned into a graph-rich teaching dataset by appending synthetic source and destination IP addresses, computing centrality and community features, and making the result publicly available. The intended payoff is educational: students and professionals can practice graph-theoretical techniques such as degree and betweenness centrality and Louvain community detection on a recognized benchmark while learning to interpret network vulnerabilities. The paper also claims that the enriched features support automated classification: feature selection ranked out-degree and betweenness centrality among the top 50 features, and an Auto AI classification experiment reported high prediction performance with no errors. If the claim holds, the artifact addresses a concrete gap, namely the scarcity of datasets that let learners apply graph methods to intrusion data in a controlled way.","feed_headline":"Graph-enriched benchmark dataset brings attack analysis to class","feed_subtitle":"Students practice centrality and community detection on a benchmark intrusion dataset.","key_machinery":"The load-bearing object is the enriched NSL-KDD artifact: a new feature set that appends synthetic source and destination IP addresses, graph centrality scores, and community labels to each NSL-KDD record. The graph behind these features is built from the synthetic IP addresses using a sparse edge-generation probability of 0.2, then partitioned with the Louvain modularity method. The machinery works by quantifying the position of each connection's endpoints in the constructed graph, so that each tabular row inherits a network-level description; feature selection then judges whether those descriptions help separate attack classes from normal traffic.","core_discovery":"On the paper's own terms, the central discovery is that a widely used tabular intrusion-detection dataset can be enriched with graph-derived features without losing its practical value. The paper generates synthetic IPv4 addresses for source and destination hosts, connects them with an edge probability of 0.2, builds directed and bipartite graph views, computes degree, betweenness, closeness, and PageRank centrality, and detects communities using the Louvain method. These graph features are appended to the original NSL-KDD features; two centrality metrics land in the top 50 selected features, and automated classification on the enriched set is reported to run without errors and with high prediction performance. The paper presents the enriched dataset as a reusable educational artifact and as a demonstration that graph-theoretic features can be added to benchmark cybersecurity data.","pith_inferences":["An unstated consequence is that the same enrichment pipeline could be applied to flow-based datasets that carry real IP fields; if centrality features still rank near the top there, it would show the graph signal survives when the topology is real rather than randomly generated.","Because the IP addresses and edges are synthetic, a direct check would be shuffling the graph features and repeating the Auto AI evaluation; if performance does not drop, the top-feature ranking and the reported no-error result reflect overfitting rather than signal.","The educational payoff has not been measured directly: a before-and-after comparison of students' ability to interpret centrality and community scores would test whether the enriched dataset teaches graph reasoning, not just classification.","A natural follow-up would be to publish the exact feature values and generation seed so the centrality features can be audited against the original labels, turning the artifact into a reproducible benchmark."],"forward_implications":["Students can practice degree centrality, betweenness centrality, PageRank, and Louvain community detection on a standard intrusion-detection benchmark instead of on toy graphs.","The dataset gives instructors a privacy-preserving starting point, since synthetic pseudonymized IP addresses avoid exposing real network identifiers while still supporting graph analysis.","If the reported Auto AI performance is taken at face value, graph-derived features do not interfere with automated classification and may improve it, so similar enrichment could be layered onto other benchmarks.","The feature-selection result suggests that bridge-like and outward-connecting nodes carry useful structural information for attack detection.","Public availability of the enrichment code lets other instructors and researchers reproduce the artifact and adapt it to new datasets."],"supporting_citations":[{"why":"Supplies the Design Science Research methodology whose guidelines organize artifact construction and evaluation.","marker":"[2]"},{"why":"Positions NSL-KDD in the taxonomy of network-based intrusion-detection datasets, justifying the choice of dataset for enrichment.","marker":"[14]"},{"why":"Supplies the principle that practitioners must build a network representation of their data to use graph techniques, grounding the graph-construction step.","marker":"[16]"},{"why":"Provides the networkx library used to generate IP graphs and compute centrality measures.","marker":"[17]"},{"why":"Defines the Louvain modularity method used for community detection on the generated graph.","marker":"[18]"},{"why":"Provides the automated machine-learning service used to evaluate the enriched dataset for classification and prediction.","marker":"[19]"},{"why":"Makes the enrichment artifact publicly available so the educational resource can be reproduced and reused.","marker":"[20]"}],"fun_headline_variants":["Graph features bolster cyber attack dataset for classrooms","NSL-KDD gets graph theory makeover for hands-on cyber training","Cyber students gain graph-powered attack data with IBM AI check","Enriched dataset adds graph metrics to classic cyber intrusion set","New cyber dataset mixes graph theory with AI for student labs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a graph built from randomly generated IP addresses with a fixed edge probability of 0.2 produces centrality and community features that genuinely correspond to the network structure behind the NSL-KDD connection records, rather than adding arbitrary numbers.","fun_headline_variants_meta":{"raw":{"variants":["Graph features bolster cyber attack dataset for classrooms","NSL-KDD gets graph theory makeover for hands-on cyber training","Cyber students gain graph-powered attack data with IBM AI check","Enriched dataset adds graph metrics to classic cyber intrusion set","New cyber dataset mixes graph theory with AI for student labs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000376,"raw_usage":{"total_tokens":1945,"prompt_tokens":828,"completion_tokens":1117,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":444,"completion_tokens_details":{"reasoning_tokens":1036}},"tokens_in":444,"tokens_out":1117,"duration_ms":7548,"temperature":1.0,"reasoning_tokens":1036,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:49:53.121043+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the released enrichment code and compare the same Auto AI pipeline on the original NSL-KDD features versus the enriched features using a strict train/test split; if a shuffled version of the graph features preserves the reported accuracy, or if the enriched set fails to beat the original on held-out classes, the claim that the graph-derived features support high-performance attack classification is refuted.","supporting_citations":[{"cited_title":"Design science in information systems research,","cited_arxiv_id":null,"evidence_quote":"Supplies the Design Science Research methodology whose guidelines organize artifact construction and evaluation."},{"cited_title":"A survey of network-based intrusion detection data sets,","cited_arxiv_id":null,"evidence_quote":"Positions NSL-KDD in the taxonomy of network-based intrusion-detection datasets, justifying the choice of dataset for enrichment."},{"cited_title":"Graph based anomaly detection and description: a survey,","cited_arxiv_id":null,"evidence_quote":"Supplies the principle that practitioners must build a network representation of their data to use graph techniques, grounding the graph-construction step."},{"cited_title":"Exploring network structure, dynamics, and function using networkx,","cited_arxiv_id":null,"evidence_quote":"Provides the networkx library used to generate IP graphs and compute centrality measures."},{"cited_title":"Fast unfolding of communities in large networks,","cited_arxiv_id":null,"evidence_quote":"Defines the Louvain modularity method used for community detection on the generated graph."},{"cited_title":"An approach for predicting heart failure rate using ibm auto ai service,","cited_arxiv_id":null,"evidence_quote":"Provides the automated machine-learning service used to evaluate the enriched dataset for classification and prediction."},{"cited_title":"Data enrichment solution for cybersecurity","cited_arxiv_id":null,"evidence_quote":"Makes the enrichment artifact publicly available so the educational resource can be reproduced and reused."}],"review_version":1}