{"id":"b9123d37-f9fc-4197-a0b1-bd77b436419d","arxiv_id":"2411.14550","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A K-means-based unsupervised intrusion detection experiment whose core claim, that the model correctly counts unknown attack types, is neither validated nor reproducible from the paper as written.","lead":"This paper applies K-means clustering to network traffic data in an attempt to group unknown cyber attacks into clusters. The claimed success is not supported by the evidence presented, and the text shows signs of being assembled from unrelated sources.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The counting claim is unsupported: K-means requires a preset K, yet no K value or selection method is reported for either dataset, and the only displayed K-means output (Figure 2) has five clusters, not the seven claimed attack types; the central 'detected the number of attacks' claim is thus…","rationale":"The reader's weakest assumption is exactly the one that carries the paper: K-means cannot infer the number of clusters. I agree, and the paper itself provides a stronger reason to reject: Section 1.3.2 defines K as a predetermined number of clusters, so 'the model counted the attacks' cannot be a property of K-means without an additional model-selection step. The reported value_counts has five clusters, while the Contributions claim seven; this internal inconsistency means even the descriptive claim is not coherent. I do not rely on the provenance of the text or the authors; the argument fails on its own terms. No independent verification, code, or formal proof is offered, and the Kaggle result is asserted without displayed cluster outputs. A reproduction with hidden labels and adjusted Rand index across K would settle whether any unsupervised signal recovers the attack taxonomy; without it, the central claim is unfalsifiable. Since the reader already rejected the paper and my review identifies the same load-bearing flaw plus a concrete contradiction, no verdict change is needed.","tokens_in":12934,"tokens_out":3373,"duration_ms":32668,"concrete_test":"Obtain (or re-run from the paper's description) the Kaggle IoT Intrusion Detection dataset, apply the stated preprocessing (drop Flow ID, Src IP, Dst IP, Timestamp; normalize), and run K-means with K = 2,...,10 while withholding the true labels. For each K, compute the adjusted Rand index between the cluster assignment and the known attack labels. If the best ARI occurs at K equal to the true number of classes and is attained only by giving K that value, the counting claim collapses; if no K-selection method is supplied, the claim is unfalsifiable. Also reproduce the self-generated dataset and report whether the K-means output contains 5 or 7 clusters.","verdict_should_be":"UNCHANGED","load_bearing_attack":"K-means (Section 1.3.2) partitions data into K pre-defined clusters; K is an input, not an output. For the central claim that the model 'counted the attacks' to be valid, the paper must show K was chosen from the data (e.g., silhouette, gap statistic, stability) and that clusters align with attack types. Neither is provided. The only quantitative unsupervised output in the paper is the value_counts under Figure 2: cluster labels 0–4 with counts (7361, 5801, 3390, 1349, 1080), i.e., five clusters. This directly contradicts the Contributions statement that seven attack types 'were actually classified by the clustering system.' If K was set to seven because the authors knew the true taxonomy, then the 'detection of new types' is circular: K-means will return seven clusters by construction, and the later supervised classification report (Table 1, seven classes) validates labels supplied to it, not novel structure discovered from unlabeled traffic. For the Kaggle IoT dataset, the paper gives no cluster output or K at all, so 'detected the number of attacks correctly' is unverifiable. The load-bearing assumption—that K emerges from clustering rather than being injected—is therefore both undisclosed and contradicted by the reported result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper claims that K-means clustering can discover and correctly count attack types in unlabeled network traffic, using two datasets: a self-collected set built with Kali Linux, tcpdump, and CICFlowMeter, and the Kaggle 'Intrusion Detection in Internet of Things Network' dataset. The authors state in the Contributions section that seven attack types 'were actually classified by the clustering system using the K-means algorithm' and that the model 'detected the number of attacks correctly' on both datasets. The paper also describes an XGBoost classification stage and reports a classification report with very high precision, recall, and Cohen's Kappa. The central claim, however, is unsupported because the paper never discloses the value of K or how it was chosen; the only displayed K-means output shows five clusters, not seven; and no ground-truth comparison is provided for either dataset.","tokens_in":13234,"tokens_out":2457,"duration_ms":23644,"significance":"If the central claim were established, the paper would address a real need: unsupervised detection of novel attack types in unlabeled traffic. The authors also attempted a practical data collection effort and provided a detailed description of their testbed. However, the significance is currently negative because the load-bearing evidence is absent. The paper supplies no machine-checked proofs, no reproducible code, no parameter-free derivation, and no falsifiable prediction. The claim that K-means 'counted the attacks' is not supported by any disclosed experimental output; in fact, the only quantitative clustering result contradicts it. The contribution, as presented, reduces to a circular statement if K was set to the known number of attack types.","major_comments":[{"comment":"The central claim that K-means 'detected the number of attacks correctly' is unsupported because K is an input parameter, not an output. Section 1.3.2 states that 'K indicates how many pre-defined clusters must be formed during the process,' yet the paper never reports the value of K used for either dataset, nor any method for selecting it (e.g., silhouette, gap statistic, or stability analysis). If K was set to the known seven attack types, then the claim that the model 'counted the attacks' is circular: K-means will return exactly K clusters by construction. The paper must disclose K and demonstrate that it emerged from the data.","section":"Contributions section (p. 2) and §1.3.2"},{"comment":"The only unsupervised clustering output displayed for the self-collected dataset is the value_counts shown under Figure 2, which lists cluster labels 0 through 4 with counts (7361, 5801, 3390, 1349, 1080). That is five clusters, not the seven attack types claimed in the Contributions section. This direct discrepancy undermines the statement that 'the seven attacks were actually classified by the clustering system.' The authors need to explain why the displayed clustering produced five groups and how this result supports the claim of seven detected attacks.","section":"Figure 2 (p. 5)"},{"comment":"Equation (1) is unusable as presented because the text surrounding it is copied from a different domain. The formula for Euclidean distance is immediately followed by 'In the presented work, n is fixed to 1 since the occurrence probability of the landslide serves as the data property for modeling the susceptibility to landslides.' No landslide data appear anywhere in this paper, and the centroid update equation is missing entirely. This garbled text means the clustering algorithm is not actually described, which is a load-bearing methodological gap for a paper whose entire contribution is the application of K-means.","section":"§1.3.2, Eq. (1)"},{"comment":"No ground-truth comparison is provided for either dataset. For the self-collected dataset, the paper does not show how the cluster labels correspond to the seven known attack types (DoS, Brute Force, TCP Flooding, UDP Flooding, etc.) that were generated in the testbed. For the Kaggle IoT dataset, no clustering output or K value is reported at all. Table 1 reports precision, recall, F1-score, and support for seven classes, but these are supervised classification metrics; the paper does not explain how clusters were mapped to class labels, whether the reported classes came from cluster pseudo-labels or from true labels, or how the classification report relates to the unsupervised claim. Without a confusion matrix or cluster-to-class correspondence, the statement that the model 'detected the number of attacks correctly' is unverifiable.","section":"Table 1 and Dataset Description (pp. 6-8)"}],"minor_comments":[{"comment":"The section numbering is inconsistent: '1.1.2 Building the model' appears after '1.4 METHODOLOGY', and '1.2 CONCLUSION' appears after the results figures. The paper would benefit from a coherent structure with sequential numbering.","section":"Section numbering and organization"},{"comment":"The dataset dimensions are reported inconsistently: the paper mentions 19,677 rows and 84 columns, then 83 columns, then states that after dropping one missing-value column and four other columns the dataset became 78 columns. The arithmetic (83 - 5 = 78) is consistent, but the initial 84-column mention is not explained, and the names of dropped columns are repeated verbatim twice.","section":"Dataset description (p. 5)"},{"comment":"Figure (1) is referenced as showing a sample of the dataset, but the figure is not present in the text. Figure (3) and Figure (4) are similarly referenced without being displayed or described. The reader cannot assess the clustering or classification results from the visualizations.","section":"Figure references"},{"comment":"The abstract states 'The model counted the attacks and assigned numbers to each one of them,' but the paper does not explain what 'assigned numbers' means in terms of attack types. The phrase 'with the use of XG-boost algorithm following completing the aggregation with K-means algorithm' is also grammatically unclear.","section":"Abstract and Contributions wording"},{"comment":"Several references are incomplete or incorrectly numbered. For example, reference [13] is listed as 'Dhaliwal, S. S., Nahid, A. A, 2018' but the text cites Farhan et al. for the same citation number, and the reference list contains duplicate entries for the same work (e.g., references [20] and [21] duplicate Salem et al.).","section":"References"}],"recommendation":"reject","confidential_remarks":"This manuscript appears to be a very rough draft with substantial copy-paste errors, missing figures, and an inconsistent narrative. The central claim is not merely under-evidenced; the presented evidence (five clusters, a supervised classification report, and a garbled equation from a landslide-susceptibility paper) directly undercuts it. The paper does not meet the standard for a peer-reviewed publication in its current form, and the load-bearing issues are not local fixes. The journal may also wish to review the editorial process that accepted this article for publication in its current state."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain take: the central claim is not supported. K-means takes K as an input; the paper never discloses K, and the only unsupervised output shown (Figure 2, value_counts) contains five clusters, not the seven attack types claimed. So \"the model counted the attacks correctly\" is either circular or contradicted.\n\nWhat is new and decent: the authors did collect a small custom dataset (roughly 20k rows, seven attack types) using Kali Linux, tcpdump, and Putty, and they ran both K-means and XGBoost on it plus a public Kaggle IoT set. That is the only original asset. If the dataset were released and described more carefully, it could be useful for unsupervised intrusion detection. The paper also frames a real problem: detecting newly emerging attacks without labels.\n\nSoft spots, in order of severity. The biggest is the undisclosed K. K-means partitions into K clusters; it cannot \"detect the number of attacks\" unless you choose K appropriately. No silhouette, gap statistic, or stability analysis is reported. The one visible output has labels 0–4 (five clusters), which contradicts the seven claimed categories. The classification report in Table 1 uses supervised metrics; it does not show that clustering recovered the attack taxonomy. For the Kaggle dataset there is no cluster output at all, so the claim that it \"detected the number of attacks correctly\" is unverifiable. Second, Equation (1) is copy-pasted: it refers to \"landslide susceptibility\" and \"n is fixed to 1,\" which is unrelated to this paper. That artifact destroys confidence that the rest of the text was checked. Third, the references are in disarray (numbering like [38,39] appears while only 33 are listed, and some citations do not match). Fourth, no code or data link is provided, so the work is not reproducible.\n\nOn the positive side, no obvious fabricated numbers in the table; the metrics are plausible for a supervised seven-class problem. But they do not support the unsupervised claim.\n\nWho this is for: maybe a student or practitioner wanting a simple example of K-means + XGBoost on a small IDS dataset. It is not a serious research contribution in its current form.\n\nRecommendation: desk reject, with clear feedback to the authors: report K selection, include ground-truth alignment, fix the copy-paste, and release the dataset. I would not spend referee time on this version.","headline":"K-means counts nothing on its own, and since the paper never says what K was—and its own figure shows five clusters for seven claimed attacks—the central detection claim is unsupported and the rest is a thin, sloppy workflow.","tokens_in":13711,"tokens_out":2691,"would_cite":false,"duration_ms":26135,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that K-means clustering, applied without labels, can recover the true attack categories in network traffic, including previously unseen types.","keywords":["intrusion detection","K-means clustering","unlabeled network traffic","XGBoost","anomaly detection","IoT intrusion dataset","cyber security"],"falsifier":"Re-run the pipeline on the same 19,677-row dataset with K chosen by a data-driven rule such as the elbow method or silhouette score, and check whether the resulting number of clusters equals seven and whether the cluster memberships align with the known attack labels; any mismatch would refute the central claim.","tokens_in":12734,"feed_emoji":"🛡️","tokens_out":9151,"duration_ms":74753,"temperature":0.7,"pith_summary":"This paper argues that K-means clustering, run on network traffic without any labels, can discover the attack categories hiding in the data. The authors collected 19,677 traffic records spanning seven common attack types, applied K-means after cleaning and normalizing the features, and report that the clusters match those seven types. They also applied the same approach to a public Kaggle IoT intrusion dataset and report that the model 'detected the number of attacks correctly.' The wider point matters because new attack types appear constantly and labeled examples for them do not exist; if clustering can recover the taxonomy on its own, intrusion detection would not have to wait for manual labeling.","feed_headline":"K-means clustering could recover attack types from unlabeled traffic","feed_subtitle":"If true, intrusion detection could flag unknown attack types without labeled training data, a key need as threats evolve.","key_machinery":"The load-bearing mechanism is K-means clustering. Given a pre-specified number K, the algorithm picks K initial centroids, assigns every data point to the nearest centroid by Euclidean distance $$d(X_t, X_\\xi) = \\sqrt{\\sum_{u=1}^{n}(X_{u t} - X_{u \\xi})^2}$$ and then recomputes each centroid as the mean of its assigned points, repeating until assignments stabilize. Before clustering, the paper drops flow identifiers, IP addresses, timestamps, and missing-value columns, converts categorical features to numbers, and normalizes scales. The resulting cluster labels are treated as pseudo-labels and passed to XGBoost for the classification and evaluation stage; K-means is what carries the claimed ability to count and enumerate attack types.","core_discovery":"On the paper's own terms, the central discovery is that K-means clustering, applied to unlabeled network-flow data, groups traffic into clusters that correspond one-to-one with known attack types and assigns each cluster a numerical label. The authors state that on their collected dataset 'the seven attacks were actually classified by the clustering system using the K-means algorithm,' and on the Kaggle IoT intrusion dataset 'the clustering model worked well and detected the number of attacks correctly.' The intended consequence is that when a genuinely new attack appears, it will form a new cluster and receive a new number, allowing analysts to flag it before any labeled examples exist.","pith_inferences":["A fair test of the claim would require choosing K without access to the known attack count (for example, by silhouette score or elbow method) and then comparing the resulting clusters to held-out labels; the paper never discloses how K was selected, so the reported exact match could be a self-fulfilling choice.","The paper reports no cluster-quality metrics such as adjusted Rand index, normalized mutual information, or purity against ground truth; reporting those would show whether clusters correspond to attack semantics or merely separate traffic by volume, timing, or protocol.","A direct test of the 'new attack type' claim would be to withhold one known attack family from the data, cluster the remainder, and see whether the withheld family forms its own distinct cluster and receives its own number."],"forward_implications":["Intrusion detection systems could flag genuinely new attack types as soon as they appear, because a previously unseen cluster would signal an unknown behavior.","The need for manually labeled traffic data would fall, since cluster labels can serve as pseudo-labels for training downstream classifiers such as XGBoost.","The same pipeline could be applied to any unlabeled traffic capture, including IoT networks, to enumerate how many distinct attack behaviors are present.","If cluster counts match true attack counts, K-means could estimate attack diversity without any ground-truth labels."],"supporting_citations":[{"why":"Defines the K-means algorithm and its objective of finding K divisions that minimize the error criterion, supplying the core method.","marker":"[22]"},{"why":"Establishes K-means as the unsupervised technique used to group unlabeled datasets in intrusion detection.","marker":"[25]"},{"why":"Provides the Euclidean-distance formula used to assign each data point to the nearest centroid.","marker":"[28]"},{"why":"Supplies the centroid-update and convergence rule that terminates the K-means iterations.","marker":"[29]"},{"why":"Supports the premise that clustering is suited to unclassified and qualitative cybersecurity data.","marker":"[17]"},{"why":"Provides the supervised-versus-unsupervised workflow that frames the paper's experimental design.","marker":"[12]"},{"why":"Situates XGBoost as a classification tool for intrusion detection, used after clustering in the pipeline.","marker":"[30]"}],"fun_headline_variants":["K-means flags novel intrusions as new clusters","Unlabeled traffic clustering pinpoints attack types","K-means recovers attack types from unlabeled data","Clustering model spots unknown attacks in traffic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim depends on the number of clusters K being chosen without knowing how many attack types are actually present, and the paper never states how K was selected; if it was set to the known seven attack types, then the reported match is circular.","fun_headline_variants_meta":{"raw":{"variants":["K-means flags novel intrusions as new clusters","Unlabeled traffic clustering pinpoints attack types","K-means recovers attack types from unlabeled data","Clustering model spots unknown attacks in traffic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000681,"raw_usage":{"total_tokens":3103,"prompt_tokens":968,"completion_tokens":2135,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":2075}},"tokens_in":584,"tokens_out":2135,"duration_ms":15419,"temperature":1.0,"reasoning_tokens":2075,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:08:22.064002+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline on the same 19,677-row dataset with K chosen by a data-driven rule such as the elbow method or silhouette score, and check whether the resulting number of clusters equals seven and whether the cluster memberships align with the known attack labels; any mismatch would refute the central claim.","supporting_citations":[{"cited_title":"Unsupervised K - Means Clustering Algorithm","cited_arxiv_id":null,"evidence_quote":"Defines the K-means algorithm and its objective of finding K divisions that minimize the error criterion, supplying the core method."},{"cited_title":"Intrusion Detection System: A Comprehensive Review","cited_arxiv_id":null,"evidence_quote":"Establishes K-means as the unsupervised technique used to group unlabeled datasets in intrusion detection."},{"cited_title":"Landslide Susceptibility Zonation Method Based on C5.0 Decision Tree and K - Means Cluster Algorithms to Improve the Efficiency of Risk Management","cited_arxiv_id":null,"evidence_quote":"Provides the Euclidean-distance formula used to assign each data point to the nearest centroid."},{"cited_title":"A Comparative Analysis of Supervised and Unsupervised Models for Detecting Attacks on the Intrusion Detection Systems","cited_arxiv_id":null,"evidence_quote":"Provides the supervised-versus-unsupervised workflow that frames the paper's experimental design."},{"cited_title":"Detecting Cybersecurity Attacks Using Different Network Features with LightGBM and XGBoost Learners","cited_arxiv_id":null,"evidence_quote":"Situates XGBoost as a classification tool for intrusion detection, used after clustering in the pipeline."}],"review_version":1}