{"id":"17f525b6-a12c-4e02-83a8-4365cef7213b","arxiv_id":"2502.04341","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A benchmark comparison of six community detection algorithms on a Facebook graph finds Louvain and Label Propagation strongest by internal metrics, with no ground-truth validation.","lead":"This paper compares six community detection algorithms on the SNAP Facebook Social Circles dataset using internal clustering metrics. It reports that Louvain and Label Propagation look most robust, but the evaluation omits the dataset's ground-truth circles and several methodological details.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central ranking of Louvain and Label Propagation as robust rests on internal metrics that are also the objectives those algorithms optimize; on a dataset with provided ground-truth circles, no external validation or error bars support the claim.","rationale":"The reader rejected the paper because the central claim is not supported by the evidence: ground-truth circles are never used, metrics overlap with algorithm objectives, stochastic runs lack error bars, and key implementation details are missing. My stress-test converges on the same load-bearing concern, sharpened as a circularity problem: the metric table in Section 6 ranks algorithms partly on the very quantities those algorithms optimize, while the dataset provides external ground truth that would break the circularity. I do not accuse the authors of any manipulation; the paper is a straightforward benchmark with shared code, and the descriptive degree-distribution and visualization work is useful context. The concern is about experimental design, not integrity. The concrete test—external agreement against the provided circles—directly settles whether Louvain and Label Propagation are genuinely robust or merely score well on self-aligned objectives. Because the reader's verdict was already REJECT and this concern reinforces rather than redirects that verdict, I recommend no change.","tokens_in":7508,"tokens_out":4582,"duration_ms":52734,"concrete_test":"For each of the 10 ego networks in the SNAP Social Circles dataset, run Louvain, Label Propagation, spectral clustering, K-Means, Infomap, and leading eigenvector with stated parameter choices and at least 20 random seeds, then compute NMI and adjusted Rand index against the provided ground-truth circles and aggregate by mean or median. If Louvain and Label Propagation do not rank in the top two under external agreement, or if the top two differ across ego networks, the Section 7 'robust' conclusion is an artifact of internal-metric circularity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Section 7) is that Louvain and Label Propagation are 'particularly robust choices' because they score well on the metrics in Section 6. For that claim to be supported, the metric table (Figure 12) must be an unbiased measure of community quality. That condition is the least secure part of the argument. First, the metrics are not independent judges: modularity is the objective Louvain maximizes; normalized cut is what spectral clustering is designed to minimize; compactness and Calinski-Harabasz are K-Means objectives; and for graph partitions, the 'distances' in Equations (3)-(6) and (8) require a node embedding or distance model that the paper never specifies. A benchmark that ranks algorithms partly by their own objective functions cannot establish robustness. Second, the SNAP Social Circles dataset ships ground-truth circles (Section 5.1), yet no agreement measure such as NMI or adjusted Rand index against those circles is reported, so there is no external anchor validating that high internal scores correspond to 'meaningful communities' as the abstract claims. This is especially acute because the dataset is ego-network-based; the paper silently analyzes one aggregated 4,039-node graph without justifying how ground truth would transfer. The absence of error bars on stochastic algorithms (Louvain, Label Propagation, spectral clustering, K-Means) further means the observed metric gaps could be run-to-run noise. If the Section 6 numbers are not externally validated, Section 7's recommendation is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical comparison of six community detection algorithms (K-Means, Louvain, spectral clustering, label propagation, Infomap, and leading eigenvector) on the SNAP Facebook Social Circles dataset. The authors compute modularity, normalized cut ratio, silhouette score, compactness, Calinski-Harabasz score, and separability for each algorithm, and conclude in Section 7 that Louvain and label propagation are 'particularly robust choices.' The paper provides visualizations and a link to code.","tokens_in":7831,"tokens_out":8443,"duration_ms":89230,"significance":"The topic is of practical interest, and the paper has the merit of making its code public and covering standard algorithms. If the central ranking were established through external validation against the ground-truth circles, the paper could be a useful practical reference. However, the current design—internal metrics that partly coincide with the algorithms' objectives, no ground-truth baseline, no error bars, and an unspecified node embedding—means the headline conclusion is not currently established. The significance is therefore limited unless the evaluation is substantially strengthened.","major_comments":[{"comment":"The headline conclusion that Louvain and Label Propagation are 'particularly robust choices' is not supported by the reported evidence, because several of the metrics are the objective functions of the algorithms being compared. Modularity (Eq. 1) is the objective optimized by Louvain; normalized cut (Eq. 2) is the criterion spectral clustering is designed to minimize; compactness (Eq. 6) and, indirectly, the Calinski-Harabasz score (Eq. 7) are the K-Means objective. A ranking that rewards each algorithm partly for optimizing its own objective cannot establish robustness. Since the SNAP Social Circles dataset includes ground-truth circles, the paper should report agreement measures such as NMI or adjusted Rand index against those circles; without this external anchor, the claim that the detected communities are 'meaningful' (Abstract) is unjustified.","section":"Section 6, Fig. 12; Section 7"},{"comment":"The K-Means and spectral clustering results depend on a vector embedding of nodes that is never described. The silhouette score (Eq. 3), compactness (Eq. 6), and separability (Eq. 8) all require pairwise distances d(i,j) or distances to centroids; for a graph, these quantities are undefined unless an embedding or distance model is specified. The paper must state the embedding used and justify it; otherwise the metric values for K-Means and spectral clustering cannot be interpreted or reproduced.","section":"Section 5.5A; Eqs. (3)-(6), (8)"},{"comment":"The dataset is described only as 'more than 4000 nodes having more than 88,000 edges.' The SNAP Social Circles dataset is a collection of ego networks, each with its own ground-truth circles. The paper appears to analyze a single aggregated graph, but it never states how the ego networks were combined, whether isolated nodes were removed, or how ground-truth circles would transfer to the aggregated graph. This omission affects both the internal-metric analysis and any external validation; the authors should describe the graph construction precisely.","section":"Section 5.1; Section 7"},{"comment":"Louvain, Label Propagation, K-Means, spectral clustering, and the leading eigenvector method are stochastic or initialization-dependent. The paper reports a single value per metric with no error bars, no repeated runs, and no significance tests. The observed differences between algorithms could therefore be run-to-run noise. The authors should report means and standard deviations over multiple random seeds, or otherwise demonstrate stability.","section":"Section 6, Fig. 12; Section 5.5"}],"minor_comments":[{"comment":"There is a typo: 'number od edges' should read 'number of edges.'","section":"Section 4.2, Eq. (2)"},{"comment":"Section 6.1 says six algorithms were tested on 'five separate metrics,' while Section 7 counts 'six key metrics'; the metric list actually contains six entries. Please make the count consistent.","section":"Section 6.1 vs. Section 7"},{"comment":"The 'Leading Eigenvector Algorithm' is described as 'frequently called the Newman-Girvan algorithm,' but Newman-Girvan is a different betweenness-based divisive algorithm. The lead eigenvector method is a separate spectral method (Newman 2006). Also, the sentence 'The Infomap Algorithm identified 18 communities' in this section appears to be a typo; it should refer to the leading eigenvector method.","section":"Section 5.5F"},{"comment":"The captions are inconsistent with the definitions in Section 4.2: normalized cut is better when lower, compactness is better when lower, and silhouette score is better when higher. The captions state the opposite directions and should be corrected.","section":"Figure captions 14, 15, 18"},{"comment":"Figure 12 is a screenshot; the numeric metric values should be provided in a table so that the results are reproducible from the written paper alone.","section":"Section 6.1, Fig. 12"},{"comment":"'Commuity Detection' should be 'Community Detection.'","section":"Keywords"}],"recommendation":"major_revision","confidential_remarks":"This is a very preliminary study. I would not support acceptance on the current evidence. If the editor is willing to consider a major revision, the authors should redo the evaluation with the four points in the major comments; otherwise the paper is more suited to a workshop or as a technical report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a competent implementation of six standard algorithms on a standard dataset, but the central claim that Louvain and Label Propagation are 'robust' doesn't survive contact with the evaluation.\n\nWhat's actually new: nothing algorithmically. The paper applies Louvain, label propagation, spectral clustering, K-Means, Infomap, and leading eigenvector to the SNAP Facebook graph and reports internal metrics. That's a standard comparative exercise seen many times before; the only novelty is the specific dataset instance.\n\nWhat the paper does well: it's clearly written, the literature review is adequate, and the code is public on GitHub, so the results are reproducible. The degree-distribution analysis and visualizations are fine as descriptive material.\n\nWhere it falls down: the evaluation is circular in a load-bearing way. Modularity is the objective Louvain optimizes; normalized cut is what spectral clustering is designed to minimize; compactness and Calinski-Harabasz are K-Means objectives. Ranking algorithms largely by their own objective functions doesn't establish robustness. The dataset ships ground-truth circles, yet the paper never computes NMI or adjusted Rand index against them, so there's no external anchor. The SNAP data is ego networks, but the paper silently analyzes one aggregated 4,039-node graph without justifying how ground truth would transfer. Stochastic algorithms are run once with no error bars, so the observed metric gaps could be run-to-run noise. The node embedding used for K-Means and spectral clustering is never defined, and the spectral clustering cluster count is never stated. There's also an internal inconsistency: the text says lower compactness is better, but Figure 15's caption says higher values indicate more cohesion.\n\nThe Section 7 recommendation that Louvain and Label Propagation are 'particularly robust' is therefore unsupported. That's the paper's central claim, and it fails. The flaws are fixable in principle: add ground-truth comparison, run multiple seeds, report error bars, and disclose all hyperparameters. But as submitted, the evidence does not back the conclusion.\n\nWho is this for? Beginners in network science might find it a readable walkthrough, but it won't change practice and I wouldn't cite it. If this crosses my desk for a serious venue, I'd send it back for major revision or reject, depending on the journal. It doesn't deserve a full referee cycle until the evaluation is fixed.","headline":"A routine benchmark that ships code but whose headline ranking rests on circular metrics and ignores the dataset's ground-truth labels.","tokens_in":8331,"tokens_out":2983,"would_cite":false,"duration_ms":28938,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Louvain and label propagation emerge as the most robust community detection algorithms on the SNAP Facebook graph.","keywords":["community detection","clustering","social networks","social media analytics","Facebook network","SNAP Social Circles dataset","Louvain algorithm","label propagation"],"falsifier":"Re-score all six partitions against the dataset's ground-truth circles using normalized mutual information or the adjusted Rand index; if Infomap or the leading eigenvector method matches the known circles better than Louvain and Label Propagation, the paper's conclusion that the latter two are the most robust choices would be contradicted.","tokens_in":7319,"feed_emoji":"🕸️","tokens_out":5910,"duration_ms":58512,"temperature":0.7,"pith_summary":"This paper tries to answer a practical question: which community detection algorithm should someone use on a real social network when no known answer exists? To find out, it runs six algorithms on the SNAP Social Circles Facebook dataset and scores each partition with six internal quality metrics. The main finding is that Louvain and Label Propagation perform well across the most metrics, while other algorithms win on specific measures but not overall. If the finding holds, it gives practitioners a defensible default choice and a template for comparing algorithms on other social graphs.","feed_headline":"Louvain and label propagation win the Facebook community test","feed_subtitle":"Six algorithms, six metrics, one Facebook graph: two come out ahead for real-world use.","key_machinery":"The carrying object is a comparison protocol rather than a new algorithm or theorem: the SNAP Social Circles Facebook graph, treated as a single network of 4,039 nodes and more than 88,000 edges, is partitioned by each algorithm and then each partition is scored by six internal quality metrics. The metrics translate qualitative notions of a good community—dense internal connections, few edges between communities, compact clusters, well-separated centers—into numbers that can be ranked. The protocol works because it treats the algorithms symmetrically, so differences in metric scores are attributed to algorithmic behavior rather than to the evaluator.","core_discovery":"The paper's central claim is that, among K-Means, Louvain, Spectral Clustering, Label Propagation, Infomap, and Leading Eigenvector, the Louvain and Label Propagation algorithms are the most robust choices for detecting communities in the SNAP Social Circles Facebook network. The claim rests on comparing each algorithm's partition with modularity, normalized cut ratio, silhouette score, compactness, Calinski-Harabasz score, and separability. No single algorithm dominates every metric; each has strengths, but Louvain and Label Propagation score well across a wider range, leading the authors to recommend them for general use and to caution that algorithm selection should depend on the dataset and objective.","pith_inferences":["Because the comparison ignores the ground-truth circles that ship with the SNAP dataset, the ranking reflects internal cohesion and separation, not agreement with real friend lists; measuring agreement with those circles could plausibly reorder the winners.","The paper treats the dataset as one aggregated 4,039-node graph, although the ground truth is defined per ego network; running the same metrics separately on each ego network is a natural test of whether the ranking survives the aggregation.","The number of communities each algorithm finds varies widely—from around 13 for Louvain to 93 for Infomap—so part of what the metrics capture may be partition granularity; a fairer comparison might hold the number of communities fixed."],"forward_implications":["On Facebook-like social graphs, Louvain and Label Propagation can serve as safe defaults when no ground-truth communities are available, with less risk of trailing badly on any single metric.","Algorithms such as Infomap and the leading eigenvector method produce many small communities in this dataset, so they are better suited to tasks that need fine-grained structure rather than a few large clusters.","Researchers comparing algorithms on their own networks can reuse the same metric suite to rank methods without relying on labels.","Because no algorithm dominates every metric, evaluation should report several metrics rather than a single headline score when choosing a community detection method."],"supporting_citations":[{"why":"Frames the evaluation problem and the range of community detection approaches the paper builds on.","marker":"[1]"},{"why":"Establishes the comparative benchmark approach of testing algorithms on diverse networks.","marker":"[3]"},{"why":"Supplies the SNAP Social Circles Facebook dataset that all algorithms are run on.","marker":"[6]"},{"why":"Defines the Louvain method whose output is evaluated.","marker":"[9]"},{"why":"Provides one of the spectral clustering references behind the implemented method.","marker":"[12]"},{"why":"Defines the label propagation algorithm whose output is evaluated.","marker":"[15]"},{"why":"Defines Infomap and the map equation it optimizes.","marker":"[16]"},{"why":"Defines the leading eigenvector algorithm whose output is evaluated.","marker":"[17]"}],"fun_headline_variants":["Louvain and label propagation lead Facebook community detection","Facebook communities: Louvain and label propagation win on balance","Best community finders for SNAP Facebook: Louvain and label propagation","Community detection showdown: Louvain and label propagation robust","Which algorithm finds Facebook groups best? Louvain and label propagation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ranking assumes that internal quality metrics computed on an algorithm's own output are sufficient to judge community detection, so the ground-truth circles in the SNAP dataset can be ignored.","fun_headline_variants_meta":{"raw":{"variants":["Louvain and label propagation lead Facebook community detection","Facebook communities: Louvain and label propagation win on balance","Best community finders for SNAP Facebook: Louvain and label propagation","Community detection showdown: Louvain and label propagation robust","Which algorithm finds Facebook groups best? Louvain and label propagation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000115,"raw_usage":{"total_tokens":1009,"prompt_tokens":820,"completion_tokens":189,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":436,"completion_tokens_details":{"reasoning_tokens":102}},"tokens_in":436,"tokens_out":189,"duration_ms":3112,"temperature":1.0,"reasoning_tokens":102,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:23:35.947804+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-score all six partitions against the dataset's ground-truth circles using normalized mutual information or the adjusted Rand index; if Infomap or the leading eigenvector method matches the known circles better than Louvain and Label Propagation, the paper's conclusion that the latter two are the most robust choices would be contradicted.","supporting_citations":[{"cited_title":"Community detection in graphs","cited_arxiv_id":null,"evidence_quote":"Frames the evaluation problem and the range of community detection approaches the paper builds on."},{"cited_title":"Community detection algorithms: a comparative analysis","cited_arxiv_id":null,"evidence_quote":"Establishes the comparative benchmark approach of testing algorithms on diverse networks."},{"cited_title":"Snap: A general-purpose network analysis and graph-mining library","cited_arxiv_id":null,"evidence_quote":"Supplies the SNAP Social Circles Facebook dataset that all algorithms are run on."},{"cited_title":"Fast unfolding of communities in large networks","cited_arxiv_id":null,"evidence_quote":"Defines the Louvain method whose output is evaluated."},{"cited_title":"On spectral clustering: Analysis and an algorithm","cited_arxiv_id":null,"evidence_quote":"Provides one of the spectral clustering references behind the implemented method."},{"cited_title":"Near linear time algorithm to detect community structures in large-scale networks","cited_arxiv_id":null,"evidence_quote":"Defines the label propagation algorithm whose output is evaluated."},{"cited_title":"Finding community structure in networks using the eigenvec- tors of matrices","cited_arxiv_id":null,"evidence_quote":"Defines the leading eigenvector algorithm whose output is evaluated."}],"review_version":1}