{"id":"ef7ec920-ffe3-4ed9-a0cf-8b485a8a4884","arxiv_id":"2504.13538","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A supervised node-pair classifier, fed to Louvain, Leiden, Infomap, or Fast-Greedy as edge weights, is claimed to improve community detection, but the evaluation is compromised by transductive label use and best-of-16 selection.","lead":"The authors train machine-learning classifiers to guess whether pairs of nodes belong to the same community, then use those guesses as link weights for standard community-detection algorithms. On six networks, they report higher modularity and accuracy scores, but the evaluation design relies on training with the very labels being predicted.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Ground-truth mode's NMI/ARI gains are circular: the same labels train the similarity classifier and score the final partition.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing flaw: the ground-truth mode trains on the target labels and is then evaluated against those labels. My reading of Sections 3 and 4.3 confirms this. The framework has a genuine unsupervised variant (statistical-physics mode), but the headline 'most accurate' claim is tied to the ground-truth mode, so the leakage undermines the central claim rather than a peripheral one. The proposed permutation test would settle the question cleanly: if the learned similarities carry no information beyond what is already in the graph structure, the method may still work without target labels; if they do, the supervised comparison is circular. I do not see a way to rescue the ground-truth accuracy claim from the current protocol, because 5-fold cross-validation over node pairs is not a held-out evaluation at the network level. The best-of-16 selection is a second independent problem, but I do not need to rely on it to recommend keeping the reader's REJECT verdict. No code or data are provided, so the reported numbers cannot be independently checked; that is an additional barrier, but the analytical concern stands on its own. I agree with the reader: the central claim is not supported as presented.","tokens_in":12663,"tokens_out":6975,"duration_ms":66980,"concrete_test":"Run a label-permutation control for the ground-truth mode: keep the graph and all node-pair features fixed, randomly permute the ground-truth community labels used to train the classifier, then apply the identical 5-fold cross-validation, weighted-network construction, and community detection pipeline, and compute NMI/ARI against the true labels. If permuting the training labels drops NMI/ARI to near the unsupervised baselines, the reported gains depend on direct access to the target labels. If performance stays high, the target labels are not the source of the advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the framework 'consistently outperforms conventional methods' and is 'most accurate' when ground-truth labels are available rests on Section 4.3 (Tables 3-4), where the ground-truth mode trains on the target labels. In Fig. 3(b), the paper notes that when ground-truth communities are available, step (b) is omitted and the classifier is trained directly on those labels. Those same labels are then used to compute NMI and ARI for the final detection. Five-fold cross-validation over node pairs does not remove the leakage because every fold shares the same network and the same community structure; the model still sees labels from the target partition before scoring a partition of that same network. The comparison against Louvain, Leiden, Infomap, and Fast-Greedy is therefore not a comparison among detection methods: one competitor is given the target labels during training and is then scored against those same labels. Even if the features are purely structural, the learned edge weights can encode the target partition, making the reported NMI/ARI a measure of fit to the training labels rather than an independent accuracy result. The best-of-16 reporting in the table captions compounds this by selecting the most favorable configuration without correction, but the label-leakage issue alone is sufficient to invalidate the supervised accuracy claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a community-detection framework that uses machine-learning classifiers on node-pair similarity features (degree heterogeneity, clustering-coefficient heterogeneity, and common neighbors) to assign weights to edges, after which standard algorithms (Louvain, Leiden, Infomap, Fast-Greedy) are applied to the resulting weighted similarity network. Two training regimes are considered: a “statistical-physics” mode in which an initial detection supplies pseudo-labels, and a “ground-truth” mode in which the true community labels are used as training labels. Experiments on one synthetic and five real networks report weighted modularity, NMI, and ARI, along with a correlation analysis between node-pair similarity and these metrics. The abstract claims the framework consistently outperforms conventional methods and is most accurate when ground-truth labels are available.","tokens_in":12913,"tokens_out":4722,"duration_ms":39776,"significance":"The topic is timely and the pipeline is simple enough to be useful if validated; the authors test on several large SNAP datasets and compare against four standard detectors. However, the evidence offered for the central claim is not sound: the ground-truth evaluation is circular because the classifier is trained on the same labels used for scoring, the reported numbers are selected as best-of-16 without correction, no uncertainty estimates are given, and the correlation results are not statistically significant at conventional levels. The paper therefore does not currently substantiate its headline claims. The idea of using machine-learned edge weights informed by mesoscopic structure remains worth exploring, and the modularity improvements in the statistical-physics mode might survive a more careful evaluation, but the present manuscript does not demonstrate that.","major_comments":[{"comment":"The “ground-truth” mode trains the node-pair classifier on the exact community labels that are later used to compute NMI and ARI. The five-fold cross-validation shown in Fig. 3(f) is applied to node pairs, not to networks, so every fold shares the same network and the same target partition. The model therefore sees the target labels before the final partition is scored. This makes the reported improvements, such as DBLP VC-FG NMI 0.640 versus Infomap 0.410 and Amazon RF-Infomap ARI 0.342 versus Leiden 0.143, a measure of fit to the training labels rather than independent detection accuracy. The comparison against Louvain, Leiden, Infomap, and Fast-Greedy is thus not a fair comparison among detection methods, and the Abstract's claim that the approach is “most accurate” when ground-truth labels are available is not supported.","section":"Section 3, Fig. 3(b), and Section 4.3, Tables 3-4"},{"comment":"Each caption states that the best result among 16 configurations is reported. Selecting the best of 16 outcomes for each dataset and metric, without correction for multiple comparisons and without reporting the variance or the full set of results, inflates the apparent advantage of the proposal. This applies to both the statistical-physics and ground-truth sections and makes the quantitative superiority claims unverifiable. Please report averages and standard deviations over configurations, or pre-specify the configuration choice on a validation set.","section":"Tables 2-4 captions"},{"comment":"The Abstract claims “a strong and statistically significant correlation,” but Table 5 reports p-values of 0.084 and 0.052 for the ground-truth approach and 0.159 and 0.226 for the statistical-physics approach. None of these reaches the 0.05 level, and with only six networks the test has very low power. Moreover, the correlation is computed from the best detection results, which were themselves selected from the same 16 configurations, so the analysis is post-selection. The significance claim must be removed or replaced with a properly powered test.","section":"Table 5"},{"comment":"In the statistical-physics mode, the initial detector's output is used to label the training node pairs, and the same detectors are then re-run on the reweighted network. If the learned weighting mostly reinforces the teacher's partition, high Qw or NMI relative to the unweighted baseline is an expected self-confirmation rather than evidence of improved detection. A control experiment, such as weighting edges by the teacher's own within-community counts or by random weights matched to the same marginal distribution, is needed to show that the gains are not an artifact of this closed loop.","section":"Section 4.2 and Fig. 3(b)"},{"comment":"No error bars or repeated runs are reported anywhere; for example, the Tree ARI improvement from 0.020 to 0.060 in Table 4 rests on single-point estimates. Since several of the underlying detectors are stochastic and the network-construction procedure in Section 4.1.2 involves random tie-breaking, the reported single numbers are insufficient to establish consistent improvements.","section":"Throughout Section 4"}],"minor_comments":[{"comment":"The symbol R_{ij} is used for both degree heterogeneity and clustering-coefficient heterogeneity; please use distinct symbols to avoid confusion.","section":"Section 3, Eqs. (3)-(4)"},{"comment":"The parameter description is confusing: m=1 and n=100 are described as governing links, while Table 1 lists the Tree dataset with 100 nodes and 200 links. Please clarify the generation process and the meaning of each parameter.","section":"Section 4.1.1"},{"comment":"The label “P-value = 0.00” should be reported as “P < 0.001”, since a p-value cannot equal exactly zero.","section":"Fig. 2"},{"comment":"There is a typo in “communityi” in the sentence following Eq. (2); it should read “community i”.","section":"Section 2.3"},{"comment":"The concluding paragraph on a “teacher-student-teacher” cycle is speculative and is not tested by any experiment in the paper; it should be removed or supported with evidence.","section":"Section 5"},{"comment":"The captions say “best detection performance among 16 results” but do not define the 16 configurations; please state exactly which classifiers, detectors, feature sets, and voting schemes are included in the search.","section":"Tables 2-4"},{"comment":"The data-availability statement says that code will be released only after acceptance; for a computational paper, providing the code at submission time would materially aid reproducibility.","section":"Section 7"}],"recommendation":"reject","confidential_remarks":"The evaluation protocol is the central problem: the ground-truth mode trains on the labels used for scoring, and the statistical-physics mode is a closed loop with the same detectors as teachers and finalists. Combined with best-of-16 reporting and non-significant correlation p-values, the current evidence does not support the paper's headline claims. Even a major revision would require re-running the entire comparison with a non-circular protocol, which is substantial work within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline claim—\"consistently outperforms\" with better NMI/ARI—does not survive reading the evaluation section. The ground-truth mode trains the pairwise classifier on exactly the labels that are then used to compute NMI and ARI (Fig 3, step (b) omitted per the caption; Section 4.3). That is leakage, and five-fold CV over node pairs doesn't fix it: every fold carries the same partition. Tables 3 and 4 are therefore not a comparison of detection methods; one competitor was given the answer key. The stress-test note is correct on this point.\n\nWhat is actually new and worth taking seriously: the unsupervised \"statistical physics\" loop—run a detector, learn edge weights from the detected partition, reweight, re-run the detector. It is a legitimate self-training idea, clearly described, and the motivating observation in Fig. 2 (r=0.83, p<0.05 on DBLP) is concrete and checkable from public data. The LOL roster dataset is small but real. The paper is transparent about its own setup; the flaw is in how the results are framed.\n\nThe soft spots beyond the leakage: Table 2's modularity gains compare Q on the original graph with Qw on the reweighted graph, which are different quantities on different graphs. The captions admit each number is the best of 16 configurations, with no multiple-comparison correction. The correlation analysis (Table 5) uses six data points with p-values of 0.084 and 0.052—the abstract's \"statistically significant\" overstates even the borderline 0.052. No error bars, and code/data are promised only after acceptance. The novelty is modest: the features are standard link-prediction features, and prior work on using predicted similarities to reweight edges for community detection is not cited, so the leap claimed over the literature is overstated.\n\nWho this is for: someone working on self-supervised weighting for community detection, or anyone who wants a clean example of evaluation leakage for a methods class. It does not deserve acceptance as is; the empirical core of the paper would need to be rebuilt—proper unsupervised NMI/ARI, honest reporting of the 16 runs, error bars, code release. I would still send it to review rather than desk-reject, because the repair path is clear and the underlying question is worth a careful answer. But if a reviewer takes the abstract at face value, the paper misleads; the rejection should say exactly why.","headline":"Useful self-training idea, but the headline NMI/ARI gains come from training on the same labels used for scoring—the stress-test concern holds.","tokens_in":13411,"tokens_out":6048,"would_cite":false,"duration_ms":57568,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a machine-learning layer that weights node pairs by predicted same-community likelihood improves community detection on both synthetic and real networks, and that the largest accuracy gains appear when the layer is…","keywords":["community detection","node-pair similarity","ensemble learning","statistical physics","modularity","normalized mutual information","adjusted rand index","weighted similarity network"],"falsifier":"Hold out a set of ground-truth communities entirely from training, train the classifier only on the remaining labels, then evaluate NMI and ARI on the held-out communities; if the gain over the original algorithms disappears or shrinks to the statistical-physics mode's gain, the ground-truth mode's advantage comes from label reuse rather than from node-pair similarity.","tokens_in":12487,"feed_emoji":"🕸️","tokens_out":12925,"duration_ms":105697,"temperature":0.7,"pith_summary":"This paper tries to show that community detection improves when node-pair similarity is treated as a learnable signal and injected into standard detection algorithms. The proposed framework first obtains a preliminary partition of the network, or uses known community labels when those are available, then trains an ensemble of decision trees, random forests, and XGBoost to predict whether two nodes belong to the same community from three micro-level features. The predicted likelihoods become edge weights in a weighted similarity network, and four classic algorithms (Louvain, Leiden, Infomap, and Fast-Greedy) are re-run on that re-weighted graph. Across one artificial and five real-world networks, the paper reports higher modularity, normalized mutual information, and adjusted rand index than the same algorithms achieve on the original graph. The paper also reports a strong correlation between the internal-versus-external node-pair similarity gap and detection accuracy, and offers that correlation as the mechanism behind the gains.","feed_headline":"Trained node-pair weights improve community detection on six networks","feed_subtitle":"Ensemble-learned similarities feed standard detectors; labeled networks gain most in accuracy.","key_machinery":"The load-bearing mechanism is the weighted similarity network: node-pair likelihoods predicted by an ensemble of Decision Trees, Random Forests, and XGBoost, combined through voting and 5-fold cross-validation, are squared and used as edge weights, and the four community-detection algorithms are then applied to this weighted network. Three node-pair features carry the micro-level information: degree heterogeneity for sparse networks, clustering-coefficient heterogeneity, and common-neighbor count for dense networks. The squaring step converts a probabilistic same-community score into a symmetric similarity weight, which is the object that changes the input graph seen by the detector. In the statistical-physics mode, a preliminary community partition supplies the training labels; in the ground-truth mode, real labels replace that preliminary partition.","core_discovery":"The central claim is that micro-level node-pair similarity is a usable signal, not a side detail: a low-complexity machine-learning step that maps node-pair features to same-community likelihoods, then re-weights the network with the squared likelihoods, makes standard community detectors find partitions that score higher on modularity, NMI, and ARI. When ground-truth communities are known, training the classifier on them and then measuring accuracy against the same labels yields the most accurate recovery in the paper's experiments. When labels are unavailable, using one of the four baseline detectors to produce the training labels still improves modularity and, on several networks, improves NMI and ARI as well. The paper interprets the improvement as a bridge between local node interactions and global network organization, and supports the interpretation with a Pearson-correlation analysis linking the similarity gap to the evaluation metrics.","pith_inferences":["Because the re-weighting step only changes the input graph, the same wrapper could be applied to spectral, stochastic-block-model, or deep-learning community detectors; the paper demonstrates it only with the four classic algorithms.","The decision to square each predicted likelihood before treating it as an edge weight is unexamined; testing other monotone transformations (identity, square root, log) would reveal how much of the gain comes from that specific calibration.","The correlation evidence points to a direct optimization target—the internal-versus-external similarity gap—that could be trained as a loss function instead of a classification likelihood; the paper does not explore this.","The framework could be extended to attributed or temporal networks by adding node attributes or time-respecting similarity features, since the micro-feature set in the paper is purely topological."],"forward_implications":["Any of the four tested detectors—Louvain, Leiden, Infomap, or Fast-Greedy—can be improved by replacing the original graph with the weighted similarity network, so the reported gain is not tied to a single algorithm.","When ground-truth communities are available, the supervised mode gives the largest NMI and ARI improvements, with relative gains exceeding 125 percent on some networks.","The strong correlation between the internal-versus-external similarity gap and detection metrics implies that node-pair similarity is a meaningful predictor of community membership, not a randomly chosen feature.","The framework is low-complexity: it uses three simple topological features and standard ensemble classifiers, so the preprocessing cost stays modest on networks with hundreds of thousands of nodes."],"supporting_citations":[{"why":"It supplies the Leiden algorithm that serves as both a baseline and a component of the proposed framework.","marker":"[35]"},{"why":"It supplies the Infomap algorithm, the information-theoretic detector used as a baseline and in the statistical-physics mode.","marker":"[28]"},{"why":"It supplies the Fast-Greedy algorithm used as one of the four baseline detectors.","marker":"[3]"},{"why":"It motivates the tree-like benchmark network and the degree- and clustering-based node-pair features.","marker":"[29]"},{"why":"It supplies the tree-like network generation method and the related similarity features used in the experiments.","marker":"[30]"},{"why":"It provides the common-neighbor feature and the analysis of topological feature capability that the framework relies on.","marker":"[26]"},{"why":"It is the source of the Email, Amazon, DBLP, and YouTube network datasets used in the evaluation.","marker":"[16]"},{"why":"It defines the ground-truth community labels that the supervised mode trains on and that NMI and ARI are computed against.","marker":"[42]"},{"why":"It supplies the normalized mutual information metric used to score the detected communities.","marker":"[45]"}],"fun_headline_variants":["ML merges node-pair details into community structure for better detection","Micro-level similarities boost community detection when fused into mesoscale","Ensemble-learned similarities sharpen community detection results","Low-complexity ML adds micro details to mesoscale community detection","Statistical physics plus ML: better community partitions from node pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ground-truth mode assumes it is legitimate to train the node-pair classifier on the same known community labels that are later used to score the detected communities; if that reuse is information leakage, the reported accuracy gains are partly self-fulfilling.","fun_headline_variants_meta":{"raw":{"variants":["ML merges node-pair details into community structure for better detection","Micro-level similarities boost community detection when fused into mesoscale","Ensemble-learned similarities sharpen community detection results","Low-complexity ML adds micro details to mesoscale community detection","Statistical physics plus ML: better community partitions from node pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000656,"raw_usage":{"total_tokens":2987,"prompt_tokens":913,"completion_tokens":2074,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":1991}},"tokens_in":529,"tokens_out":2074,"duration_ms":13681,"temperature":1.0,"reasoning_tokens":1991,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:06:03.991195+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a set of ground-truth communities entirely from training, train the classifier only on the remaining labels, then evaluate NMI and ARI on the held-out communities; if the gain over the original algorithms disappears or shrinks to the statistical-physics mode's gain, the ground-truth mode's advantage comes from label reuse rather than from node-pair similarity.","supporting_citations":[{"cited_title":"Finding community structure in very large net- works","cited_arxiv_id":null,"evidence_quote":"It supplies the Fast-Greedy algorithm used as one of the four baseline detectors."},{"cited_title":"The maximum capability of a topological feature in link prediction","cited_arxiv_id":null,"evidence_quote":"It provides the common-neighbor feature and the analysis of topological feature capability that the framework relies on."},{"cited_title":"Snap datasets: Stanford large network dataset collection","cited_arxiv_id":null,"evidence_quote":"It is the source of the Email, Amazon, DBLP, and YouTube network datasets used in the evaluation."},{"cited_title":"Defining and evaluating network communities based on ground-truth","cited_arxiv_id":null,"evidence_quote":"It defines the ground-truth community labels that the supervised mode trains on and that NMI and ARI are computed against."},{"cited_title":"Evaluating accuracy of community detection using the relative normalized mutual information","cited_arxiv_id":null,"evidence_quote":"It supplies the normalized mutual information metric used to score the detected communities."}],"review_version":1}