{"id":"7dc954ef-0733-43cb-b3d6-b80d24312c32","arxiv_id":"2411.16787","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"ConNHS, a multi-graph contrastive learning method with neighbor hierarchical sifting, reports accuracy 95.86%, 97.52%, 87.43%, and 70.65% on ThuCNews, SogouNews, 20NG, and Ohsumed.","lead":"This paper builds graphs that link text documents through their titles, keywords, and events, and trains a semi-supervised classifier with a contrastive loss that avoids treating similar documents as negative examples. It reports accuracy gains of 0.30 to 2.12 points over eight baselines on four text classification datasets, which matters for applications with scarce labels.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains are confounded: ConNHS uses BGE-M3 node and edge features while baselines do not, so the claimed superiority in Table 2 is not attributable to the proposed components.","rationale":"The paper's stated contribution is a method, not a new encoder, so a fair test must hold input features fixed across methods. Currently, ConNHS's advantage could come entirely from BGE-M3, and the ablations do not control for this. I also note Eq. (10) as printed sums terms h(x_j - x_j) * (x_j - x_j) = 0, which would nullify RW-GCN; this is likely a typo for x_j - x_i, but it must be corrected. Hyperparameter sensitivity is reported on test accuracy, no variance or significance is given, and the comparison omits the authors' closely related cluster-refined negative-sampling preprint. The reader's stated weakest assumption, homophily, is less decisive because the NHS-gs ablation shows that including first-order neighbors as negatives hurts accuracy, which is indirect evidence that graph homophily holds sufficiently on these benchmarks. The encoder confound is therefore the most load-bearing concern. These issues do not refute the idea, but they make acceptance impossible without revision, so the conditional verdict is retained.","tokens_in":23281,"tokens_out":6910,"duration_ms":98016,"concrete_test":"Re-run Table 2 with encoder-matched baselines: initialize HAN, RGCN, and TGNCL with the same BGE-M3 document vectors and the same title, keyword, and event adjacency matrices used by ConNHS, keeping their original training objectives and using validation-based hyperparameter selection, then compare final test accuracy and F1. If the best encoder-matched baseline approaches or exceeds ConNHS, the central claim fails; if ConNHS still leads by the same margins, the encoder confound is ruled out.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 4.4.1 and Table 2 is that ConNHS outperforms eight baselines, but the comparison is not feature-matched. Section 3.1 and Section 4.3 state that document nodes are initialized with BGE-M3 embeddings, and the graph edges (Eqs. 3-9) are built from BGE-M3 cosine similarities over titles, keywords, and events. The baselines are reported with their standard features, which are typically bag-of-words, shallow, or task-specific representations; BERT-large is the only pretrained-transformer baseline, and even it is not initialized with BGE-M3. The reported margins are small (1.12, 0.30, 1.51, and 2.12 points), so the stronger encoder alone could explain the gap. Table 4 does not fix the attribution: all ConNHS ablations share the BGE-M3 initialization, so they only show that NHS beats NT-Xent within that encoder. Unless the baselines are rerun with the same BGE-M3 features and graph (or ConNHS is rerun with baseline features), the claimed superiority cannot be assigned to the proposed RW-GCN, CGAN, or NHS components.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ConNHS, a semi-supervised text classification method that constructs a multi-relational document graph from titles, keywords, and events, separates it into semantic subgraphs, and learns node representations via a relation-aware graph convolutional network (RW-GCN) with cross-graph attention (CGAN). The contributions are an augmentation-free contrastive learning setup and a neighbor hierarchical sifting (NHS) loss that masks first-order neighbors and similar high-order neighbors from negative sampling. Experiments on ThuCNews, SogouNews, 20NG, and Ohsumed report accuracies of 95.86%, 97.52%, 87.43%, and 70.65%, which the authors claim outperform eight baselines.","tokens_in":23494,"tokens_out":4103,"duration_ms":38505,"significance":"The conceptual direction is interesting: replacing explicit graph augmentation with relation-specific subgraphs, and refining negative-sample selection using graph structure plus attribute similarity, are both reasonable ideas that could contribute to graph-based semi-supervised text classification. The paper also provides a fairly detailed description of the graph construction pipeline and a large ablation study. However, the central empirical claim is currently not supported because the comparison in Table 2 is not feature-matched and because key hyperparameters appear to be tuned on test accuracy. The manuscript also contains a core equation (Eq. 10) that, as printed, is identically zero. If these issues are fixed with a properly validated, feature-matched comparison, the method could be a useful contribution.","major_comments":[{"comment":"Eq. (10) as printed defines the RW-GCN aggregation operator as h(x_j^l - x_j^l; θ_h) · (x_j^l - x_j^l), which is identically zero for every neighbor. The accompanying prose states that the input should be the difference (x_j^l - x_i^l), so this is evidently a typographical error, but as written the core component of the method cannot be implemented. The equation must be corrected and the notation for the neighbor set consistently defined before the method is reproducible.","section":"§3.3, Eq. (10)"},{"comment":"The hyperparameter analysis in Section 4.6 selects the similarity thresholds ρ_t, ρ_e, ρ_k, the association coefficients γ_e, γ_k, and the temperature τ per dataset by directly maximizing the reported accuracy on the test set (e.g., \"the optimal ρ_t is 0.7\", \"ConNHS achieved the best performance across all four datasets with these settings\"). If these choices are made on the test data and then used for the final numbers in Table 2, the reported accuracies are optimistic selection results rather than honest test-set performance. The authors must use a held-out validation split for hyperparameter selection and report only the corresponding test numbers.","section":"§4.6, Figures 4-6, Table 5"},{"comment":"The comparison in Table 2 is confounded: ConNHS initializes node features with BGE-M3 embeddings and constructs graph edges from BGE-M3 cosine similarities over titles, keywords, and events, while the eight baselines use their standard, weaker features (bag-of-words, word vectors, or BERT-large in one case). The reported accuracy margins (1.12, 0.30, 1.51, and 2.12 points) are small enough that the stronger encoder alone could explain the observed gains. The ablation study in Table 4 does not resolve this, because all ConNHS variants share the BGE-M3 initialization and graph. To attribute the improvement to RW-GCN, CGAN, or NHS, the authors need to rerun baselines with the same BGE-M3 features and graph, or run ConNHS with the baseline feature sets.","section":"§3.1, §4.3, Table 2"},{"comment":"The NHS loss relies on the homophily assumption stated in Section 4.5, i.e., that connected document nodes tend to share the same label. This assumption is load-bearing because first-order neighbors are masked from being negatives, and the graph edges are created from high feature similarity. The paper provides no quantitative homophily analysis (e.g., the fraction of edges connecting same-label documents) and no test on heterophilic or high-noise graphs, even though the conclusion acknowledges that the multi-relational graphs \"inevitably contain some noisy edges.\" A homophily/edge-noise sensitivity analysis is needed to delimit when the proposed negative-sifting mechanism is beneficial.","section":"§4.5, §5"}],"minor_comments":[{"comment":"The subsection heading \"Intra-graph propagation\" appears twice; the second occurrence, which introduces cross-graph attention, should be \"Inter-graph propagation.\"","section":"§3.3"},{"comment":"The vector k in the attention formula α_r = softmax(k^T tanh(p(x_{i,r}; θ_p))) is not defined; please specify its dimension and whether it is a learned parameter or an input feature.","section":"Eq. (12)"},{"comment":"After concatenation, the claim that x_i^{l+1} ∈ R^{2×d} is ambiguous; the concatenation of two d-dimensional vectors should lie in R^{2d}, unless a specific reshaping is intended.","section":"Eq. (11)"},{"comment":"The text refers to \"TGNLCL\" but the comparison method is named TGNCL; please make the notation consistent.","section":"§4.4.1"},{"comment":"There are numerous spacing and grammatical errors, e.g., \"methodofcontrastivemulti-graphlearning\" in the abstract and \"the contrastive loss suffer from false negatives\"; a careful proofreading pass is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper cites a large number of its own arXiv preprints, several of which appear tangential to the present work; this should be trimmed to published, directly relevant references. The central empirical claim is currently under-supported because of the feature mismatch and the test-set hyperparameter selection, but both issues are fixable with additional experiments. I do not see evidence of intentional misconduct, only methodological weakness in the evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible, incremental combination of known graph-contrastive components, and the authors did real ablation work. But as submitted, the reported gains over baselines are confounded by the use of BGE-M3 node and edge features, and the hyperparameter choices look like they were made on test accuracy. Eq. (10) as printed cancels to zero—almost certainly a typo, but it must be fixed before the method is implementable.\n\nWhat is actually new: the specific pipeline—title/keyword/event multi-relational graph, relation-aware intra-graph propagation (RW-GCN), cross-graph attention fusion (CGAN), and neighbor hierarchical sifting (NHS) loss—is a fresh combination, even though each ingredient exists in the literature. The paper is clearly written, and the ablations in Table 4 usefully separate structure-guided from attribute-guided negative masking. Reporting 10-run averages is good practice.\n\nThe soft spots are serious and mostly correctable. First, the BGE-M3 confound: the node embeddings and the graph edges are both built from BGE-M3 cosine similarities, while the baselines use their standard (usually bag-of-words or shallow) features. The margins in Table 2 are small enough (0.30–2.12 points) that the stronger encoder alone could explain them. Table 4 cannot fix this, because every ablation shares the BGE-M3 initialization. The authors need to run baselines with BGE-M3 features, or run ConNHS with baseline features, to isolate the contribution. Second, the sensitivity analysis in Section 4.6 and Figures 4–6 appears to select thresholds and temperature by maximizing test accuracy; there is no validation split and no significance testing. Third, they cite their own closely related arXiv preprint on cluster-refined negative sampling (Ai et al., 2024b) but do not compare with it or explain the relationship—this is an obvious missing baseline. Fourth, the homophily assumption is acknowledged but never stress-tested on heterophilic data; the paper itself notes the graph may contain noisy edges, but that point is not explored.\n\nIf these issues are addressed—a corrected Eq. (10), feature-matched baselines, validation-based tuning, and a comparison with the related preprint—the central claim could hold. As is, the evidence is not sufficient to accept the reported superiority.\n\nI would send this to peer review, expecting major revision. The core idea is worth the referee's time, but the experiments need to be redone.","headline":"Plausible incremental method, but the BGE-M3 confound and test-set tuning undercut the headline gains; worth a careful revision.","tokens_in":24106,"tokens_out":3698,"would_cite":false,"duration_ms":62976,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Building a multi-relational document graph from titles, keywords, and events, and sifting false negatives out of contrastive learning, improves semi-supervised text classification on four datasets.","keywords":["semi-supervised text classification","graph contrastive learning","multi-relational text graph","negative sample selection","false negatives","graph neural networks"],"falsifier":"Run ConNHS on a dataset where class labels cut across strong surface similarity, such as 'stock' appearing in finance and real-estate news, and compare NHS against NT-Xent; if ConNHS loses its advantage or falls below NT-Xent, the homophily-based masking is removing true negatives rather than false ones.","tokens_in":23024,"feed_emoji":"🔗","tokens_out":9625,"duration_ms":77971,"temperature":0.7,"pith_summary":"This paper claims that semi-supervised text classification can be improved by linking documents through titles, keywords, and events and then training graph contrastive learning without graph augmentation. To reduce false negatives, it introduces a neighbor hierarchical sifting (NHS) loss that removes first-order neighbors and high-similarity high-order neighbors from the negative set. The resulting ConNHS method reports accuracy of 95.86%, 97.52%, 87.43%, and 70.65% on ThuCNews, SogouNews, 20 Newsgroups, and Ohsumed, with gains from 0.30 to 2.12 accuracy points over the best baseline on each dataset. It also reports 70.21% accuracy on 20NG with only 1% of labels available. If these results hold, the main bottleneck in graph contrastive text classification is negative-sample selection and semantic graph construction, not view generation.","feed_headline":"Graph contrastive learning without augmentation beats baselines","feed_subtitle":"Titles, keywords, and events link documents; a neighbor-sifting loss cuts false negatives and lifts accuracy on four datasets.","key_machinery":"The load-bearing mechanism is the Neighbor Hierarchical Sifting (NHS) loss, a contrastive loss whose negative set is pruned twice: graph structure masks first-order neighbors of the anchor and its positives, and node attributes exclude high-order neighbors with high similarity to the anchor in the fused embedding space. The multi-relational text graph is the second essential mechanism; its title, keyword, and event edge types provide the diversity that replaces augmentation and the structure signal that NHS relies on. The paper attributes the accuracy gains to these two parts, with ablations showing that switching NHS to NT-Xent costs 3.33–5.55 accuracy points, dropping the structure signal costs 1.03–1.96, and dropping the attribute signal costs smaller amounts.","core_discovery":"The central claim is that separating a multi-relational text graph into relation-specific subgraphs gives contrastive views without the semantic distortion of random edge dropping or feature masking, and that a sifted negative set makes those views useful. ConNHS builds document-to-document edges from title, keyword, and event similarities, propagates within each subgraph with a relation-aware GCN that weights edges, fuses the subgraphs with cross-graph attention, and trains contrastively with a loss that masks first-order neighbors of the anchor and its positives and also excludes high-order neighbors whose fused representations are highly similar to the anchor. Ablation results show that replacing NHS with NT-Xent lowers accuracy on every dataset, and that removing either the structure-guided or the attribute-guided sifting signal also lowers accuracy.","pith_inferences":["Part of the reported gain may come from the stronger text encoder used for initial node representations rather than from the graph construction and NHS loss; a direct test would initialize all baselines with the same encoder.","The homophily-based masking is a liability on datasets where similar wording spans multiple classes; on such heterophilous text, NHS could mask true negatives and put ConNHS below NT-Xent.","The graph-construction thresholds and association coefficients are tuned per dataset, so a testable extension is to learn them or make the relation weights adaptive instead of fixed.","Because the attribute-guided sifting uses the fused representations themselves to decide exclusions, the loss can be circular in principle: if the encoder separates classes poorly in early training, similarity-based exclusion may reinforce the error."],"forward_implications":["Graph contrastive text classification can avoid augmentation entirely, because separating a multi-relational graph into relation-specific views preserves structural information and removes the need to tune augmentation parameters.","False negatives, not just view quality, limit contrastive learning for text, since pruning negatives by graph structure and embedding similarity sharpens cluster boundaries and lifts downstream accuracy.","Document graphs built from titles, keywords, and events provide richer supervision than word-document graphs built from PMI and TF-IDF, because they connect documents through latent semantic content.","The method remains effective in label-scarce regimes, with 70.21% accuracy on 20NG at a 1% label rate, well above Bi-LSTM and TextGCN.","Each sifting signal contributes independently, and the structure-guided mask contributes more than the attribute-guided exclusion in the reported ablations."],"supporting_citations":[{"why":"Defines the corpus-level word-document graph construction (PMI/TF-IDF) and serves as a graph baseline that ConNHS extends and compares against.","marker":"[Yao et al., 2019]"},{"why":"Introduces multi-graph text representation with intra-graph and inter-graph propagation, which ConNHS refines with relation-aware and attention-based propagation.","marker":"[Liu et al., 2020]"},{"why":"Provides the graph-contrastive text classification baseline whose NT-Xent-style negative selection and augmentation use ConNHS targets.","marker":"[Zhao and Song, 2023]"},{"why":"Supplies the BGE-M3 text embeddings used to initialize document nodes and to embed titles, keywords, and events.","marker":"[Chen et al., 2024]"},{"why":"Supplies the spectral convolution layer that RW-GCN adapts by weighting neighbor messages with edge information.","marker":"[Kipf and Welling, 2016]"},{"why":"Represents the contrastive text classification method that uses adaptive augmentation and NT-Xent loss, illustrating the false-negative problem ConNHS addresses.","marker":"[Yang et al., 2022b]"}],"fun_headline_variants":["Hierarchical neighbor sifting cuts false negatives in contrastive learning","Relation-specific views from multi-graph learning without augmentation","ConNHS: sifted neighbors improve semi-supervised text classification","Graph contrastive learning without augmentation via hierarchical sifting","Neighbor sifting loss reduces false negatives in contrastive text learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that documents joined by similar titles, keywords, or events tend to carry the same label, so removing those neighbors from the negative set helps; if similar-feature documents often belong to different classes, the graph edges and the masking would mislead learning instead of helping.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical neighbor sifting cuts false negatives in contrastive learning","Relation-specific views from multi-graph learning without augmentation","ConNHS: sifted neighbors improve semi-supervised text classification","Graph contrastive learning without augmentation via hierarchical sifting","Neighbor sifting loss reduces false negatives in contrastive text learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000499,"raw_usage":{"total_tokens":2479,"prompt_tokens":1020,"completion_tokens":1459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":636,"completion_tokens_details":{"reasoning_tokens":1375}},"tokens_in":636,"tokens_out":1459,"duration_ms":11009,"temperature":1.0,"reasoning_tokens":1375,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:24:37.876128+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ConNHS on a dataset where class labels cut across strong surface similarity, such as 'stock' appearing in finance and real-estate news, and compare NHS against NT-Xent; if ConNHS loses its advantage or falls below NT-Xent, the homophily-based masking is removing true negatives rather than false ones.","supporting_citations":[{"cited_title":"Textgcl:Graphcontrastivelearningfortransduc- tivetextclassification,in:2023InternationalJointConferenceonNeural Networks (IJCNN), IEEE","cited_arxiv_id":null,"evidence_quote":"Provides the graph-contrastive text classification baseline whose NT-Xent-style negative selection and augmentation use ConNHS targets."}],"review_version":1}