{"id":"a5f270b2-7e31-4aae-afba-ed1faa2a343c","arxiv_id":"2507.18153","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GraphALP combines LLM-generated minority node oversampling with confidence-weighted pseudo-labeling to improve node classification on graphs that have both class imbalance and label noise.","lead":"This paper presents GraphALP, a graph augmentation method that uses a large language model to create synthetic minority-class nodes and a weighted pseudo-labeling step to handle noisy labels in class-imbalanced graphs. On four citation datasets, it reports gains in accuracy, F1, and G-mean over existing class-imbalance methods, which matters because real-world graphs often combine skewed classes with mislabeled nodes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pseudo-labeling may train on test-set nodes: Eq. (20) takes pseudo-labels from all unlabeled nodes, and the paper never excludes the test split before integrating them into the training set, so reported gains could be inflated.","rationale":"The reader's conditional verdict is a fair starting point, but the weakest_assumption they name, namely the Eq. (7) alignment between LLM embeddings and bag-of-words features, is not the most decisive risk. Even if the projection is imperfect, the shared MLP and reconstruction losses might provide enough implicit alignment for the method to work empirically. The more dangerous vulnerability is in the evaluation protocol: pseudo-labels are sourced from all unlabeled nodes and folded into the training set. In the transductive benchmarks used here, unlabeled nodes include test nodes; the paper gives no indication that test nodes are excluded from Eq. (20). If a test node is pseudo-labeled, the model's final predictions for that node are influenced by a training target derived from the model's own earlier predictions. This can produce optimistic measurements, particularly under class imbalance where high-confidence pseudo-labels concentrate in the majority class. It also creates an unfair comparison with baselines that do not use pseudo-labels. This is straightforwardly checkable from the code or by rerunning with an exclusion mask, which is why the verdict should remain CONDITIONAL rather than being hardened now. The feature-alignment concern is valid as a secondary limitation and should also be addressed, but the pseudo-label split issue is the one that can overturn the headline numbers.","tokens_in":11746,"tokens_out":9745,"duration_ms":102394,"concrete_test":"Inspect the released code or splits and check whether any node in the test split appears in the pseudo-labeled set YP from Eq. (20). Then rerun Table 1 with an added constraint that test nodes are never pseudo-label candidates, keeping all hyperparameters fixed; if ACC/G-mean drops materially or any baseline outperforms GraphALP, the reported superiority is an artifact of the evaluation protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the evaluation protocol for pseudo-labeling, not the feature-alignment assumption. In the Fine-tuning Module with Pseudo-label, Eq. (20) defines YP from YU, described as 'the pseudo-label predictions for the unlabeled nodes,' and the text then states 'we integrate pseudo labels into the training set' before reporting final results. The experimental section never states that test nodes are excluded from YU. On Cora, CiteSeer, and PubMed, standard transductive splits treat all non-training nodes as unlabeled, which includes the test set; for Wiki-CS the paper explicitly reserves 50% for testing after using 25% for training and validation. If high-confidence test nodes receive pseudo-labels and are used as training supervision, the reported ACC/F1/G-mean are computed on nodes whose predictions were directly optimized during training. This is not label leakage in the usual sense, but it is a self-training-on-the-evaluation-set protocol that can inflate accuracy, especially when the confidence threshold selects easy majority-class nodes, and it makes the comparison to baselines that do not pseudo-label unlabeled nodes unfair. The central empirical claim therefore rests on an unverified split assumption.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies node classification on graphs that simultaneously exhibit class imbalance and label noise, a setting the authors argue is underexplored. They propose GraphALP, a framework with three components: LLM-based oversampling of minority nodes (with SentenceBERT-style embedding and MLP projection), a self-supervised pre-training module (AE/GAE) on the augmented class-balanced graph, and a fine-tuning module that assigns pseudo-labels to high-confidence unlabeled nodes and uses them as additional training supervision, followed by a second LLM-based rebalancing step. Experiments on Cora, CiteSeer, PubMed, and Wiki-CS report consistent improvements in ACC, F1, and G-mean over several class-imbalance baselines, including gains of up to 8.03 G-mean points over GraphDAO. The paper also includes robustness studies varying imbalance and noise ratios, ablations, and a cost comparison.","tokens_in":12040,"tokens_out":3801,"duration_ms":42577,"significance":"The problem addressed—joint class imbalance and label noise on graphs—is practically important and relatively underexplored, so a well-validated method would be a useful contribution. The design of using LLM-generated synthetic minority nodes is interesting and avoids some limitations of feature-space oversampling. The pseudo-labeling with dynamic weighting is also a reasonable mechanism. However, the empirical evidence as presented is not fully convincing: the main claim rests on averaged results without error bars or significance tests, and the pseudo-labeling protocol risks training on the evaluation set. If those issues are resolved, the contribution could be significant, but at present the central empirical claim is not yet established.","major_comments":[{"comment":"The paper never states that test nodes are excluded from the set YU used for pseudo-label selection. Under the standard transductive splits for Cora, CiteSeer, and PubMed, all non-training nodes—including the test split—are typically treated as unlabeled. If high-confidence test nodes are assigned pseudo-labels and then integrated into the training set (as stated after Eq. (22)), the reported ACC/F1/G-mean are computed on nodes whose predictions were directly optimized during training. This can inflate the reported gains and makes the comparison to baselines unfair. The authors must specify the exact split used for YU and rerun all experiments with the test split explicitly excluded from pseudo-labeling. If the results change materially, the conclusions must be revised.","section":"Fine-tuning Module with Pseudo-label, Eq. (20)"},{"comment":"All reported numbers in Table 1 are averages over 5 runs without standard deviations, confidence intervals, or statistical significance tests. The claimed improvements range from about 1 to 8 percentage points, which may be within run-to-run variation for GNNs on small graph datasets. The authors should report per-seed results with error bars and apply pairwise significance tests (e.g., paired t-test or Wilcoxon signed-rank test) between GraphALP and each baseline, especially for the headline G-mean gains.","section":"Experiments, Table 1"},{"comment":"The text in \"Influence of Label Noise Ratio\" states: \"As the label noise ratio decreases, the performance of all methods deteriorates.\" This contradicts the expected and subsequently described behavior: lower noise should improve performance, and the later statement that at noise ratio 0.1 GraphALP beats GraphDAO by 1.55% while at 0.5 the gap is 9.78% implies the opposite trend. This is a load-bearing inconsistency in the robustness narrative and must be corrected and the corresponding numbers verified.","section":"Robust Analysis, Figure 4"},{"comment":"Several free hyperparameters that directly control the method's behavior are not reported: the edge similarity threshold tau in Eq. (11), the pseudo-label confidence threshold tau in Eq. (20), the loss weights alpha, beta, gamma in Eq. (21), and the exact oversampling scale used for each dataset (e.g., Wiki-CS). Without these values, the method cannot be reproduced, and the sensitivity of the reported gains to these thresholds is unknown. The authors should provide the values and include a sensitivity analysis for at least the two thresholds and the loss weights.","section":"Experiment Setup and Loss Function, Eqs. (11), (20), (21)"}],"minor_comments":[{"comment":"The claims \"first explicit study\" and \"first work to tackle these challenges via LLM\" are too strong given existing LLM-based augmentation for imbalanced node classification (e.g., Wang et al. 2024a, cited in the paper) and the existence of noisy-label benchmarks like NoisyGL. Please soften the novelty statements and clearly position against these works.","section":"Introduction and Related Work"},{"comment":"In Eq. (22), the notation yP_i is used without explicitly defining it as the one-hot encoding of the selected pseudo-label from YP. Please define all symbols and specify how the pseudo-label loss term is masked for nodes that are not selected.","section":"Fine-tuning Module with Pseudo-label, Eq. (22)"},{"comment":"The ablation names Ours-PL, Ours-RB, Ours-W are not fully consistent with the text: the text says removing (1) pseudo-labeling module, (2) rebalancing module, and (3) weighted cross-entropy loss, but the table caption uses RB and W without defining which is which. Please clarify the definitions and make the table self-contained.","section":"Ablation Study, Table 3"},{"comment":"There are multiple typos and formatting artifacts: \"effciency\" appears twice, Table 1 has \"0.3921\" instead of \"39.21\", and equations contain repeated letters such as \"XXX\", \"AAA1\", and \"ZZZ\". These should be cleaned up before submission.","section":"Throughout"},{"comment":"The captions do not specify what is being visualized or how the reported noise ratio is computed. For Figure 6, please state the exact procedure used to compute the noise ratio after pseudo-labeling and whether the visualization is on training or test nodes.","section":"Case Study and Visualization, Figures 5 and 6"}],"recommendation":"major_revision","confidential_remarks":"The most consequential issue is the pseudo-label evaluation protocol: if test nodes were included in the pseudo-labeled training set, the central empirical claim is invalid and the paper would need a fundamental re-evaluation. The authors must clarify this before the paper can be considered. Even if test nodes were excluded, the lack of statistical significance testing and the contradictory robustness statements are serious. The novelty relative to existing LLM augmentation for imbalanced graphs is modest, and the paper would benefit from stronger positioning and additional baselines that combine label-noise handling with class-imbalance methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: GraphALP is a coherent engineering combination of LLM-based minority oversampling and pseudo-labeling for node classification under simultaneous class imbalance and label noise. The problem is real and the framework makes sense, but the empirical claim of consistent superiority is under-supported: the pseudo-label evaluation protocol is ambiguous in a way that could inflate results, and the experimental reporting is too thin.\n\nWhat is actually new: the joint problem formulation is underexplored, and the specific pipeline—LLM-generated synthetic minority nodes, self-supervised pre-training on the balanced graph, then confidence-thresholded pseudo-labeling with reweighted loss and a second LLM oversampling step to fix pseudo-label-induced skew—is a legitimate new combination. The ablation study suggests each component contributes. The gains over GraphDAO are consistent across four datasets, which is worth taking seriously.\n\nSoft spots, in rough order of importance. First, the pseudo-label protocol. Eq. (20) takes pseudo-labels from all “unlabeled nodes” and the text says they are integrated into the training set; the experimental section never states that the test split is excluded from that pool. In transductive splits of Cora/CiteSeer/PubMed, test nodes are unlabeled, so if they receive pseudo-labels and are used as supervision, the reported ACC/F1/G-mean are computed on nodes the model was explicitly trained to predict. That would make the comparison to baselines (which do not pseudo-label unlabeled nodes) unfair. This is a load-bearing ambiguity, not a minor omission. Second, Table 1 reports averages over 5 runs without error bars or significance tests, so the 2–8 point gaps could be noise. Third, no combined baseline: we don’t see GraphDAO or GraphMixup augmented with pseudo-labeling, or a direct comparison to Wang et al. 2024a’s LLM augmentation. Fourth, key hyperparameters (confidence threshold τ, loss weights α/β/γ, oversampling scale) and LLM prompts are not reported, which hurts reproducibility.\n\nOn circularity: the pseudo-labeling module uses the model’s own confidence to select training labels, which can reinforce initial errors. The LLM oversampling is external and reduces that burden, but the self-training loop remains a concern. It’s not fatal, but it should be discussed.\n\nBottom line: This paper is for graph learning researchers working on imbalance or noisy labels. It deserves a serious referee—the problem is relevant, the method is coherent, and the results are promising. But it needs a clear statement of the split protocol, stronger statistics, and more thorough baselines before the central claim is accepted.","headline":"A coherent LLM+pseudo-label pipeline for a real problem, but the evaluation protocol is ambiguous enough that the headline gains may not be real.","tokens_in":12519,"tokens_out":4336,"would_cite":true,"duration_ms":42427,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GraphALP proposes that class-imbalanced graphs with noisy labels are best handled by LLM-generated minority nodes, self-supervised pre-training on the balanced graph, and reweighted pseudo-labeling followed by a second LLM oversampling…","keywords":["graph node classification","class imbalance","label noise","graph augmentation","large language models","pseudo-labeling","oversampling","robust learning"],"falsifier":"Train a discriminator to separate projected LLM-generated minority embeddings from real minority node features; if the two sets are cleanly separable, the alignment assumption fails and the reported oversampling gains cannot be attributed to semantic fidelity.","tokens_in":11576,"feed_emoji":"📈","tokens_out":11902,"duration_ms":110518,"temperature":0.7,"pith_summary":"Class-imbalanced graphs in the real world tend to arrive with noisy labels, yet prior graph methods typically tackle one of those problems and assume the other away. This paper proposes GraphALP, a single pipeline that treats both at once: an LLM writes realistic text for minority nodes with reliable labels, a sentence-embedding model and an MLP projection insert those synthetic nodes into the graph, and a confidence-based pseudo-labeling stage with class reweighting cleans up residual noise. The paper reports consistent gains in ACC, F1, and G-mean over state-of-the-art class-imbalance baselines on Cora, CiteSeer, Pubmed, and Wiki-CS, including G-mean improvements of 8.03 and 6.03 points over GraphDAO. If these results hold, graph classifiers no longer have to choose between fixing skewed classes and tolerating mislabeled nodes.","feed_headline":"LLM-made nodes lift graph accuracy under noise and skew","feed_subtitle":"LLM-generated minority nodes plus pseudo-label rebalancing lift G-mean by up to 8 points on four benchmarks.","key_machinery":"The load-bearing object is the two-stage augmentation loop. In stage one, LLM-written minority text is turned into node representations by jina-embedding-v3 and aligned through an MLP projection with the original node attributes; the resulting class-balanced graph is encoded by a GraphSage autoencoder whose reconstruction losses include attribute reconstruction, structural reconstruction, and an edge predictor that wires synthetic nodes to similar real nodes. In stage two, pseudo-labels are selected only when confidence exceeds a threshold and the cross-entropy loss uses class weights $w_{C_i}=\\max(1,N/n_i)$, so minority nodes are neither mistaken for noise nor starved of gradient. A second LLM oversampling pass rebalances the pseudo-label-skewed distribution. The mechanism that carries the argument is the combination: accurate synthetic minority labels lower the noise ratio at the source, and then rebalanced pseudo-labels keep the classifier from regressing toward the majority.","core_discovery":"GraphALP's central claim is that the interaction between class imbalance and label noise is the actual obstacle, and a coordinated augmentation loop can remove both. Synthetic minority nodes are generated by prompting an LLM to produce title-and-abstract style text for the minority classes, embedded with jina-embedding-v3, and projected by an MLP to match the original feature space; this class-balanced graph is pre-trained with an autoencoder and GraphSage before fine-tuning with a dynamically weighted cross-entropy loss that selects high-confidence pseudo-labels. A second LLM oversampling step then compensates for the majority-class skew that pseudo-labeling itself introduces. Across four real datasets with controlled imbalance and uniform label noise, the paper reports that GraphALP outperforms all baselines on ACC, F1, and G-mean, with the largest gains concentrated in balanced metrics like G-mean.","pith_inferences":["If the alignment trick generalizes, LLM oversampling could replace interpolation-based synthesis on any text-attributed graph, including knowledge graphs or item-description recommendation graphs, which this paper does not evaluate.","A cheap extension would be to report pseudo-label noise after each stage on all datasets, which would separate the contribution of label cleaning from class rebalancing.","Because the LLM's knowledge is domain-bound, synthetic nodes for specialized vocabularies such as legal or biomedical text may drift off target; holding the pipeline fixed while varying the prompt template or generator would reveal how much of the gain depends on the specific LLM.","The two-stage rebalancing recipe can be bolted onto existing noisy-label GNNs: select confident pseudo-labels, measure the class skew they create, then oversample the shrunken classes."],"forward_implications":["On the four benchmark graphs, GraphALP reports the highest ACC, F1, and G-mean among all tested baselines, with the largest G-mean advantage over GraphDAO being 8.03 points on Cora and 6.03 points on Wiki-CS.","The pseudo-labeling stage cuts the measured label noise ratio on Cora from 30.23% to 10.15%, so most of the supervision the GNN finally trains on is cleaner than the original labels.","In robustness sweeps, GraphALP stays ahead as both imbalance and noise get worse, and its edge over GraphDAO on CiteSeer grows from 1.55 points at 10% noise to 9.78 points at 50% noise.","The reported runtimes are lower than GraphMixup and GraphDAO on Cora, CiteSeer, and Wiki-CS, so the combined pipeline does not buy robustness at the cost of prohibitive training time."],"supporting_citations":[{"why":"the class-imbalance graph oversampling baseline that GraphALP must beat; its settings are reused from the original paper and it anchors the main comparison.","marker":"GraphDAO (Xia et al. 2024b)"},{"why":"the self-supervised graph oversampling baseline whose official parameter settings are used for all shared baselines.","marker":"GraphMixup (Wu et al. 2022)"},{"why":"the classic interpolation-based oversampling method whose failure to handle noisy minority nodes motivates LLM-based synthesis.","marker":"SMOTE (Chawla et al. 2002)"},{"why":"the LLM that writes synthetic minority node text and supplies labels for those nodes.","marker":"DeepSeek-AI et al. 2024"},{"why":"the sentence-embedding model used to turn LLM-generated node text into initial node representations.","marker":"Sturua et al. 2024"},{"why":"defines the uniform label-noise injection protocol used in the main and robustness experiments.","marker":"Song et al. 2023"},{"why":"pseudo-labeling work that motivates the confidence-based label-cleaning stage and represents the noisy-label baseline class that GraphALP extends.","marker":"Xia et al. 2024a"}],"fun_headline_variants":["LLM-generated nodes fix noisy, imbalanced graphs","GraphALP: LLM oversampling beats noise and skew","Synthetic minority nodes via LLM lift balanced accuracy","LLM plus pseudo-labels tame noisy graph imbalance","Boosting G-mean: LLM-made nodes counter noise and skew"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire pipeline depends on the assumption that after an MLP projection, LLM-written synthetic node texts land in the same feature space as the graph's real node attributes, so that oversampling with them improves minority-class learning instead of injecting out-of-distribution examples.","fun_headline_variants_meta":{"raw":{"variants":["LLM-generated nodes fix noisy, imbalanced graphs","GraphALP: LLM oversampling beats noise and skew","Synthetic minority nodes via LLM lift balanced accuracy","LLM plus pseudo-labels tame noisy graph imbalance","Boosting G-mean: LLM-made nodes counter noise and skew"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1463,"prompt_tokens":915,"completion_tokens":548,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":531,"tokens_out":548,"duration_ms":5258,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:38:25.163835+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a discriminator to separate projected LLM-generated minority embeddings from real minority node features; if the two sets are cleanly separable, the alignment assumption fails and the reported oversampling gains cannot be attributed to semantic fidelity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the uniform label-noise injection protocol used in the main and robustness experiments."}],"review_version":1}