Pith. sign in

REVIEW 3 major objections 6 minor 84 references

Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces BeGIN, a graph label-noise benchmark, and argues that LLM-generated, instance-dependent mislabels hurt GNNs more than the class-dependent noise used in prior work, with node-specific center/neighbor parameterization…

desk verdict A genuinely useful graph-label-noise benchmark that deserves citation, but the headline 'LLM noise is hardest' result is partly an artifact of hard-example selection in the LLM-Refined construction. read the letter →

arxiv 2506.12468 v2 pith:BDFAXJTY submitted 2025-06-14 cs.LG cs.AIstat.ML

Suyeon Kim , SeongKu Kang , Dongwoo Kim , Jungseul Ok , Hwanjo Yu This is my paper
classification cs.LGcs.AIstat.ML
keywords labelnoisegraphneuralnetworksnodeclassificationinstance-dependentLLM-basedsimulationnoisydetectionnoise-robustlearningbenchmarkdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds BeGIN, a benchmark that injects six kinds of label noise into ten graph datasets, including three algorithmic instance-dependent corruptions (topology-, feature-, and confidence-based) and one LLM-based simulation meant to mimic human annotation errors. It argues that real-world mislabeling is instance-dependent, not class-dependent, and that prior graph label-noise research has therefore been testing on an unrealistically easy noise model. Across architectures, detectors, and more than twenty robust learning methods, the paper finds LLM noise consistently hardest, with corruption concentrated on a few contextually plausible wrong classes and on the same high-uncertainty nodes again and again. The paper's main design insight is that separating the parameters for the center node from those for its neighbors, as GraphSAGE does, curbs error propagation and gives the most reliable average gains.

What carries the argument

The machinery is a two-stage label-corruption pipeline. First, transition probabilities are computed per node: for topology noise via Personalized PageRank weights over neighbor labels, for feature noise via cosine similarity between node features and class mean representations, for confidence noise via a trained GNN's output distribution, for class-dependent noise via the standard uniform or pairwise class-flip matrices, and for LLM noise via GPT-4o-mini's classifications of the node text, refined to retain only cases where both naive and reasoned LLM outputs disagree with the ground truth. Second, Algorithm 1 selects nodes for corruption in proportion to their total off-diagonal transition mass and samples the new label from the renormalized transition row. The central identity doing analytic work is the correlation between the entropy of off-diagonal transition probabilities and downstream accuracy: lower transition entropy, meaning concentrated wrong-class choices, predicts worse performance, and LLM noise has the lowest off-diagonal entropy among instance-dependent noises.

What would settle it

Take any BeGIN dataset with LLM noise and permute the wrong-class targets of the corrupted nodes so that transitions are spread uniformly over all classes at the same noise rate; if GNN accuracy recovers to the uniform-noise level, the 'structured, concentrated transitions' mechanism is the cause of LLM noise's difficulty. To test realism, replace GPT-4o-mini's labels with human annotators' labels on the same nodes and check whether humans mislabel the same nodes to the same classes; if agreement is no better than chance, the benchmark's external validity as a human-noise proxy fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that instance-dependent label noise, especially LLM-generated noise, is a substantively harder and more realistic failure mode for GNNs than the class-dependent uniform or pairwise noise previously used in the graph label-noise literature. Equality of noise rates is the control: LLM-refined labels are applied at the same rates as all other noise types, yet they produce the largest accuracy drops. The paper traces this to structured mislabeling: transitions concentrate on a few contextually plausible classes, and the same nodes tend to be corrupted repeatedly, correlating with the model's prediction entropy. The paper further claims these patterns are what make many existing label-noise methods underperform, and that GraphSAGE's separate center/neighbor parameterization helps by limiting error propagation during aggregation. The benchmark itself, ten datasets spanning citation, e-commerce, co-purchase, and webpage graphs with homophily ratios from 0.07 to 0.81, is the supporting apparatus that makes these comparisons possible.

Load-bearing premise

The entire comparison treats the LLM-refined noisy labels, labels that both naive and reasoned GPT-4o-mini runs get wrong, as a faithful proxy for real human annotation errors, so the claim that LLM noise is the hardest realistic noise stands or falls with that proxy.

Editorial extensions

If this is right

  • If LLM-style instance-dependent noise is representative, evaluations that only use uniform or pairwise noise will overstate how well GNNs and label-noise methods cope with real mislabeled graphs.
  • GraphSAGE's separate center/neighbor parameterization should become a default architectural component or strong baseline when training GNNs under label noise.
  • Noisy-label detectors that use the maximum epoch-wise loss of a model detect LLM-corrupted nodes better than average-loss baselines, but detection scores fluctuate over training, so early stopping or loss-dynamics modeling is needed.
  • Robustness to class-dependent noise does not reliably transfer to LLM noise on datasets like Cora-ML and WikiCS, so new methods should be tested against realistic instance-dependent noise.
  • LLM-based noise generation gives a scalable way to create challenging noisy graph benchmarks with controllable noise rates across domains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because BeGIN's LLM-refined noise keeps only nodes that both naive and reasoned LLM runs mislabel, the retained noise is hard by construction; the 'LLM is hardest' ranking may partly reflect instance difficulty rather than the error distribution of real annotators, and a human-annotation comparison would separate these.
  • The paper's off-diagonal entropy correlation suggests a design rule the paper does not state: any noise generator that concentrates wrong-class transitions should predictably hurt GNNs more, so noise 'hardness' could be charted as a function of transition entropy rather than only noise rate.
  • The center/neighbor decoupling insight could be tested as a plug-in modification to other message-passing architectures, not just GraphSAGE, and could also improve noisy-label detection rather than only classification.
  • The heterophily subset contains very small graphs (few hundred nodes) with high variance, so ranking conclusions there would need larger heterophilic graphs before being treated as settled.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces BeGIN, a benchmark for instance-dependent label noise in graph data. It provides 10 graph datasets across homophilic and heterophilic domains, six noise types (uniform, pairwise, topology-based, feature-based, confidence-based, and LLM-based), and extensive evaluations of GNN architectures, noisy-label detection methods, and robust-learning algorithms. The authors propose algorithmic noise-injection strategies and an LLM-based simulation using GPT-4o-mini, and they report that LLM-based noise is the most challenging for GNNs. They also claim that GraphSAGE's separate parameterization of center and neighbor nodes enhances robustness, and they evaluate 23 learning methods and several detection protocols under the LLM-based noise setting.

Significance. If the benchmark and its headline comparisons are validated, BeGIN addresses a real gap: prior graph label-noise work relies almost exclusively on class-dependent noise, and the community lacks a shared set of instance-dependent noisy graph datasets with code, noise-generation protocols, and baseline results. The paper's strengths include the breadth of the evaluation (10 datasets, 6 noise types, 23 training methods, multiple detection strategies), the release of code at a public repository, and the attempt to connect noise structure (transition-matrix skewness, corruption frequency) to GNN vulnerability. The identification of structured, instance-dependent noise as more damaging than uniform or pairwise noise is practically relevant. However, the headline claims currently rest on a few design choices that conflate the corruption mechanism with instance difficulty and on an unsupported architectural attribution, so the benchmark's central conclusions require additional controls before they can be taken as established.

major comments (3)
  1. [Appendix B; Section 3.1, Table 1, Figure 1] In Appendix B, the default LLM-Refined noise is constructed by retaining only instances for which both LLM-Naive and LLM-Reasoned outputs disagree with the ground-truth label, which explicitly filters out easy misclassifications. Table 1 and Figure 1 then compare this noise with other noise types at the same aggregate noise rate but over different node subsets. The conclusion that LLM-based noise is the most challenging therefore conflates the corruption mechanism with the selection of intrinsically hard instances: nodes that are consistently hard for GPT-4o-mini may also be hard for GNNs because of feature or topological ambiguity. The transition-matrix analysis in Section 3.3 is correlational and does not control for this selection effect. Please add a control using unfiltered LLM-Reasoned or LLM-Naive noise at the same noise rate, and/or a difficulty-matched sample of other noise types, to separate the noise mechanism from hard-example selection.
  2. [Section 4.1.1, Table 3] The 'Maximum' detection scores in Table 3 are obtained by selecting the epoch with the highest observed ROC-AUC, which requires access to the clean labels at evaluation time. This is an oracle post-hoc selection and does not represent a feasible detection protocol; its inclusion inflates the reported detection baselines (e.g., 74.5% average for GraphSAGE under Maximum vs. 59.8% under Average). Please remove this metric or explicitly relabel it as an oracle upper bound, and provide a realistic model-selection rule (e.g., validation-based early stopping) for a feasible detection protocol.
  3. [Section 3.1, observation (3); Abstract] The abstract and conclusion state that 'parameterizing the center node separately from its neighbors, as done in GraphSAGE, can significantly enhance robustness.' This causal claim is not supported by the experiments: GraphSAGE is compared against GCN, GAT, GIN, and MLP as whole architectures, and no ablation isolates the separate center-node parameters from other differences such as sampling, aggregation function, or parameter count. The observation that GraphSAGE performs well on heterophilic graphs is consistent with the claim but does not establish the mechanism. Please provide a controlled comparison (e.g., a GCN variant with a separate center-node weight, or a GraphSAGE variant without the separate parameterization) before claiming this as a key insight.
minor comments (6)
  1. [Section 2.3.1] The PPR restart probability alpha is fixed to 0.9 throughout, but no sensitivity analysis is reported; since PPR is central to the topology-based noise, a brief ablation over alpha would strengthen the benchmark's robustness.
  2. [Section 2.3.2 and Table 1] Equalizing noise rates across noise types is a reasonable comparability choice, but the paper should state explicitly that the relative difficulty of noise types may depend on this equalization and on the exclusion of pairwise noise for the Children dataset at noise rates above 0.5.
  3. [Figure 3] The correlation between prediction entropy and corruption frequency is described as strong, but no correlation coefficients, confidence intervals, or p-values are reported; please add these values.
  4. [Table 4] The name 'GraphSage' appears with inconsistent capitalization; use 'GraphSAGE' consistently throughout the table and text.
  5. [Table 3] The footnote 'with the best in each method bolded' is ambiguous because multiple entries are bolded; clarify whether bold indicates the best per backbone, per dataset, or top-three.
  6. [Section 4.2] The statement 'we apply the same hyperparameter tuning process across all baseline models' is confusing because the search spaces in Table 6 are model-specific; it would be clearer to say that the same tuning budget and protocol were used.

Circularity Check

2 steps flagged · score 6.0 of 10

LLM-Refined noise is constructed to contain only instances the LLM gets wrong twice, so the headline 'LLM noise is hardest' partially restates that construction; the rest of the benchmark is independent.

  1. fitted input called prediction [Appendix B ('LLM-based label noise'); Section 3.1 observation (1); Section 3.3 conclusion]
    "Then, we only adopt the LLM-reasoned labels in cases where both the LLM-naive and LLM-reasoned outputs disagree with the ground-truth label. This ensures that the retained noisy labels reflect more complex cases by filtering out simple misclassifications that could be corrected without deeper analysis. ... LLM-based label noise leads to significant performance degradation. This is particularly noteworthy given that the noise ratio remains the same across all noise types."

    The default 'LLM-Refined' noise is not a random sample of LLM mistakes; it is the intersection of two LLM error sets (LLM-Naive and LLM-Reasoned). Thus by construction every corrupted node is one the LLM misclassifies twice, i.e., a consistently hard instance. Table 1 and Figure 1 then compare this noise with other types at the same aggregate noise rate, but over different node subsets, and the paper reports 'LLM-based label noise leads to significant performance degradation.' The claimed discovery that LLM noise is most challenging therefore encodes the construction choice: hard-for-the-LLM instances are selected, and then GNN difficulty is measured on that selected set.

  2. self definitional [Section 3.2 (Figure 2) vs. Algorithm 1 in Section 2.3.2]
    "Class-dependent noise follows a normal distribution with no selection patterns, applying uniform mislabeling across all instances in a class. In contrast, instance-dependent noise exhibits a skewed distribution, repeatedly corrupting the same instances."

    Algorithm 1 sets P_cor(v_i)=1-T_D[i,Y_i] and samples N_c nodes in proportion to P_cor. For class-dependent uniform noise, T_D is constant, so P_cor is the same for every node and ten independent runs yield a binomial (near-Gaussian) corruption-frequency distribution. For any instance-dependent T_D, P_cor varies by node, so high-P_cor nodes are re-selected in every run, mechanically generating the reported 'skewed distribution, repeatedly corrupting the same instances.' Figure 2's contrast is therefore a corollary of the definition of instance-dependent noise, not an independently discovered behavioral difference.

full rationale

BeGIN is mostly a self-contained empirical contribution: it releases new noisy datasets, runs a 10-seed evaluation, tunes hyperparameters with wandb, and compares 23 methods on equal footing. The GraphSAGE center/neighbor-parameterization insight is an honest reading of Table 1 and Table 4, not a fitted parameter; the self-citation [29] is a non-load-bearing pointer to loss-dynamics modeling. The central 'LLM noise is hardest' claim, however, is partially construction-bound. Appendix B defines LLM-Refined (the default noise) as the set of nodes where both LLM-Naive and LLM-Reasoned outputs disagree with ground truth, i.e., deliberately hard cases, and then forces the same aggregate noise rate on all other noise types but not the same node set. The observed difficulty ranking is therefore a selection effect as much as a property of LLM mislabeling: hard-for-LLM instances are chosen, and GNNs are found to struggle on them. The secondary observation that instance-dependent noise has a skewed corruption-frequency distribution is likewise an algebraic consequence of Algorithm 1's proportional sampling. These two steps make the headline partially circular, but the benchmark data and most method-comparison results stand independently.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper contributes no new theory, so the ledger captures benchmark-design choices and domain assumptions it relies on. The main items are the fixed PPR alpha, the equalized noise rates, and the assumptions that LLM mistakes mimic human mistakes and that small-loss GMM separation is valid.

free parameters (3)
  • PPR restart probability alpha = 0.9
    Fixed to 0.9 in Section 2.3.1 for topology-based noise; no sensitivity analysis reported.
  • Per-dataset noise rate eta = 0.246 to 0.575, equal to LLM-Refined rate
    All noise types use the LLM-Refined noise rate so comparisons are made at equal rates; this design choice shapes the difficulty comparison.
  • Maximum-epoch detection selection = not reported; argmax over epochs
    Table 3 reports Maximum ROC-AUC over training epochs, an oracle choice not attainable in practice.
assumptions (5)
  • domain assumption LLM errors are representative of human annotation errors
    Section 2.4 asserts this without a human annotation study.
  • ad hoc to paper Equalizing noise rates across noise types makes difficulty comparable
    Section 2.4 and Appendix B set the same eta for all noise types; this assumes the only relevant difference is the noise pattern.
  • domain assumption Algorithm 1's transition model adequately represents instance-dependent corruption
    The corruption process samples a fixed number of nodes in proportion to 1 minus the diagonal transition probability; this is an arbitrary design.
  • domain assumption Supervised clean-label detector is a meaningful benchmarking setup
    Section 4.1.2 assumes access to clean labels for training the detector, which is unavailable in practical noise-handling scenarios.
  • domain assumption Small-loss GMM separates noisy from clean nodes
    Section 4.1.1 relies on the standard small-loss trick and a Gaussian Mixture Model without validating the fit on each dataset.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark." pith.science (2026). https://pith.science/paper/BDFAXJTY

@misc{pith2026250612468,
  author       = {Pith},
  title        = {Pith review of: Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BDFAXJTY}},
  note         = {Machine review of arXiv:2506.12468}
}
read the original abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in node classification tasks but struggle with label noise in real-world data. Existing studies on graph learning with label noise commonly rely on class-dependent label noise, overlooking the complexities of instance-dependent noise and falling short of capturing real-world corruption patterns. We introduce BeGIN (Benchmarking for Graphs with Instance-dependent Noise), a new benchmark that provides realistic graph datasets with various noise types and comprehensively evaluates noise-handling strategies across GNN architectures, noisy label detection, and noise-robust learning. To simulate instance-dependent corruptions, BeGIN introduces algorithmic methods and LLM-based simulations. Our experiments reveal the challenges of instance-dependent noise, particularly LLM-based corruption, and underscore the importance of node-specific parameterization to enhance GNN robustness. By comprehensively evaluating noise-handling strategies, BeGIN provides insights into their effectiveness, efficiency, and key performance factors. We expect that BeGIN will serve as a valuable resource for advancing research on label noise in graphs and fostering the development of robust GNN training methods. The code is available at https://github.com/kimsu55/BeGIN.

Figures

Figures reproduced from arXiv: 2506.12468 by the authors.

Figure 1
Figure 1. Average node classification accuracy for homophily [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. The distribution of instances’ corruption frequency [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 5
Figure 5. Correlation between the entropy of off-diagonal [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (4 more)
Figure 7
Figure 7. Figure 7: Feature similarity distribution and topological label [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Correlation between classification accuracy on LLM [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Time and memory usage comparison on the WikiCS [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Average node classification accuracy on both ho [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

84 extracted references · 44 canonical work pages

  1. [1]

    Toufique Ahmed, Premkumar Devanbu, Christoph Treude, and Michael Pradel

  2. [2]

    Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. 2021. Understanding and improving early stopping for learning with noisy labels.Advances in Neural Information Processing Systems34 (2021), 24392–24403

    2021

  3. [3]

    Antonin Berthon, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. 2021. Confidence scores make instance-dependent label-noise learning possible. In International conference on machine learning. PMLR, 825–836

    2021

  4. [4]

    Tsang Masashi Sugiyama Bo Han, Quanming Yao

    Xingrui Yu Gang Niu Miao Xu Weihua Hu Ivor W. Tsang Masashi Sugiyama Bo Han, Quanming Yao. 2018. Co-teaching: Robust training of deep neural networks with extremely noisy labels.Advances in neural information processing systems31

    2018

  5. [5]

    Aleksandar Bojchevski and Stephan Günnemann. 2017. Deep gaussian embed- ding of graphs: Unsupervised inductive learning via ranking.arXiv preprint arXiv:1707.03815(2017)

    arXiv 2017

  6. [6]

    Yao Cheng, Caihua Shan, Yifei Shen, Xiang Li, Siqiang Luo, and Dongsheng Li

    work page

  7. [7]

    Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, and Cho-Jui Hsieh

  8. [8]

    InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Resurrecting label propagation for graphs with heterophily and label noise. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 433–444

    work page

Show all 84 references
  1. [9]

    Xuefeng Du, Tian Bian, Yu Rong, Bo Han, Tongliang Liu, Tingyang Xu, Wenbing Huang, Yixuan Li, and Junzhou Huang. 2021. Noise-robust graph learning by estimating and leveraging pairwise interactions.arXiv preprint arXiv:2106.07451 (2021)

    2021 arXiv

  2. [10]

    Jiarui Feng, Hao Liu, Lecheng Kong, Mingfang Zhu, Yixin Chen, and Muhan Zhang. 2024. TAGLAS: An atlas of text-attributed graph datasets in the era of large graph and language models.arXiv preprint arXiv:2406.14683(2024)

    2024 arXiv

  3. [11]

    Enyan Dai, Charu Aggarwal, and Suhang Wang. 2021. Nrgnn: Learning a label noise resistant graph neural network on sparsely and noisily labeled graphs. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 227–236

    2021

  4. [12]

    Johannes Gasteiger, Stefan Weißenberger, and Stephan Günnemann. 2019. Diffu- sion improves graph learning.Advances in neural information processing systems 32 (2019)

    2019

  5. [13]

    Spiros V Georgakopoulos, Dimitris K Iakovidis, Michael Vasilakakis, Vassilis P Plagianakos, and Anastasios Koulaouzidis. 2016. Weakly-supervised convolu- tional learning for detection of inflammatory gastrointestinal lesions. In2016 IEEE international conference on imaging sys...

    2016

  6. [14]

    Matthias Fey and Jan Eric Lenssen. 2019. Fast graph representation learning with PyTorch Geometric.arXiv preprint arXiv:1903.02428(2019)

    2019 arXiv

  7. [15]

    Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks.Proceedings of the National Academy of Sciences120, 30 (2023), e2305016120

    2023

  8. [16]

    Jacob Goldberger and Ehud Ben-Reuven. 2017. Training deep neural-networks using a noise adaptation layer. InInternational conference on learning representa- tions. KDD ’25, August 3–7, 2025, Toronto, ON, Canada Suyeon Kim, SeongKu Kang, Dongwoo Kim, Jungseul Ok, & Hwanjo Yu

    2017

  9. [17]

    Aritra Ghosh, Himanshu Kumar, and P Shanti Sastry. 2017. Robust loss functions under label noise for deep neural networks. InProceedings of the AAAI conference on artificial intelligence, Vol. 31

    2017

  10. [18]

    Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs.Advances in neural information processing systems30 (2017)

    2017

  11. [19]

    Zellig S. Harris. 1954. Distributional Structure.WORD10, 2-3 (1954), 146–162

    1954

  12. [20]

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017. On calibration of modern neural networks. InInternational conference on machine learning. PMLR, 1321–1330

    2017

  13. [21]

    Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. 2023. Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning.arXiv preprint arXiv:2305.19523 (2023)

    2023 arXiv

  14. [22]

    NT Hoang, Jun Jin Choong, and Tsuyoshi Murata. 2019. Learning graph neural networks with noisy labels. (2019)

    2019

  15. [23]

    Alex Havrilla and Maia Iyer. 2024. Understanding the Effect of Noise in LLM Train- ing Data with Algorithmic Chains of Thought.arXiv preprint arXiv:2402.04004 (2024)

    2024 arXiv

  16. [24]

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2020. Open graph benchmark: Datasets for machine learning on graphs.Advances in neural information processing systems 33 (2020), 22118–22133

    2020

  17. [25]

    Jinchi Huang, Lie Qu, Rongfei Jia, and Binqiang Zhao. 2019. O2u-net: A simple noisy label detection approach for deep neural networks. InProceedings of the IEEE/CVF international conference on computer vision. 3326–3334

    2019

  18. [26]

    Guanhao Hou, Xingguang Chen, Sibo Wang, and Zhewei Wei. 2021. Massively parallel algorithms for personalized pagerank.Proceedings of the VLDB Endow- ment14, 9 (2021), 1668–1680

    2021

  19. [27]

    Yuanfeng Ji, Lu Zhang, Jiaxiang Wu, Bingzhe Wu, Lanqing Li, Long-Kai Huang, Tingyang Xu, Yu Rong, Jie Ren, Ding Xue, et al . 2023. Drugood: Out-of- distribution dataset curator and benchmark for ai-aided drug discovery–a focus on affinity prediction problems with noise annotat...

    2023

  20. [28]

    Lu Jiang, Di Huang, Mason Liu, and Weilong Yang. 2020. Beyond synthetic noise: Deep learning on controlled noisy labels. InInternational conference on machine learning. PMLR, 4804–4815

    2020

  21. [29]

    Adrián Javaloy, Pablo Sanchez Martin, Amit Levi, and Isabel Valera. 2023. Learn- able Graph Convolutional Attention Networks. InInternational Conference on Learning Representations (ICLR). https://openreview.net/forum?id=WsUMeHPo- 2

    2023

  22. [30]

    Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907(2016)

    2016 arXiv

  23. [31]

    Bhawesh Kumar, Jonathan Amar, Eric Yang, Nan Li, and Yugang Jia. 2024. Selec- tive Fine-tuning on LLM-labeled Data May Reduce Reliance on Human Anno- tation: A Case Study Using Schedule-of-Event Table Detection.arXiv preprint arXiv:2405.06093(2024)

    2024 arXiv

  24. [32]

    Suyeon Kim, Dongha Lee, SeongKu Kang, Sukang Chae, Sanghwan Jang, and Hwanjo Yu. 2024. Learning Discriminative Dynamics with Label Corruption for Noisy Label Detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 22477–22487

    2024

  25. [33]

    Junnan Li, Richard Socher, and Steven CH Hoi. 2020. Dividemix: Learning with noisy labels as semi-supervised learning.arXiv preprint arXiv:2002.07394(2020)

    2020 arXiv

  26. [34]

    Xianxian Li, Qiyu Li, Haodong Qian, Jinyan Wang, et al . 2024. Contrastive learning of graphs under label noise.Neural Networks172 (2024), 106113

    2024

  27. [35]

    Md Tahmid Rahman Laskar, Mizanur Rahman, Israt Jahan, Enamul Hoque, and Jimmy Huang. 2023. Can large language models fix data annotation errors? an empirical study using debatepedia for query-focused text summarization. In Findings of the Association for Computational Linguist...

    2023

  28. [36]

    Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. 2023. One for all: Towards training one graph model for all classification tasks.arXiv preprint arXiv:2310.00149(2023)

    2023 arXiv

  29. [37]

    Xingjun Ma, Hanxun Huang, Yisen Wang, Simone Romano, Sarah Erfani, and James Bailey. 2020. Normalized loss functions for deep learning with noisy labels. InInternational conference on machine learning. PMLR, 6543–6553

    2020

  30. [38]

    Yayong Li, Jie Yin, and Ling Chen. 2021. Unified robust training for graph neural networks against label noise. InPacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 528–540

    2021

  31. [39]

    Andrew McCallum. 2021. The 4 Universities Data Set. Available online: http: //www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/data/ (accessed on 222 Feb. 2025)

    2021

  32. [40]

    Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore

  33. [41]

    Seiji Maekawa, Koki Noda, Yuya Sasaki, et al. 2022. Beyond real-world benchmark datasets: An empirical study of node classification with GNNs.Advances in Neural Information Processing Systems35 (2022), 5562–5574

    2022

  34. [42]

    Péter Mernyei and Cătălina Cangea. 2020. Wiki-cs: A wikipedia-based benchmark for graph neural networks.arXiv preprint arXiv:2007.02901(2020)

    2020 arXiv

  35. [43]

    Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural...

    2019

  36. [44]

    Diane Oyen, Michal Kucer, Nicolas Hengartner, and Har Simrat Singh. 2022. Robustness to label noise depends on the shape of the noise distribution.Advances in Neural Information Processing Systems35 (2022), 35645–35656

    2022

  37. [45]

    Aditya Krishna Menon, Brendan Van Rooyen, and Nagarajan Natarajan. 2018. Learning from binary labels with instance-dependent noise.Machine Learning 107 (2018), 1561–1595

    2018

  38. [46]

    Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. 2017. Making deep neural networks robust to label noise: A loss correction approach. InProceedings of the IEEE conference on computer vision and pattern recognition. 1944–1952

    2017

  39. [47]

    Maja Pavlovic and Massimo Poesio. 2024. The Effectiveness of LLMs as Annota- tors: A Comparative Overview and Empirical Analysis of Direct Representation. arXiv preprint arXiv:2405.01299(2024)

    2024 arXiv

  40. [48]

    Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang

  41. [49]

    Lawrence Page, Sergey Brin, Rajeev Motwani, Terry Winograd, et al. 1999. The pagerank citation ranking: Bringing order to the web. (1999)

    1999

  42. [50]

    Siyi Qian, Haochao Ying, Renjun Hu, Jingbo Zhou, Jintai Chen, Danny Z Chen, and Jian Wu. 2023. Robust training of graph neural networks via noise governance. InProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 607–615

    2023

  43. [51]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learnin...

    2021

  44. [52]

    Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2019. Dropedge: Towards deep graph convolutional networks on node classification.arXiv preprint arXiv:1907.10903(2019)

    2019 arXiv

  45. [53]

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1986. Learning representations by back-propagating errors.nature323, 6088 (1986), 533–536

    1986

  46. [54]

    Joshua C Peterson, Ruairidh M Battleday, Thomas L Griffiths, and Olga Rus- sakovsky. 2019. Human uncertainty makes classification more robust. InProceed- ings of the IEEE/CVF international conference on computer vision. 9617–9626

    2019

  47. [55]

    Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation and synthesis: A survey. InProceedings of the 2024 Conference on Empirical Methods in Natu...

    2024

  48. [56]

    Brendan Van Rooyen, Aditya Menon, and Robert C Williamson. 2015. Learning with symmetric label noise: The importance of being unhinged.Advances in neural information processing systems28 (2015)

    2015

  49. [57]

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. InICLR

    2017

  50. [58]

    Bin Wang, Chengwei Wei, Zhengyuan Liu, Geyu Lin, and Nancy F Chen. 2024. Resilience of Large Language Models for Noisy Instructions.arXiv preprint arXiv:2404.09754(2024)

    2024 arXiv

  51. [59]

    Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan Günnemann. 2018. Pitfalls of graph neural network evaluation.arXiv preprint arXiv:1811.05868(2018)

    2018 arXiv

  52. [60]

    Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. 2019. Symmetric cross entropy for robust learning with noisy labels. InProceedings of the IEEE/CVF international conference on computer vision. 322–330

    2019

  53. [61]

    Zhonghao Wang, Danyu Sun, Sheng Zhou, Haobo Wang, Jiapei Fan, Longtao Huang, and Jiajun Bu. 2024. NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise.arXiv preprint arXiv:2406.04299(2024)

    2024 arXiv

  54. [62]

    Hongxin Wei, Lei Feng, Xiangyu Chen, and Bo An. 2020. Combating noisy labels by agreement: A joint training method with co-regularization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition. 13726–13735

    2020

  55. [63]

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu

  56. [64]

    Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. 2020. Microsoft academic graph: When experts are not enough.Quantitative Science Studies1, 1 (2020), 396–413

    2020

  57. [65]

    Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. 2020. Part-dependent label noise: Towards instance-dependent label noise.Advances in Neural Information Processing Systems33 (2020), 7597–7610

    2020

  58. [66]

    Shunxin Xiao, Shiping Wang, Yuanfei Dai, and Wenzhong Guo. 2022. Graph neural networks in node classification: survey and evaluation.Machine Vision Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark KDD ’25, August 3–7, 2025, Toronto...

    2022

  59. [67]

    Tong Xiao, Tian Xia, Yi Yang, Chang Huang, and Xiaogang Wang. 2015. Learning from massive noisy labeled data for image classification. InProceedings of the IEEE conference on computer vision and pattern recognition. 2691–2699

    2015

  60. [68]

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. 2019. How powerful are graph neural networks?. InICLR

    2019

  61. [69]

    Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. 2023. A comprehensive study on text-attributed graphs: Benchmarking and rethinking.Advances in Neural Information Processing Systems36 (2023), 17238–17264

    2023

  62. [70]

    Xiaowen Wei, Xiuwen Gong, Yibing Zhan, Bo Du, Yong Luo, and Wenbin Hu

  63. [71]

    Jingyang Yuan, Xiao Luo, Yifang Qin, Yusheng Zhao, Wei Ju, and Ming Zhang

  64. [72]

    Mengmei Zhang, Linmei Hu, Chuan Shi, and Xiao Wang. 2020. Adversarial label- flipping attack and defense for graph neural networks. In2020 IEEE International Conference on Data Mining (ICDM). IEEE, 791–800

    2020

  65. [73]

    Xin Zheng, Yi Wang, Yixin Liu, Ming Li, Miao Zhang, Di Jin, Philip S Yu, and Shirui Pan. 2022. Graph neural networks for graphs with heterophily: A survey. arXiv preprint arXiv:2202.07082(2022)

    2022 arXiv

  66. [74]

    Yonghua Zhu, Lei Feng, Zhenyun Deng, Yang Chen, Robert Amor, and Michael Witbrock. 2024. Robust Node Classification on Graph Data with Graph and Label Noise. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17220–17227

    2024

  67. [75]

    Zhaowei Zhu, Zihao Dong, and Yang Liu. 2022. Detecting corrupted labels without training a model to predict. InInternational conference on machine learning. PMLR, 27412–27427

    2022

  68. [76]

    Children

    Daniel Zoran and Yair Weiss. 2011. From learning models of natural image patches to whole image restoration. In2011 international conference on computer vision. IEEE, 479–486. A Additional Dataset Information We present 10 benchmark datasets, all including text attributes on n...

    2011

  69. [77]

    Chenglin Yu, Xinsong Ma, and Weiwei Liu. 2023. Delving into noisy label detection with clean data. InInternational Conference on Machine Learning. PMLR, 40290–40305

    2023

  70. [79]

    InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Learning on graphs under label noise. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5

    2023

  71. [2000]

    Information Retrieval3 (2000), 127–163

    Automating the construction of internet portals with machine learning. Information Retrieval3 (2000), 127–163

    2000

  72. [2019]

    InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining

    Cluster-gcn: An efficient algorithm for training deep and large graph convolutional networks. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. 257–266

  73. [2020]

    Geom-gcn: Geometric graph convolutional networks.arXiv preprint arXiv:2002.05287(2020)

    2020 arXiv

  74. [2021]

    Learning with noisy labels revisited: A study using real-world human annotations.arXiv preprint arXiv:2110.12088(2021)

    2021 arXiv

  75. [2023]

    InProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining

    Clnode: Curriculum learning for node classification. InProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 670–678

  76. [2024]

    Can LLMs Replace Manual Annotation of Software Engineering Artifacts? arXiv preprint arXiv:2408.05534(2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.