Pith. sign in

REVIEW 4 major objections 5 minor 87 references

No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RINGS tests whether graph benchmarks need both structure and features; most of 13 popular datasets fail its quality test.

desk verdict A genuinely useful mode-perturbation framework for evaluating graph datasets, with clean math and extensive experiments; the taxonomy recommendations rest on an unvalidated structural-diversity metric and should be treated as hypotheses, not verdicts. read the letter →

arxiv 2502.02379 v3 pith:MIDOXWEP submitted 2025-02-04 cs.LG cs.SIstat.ML

Corinna Coupette , Jeremy Wayland , Emily Simons , Bastian Rieck This is my paper · ORCID
classification cs.LGcs.SIstat.ML
keywords graph-learningdatasetsdatasetqualityevaluationmodeperturbationsperformanceseparabilitycomplementaritybenchmarkgraphneuralnetworksofevaluations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish a principled way to judge the quality of graph-learning benchmark datasets, rather than judging the models trained on them. Its starting point is that attributed graphs combine two modes — graph structure and node features — and a good benchmark should require information from both. To test this, the paper introduces RINGS, which replaces one mode with an uninformative surrogate (empty, complete, random, or shuffled) and measures two things: performance separability, whether models trained on the original data statistically outperform models trained on the perturbations, and mode complementarity, the geometric distance between the graph's diffusion-distance space and the features' Euclidean space. Applied to 13 popular graph-classification datasets, the framework finds that only 3 satisfy the expected separability relations and that several originals are separably outperformed by their own perturbed versions, and it distills these findings into an actionable taxonomy: keep MolHIV, NCI1, and Peptides; realign five datasets; deprecate five. The contribution matters because it offers a task-independent, model-agnostic diagnostic for dataset quality to complement costly model-based evaluation.

What carries the argument

The central object is the mode perturbation — a map that replaces the edge set or feature set of an attributed graph with a controlled surrogate, applied element-wise to an entire dataset. Two measures sit on top of it: performance separability, the statistically tested distance between performance distributions of tuned models on the original versus perturbed datasets, and mode complementarity, the normalized $L_{1,1}$ norm between the graph's diffusion-distance matrix and the features' Euclidean distance matrix. A duality theorem, $\gamma_{c*} = 1 - \gamma_{e*}$, links the complete and empty perturbations under this comparator, and the derived mode diversity $\Delta = 1 - |1 - 2\gamma_{e*}|$ scores how much non-trivial geometric structure a single mode carries. The diffusion distance is sampled over $t \in [10]$ diffusion steps, so complementarity is a curve rather than a point estimate.

What would settle it

Build a dataset whose labels depend on only one mode (or neither) but whose structure and features are geometrically complementary: if mode complementarity still ranks it alongside MolHIV and NCI1, the diagnostic is measuring geometric variety rather than task-relevant information. A cheaper check is to recompute the taxonomy with resistance distance or shortest-path distance as the structural metric; any dataset that switches category under that swap exposes the verdicts' dependence on the diffusion-distance choice.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that a good graph-learning dataset is one whose structure and features both carry task-relevant information and whose structural and feature geometries are complementary — and that both properties can be tested by perturbing one mode while leaving the other intact. The paper formalizes mode perturbations (empty, complete, random, and shuffled graphs and features) and two measures built on them. Performance separability compares performance distributions of tuned GNNs on original versus perturbed datasets using permutation tests with the Kolmogorov-Smirnov statistic; when a perturbation matches or beats the original, the perturbed mode is not carrying the information the task needs. Mode complementarity compares the normalized distance matrices of the two modes using the $L_{1,1}$ norm, with a duality theorem relating complete and empty perturbations and a derived mode-diversity score. Across 13 real-world graph-classification datasets, only COLLAB, MolHIV, and NCI1 (with Peptides close behind) pass the expected separability relations, while DD is separably outperformed by its random-graph perturbation, Enzymes by its empty-graph perturbation, and Reddit-B by complete-features and random-graph perturbations. Higher mean mode complementarity is associated with higher AUROC across perturbations, supporting complementarity as a task-independent diagnostic.

Load-bearing premise

The load-bearing premise is that the geometric gap between node-feature distances and graph-diffusion distances, summed with the $L_{1,1}$ norm, captures the complementary information that graph-learning models actually exploit, so that a dataset scoring well on mode complementarity is genuinely a better benchmark; if another pair of metric choices reverses the complementarity orderings, the dataset verdicts would change.

Editorial extensions

If this is right

  • Claims of new GNN capability should come with separability evidence: a model that performs well on the original data but also on random-graph or complete-features perturbations has not shown that it needs the graph structure.
  • Mode complementarity can serve as a cheap screening diagnostic — it correlates with AUROC across perturbations — so dataset builders can assess candidates before expensive hyperparameter tuning.
  • The taxonomy gives concrete guidance: keep MolHIV, NCI1, and Peptides as-is; realign AIDS, DD, MUTAG, Reddit-B, and Reddit-M toward better-aligned tasks; deprecate COLLAB, IMDB-B, IMDB-M, Enzymes, and Proteins as graph benchmarks.
  • The framework extends beyond classification: the appendix applies it to regression datasets (QM9, ZINC-12k) and transformer architectures without changing the core conclusions.
  • Graph-level analysis of correctly classified sets can expose shortcut learning — Proteins' complete-features perturbation correctly classifies a distinct outlier subset, a sign that models exploit superficial cues rather than meaningful patterns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because mode complementarity scores depend on the chosen structural and feature metrics, a different metric pair could reorder the taxonomy; the paper tests a small set of alternatives, so its verdicts on individual datasets are less stable than the framework itself.
  • The framework suggests a constructive use the authors only gesture at: steer dataset construction by maximizing the drop in complementarity under perturbation, actively building benchmarks that make structural and feature information non-redundant rather than auditing existing ones afterward.
  • The correlation between complementarity and performance is established on 13 datasets and one perturbation family; a stress test on adversarially constructed graphs with high complementarity but useless labels would show whether the diagnostic reflects benchmark utility or just geometric diversity.
  • A two-stage pipeline is left implicit: screen candidate datasets with complementarity (cheap), then verify with separability (expensive), dividing labor between the task-independent and task-dependent measures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper introduces RINGS, a mode-perturbation framework for evaluating the quality of graph-learning datasets. The framework applies perturbations to the graph structure or node features and uses two measures: performance separability, which compares performance distributions of models trained on original versus perturbed data via permutation tests, and mode complementarity, which computes a normalized L1,1 distance between metric spaces induced by the structure and features. The authors evaluate 13 graph-classification datasets, report that only COLLAB, MolHIV, and NCI1 satisfy the expected separability relations, and derive a taxonomy classifying datasets as Keep, Realign, or Deprecate. They also demonstrate extensions to regression, transformer architectures, and graph-level analyses, and publicly release code.

Significance. If the framework is valid, it addresses a real need in data-centric graph learning: systematic evaluation of benchmark datasets rather than models. The authors provide a reproducible pipeline (Zenodo/GitHub), thorough statistical testing (10,000-permutation tests, Bonferroni correction, robustness checks across test statistics and alpha levels), and a clean theoretical result in Theorem 2.15. The proposed mode-complementarity measure could serve as a fast, task-independent diagnostic. However, the practical value hinges on the unvalidated assumption that the chosen structural metric (diffusion distance at t=1 with the L1,1 norm) captures the structural variation that graph-learning methods actually exploit; the paper's own performance-separability results for COLLAB and IMDB contradict the taxonomy's Deprecate labels. The headline taxonomy should be treated as suggestive rather than established.

major comments (4)
  1. [Section 4.3, Table 2, Definition 2.14] The Deprecate (‡) category for COLLAB, IMDB-B, and IMDB-M is based on low structural diversity Δ_S, computed from the diffusion distance at t=1 with the L1,1 comparator. This conflicts with the paper's own P1 results: Table 1 reports COLLAB as having informative structure and IMDB-B/IMDB-M as having at least separable structure. The authors never establish that Δ_S measures the structural information used by message-passing GNNs; for diameter-2 ego-networks the all-pairs diffusion distance is nearly degenerate even though local adjacency patterns are informative. Appendix B.2 (Figure 7) illustrates alternative metrics only on Peptides and does not re-derive Δ_S or the taxonomy. As written, the Deprecate recommendations are an artifact of a particular, unvalidated metric choice.
  2. [Figure 5, Section 4.2] The claim that mode complementarity is a useful task-independent diagnostic is supported by a correlation computed on the same 13 datasets used to construct the taxonomy. This is a circular validation: the data that motivated the measure are also used to show it predicts performance. An external validation—for example, on the OGB datasets shown in Figure 9, or on held-out datasets with known rankings—is needed before the diagnostic claim can be accepted.
  3. [Definition 2.11 and proof of Theorem 2.15] The comparator C_{p,q} in Definition 2.11 divides by the q-th root of n^2−n, but the proof of Theorem 2.15 uses the norm without this factor, and the numerical results in Figure 4 are consistent with the unnormalized version. This is not merely cosmetic; it changes all complementarity values and the derived diversity scores. The definition should be corrected or the proof and results reconciled.
  4. [Figure 3, Tables 6 and 7] The conclusion that Reddit-M 'lacks performance separability' is based on experiments where the complete-graph and complete-features perturbations failed to train, and Reddit-B has missing GIN runs. Missing two of six conditions on Reddit-M means the separability ordering is only partially observed; the failure may reflect model or resource constraints rather than dataset quality. Please provide the missing runs or temper the conclusion.
minor comments (5)
  1. [Abstract] The term 'Rings' appears in the abstract but should be 'RINGS' for consistency with the rest of the paper.
  2. [Table 14] The word 'unformative' should be 'uninformative'.
  3. [Appendix C.2.2] The text says 'AURUC' but should say 'AUROC'.
  4. [Definition 2.14] The text states that Δ_* scores the ability of d_* to produce non-trivial geometric structure, but for * = f the quantity γ_{e*} measures the structure metric (since φ_{ef} zeroes the features). Clarify the subscript convention to avoid a swapped interpretation of Δ_S and Δ_F.
  5. [Section 4.3] The threshold for 'high' is based on coarse bins (e.g., ◦ is grouped with + and ++); a sensitivity analysis over these thresholds would strengthen the taxonomy.

Circularity Check

1 steps flagged · score 6.0 of 10

Deprecate (‡) rests on 'structural diversity' that Proposition A.10 defines as a feature-distance norm, so the structural claim reduces to a feature statistic.

  1. renaming known result [Definition 2.14; Proposition A.10 (Eq. 16); Table 2; Section 4.3]
    "Definition 2.14 ... ∆^{p,q}_∗(G, X) := 1 − |1 − 2 γ^{p,q}_{e∗}(G, X)| ... Proposition A.10 ... γ^{p,q}_{e∗}(G, X) = 1/(q√(n²−n)) ||D_{d*}||_{p,q}. Table 2: '∆S ∆F Structure Features'. Section 4.3: 'Deprecate (‡). This category contains datasets with high performance separability but low structural diversity.'"

    For ∗=g, γ_{e_g} is the complementarity under the empty-graph perturbation, which lifts to 0n; by Eq. (16) it equals the normalized L1,1 norm of D_{d_F}, the Euclidean node-feature distance matrix. Hence ∆S = ∆_g is a function of feature distances alone. Table 2 labels this 'Structural Diversity,' and Section 4.3 uses low ∆S to place COLLAB, IMDB-B, and IMDB-M in Deprecate (‡), asserting they 'do not contain interesting structural variation.' That assertion is therefore not derived from any structural measurement; it is a renaming of a feature-dispersion statistic, and the deprecation recommendation reduces by construction to a threshold on ||D_F||.

full rationale

Performance separability (Section 4.1) is an empirical, non-circular measurement: no parameters are fitted to force rankings, and the permutation-test comparisons are standard. Mode complementarity (Definition 2.12) is a direct geometric score with no fitted inputs. However, the taxonomy's Deprecate (‡) categories for COLLAB, IMDB-B, and IMDB-M rest on 'structural diversity' ∆S, which by Definition 2.14 and Proposition A.10 (Eq. 16) is computed from γ_{e_g}, i.e., the normalized L1,1 norm of the feature distance matrix D_{d_F}, not from any graph-structural distance. Thus the conclusion that these datasets 'do not contain interesting structural variation' is a renaming of a feature-statistic threshold, and the headline deprecation recommendation reduces by construction to a property of the node features. The rest of the RINGS framework is self-contained against external benchmarks, so the circularity is partial and localized to the structural-diversity column and the Deprecate (‡) recommendation.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The framework's validity rests on several modeling choices and domain assumptions rather than on fitted parameters: the metric lifts, the comparator, the choice of probe architectures, and the post hoc taxonomy thresholds. None of these are adversarial; they are standard practice, but they are assumptions the reader must accept for the recommendations to hold.

free parameters (6)
  • Diffusion steps t = 1 (for main results), varied over 1..10
    Mode complementarity depends on the number of diffusion steps in the structural metric; the authors choose t=1 for headline results and show some sensitivity in Figure 5.
  • Structural metric = Diffusion distance (Coifman-Lafon)
    Choice of lift for graph structure; other metrics (heat kernel, resistance, shortest path) are tested in the appendix and change complementarity values.
  • Feature metric = Euclidean distance
    Choice of lift for node features; cosine distance is also tested.
  • Comparator norm = L1,1
    The Lp,q norm with p=q=1 is chosen, motivated by the duality theorem (Theorem 2.15).
  • Evaluation score weighting = 1.5*S + 1*F
    Ad hoc weighting of structural vs feature informativeness used to bin datasets into taxonomy categories (Appendix D.2.2).
  • Taxonomy thresholds = bins [0,0.2),..., plus 'high' defined as ◦,+,++
    Hand-chosen bin boundaries for mode-diversity scores and performance-separability categories.
assumptions (4)
  • domain assumption Graph learning datasets can be faithfully evaluated by the two selected lifts: Euclidean distance on features and diffusion distance on structure.
    The framework's mode complementarity relies on these metric choices representing the information used by graph-learning methods (Section 2.3, Appendix B.2).
  • domain assumption Performance separability measured with GAT, GCN, and GIN generalizes to other graph-learning models.
    The paper uses three message-passing architectures to draw conclusions about dataset quality, and acknowledges this limitation in Section 5.
  • domain assumption The desired properties P0, P1, P2 (complementarity and informativeness of modes) are appropriate criteria for dataset quality.
    These principles are asserted in the introduction, not derived; the entire framework is built on them.
  • standard math Standard mathematical background for metric spaces, norms, and permutation tests.
    Definitions and proofs use standard results; no machine-checked formalization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets." pith.science (2026). https://pith.science/paper/MIDOXWEP

@misc{pith2026250202379,
  author       = {Pith},
  title        = {Pith review of: No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MIDOXWEP}},
  note         = {Machine review of arXiv:2502.02379}
}
read the original abstract

Benchmark datasets have proved pivotal to the success of graph learning, and good benchmark datasets are crucial to guide the development of the field. Recent research has highlighted problems with graph-learning datasets and benchmarking practices -- revealing, for example, that methods which ignore the graph structure can outperform graph-based approaches. Such findings raise two questions: (1) What makes a good graph-learning dataset, and (2) how can we evaluate dataset quality in graph learning? Our work addresses these questions. As the classic evaluation setup uses datasets to evaluate models, it does not apply to dataset evaluation. Hence, we start from first principles. Observing that graph-learning datasets uniquely combine two modes -- graph structure and node features --, we introduce Rings, a flexible and extensible mode-perturbation framework to assess the quality of graph-learning datasets based on dataset ablations -- i.e., quantifying differences between the original dataset and its perturbed representations. Within this framework, we propose two measures -- performance separability and mode complementarity -- as evaluation tools, each assessing the capacity of a graph dataset to benchmark the power and efficacy of graph-learning methods from a distinct angle. We demonstrate the utility of our framework for dataset evaluation via extensive experiments on graph-level tasks and derive actionable recommendations for improving the evaluation of graph-learning methods. Our work opens new research directions in data-centric graph learning, and it constitutes a step toward the systematic evaluation of evaluations.

Figures

Figures reproduced from arXiv: 2502.02379 by the authors.

Figure 1
Figure 1. Overview of our main mode perturbations. Given a ring of cliques on 12 nodes (top left panel) with 12-dimensional node features (bottom left panel), we show our input data for the original and perturbed states of the graph structure (top row) and the features (bottom row). turbations: performance separability, addressing P1, and mode complementarity, addressing P2. C3 Experiments. We demonstrate the value of our fra… view at source ↗
Figure 2
Figure 2. Setup for mode-complementarity assessment. For(G, X)from [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparing GNN performance across different versions of the same dataset. We show the mean (dot) and 95th percentile interval (bars) of accuracy and AUROC across 100 runs of the best (as measured by the respective performance mean) among our tuned GAT, GCN, and GIN models, for the original version and 5 perturbations of 13 graph-learning datasets. Black resp. silver horizontal lines show best mean trained resp. untra… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Comparing levels of mode complementarity across different versions of the same dataset. We show the mean (dot) and 95th percentile intervals (bars) of complementarity scores for the original version as well as 4 deterministic perturbations (top) and 4 randomized pertur…
Figure 5
Figure 5. Figure 5: Mode complementarity and performance. We show mean AUROC (y) as a function of mean mode comple￾mentarity (x), for the original version and 5 perturbations of our 13 main datasets, based on our best-on-average models (as in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Mode complementarity under different graph distances and varying density. We show the evolution of mode complementarity for 5 graphs on 10 nodes with normally distributed one-dimensional features as we randomly add edges to increase the density from the minimum for a c…
Figure 7
Figure 7. Figure 7: Real-world mode complementarity under dif￾ferent graph distances dS and feature distances dF . We show the 95th percentile of the mode-complementarity dis￾tribution for the Peptides dataset, for varying combinations of graph distance dS and feature distance dF , using …
Figure 8
Figure 8. Figure 8: Graph-level agreement between (mode, architecture) pairs. Jaccard similarity (upper triangular) and asymmet￾ric overlap (lower triangular) quantify the similarity between sets of graphs correctly classified by different models on the MUTAG (a) and Proteins (b) datasets…
Figure 9
Figure 9. Figure 9: Comparing levels of mode complementarity across different versions of the same dataset. We show the mean (dot) and 95th percentile intervals (bars) of com￾plementarity scores for the original version as well as 4 deterministic perturbations (top) and 4 randomized pertu…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

87 extracted references · 72 canonical work pages

  1. [1]

    Evaluating explainability for graph neural networks

    Agarwal, C., Queen, O., Lakkaraju, H., and Zitnik, M. Evaluating explainability for graph neural networks. Scientific Data, 10 0 (1): 0 144, 2023

    2023

  2. [2]

    Graphtester: Exploring theoretical boundaries of gnns on graph datasets

    Akbiyik, E., Gr \"o tschla, F., Egressy, B., and Wattenhofer, R. Graphtester: Exploring theoretical boundaries of gnns on graph datasets. In Data-centric Machine Learning Research (DMLR) Workshop at ICML, 2023

    2023

  3. [3]

    and Yahav, E

    Alon, U. and Yahav, E. On the bottleneck of Graph Neural Networks and its practical implications. In International Conference on Learning Representations, 2021

    2021

  4. [4]

    and Hancock, E

    Bai, X. and Hancock, E. R. Heat kernels, manifolds and graph embedding. In Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshops, pp.\ 198--206. Springer, 2004

    2004

  5. [5]

    Network science

    Barab \'a si, A.-L. Network science. Cambridge University Press, 2016

    2016

  6. [6]

    Graph neural networks use graphs when they shouldn't

    Bechler-Speicher, M., Amos, I., Gilad-Bachrach, R., and Globerson, A. Graph neural networks use graphs when they shouldn't. In International Conference on Learning Representations, 2024

    2024

  7. [7]

    I Can't Believe It's Not Better!

    Biderman, S. and Scheirer, W. J. Pitfalls in machine learning research: Reexamining the development cycle. In Proceedings on "I Can't Believe It's Not Better!" at NeurIPS Workshops, volume 137 of Proceedings of Machine Learning Research, pp.\ 106--117. PMLR, 2020

    2020

  8. [8]

    and Khan, A

    Bonabi Mobaraki, E. and Khan, A. A demonstration of interpretability methods for Graph Neural Networks . In Proceedings of the 6th Joint Workshop on Graph Data Management Experiences & Systems ( GRADES ) and Network Data Analytics ( NDA ) , pp.\ 1--5, 2023

    work page 2023

Show all 87 references
  1. [9]

    Graph kernels: State-of-the-art and future challenges

    Borgwardt, K., Ghisu, E., Llinares-L \'o pez, F., O'Bray, L., and Rieck, B. Graph kernels: State-of-the-art and future challenges. Foundations and Trends in Machine Learning, 13 0 (5--6): 0 531--712, 2020

    2020

  2. [10]

    M., Ong, C

    Borgwardt, K. M., Ong, C. S., Sch\" o nauer, S., Vishwanathan, S. V. N., Smola, A. J., and Kriegel, H.-P. Protein function prediction via graph kernels. Bioinformatics, 21 0 (1): 0 47–56, 2005

    2005

  3. [11]

    and Wang, Y

    Cai, C. and Wang, Y. A simple yet effective baseline for non-attributed graph classification. In Representation Learning on Graphs and Manifolds Workshop at ICLR, 2018

    2018

  4. [12]

    SA - MLP : Distilling Graph Knowledge from GNNs into Structure - Aware MLP , 2022

    Chen, J., Chen, S., Bai, M., Gao, J., Zhang, J., and Pu, J. SA - MLP : Distilling Graph Knowledge from GNNs into Structure - Aware MLP , 2022. arXiv:2210.09609 [cs]

    2022 arXiv

  5. [13]

    Are powerful graph neural nets necessary? A dissection on graph classification, 2020

    Chen, T., Bian, S., and Sun, Y. Are powerful graph neural nets necessary? A dissection on graph classification, 2020. arXiv:1905.04579 [cs]

    2020 arXiv

  6. [14]

    and Jegelka, S

    Chuang, C.-Y. and Jegelka, S. Tree mover's distance: Bridging graph metrics and stability of graph neural networks. In Advances in Neural Information Processing Systems, pp.\ 2944--2957, 2022

    2022

  7. [15]

    Coifman, R. R. and Lafon, S. Diffusion maps. Applied and Computational Harmonic Analysis, 21 0 (1): 0 5--30, 2006. Special Issue: Diffusion Maps and Wavelets

    2006

  8. [16]

    K., Lopez de Compadre, R

    Debnath, A. K., Lopez de Compadre, R. L., Debnath, G., Shusterman, A. J., and Hansch, C. Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity. Journal of Medicinal Chemistry, 34...

    1991

  9. [17]

    Di Giovanni, F., Giusti, L., Barbero, F., Luise, G., Lio, P., and Bronstein, M. M. On over-squashing in message passing neural networks: The impact of width, depth, and topology. In International Conference on Machine Learning, pp.\ 7865--7885, 2023

    2023

  10. [18]

    and Li, X

    Ding, J. and Li, X. An approach for validating quality of datasets for machine learning. In IEEE International Conference on Big Data, pp.\ 2795--2803, 2018

    2018

  11. [19]

    Dobson, P. D. and Doig, A. J. Distinguishing enzyme structures from non-enzymes without alignments. Journal of Molecular Biology, 330 0 (4): 0 771--783, 2003

    2003

  12. [20]

    and Kluger, Y

    Dong, M. and Kluger, Y. Towards understanding and reducing graph structural noise for gnns. In International Conference on Machine Learning, pp.\ 8202--8226, 2023

    2023

  13. [21]

    P., Ramp \'a s ek, L., Galkin, M., Parviz, A., Wolf, G., Luu, A

    Dwivedi, V. P., Ramp \'a s ek, L., Galkin, M., Parviz, A., Wolf, G., Luu, A. T., and Beaini, D. Long range graph benchmark. In Advances in Neural Information Processing Systems, pp.\ 22326--22340, 2022

    2022

  14. [22]

    P., Joshi, C

    Dwivedi, V. P., Joshi, C. K., Luu, A. T., Laurent, T., Bengio, Y., and Bresson, X. Benchmarking graph neural networks. Journal of Machine Learning Research, 24 0 (43): 0 1--48, 2023. URL http://jmlr.org/papers/v24/22-0567.html

    2023

  15. [23]

    A fair comparison of graph neural networks for graph classification

    Errica, F., Podda, M., Bacciu, D., and Micheli, A. A fair comparison of graph neural networks for graph classification. In International Conference on Learning Representations, 2020

    2020

  16. [24]

    Moghaddam, A., and Wattenhofer, R

    Faber, L., K. Moghaddam, A., and Wattenhofer, R. When comparing to ground truth is wrong: On evaluating GNN explanation methods. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining , pp.\ 332--341, 2021

    2021

  17. [25]

    Structured self-attention architecture for graph-level representation learning

    Fan, X., Gong, M., Xie, Y., Jiang, F., and Li, H. Structured self-attention architecture for graph-level representation learning. Pattern Recognition, 100: 0 107084, 2020

    2020

  18. [26]

    Geometry-enhanced molecular representation learning for property prediction

    Fang, X., Liu, L., Lei, J., He, D., Zhang, S., Zhou, J., Wang, F., Wu, H., and Wang, H. Geometry-enhanced molecular representation learning for property prediction. Nature Machine Intelligence, 4 0 (2): 0 127--134, 2022

    2022

  19. [27]

    Data set quality in Machine Learning : Consistency measure based on group decision making

    Fenza, G., Gallo, M., Loia, V., Orciuoli, F., and Herrera-Viedma, E. Data set quality in Machine Learning : Consistency measure based on group decision making. Applied Soft Computing, 106: 0 107366, 2021

    2021

  20. [28]

    and Lenssen, J

    Fey, M. and Lenssen, J. E. Fast graph representation learning with pytorch geometric. In Representation Learning on Graphs and Manifolds Workshop at ICLR, 2019

    2019

  21. [29]

    A survey of graph edit distance

    Gao, X., Xiao, B., Tao, D., and Li, X. A survey of graph edit distance. Pattern Analysis and Applications, 13 0 (1): 0 113--129, 2010

    2010

  22. [30]

    The HCP multi-pipeline dataset: An opportunity to investigate analytical variability in fMRI data analysis, 2023

    Germani, E., Fromont, E., Maurel, P., and Maumet, C. The HCP multi-pipeline dataset: An opportunity to investigate analytical variability in fMRI data analysis, 2023. arXiv:2312.14493 [q-bio]

    2023 arXiv

  23. [31]

    Structural Fairness -aware Active Learning for Graph Neural Networks

    Han, H., Liu, X., Ma, L., Torkamani, M., Liu, H., Tang, J., and Yamada, M. Structural Fairness -aware Active Learning for Graph Neural Networks . In International Conference on Learning Representations, 2023

    2023

  24. [32]

    Open graph benchmark: Datasets for machine learning on graphs, 2020

    Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., and Leskovec, J. Open graph benchmark: Datasets for machine learning on graphs, 2020

    2020

  25. [33]

    Junction tree variational autoencoder for molecular graph generation

    Jin, W., Barzilay, R., and Jaakkola, T. Junction tree variational autoencoder for molecular graph generation. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp.\ 2323...

    2018

  26. [34]

    M., Morris, C., Mutzel, P., and Neumann, M

    Kersting, K., Kriege, N. M., Morris, C., Mutzel, P., and Neumann, M. Benchmark data sets for graph kernels, 2016

    2016

  27. [35]

    Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017

    2017

  28. [36]

    M., Johansson, F

    Kriege, N. M., Johansson, F. D., and Morris, C. A survey on graph kernels. Applied Network Science, 5 0 (1), 2020

    2020

  29. [37]

    J., Paige, B., and Hern \'a ndez-Lobato, J

    Kusner, M. J., Paige, B., and Hern \'a ndez-Lobato, J. M. Grammar variational autoencoder. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp.\ 1945--1954. PMLR, A...

    1945

  30. [38]

    Graphs over time: densification laws, shrinking diameters and possible explanations

    Leskovec, J., Kleinberg, J., and Faloutsos, C. Graphs over time: densification laws, shrinking diameters and possible explanations. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery in Data Mining, pp.\ 177--187, 2005

    2005

  31. [39]

    Explainability in graph neural networks: An experimental survey, March 2022

    Li, P., Yang, Y., Pagnucco, M., and Song, Y. Explainability in graph neural networks: An experimental survey, March 2022. arXiv:2203.09258 [cs]

    2022 arXiv

  32. [40]

    Rethinking the effectiveness of graph classification datasets in benchmarks for assessing gnns

    Li, Z., Cao, Y., Shuai, K., Miao, Y., and Hwang, K. Rethinking the effectiveness of graph classification datasets in benchmarks for assessing gnns. In Proceedings of the International Joint Conference on Artificial Intelligence, pp.\ 2144–2152, 2024

    2024

  33. [41]

    L., Gupta, V., Bhalerao, O., and Lim, S

    Lim, D., Hohne, F., Li, X., Huang, S. L., Gupta, V., Bhalerao, O., and Lim, S. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In Advances in Neural Information Processing Systems, pp.\ 20887--20902, 2021

    2021

  34. [42]

    Metric space magnitude for evaluating the diversity of latent representations

    Limbeck, K., Andreeva, R., Sarkar, R., and Rieck, B. Metric space magnitude for evaluating the diversity of latent representations. In Advances in Neural Information Processing Systems, 2024

    2024

  35. [43]

    Graph matching with bi-level noisy correspondence

    Lin, Y., Yang, M., Yu, J., Hu, P., Zhang, C., and Peng, X. Graph matching with bi-level noisy correspondence. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 23362--23371, 2023

    2023

  36. [44]

    Revisiting heterophily for graph neural networks

    Luan, S., Hua, C., Lu, Q., Zhu, J., Zhao, M., Zhang, S., Chang, X., and Precup, D. Revisiting heterophily for graph neural networks. In Advances in Neural Information Processing Systems, 2022

    2022

  37. [45]

    When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability

    Luan, S., Hua, C., Xu, M., Lu, Q., Zhu, J., Chang, X., Fu, J., Leskovec, J., and Precup, D. When do graph neural networks help with node classification? investigating the homophily principle on node distinguishability. In Advances in Neural Information Processing Systems, 2023

    2023

  38. [46]

    Demystifying structural disparity in graph neural networks: Can one size fit all? In Advances in Neural Information Processing Systems, 2023

    Mao, H., Chen, Z., Jin, W., Han, H., Ma, Y., Zhao, T., Shah, N., and Tang, J. Demystifying structural disparity in graph neural networks: Can one size fit all? In Advances in Neural Information Processing Systems, 2023

    2023

  39. [47]

    R., Quaye, J., Rastogi, C., Kiela, D., Jurado, D., Kanter, D., Mosquera, R., Cukierski, W., Ciro, J., Aroyo, L., Acun, B., Chen, L., Raje, M., Bartolo, M., Eyuboglu, E

    Mazumder, M., Banbury, C., Yao, X., Karla s , B., Gaviria Rojas, W., Diamos, S., Diamos, G., He, L., Parrish, A., Kirk, H. R., Quaye, J., Rastogi, C., Kiela, D., Jurado, D., Kanter, D., Mosquera, R., Cukierski, W., Ciro, J., Aroyo, L., Acun, B., Chen, L., Raje, M., Bartolo, M....

    2023

  40. [48]

    Path Neural Networks : Expressive and accurate graph neural networks

    Michel, G., Nikolentzos, G., Lutzeyer, J., and Vazirgiannis, M. Path Neural Networks : Expressive and accurate graph neural networks. In International Conference on Machine Learning, 2023

    2023

  41. [49]

    M., Bause, F., Kersting, K., Mutzel, P., and Neumann, M

    Morris, C., Kriege, N. M., Bause, F., Kersting, K., Mutzel, P., and Neumann, M. Tudataset: A collection of benchmark datasets for learning with graphs. In Graph Representation Learning and Beyond Workshop at ICML, 2020

    2020

  42. [50]

    I., Levie, R., Lim, D., Bronstein, M

    Morris, C., Frasca, F., Dym, N., Maron, H., Ceylan, I. I., Levie, R., Lim, D., Bronstein, M. M., Grohe, M., and Jegelka, S. Position: Future directions in the theory of graph machine learning. In International Conference on Machine Learning, 2024

    2024

  43. [51]

    Networks

    Newman, M. Networks. Oxford University Press, 2018

    2018

  44. [52]

    AIDS antiviral screen data, 2004

    NIH National Cancer Institute . AIDS antiviral screen data, 2004

    2004

  45. [53]

    A., and Perozzi, B

    Palowitch, J., Tsitsulin, A., Mayer, B. A., and Perozzi, B. Graphworld: Fake graphs bring real insights for gnns. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.\ 3691--3701, 2022

    2022

  46. [54]

    Characterizing graph datasets for node classification: Homophily-heterophily dichotomy and beyond

    Platonov, O., Kuznedelev, D., Babenko, A., and Prokhorenkova, L. Characterizing graph datasets for node classification: Homophily-heterophily dichotomy and beyond. In Advances in Neural Information Processing Systems, 2023 a

    2023

  47. [55]

    A critical look at the evaluation of gnns under heterophily: Are we really making progress? In International Conference on Learning Representations, 2023 b

    Platonov, O., Kuznedelev, D., Diskin, M., Babenko, A., and Prokhorenkova, L. A critical look at the evaluation of gnns under heterophily: Are we really making progress? In International Conference on Learning Representations, 2023 b

    2023

  48. [56]

    Quantifying the alignment of graph and features in deep learning

    Qian, Y., Expert, P., Rieu, T., Panzarasa, P., and Barahona, M. Quantifying the alignment of graph and features in deep learning. IEEE Trans. Neural Networks Learn. Syst. , 33 0 (4): 0 1663--1672, 2022

    2022

  49. [57]

    P., Luu, A

    Rampásek, L., Galkin, M., Dwivedi, V. P., Luu, A. T., Wolf, G., and Beaini, D. Recipe for a general, powerful, scalable graph transformer. In Advances in Neural Information Processing Systems, 2022

    2022

  50. [58]

    and Klein, D

    Randi \'c , M. and Klein, D. Resistance distance. J. Math. Chem, 12: 0 81--95, 1993

    1993

  51. [59]

    BAGEL : A Benchmark for Assessing Graph Neural Network Explanations , 2022

    Rathee, M., Funke, T., Anand, A., and Khosla, M. BAGEL : A Benchmark for Assessing Graph Neural Network Explanations , 2022. arXiv:2206.13983 [cs]

    2022 arXiv

  52. [60]

    and Bunke, H

    Riesen, K. and Bunke, H. IAM graph database repository for graph based pattern recognition and machine learning. In da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J. T., Georgiopoulos, M., Anagnostopoulos, G. C., and Loog, M. (eds.), Structural, Syntactic, and Statistical ...

    2008

  53. [61]

    F., Damrich, S., and Hamprecht, F

    Sanmart \'i n, E. F., Damrich, S., and Hamprecht, F. A. The algebraic path problem for graph metrics. In International Conference on Machine Learning, 2022

    2022

  54. [62]

    BRENDA , the enzyme database: updates and major new developments

    Schomburg, I., Chang, A., Ebeling, C., Gremse, M., Heldt, C., Huhn, G., and Schomburg, D. BRENDA , the enzyme database: updates and major new developments. Nucleic Acids Research, 32: 0 D431--D433, 2004

    2004

  55. [63]

    A survey of graph neural networks for social recommender systems

    Sharma, K., Lee, Y.-C., Nambi, S., Salian, A., Shah, S., Kim, S.-W., and Kumar, S. A survey of graph neural networks for social recommender systems. ACM Comput. Surv., 56 0 (10), 2024

    2024

  56. [64]

    J., Mehlhorn, K., and Borgwardt, K

    Shervashidze, N., Schweitzer, P., van Leeuwen, E. J., Mehlhorn, K., and Borgwardt, K. M. Weisfeiler-lehman graph kernels. Journal of Machine Learning Research, 12 0 (77): 0 2539--2561, 2011

    2011

  57. [65]

    One model many scores: Using multiverse analysis to prevent fairness hacking and evaluate the influence of model design decisions

    Simson, J., Pfisterer, F., and Kern, C. One model many scores: Using multiverse analysis to prevent fairness hacking and evaluate the influence of model design decisions. In ACM Conference on Fairness, Accountability, and Transparency, pp.\ 1305--1320, 2024

    2024

  58. [66]

    and Gilbert, A

    Sonthalia, R. and Gilbert, A. Tree! I am no tree! I am a low dimensional hyperbolic embedding. Advances in Neural Information Processing Systems, 33: 0 845--856, 2020

    2020

  59. [67]

    and Irwin, J

    Sterling, T. and Irwin, J. J. Zinc 15 – ligand discovery for everyone. Journal of Chemical Information and Modeling, 55 0 (11): 0 2324--2337, 2015. doi:10.1021/acs.jcim.5b00559. URL https://doi.org/10.1021/acs.jcim.5b00559

    2015 doi

  60. [68]

    M., Yang, K., Swanson, K., Jin, W., Cubillos-Ruiz, A., Donghia, N

    Stokes, J. M., Yang, K., Swanson, K., Jin, W., Cubillos-Ruiz, A., Donghia, N. M., MacNair, C. R., French, S., Carfrae, L. A., Bloom-Ackermann, Z., et al. A deep learning approach to antibiotic discovery. Cell, 180 0 (4): 0 688--702, 2020

    2020

  61. [69]

    M., and Strube, M

    Taha, D., Zhao, W., Riestenberg, J. M., and Strube, M. Normed spaces for graph embedding. Transactions on Machine Learning Research, 2023

    2023

  62. [70]

    Robust attributed graph alignment via joint structure learning and optimal transport

    Tang, J., Zhang, W., Li, J., Zhao, K., Tsung, F., and Li, J. Robust attributed graph alignment via joint structure learning and optimal transport. In International Conference on Data Engineering, pp.\ 1638--1651, 2023

    2023

  63. [71]

    C., Dat, H

    Thang, D. C., Dat, H. T., Tam, N. T., Jo, J., Hung, N. Q. V., and Aberer, K. Nature vs. nurture: Feature vs. structure for graph neural networks. Pattern Recognition Letters, 159: 0 46--53, 2022

    2022

  64. [72]

    Where did the gap go? reassessing the long-range graph benchmark

    T \"o nshoff, J., Ritzert, M., Rosenbluth, E., and Grohe, M. Where did the gap go? reassessing the long-range graph benchmark. In Learning on Graphs Conference, 2023

    2023

  65. [73]

    and Yamada, M

    Toyokuni, A. and Yamada, M. Structural explanations for Graph Neural Networks using HSIC , February 2023. arXiv:2302.02139 [cs, stat]

    2023 arXiv

  66. [74]

    Graph attention networks

    Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., and Bengio, Y. Graph attention networks. In International Conference on Learning Representations, 2018

    2018

  67. [75]

    and Karypis, G

    Wale, N. and Karypis, G. Comparison of descriptor spaces for chemical compound retrieval and classification. In International Conference on Data Mining, pp.\ 678--689, 2006

    2006

  68. [76]

    and Shen, H.-W

    Wang, X. and Shen, H.-W. GNNInterpreter : A probabilistic generative model-level explanation for Graph Neural Networks , 2023

    2023

  69. [77]

    Mapping the multiverse of latent representations

    Wayland, J., Coupette, C., and Rieck, B. Mapping the multiverse of latent representations. In International Conference on Machine Learning, 2024

    2024

  70. [78]

    N., Gomes, J., Geniesse, C., Pappu, A

    Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learning. Chem. Sci., 9: 0 513--530, 2018 a . doi:10.1039/C7SC02664A. URL http://dx.doi.org/10.1039/C7SC02664A

    2018 doi

  71. [79]

    N., Gomes, J., Geniesse, C., Pappu, A

    Wu, Z., Ramsundar, B., Feinberg, E. N., Gomes, J., Geniesse, C., Pappu, A. S., Leswing, K., and Pande, V. Moleculenet: a benchmark for molecular machine learning. Chemical science, 9 0 (2): 0 513--530, 2018 b

    2018

  72. [80]

    Task-agnostic graph explanations

    Xie, Y., Katariya, S., Tang, X., Huang, E., Rao, N., Subbian, K., and Ji, S. Task-agnostic graph explanations. In Advances in Neural Information Processing Systems, pp.\ 12027--12039, 2022

    2022

  73. [81]

    How powerful are graph neural networks?, 2019

    Xu, K., Hu, W., Leskovec, J., and Jegelka, S. How powerful are graph neural networks?, 2019. arXiv:1810.00826 [cs]

    2019 arXiv

  74. [82]

    and Vishwanathan, S

    Yanardag, P. and Vishwanathan, S. Deep graph kernels. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.\ 1365--1374, 2015

    2015

  75. [83]

    Graphformers: Gnn-nested transformers for representation learning on textual graph

    Yang, J., Liu, Z., Xiao, S., Li, C., Lian, D., Agrawal, S., Singh, A., Sun, G., and Xie, X. Graphformers: Gnn-nested transformers for representation learning on textual graph. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Ne...

    2021

  76. [84]

    A new perspective on the effects of spectrum in graph neural networks

    Yang, M., Shen, Y., Li, R., Qi, H., Zhang, Q., and Yin, B. A new perspective on the effects of spectrum in graph neural networks. In International Conference on Machine Learning, pp.\ 25261--25279, 2022

    2022

  77. [85]

    L., and Leskovec, J

    Ying, R., He, R., Chen, K., Eksombatchai, P., Hamilton, W. L., and Leskovec, J. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.\ 974--983, 2018

    2018

  78. [86]

    Graph random neural features for distance-preserving graph representations

    Zambon, D., Alippi, C., and Livi, L. Graph random neural features for distance-preserving graph representations. In International Conference on Machine Learning, pp.\ 10968--10977, 2020

    2020

  79. [87]

    A pipeline for fair comparison of graph neural networks in node classification tasks, 2020

    Zhao, W., Zhou, D., Qiu, X., and Jiang, W. A pipeline for fair comparison of graph neural networks in node classification tasks, 2020. arXiv:2012.10619 [cs]

    2020 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.