Pith. sign in

REVIEW 4 major objections 7 minor 57 references

Measuring What Matters: A Unified Evaluation Framework for GNN Explainability

T0 review · 4 major / 7 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A ground-truth-free framework ranks GNN explainers by scoring edge and feature attributions separately and finds no method universally best.

desk verdict Solid dual-attribution, ground-truth-free G-XAI benchmark with useful Pareto rankings and practitioner guidelines; selection bias and medoid-proxy caveats are real but scoped, not fatal. read the letter →

arxiv 2607.04600 v1 pith:BKEULY7L submitted 2026-07-06 cs.LG

Francesco Paolo Nerini , Mirko Zaffaroni , Paolo Baracco , Gabriele Ciravegna , Alan Perotti This is my paper · ORCID
classification cs.LG
keywords GNNexplainabilityGraphXAIevaluationmetricsParetofrontInput×Gradientstabilitypertinenceeffectivecompactness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Practitioners face a flood of post-hoc explainers for graph neural networks yet lack objective, ground-truth-free ways to choose among them or to judge whether the outputs are trustworthy. This paper adapts four tabular quality measures—Stability, Pertinence, Effective Compactness and Time—so that topological (edge) and node-feature attributions are scored as independent components, without planted motifs or oracle rationales. Across roughly two million explanations on ten node- and edge-level classification and regression tasks, simple gradient methods such as Input×Gradient consistently sit on the Pareto front of faithfulness-versus-compactness and faithfulness-versus-speed, while several elaborate graph-native methods lag. The authors convert the empirical ranking into concrete quality thresholds and selection guidelines that teams can apply when auditing or deploying GNN pipelines.

What carries the argument

The dual-metric suite (Stability, Pertinence, Effective Compactness, Time) that evaluates feature matrices and edge vectors independently via same-input consistency, ranked deletion curves versus random baselines, and counterfactual modification counts capped at 100, all without reference explanations.

What would settle it

On a held-out proprietary graph task that also carries human-validated rationales, the Pareto ranking produced by the four metrics places explainers that auditors reject above those they accept, or Input×Gradient’s high Pertinence fails to correlate with auditor preference.

Watch

Extended reading notes

Core claim

When edge and feature attributions are scored independently with ground-truth-free metrics of stability, faithfulness, compactness and runtime, no single explainer dominates; Input×Gradient and Integrated Gradients emerge as robustly non-dominated solutions on the Pareto front across metric pairs and GraphML tasks, while more complex graph-specific methods frequently underperform.

Load-bearing premise

The ranking produced by how quickly ordered edge removals or feature substitutions flip the model’s own prediction, measured against random orderings and without any human or planted ground truth, is a sufficient proxy for industrial trustworthiness and can safely guide explainer selection.

Editorial extensions

If this is right

  • Teams can short-list explainers from the reported Pareto fronts and task tables instead of relying on anecdotal preference.
  • Stability near 1.0 flags fully deterministic methods that need only a single invocation for reproducible audits.
  • Empirically derived thresholds (e.g., Pertinence ≥ 0.6, Stability ≥ 0.8) become concrete acceptance criteria for production G-XAI components.
  • Natural-language templates can turn the dual attributions into human-readable narratives without altering the underlying scores.
  • Any new explainer that emits both edge and feature attributions can be dropped into the same modular evaluator and compared on identical axes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same dual scoring can be reused for counterfactual or concept-based explainers once they emit edge and feature attributions.
  • Sub-0.5 Pertinence under multimodal classes implies that single-medoid reference graphs are too crude; adaptive clustering of alternatives may be needed.
  • Because the metrics are model-faithful rather than human-plausible, pairing them with a separate human study would close the remaining regulatory gap.
  • Edge Effective-Compactness thresholds will transfer poorly across degree distributions, so degree-stratified revalidation should become standard on proprietary graphs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a ground-truth-free evaluation framework for post-hoc GNN explainers that scores feature and edge attributions independently via Stability, Pertinence, Effective Compactness, and wall-clock Time. Metrics are formalized with explicit algorithms (Algorithms 1–7), early-stopping for iterative estimates, and task-specific handling of node vs. link targets. The authors implement the framework in a proprietary modular library, then benchmark eight dual-attribution, open-license explainers on ~2M explanations across ten GraphML tasks (node/edge × classification/regression) using GCN and GAT models on five datasets. Results identify Input×Gradient and Integrated Gradients as consistently non-dominated on Pareto fronts for pertinence–compactness and pertinence–time, while no explainer is universally superior; findings are distilled into operational thresholds and selection guidelines in Appendix A.

Significance. The work addresses a genuine practical gap: how to compare GNN explainers on real data without planted motifs, while covering regression and link-level tasks that prior G-XAI benchmarks largely omit. Strengths include explicit dual (feature/edge) metric definitions, large-scale multi-task averaging, Pareto analysis rather than single-score ranking, and concrete practitioner guidelines with transferability notes. If the proxies and scoped ranking hold under broader validation, the framework and default recommendation of Input×Gradient would be immediately useful for regulated GNN pipelines. Credit is due for ground-truth-free design, early-stopping optimization, and honest reporting of sub-0.5 pertinence regimes and selection filters. Reproducibility is limited by the proprietary library, which reduces the paper’s archival value as a community benchmark.

major comments (4)
  1. §5.1 and Conclusion: The dual-attribution + open-license filters exclude many widely cited graph-native methods (PGExplainer, SubgraphX, GNN-LRP, GraphLIME, CLEAR, etc.). The abstract and §5.2 claim that Input×Gradient/IntGrad are robustly non-dominated “across metric pairs and tasks” and that no single explainer is universally superior. Those claims are only supported inside the eight-method subset. The manuscript acknowledges selection bias but still frames results as guidance for “state-of-the-art” G-XAI. The central recommendation and Pareto narrative need to be restated as conditional on dual-attribution, open-license methods, with an explicit non-claim about excluded families, or the benchmark must be extended with at least a subset of those methods under a relaxed dual-attribution protocol.
  2. §3 Algorithms 5–7 and §5.2: Feature Pertinence and feature EC rely on medoid reference graphs and ranked substitutions capped at 100. The paper itself reports that multimodal opposing classes make a single medoid a poor nearest alternative, so random orderings can match or beat the attribution and drive pertinence below the 0.5 baseline; ties are also counted as failures. Appendix A.2.2 then still proposes Pertinence ≥0.6 and Feature EC percentage bounds as deployment thresholds. If these failure modes are common on proprietary graphs, the Pareto fronts and Table 3 guidelines misidentify trustworthy explainers. A load-bearing revision is needed: quantify how often sub-0.5 pertinence occurs per dataset/task, add a sensitivity analysis (multiple references, k-medoids, or class-conditional sampling), and either revise the thresholds with failure-mode caveats or restrict the “default explain
  3. §4 and reproducibility: The evaluation library is proprietary and “cannot be released as open source”; only “selected snippets” are offered on request. For a paper whose primary contribution is a unified benchmarking framework and operational thresholds, this is a material obstacle. Without public code for metric implementations (especially medoid selection, computational-graph extraction, early stopping, and NLX), independent verification of Table 1, Figure 2, and Appendix A thresholds is not possible. At minimum, release metric pseudocode as executable reference implementations, full hyperparameter configs, and the exact target indices used for the 100 explanations per task-dataset, or provide a public reimplementation of the metric suite on standard PyG models.
  4. §5.1–5.2 and Appendix B: Regression and several edge results rest heavily on Movielens (dense, genre features) and a fully synthetic ER graph with an arbitrary weighted aggregation rule. The text notes that edge primacy in regression and limited feature-EC variability “may derive from the specific synthetic datasets” and “might not generalize.” Yet Appendix A still issues task-level default recommendations and transferability guidance for production. Either expand the regression/link suite with additional real datasets, or demote the regression Pareto conclusions and corresponding guideline rows to exploratory, with explicit dataset-dependence warnings in the main text rather than only in a late caveat.
minor comments (7)
  1. Abstract and §1: “∼2M explanations” is striking but not broken down (targets × explainers × models × stability/pertinence trials). A short accounting in §5.1 would help readers assess statistical power and early-stopping savings.
  2. Figure 2: Pareto fronts are clear, but axis scales and marker overlap make it hard to see which methods sit on the front for edge regression feature EC. Consider small multiples with labels or a supplementary table of non-dominated sets per panel.
  3. Table 1: Standard deviations are large for EC (often ±20–50). Report median and IQR or confidence intervals on the means, and clarify whether averaging is hierarchical (targets → datasets → models) or flat.
  4. §3: Notation for L2 normalization and ρ_edge / ρ_feat is slightly inconsistent between prose and Algorithms 2–3 (e.g., ∥·∥2 vs. /2 scaling). Unify the formulas.
  5. Related Work: GraphFramEx, GraphXAI, BAGEL, and GnnX-Bench are discussed; a compact comparison table (ground-truth requirement, tasks covered, dual attribution, production metrics) would sharpen the three-axis differentiation claimed in §2.
  6. Appendix A.2.3: Feature EC thresholds as “10–25% of total features” are not directly comparable to the absolute EC scores (capped at 100) reported in Table 1. Align units or show how percentage maps to the reported numbers per dataset.
  7. Typos / polish: “Toanswerthesequestions”, “ComputationalOptimisationWhileourmetrics…”, missing spaces in several run-on phrases in §1 and §3; “quasi-determinism” vs. Stability definition could be cross-referenced once for consistency.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical metric definitions and measured Pareto rankings are self-contained, not forced by construction or load-bearing self-citation.

full rationale

This is an empirical benchmarking paper. Stability, Pertinence, Effective Compactness and Time are operationally defined via explicit algorithms (Algs. 1–7) that score an explainer’s outputs against the model’s own behaviour under controlled re-runs, ranked deletions and medoid substitutions; the definitions do not algebraically encode which explainer will win. The ~2 M explanations, Table 1 averages, Figure 2 Pareto fronts and Appendix A thresholds are post-hoc summaries of those measurements, not predictions that reduce to fitted constants. Mild citations to the authors’ prior tabular work [6] and true-to-the-model papers supply background for the quality dimensions and the ground-truth-free stance, but the graph-specific dual-attribution implementations, multi-task experimental design and resulting rankings stand independently. No uniqueness theorem, ansatz or self-definitional loop is invoked to force the central claim that Input×Gradient / Integrated Gradients are robustly non-dominated. Potential metric limitations (medoid multimodality, 100-cap) are correctness/transfer risks, not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claims rest on a true-to-the-model evaluation stance, dual independent scoring of feature and edge attributions, and several hand-set computational thresholds (EC cap, early-stop τ/ε, sample sizes). No new physical entities; free parameters are engineering cutoffs and guideline thresholds derived from the same benchmark used to recommend explainers. Domain assumptions about what industrial quality means (determinism, cognitive load via flip-count, faithfulness vs random, latency) are imported from tabular XAI and asserted for graphs without external human-subject validation.

free parameters (4)
  • Effective Compactness iteration cap = 100
    EC is truncated at min(100, |E_rel| or |f_C|); the 100 bound is a hand-chosen comprehensibility/compute limit that shapes all EC scores and Pareto plots.
  • Early-stopping τ and ε for Stability/Pertinence = τ=0.5, ε=0.05, N∈[30,100]
    Convergence uses τ=0.5 (random baseline for Pertinence) and ε=0.05 with min 30 / max 100 trials; these control when estimates stop and thus measured means.
  • Guideline quality thresholds (Pertinence ≥0.6, Stability ≥0.8, Feature EC % bounds, Time <5s) = e.g. Pertinence≥0.6, Stability≥0.8
    Appendix A thresholds are empirically set from the same benchmark used to recommend Input×Gradient; they are free policy parameters for acceptance, not independently validated on held-out production systems.
  • Number of explained targets per task-dataset = 100
    Fixed at 100 targets per combination; sample size is a design choice affecting variance of averages.
assumptions (5)
  • domain assumption Explanations should be judged true-to-the-model (consistency with model behavior) rather than agreement with human- or motif-defined ground truth.
    Stated in Introduction and Related Work as the reason to reject planted-motif evaluation; load-bearing for claiming the framework needs no ground truth and still measures trustworthiness.
  • domain assumption Graph explanations have dual independent components—node-feature attributions and edge attributions—that should be scored separately.
    Core design of all metrics and the dual-attribution explainer filter (§3–§5).
  • domain assumption Industrial explanation quality reduces to quasi-determinism, low cognitive load, model adherence, and low runtime (following [6]).
    §3 maps these four qualities to Stability, EC, Pertinence, and Time; other desiderata (fairness, human plausibility, counterfactual minimality) are out of scope.
  • domain assumption Standard GNN message-passing models (GCN, GAT) and post-hoc attribution/perturbation explainers are the right evaluation substrate.
    Experimental setup uses 2-layer GCN/GAT only; self-interpretable and many counterfactual methods are excluded.
  • ad hoc to paper L2-normalized pairwise distances, ranked deletion/substitution curves, and medoid reference graphs are valid operationalizations of stability, compactness, and faithfulness on graphs.
    Algorithms 1–7 define the concrete scores; alternatives (e.g., different norms, non-medoid refs) would change rankings.
invented entities (2)
  • Dual graph Pertinence / Effective Compactness / Stability suite (feature and edge variants with early stopping)
    purpose: Provide ground-truth-free quantitative scores for GNN explanations at industrial scale.
    Adaptations of tabular metrics with graph-specific computational-graph filtering, medoid refs, and dual scoring; independent evidence is only the internal benchmark, not external human or causal validation.
  • Proprietary modular G-XAI evaluation library (BaseModel/BaseExplainer/Xplanation/Evaluator + NLX)
    purpose: Implement metrics and natural-language templates for deployment workflows.
    Described in §4 but not released; no independent public artifact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring What Matters: A Unified Evaluation Framework for GNN Explainability." pith.science (2026). https://pith.science/paper/BKEULY7L

@misc{pith2026260704600,
  author       = {Pith},
  title        = {Pith review of: Measuring What Matters: A Unified Evaluation Framework for GNN Explainability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BKEULY7L}},
  note         = {Machine review of arXiv:2607.04600}
}
read the original abstract

Graph eXplainable AI (G-XAI) is increasingly important for making Graph Neural Networks interpretable and accountable. While a growing number of explainers are available, choosing the right method and assessing the trustworthiness of its outputs remains unclear. Consistent evaluation practices and actionable guidance are still missing, hindering practical adoption. In this paper, we introduce a unified, quantitative benchmarking framework for G-XAI that requires no ground-truth assumptions. We formalize tabular explainability metrics for graph data, evaluating topological structure and node features as independent components. Our large-scale benchmarking study identifies explainers that consistently lie on the Pareto front across metric pairs and tasks, establishing robustly non-dominated solutions - while confirming that no single explainer achieves universal superiority. We distill our findings into actionable G-XAI usability guidelines to support Machine Learning practitioners in evaluating and deploying trustworthy GNN-based pipelines.

Figures

Figures reproduced from arXiv: 2607.04600 by the authors.

Figure 1
Figure 1. Architectural overview of the proposed XAI framework, illustrating the decou [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Pareto plots comparing the explainers across pairs of metrics. In all cases, the [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 2 linked inside Pith

  1. [1]

    Zonghan Wu, Shirui Pan, Feng Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. A comprehensive survey on graph neural networks.IEEE Transactions on Neural Networks and Learning Systems, 32(1):4–24, 2020

    2020

  2. [2]

    Inductive representation learning on large graphs

    Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. InAdvances in Neural Information Processing Systems (NeurIPS), volume 30, 2017

    2017

  3. [3]

    Gnnexplainer: Generating explanations for graph neural networks.Advances in neural information processing systems, 32, 2019

    Zhitao Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Generating explanations for graph neural networks.Advances in neural information processing systems, 32, 2019

    2019

  4. [4]

    Graphlime: Local interpretable model explanations for graph neural networks.IEEE Transactions on Knowledge and Data Engineering, 35(7):6968–6972, 2022

    Qiang Huang, Makoto Yamada, Yuan Tian, Dinesh Singh, and Yi Chang. Graphlime: Local interpretable model explanations for graph neural networks.IEEE Transactions on Knowledge and Data Engineering, 35(7):6968–6972, 2022

    2022

  5. [5]

    On explainability of graph neural networks via subgraph explorations

    Hao Yuan, Haiyang Yu, Jie Wang, Kang Li, and Shuiwang Ji. On explainability of graph neural networks via subgraph explorations. InInternational Conference on Machine Learning (ICML), pages 12241–12252. PMLR, 2021

    2021

  6. [6]

    Explainability, quantified: Benchmarking xai techniques

    Alan Perotti, Claudio Borile, Arianna Miola, Francesco Paolo Nerini, Paolo Baracco, and André Panisson. Explainability, quantified: Benchmarking xai techniques. In World Conference on Explainable Artificial Intelligence. Springer, 2024

    2024

  7. [7]

    Bagel: A benchmark for assessing graph neural network explanations

    Mandeep Rathee, Thorben Funke, Avishek Anand, and Megha Khosla. Bagel: A benchmark for assessing graph neural network explanations. InThirty-sixth Con- ference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022

    2022

  8. [8]

    Evaluating explainability for graph neural networks.Scientific Data, 10(1):144, 2023

    Chirag Agarwal, Owen Queen, Himanshu Chugh, and Marinka Zitnik. Evaluating explainability for graph neural networks.Scientific Data, 10(1):144, 2023

    2023

Show all 57 references
  1. [9]

    A true-to-the-model axiomatic benchmark for graph-based explainers.Trans

    Corrado Monti, Paolo Bajardi, Francesco Bonchi, André Panisson, and Alan Perotti. A true-to-the-model axiomatic benchmark for graph-based explainers.Trans. Mach. Learn. Res., 2024, 2024

    2024

  2. [10]

    Graphxai: a survey of graph neural networks (gnns) for explainable ai (xai).Neural Computing and Applications, 37(17):10949–11000, 2025

    Mauparna Nandan, Soma Mitra, and Debashis De. Graphxai: a survey of graph neural networks (gnns) for explainable ai (xai).Neural Computing and Applications, 37(17):10949–11000, 2025

    2025

  3. [11]

    The graph neural network model.IEEE transactions on neural net- works, 20(1):61–80, 2008

    Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model.IEEE transactions on neural net- works, 20(1):61–80, 2008

    2008

  4. [12]

    Semi-supervised classification with graph convo- lutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convo- lutional networks. InInternational Conference on Learning Representations, 2017. 17

    2017

  5. [13]

    Graph attention networks

    Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. InInternational Conference on Learning Representations, 2018

    2018

  6. [14]

    How powerful are graph neural networks? InInternational Conference on Learning Representations, 2018

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? InInternational Conference on Learning Representations, 2018

    2018

  7. [15]

    Spatial graph convolutional networks

    Tomasz Danel, Przemysław Spurek, Jacek Tabor, Marek Śmieja, Łukasz Struski, Agnieszka Słowik, and Łukasz Maziarka. Spatial graph convolutional networks. In International conference on neural information processing. Springer, 2020

    2020

  8. [16]

    A survey on self-supervised graph foundation models: Knowledge-based perspective

    Ziwen Zhao, Yixin Su, Yuhua Li, Yixiong Zou, Ruixuan Li, and Rui Zhang. A survey on self-supervised graph foundation models: Knowledge-based perspective. IEEE Transactions on Knowledge and Data Engineering, 2025

    2025

  9. [17]

    Deepinsideconvolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013

    KarenSimonyan, AndreaVedaldi, andAndrewZisserman. Deepinsideconvolutional networks: Visualising image classification models and saliency maps.arXiv preprint arXiv:1312.6034, 2013

    2013 arXiv

  10. [18]

    Learning important features through propagating activation differences

    Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. InInternational conference on machine learning, pages 3145–3153. PMlR, 2017

    2017

  11. [19]

    Axiomatic attribution for deep networks

    Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. InInternational conference on machine learning. PMLR, 2017

    2017

  12. [20]

    Striving for simplicity: The all convolutional net

    J Springenberg, Alexey Dosovitskiy, Thomas Brox, and M Riedmiller. Striving for simplicity: The all convolutional net. InICLR (workshop track), 2015

    2015

  13. [21]

    Visualizing and understanding convolutional networks

    Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. InEuropean conference on computer vision. Springer, 2014

    2014

  14. [22]

    On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.PloS one, 10(7):e0130140, 2015

    Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus- Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.PloS one, 10(7):e0130140, 2015

    2015

  15. [23]

    Higher-order explanations of graph neural networks via relevant walks.IEEE transactions on pattern analysis and ma- chine intelligence, 44(11):7581–7596, 2021

    ThomasSchnake, OliverEberle, JonasLederer, ShinichiNakajima, KristofTSchütt, Klaus-Robert Müller, and Grégoire Montavon. Higher-order explanations of graph neural networks via relevant walks.IEEE transactions on pattern analysis and ma- chine intelligence, 44(11):7581–7596, 2021

    2021

  16. [24]

    Parameterized explainer for graph neural network.Advances in neural information processing systems, 33:19620–19631, 2020

    Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network.Advances in neural information processing systems, 33:19620–19631, 2020. 18

    2020

  17. [25]

    Interpreting graph neural networks for nlp with differentiable edge masking

    Michael Sejr Schlichtkrull, Nicola De Cao, and Ivan Titov. Interpreting graph neural networks for nlp with differentiable edge masking. InInternational Conference on Learning Representations, 2021

    2021

  18. [26]

    Explaining identity-aware graph classifiers through the language of motifs

    Alan Perotti, Paolo Bajardi, Francesco Bonchi, and André Panisson. Explaining identity-aware graph classifiers through the language of motifs. In2023 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2023

    2023

  19. [27]

    Cf-gnnexplainer: Counterfactual explanations for graph neural networks

    Ana Lucic, Maartje A Ter Hoeve, Gabriele Tolomei, Maarten De Rijke, and Fabrizio Silvestri. Cf-gnnexplainer: Counterfactual explanations for graph neural networks. InInternational conference on artificial intelligence and statistics. PMLR, 2022

    2022

  20. [28]

    Clear: Generative counterfactual explanations on graphs.Advances in neural information processing systems, 35:25895–25907, 2022

    Jing Ma, Ruocheng Guo, Saumitra Mishra, Aidong Zhang, and Jundong Li. Clear: Generative counterfactual explanations on graphs.Advances in neural information processing systems, 35:25895–25907, 2022

    2022

  21. [29]

    Protgnn: To- wards self-explaining graph neural networks

    Zaixi Zhang, Qi Liu, Hao Wang, Chengqiang Lu, and Cheekong Lee. Protgnn: To- wards self-explaining graph neural networks. InProceedings of the AAAI conference on artificial intelligence, volume 36, pages 9127–9135, 2022

    2022

  22. [30]

    Global explainability of gnns via logic combination of learned concepts

    Steve Azzolin, Antonio Longa, Pietro Barbiero, Pietro Lio, and Andrea Passerini. Global explainability of gnns via logic combination of learned concepts. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  23. [31]

    Graphframex: Towards system- atic evaluation of explainability methods for graph neural networks

    Kenza Amara, Zhitao Ying, Zitao Zhang, Zhichao Han, Yang Zhao, Yinan Shan, Ulrik Brandes, Sebastian Schemm, and Ce Zhang. Graphframex: Towards system- atic evaluation of explainability methods for graph neural networks. InLearning on Graphs Conference, pages 44–1. PMLR, 2022

    2022

  24. [32]

    GNNX-BENCH: Unravelling the utility of perturbation-based GNN explainers through in-depth benchmarking

    Mert Kosan, Samidha Verma, Burouj Armgaan, Khushbu Pahwa, Ambuj Singh, Sourav Medya, and Sayan Ranu. GNNX-BENCH: Unravelling the utility of perturbation-based GNN explainers through in-depth benchmarking. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  25. [33]

    B-xaic dataset: Benchmarking explainable ai for graph neural networks using chemical data.arXiv preprint arXiv:2505.22252, 2025

    Magdalena Proszewska, Tomasz Danel, and Dawid Rymarczyk. B-xaic dataset: Benchmarking explainable ai for graph neural networks using chemical data.arXiv preprint arXiv:2505.22252, 2025

    2025 arXiv

  26. [34]

    Cure-bench.https://kaggle.com/competitions/cure-bench, 2025

    Shanghua Gao, Richard Yuxuan Zhu, Zhenglun Kong, Xiaorui Su, Curtis Ginder, Sufian Aldogom, Ishita Das, Taylor Evans, Theodoros Tsiligkaridis, and Marinka Zitnik. Cure-bench.https://kaggle.com/competitions/cure-bench, 2025. 2025 Neural Information Processing Conference (Neurip...

    2025

  27. [35]

    On quantitative aspects of model interpretability, 2020

    An phi Nguyen and María Rodríguez Martínez. On quantitative aspects of model interpretability, 2020

    2020

  28. [36]

    Revisiting semi-supervised learning with graph embeddings

    Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. InInternational conference on machine learning, pages 40–48. PMLR, 2016. 19

    2016

  29. [37]

    Multi-scale attributed node embedding.Journal of Complex Networks, 9(2):cnab014, 2021

    Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding.Journal of Complex Networks, 9(2):cnab014, 2021

    2021

  30. [38]

    The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

    F Maxwell Harper and Joseph A Konstan. The movielens datasets: History and context.Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015

    2015

  31. [39]

    Degree: Decomposition based explanation for graph neural networks

    Qizhang Feng, Ninghao Liu, Fan Yang, Ruixiang Tang, Mengnan Du, and Xia Hu. Degree: Decomposition based explanation for graph neural networks. In10th Inter- national Conference on Learning Representations, ICLR 2022, 2022

    2022

  32. [40]

    A true-to-the- model benchmark for edge-level attributions of gnn explainers

    Francesco Paolo Nerini, Francesco Bonchi, and André Panisson. A true-to-the- model benchmark for edge-level attributions of gnn explainers. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 2025. A. Operational Guidelines for G-XAI...

    2025

  33. [41]

    binomial

    After 30 trials, compute running meanµand standard error SE=σ/√n. 2.Mean convergence test: Terminate ifz=|µ−τ|/SE>1.645(one-tailed, 95% confidence), whereτis an optional threshold. 3.Precision tolerance test: Terminate ifz=ϵ/SE>1.96(two-tailed, 95% confi- dence), whereϵis the ...

  34. [42]

    ExtractMexplanations such thatM(M−1)/2≈N(target number of pairs)

  35. [43]

    Compute stability across all M 2 possible pairs

  36. [44]

    This approach requires onlyMexplanation generations instead of2N, significantly reducing computational overhead for production monitoring

    Return the average pairwise stability. This approach requires onlyMexplanation generations instead of2N, significantly reducing computational overhead for production monitoring. A.4.3. Trade-off Analysis We recommend conducting parallel experiments during development: •Exhaust...

  37. [45]

    Identify task type (node/link classification/regression)

  38. [46]

    Consult benchmark rankings (Table 2) to shortlist candidate explainers

  39. [47]

    execution time, feature vs

    Evaluate trade-offs: pertinence vs. execution time, feature vs. edge attribution quality

  40. [48]

    Document explainability requirements aligned with regulatory obligations (EU AI Act, domain-specific mandates). A.5.2. Phase 2: Development Validation During model development, validate that benchmark performance transfers to propri- etary data:

  41. [49]

    Wraptrainedmodelusingappropriatemodelclass(e.g.,NodeClassificationModel)

  42. [50]

    ConfigureEvaluatorwith dataset-specific parameters

  43. [51]

    Evaluate shortlisted explainers on representative sample (≥100instances)

  44. [52]

    Compare metrics against thresholds (Table 3)

  45. [53]

    Select final explainer based on validated performance. A.5.3. Phase 3: Production Integration Deploy selected explainer alongside ML model:

  46. [54]

    Integrate explainer into inference pipeline

  47. [55]

    Configure natural language translation for end-user consumption

  48. [56]

    Establish monitoring protocols to track metric stability over time

  49. [57]

    house-shaped

    Define re-validation triggers: model retraining, data distribution shift, regulatory updates. 25 A.6. Translating Explanations for Regulatory Compliance Technical explanations (attribution matrices, edge importance vectors) require transla- tion for non-technical stakeholders....

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.