Pith. sign in

REVIEW 3 major objections 8 minor 1 cited by

ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs

T0 review · 3 major / 8 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ProGMLP claims that a progressive chain of distilled MLP students gives graph inference an adjustable accuracy-latency dial.

desk verdict A plausible progressive G2M distillation framework whose headline efficiency claim is not yet supported, because the trade-off curves count students rather than cost. read the letter →

arxiv 2507.19031 v1 pith:P7UB6TVH submitted 2025-07-25 cs.LG

Weigang Lu , Ziyu Guan , Wei Zhao , Yaming Yang , Yujie Sun , Zheng Liang , Yibing Zhan , Dapeng Tao This is my paper · ORCID
classification cs.LG
keywords GNN-to-MLPdistillationprogressivetrainingknowledgemixupaugmentationearly-exitinferencenodeclassificationgraphneuralnetworksaccuracy-latencytrade-off
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that GNN-to-MLP knowledge distillation need not be a static, single-point trade-off. ProGMLP trains a sequence of MLP students, each initialized from the previous student's parameters and fed the previous student's hidden representation, so that later students progressively refine the distilled knowledge. The claim is that this single trained framework provides a spectrum of operating points on the accuracy-latency curve: a user can stop after a few students for low latency or run all students for maximum accuracy, without retraining. The paper supports the claim with experiments on eight real-world graph datasets, reporting both higher peak accuracy than the compared G2M baselines and large inference speedups, including a 21x speedup over the GCN teacher on ogbn-arxiv.

What carries the argument

The load-bearing mechanism is the Progressive Training Structure: a chain of MLP students in which $f_{k+1}$ inherits $\theta_k$ from $f_k$ and the input to each student is the concatenation of raw features $X$ with the previous student's hidden representation $H_{k-1}$. This turns the sequence into a recurrent-like refinement process rather than an independent ensemble. On top of it sit Progressive Knowledge Distillation, which weights each student's supervised-plus-distillation loss by $k^\beta$ so later students are pushed toward higher accuracy, and Progressive Mixup Augmentation, which makes mixed training samples progressively harder by raising the mixing ratio $\lambda_k$ when the student's moving-average mixup loss falls. The inference mechanism that realizes the trade-off is the confidence-based stopping rule: $c_k$ is the mean of per-node maximum softmax probabilities over all unlabeled nodes, and inference halts at the first student with $c_k \geq \tau_{\mathrm{conf}}$, with Eq. (12) combining the predictions of the executed students by confidence-derived weights.

What would settle it

A concrete test: run ProGMLP with a per-node early exit rule, where each node stops at the first student whose own maximum softmax probability exceeds the threshold, and compare the resulting accuracy-latency curve against the paper's batch-level rule on the same data. If the per-node rule does not match or beat the batch rule in accuracy at equal average latency, then the batch-level confidence statistic is not a faithful controller of per-request trade-offs and the on-demand claim is unsupported.

Watch

Extended reading notes

Core claim

The central claim is that progressive training of multiple MLP students yields an anytime-inference system for GNN-to-MLP distillation. Concretely, student $f_k$ is trained with the previous student's parameters as initialization, receives $\mathrm{CONCAT}(X,H_{k-1})$ as input (with $H_0$ the zero matrix), and is optimized by a loss that combines a cross-entropy-plus-KL distillation term weighted by $k^\beta$ and a mixup term whose mixing ratio $\lambda_k$ rises adaptively as the student's moving-average loss drops. At inference the students are evaluated in sequence; the chain stops at the first student whose average maximum softmax probability over unlabeled nodes, $c_k$ from Eq. (11), reaches a confidence threshold $\tau_{\mathrm{conf}}$, and the final prediction is the confidence-weighted sum of the executed students' predictions (Eq. (12)). The paper reports that on six medium-scale and two large-scale datasets, ProGMLP reaches near-peak accuracy with only a few students, continues to improve as more students run, beats the ensemble baselines GLNN and AdaGMLP on nearly all datasets, and on ogbn-arxiv improves on the teacher while being up to 21x faster.

Load-bearing premise

The load-bearing premise is that the average of the maximum softmax probabilities over all unlabeled nodes is a reliable and controllable proxy for deciding when to stop the student chain, even though this stopping signal is batch-level and the paper's on-demand per-request flexibility claims would need the decision to be made per input.

Editorial extensions

If this is right

  • If the reported results hold, a single trained ProGMLP model can replace separate fast and accurate models: setting the confidence threshold or the maximum number of executed students selects where the deployment sits on the accuracy-latency curve.
  • The progressive structure means early-exit predictions are not wasted: each executed student contributes to the confidence-weighted final prediction, so partial computation always yields a usable output.
  • On large graphs, the framework's inference cost scales with the number of students executed rather than with graph neighborhood size, which is the source of the reported speedups over GNN teachers.
  • Removing the progressive training structure hurts accuracy most (about 4% on Coauthor CS in the ablation), so the chain-like initialization and hidden-state passing, not just the ensemble size, carries the gain.
  • In the inductive setting, the distilled students can generalize to unseen nodes better than the GNN teachers, suggesting the progressive mixup and distillation improve robustness beyond transductive accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper validates the confidence threshold at the batch level, so a per-instance early-exit protocol (stopping each node when its own softmax confidence crosses the threshold) is a natural testable extension the paper does not run; without it, the on-demand per-request flexibility claim remains a batch-level property.
  • Because each student's input includes the previous student's hidden representation, the framework is close to an adaptive computation graph; it could be combined with input-dependent gating to decide per node how many students to run, which would turn the current global stopping rule into a local one.
  • The progressive-mixup schedule is driven by the student's own loss, which couples curriculum difficulty to the model's state; a direct implication is that the same schedule could be applied to other distillation settings, such as GNN-to-GNN compression, though the paper does not test that.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. ProGMLP proposes a progressive GNN-to-MLP distillation framework in which K MLP students are trained sequentially, each initialized from the previous student's parameters and fed the concatenation of the raw features and the previous student's hidden representation (Section III-C). Training combines a Progressive Knowledge Distillation term, a Progressive Mixup Augmentation term, and an optional confidence-based early exit at inference (Section III-F). The framework is evaluated on eight graph datasets with GCN, GAT, and GraphSAGE teachers, against ensemble G2M baselines (E-GLNN, AdaGMLP) and non-ensemble G2M baselines (KRD, HGMD, NOSMOG), including a large-scale latency study and an inductive setting.

Significance. The idea of offering a spectrum of accuracy-latency operating points within GNN-to-MLP distillation is timely, and the paper is generally clearly written. If the trade-off claims were established under cost-controlled comparisons, ProGMLP would be a useful contribution: it introduces concrete mechanisms (PTS, PKD, PMA), provides a code link, evaluates on eight datasets with three teacher architectures, and includes ablations that support the role of the progressive components. Comparing against AdaGMLP, a strong same-group baseline, is legitimate and not circular. However, the current evidence does not yet establish the central 'efficient trade-off' claim because inference cost is measured by student count rather than compute or latency, and the adaptive-stopping mechanism is batch-level rather than per-instance. These issues are fixable with additional experiments or by narrowing the claims.

major comments (3)
  1. [Section IV-F / Fig. 3; Section III-C / III-G] The inference-cost metric used in Figure 3 is '# Executed Students', but ProGMLP students are not cost-equivalent to the baselines. Section III-C defines X_k = CONCAT(X, H_{k-1}), so every student has input dimension d + d' rather than d, and Section III-G gives per-student forward cost O(N d'(d + d' + C)). This is larger than the O(N d'(d + C)) cost of a same-width 2-layer MLP baseline by an additional O(N d'^2) term, a gap that is largest when the feature dimension is small (e.g., ogbn-products with d=128). Equal-#students curves therefore overstate ProGMLP's accuracy-latency advantage. Table II compounds this by comparing ProGMLP (whose K is not stated) against E-GLNN_2/4 and AdaGMLP_2/4 without matching total compute, and Table IV reports latency only against teacher GNNs, not against other G2M students. Please add accuracy-versus-FLOPs and/or accuracy-versus-measured-latency curves for all methods, and include at least one cost-matched comparison (e.g., equal total forward FLOPs per operating point).
  2. [Section III-F.2 / Eq. (11) / Section I] The on-demand inference claim in the introduction and abstract is not supported by the implemented stopping rule. Eq. (11) computes c_k as the mean over all unlabeled nodes of the per-node maximum softmax probability, yielding a single batch-level scalar; the early-exit decision is therefore the same for the entire node set and cannot adapt per request or per node. This is a dataset-level budget selector rather than the per-instance dynamic mechanism suggested by the edge/mobile examples in Section I and by Figure 1. The authors should either provide a per-instance confidence protocol (with experiments measuring accuracy and latency at the instance level) or explicitly reframe ProGMLP as offering K dataset-level operating points selected before batch inference, removing the per-request interpretation. In addition, the paper should explain how tau_conf is calibrated to a latency or accuracy target; the current hyperparameter analysis on CS (Section IV-L) only shows accuracy robustness, not runtime adaptability.
  3. [Section IV-G / Table II] The claim that ProGMLP 'consistently outperforms' E-GLNN and AdaGMLP is not statistically supported. In several dataset/teacher combinations the standard deviations overlap substantially with the strongest baseline and the accuracy gap is small (e.g., GCN/Pubmed: ProGMLP 77.42±0.35 vs AdaGMLP_4 77.21±0.66; GraphSAGE/Computers: ProGMLP 80.21±1.21 vs AdaGMLP_4 79.23±1.06). Please report paired significance tests or exact confidence intervals over the 10 runs, and state in how many of the evaluated settings ProGMLP is significantly better. This matters because Table II is one of the primary empirical bases for the framework's advantage over ensemble G2M methods.
minor comments (8)
  1. [Eq. (10) / Eq. (9)] In Eq. (10), the moving-average recurrence has ell_bar_t on both sides; the right-hand side should use the instantaneous loss ell_t. In the text following Eq. (9), lambda is said to be updated from the 'current mixup loss calculated by Eq. (6)', but Eq. (6) is the PKD loss; the reference should be Eq. (8).
  2. [Eq. (6)] The notation 'kβ' in Eq. (6) is ambiguous: it should be defined explicitly as either k·β or β^k, since the surrounding text only says that later students receive higher weights.
  3. [Section IV-E / Section IV-F] The parameter search space in Section IV-E lists the number of students as {2, 3, ..., 6}, while Section IV-F sets K = 10 for all methods; these settings should be reconciled and the final K reported for every experiment table.
  4. [Algorithm 1 / Section III-F.1] Algorithm 1 uses a generic variable E for the number of epochs, whereas Section III-F.1 defines E1 and E2; the notation should be aligned.
  5. [Table I] The header 'DATASETS STATICS' contains a typo and should read 'DATASETS STATISTICS'.
  6. [Section IV-K] The ablation text reports the Computers accuracy dropping to 69.31% when PMA is removed and states the original value as 77.92%, but Table II reports GCN+ProGMLP on Computers as 78.82±0.44; please clarify which teacher/setting the ablation uses and correct the inconsistency.
  7. [Section IV-H / Table III] For the comparison with non-ensemble G2M methods, please state the number of students K used for ProGMLP and whether the reported accuracy is after executing all K students or after the early-exit criterion; this is needed to interpret the comparison with single-student methods.
  8. [Index Terms] The Index Terms line ('component, formatting, style, styling, insert') appears to be placeholder text and should be replaced with actual keywords.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: ProGMLP's architecture and losses are defined in-paper and its results are empirical comparisons against external baselines.

full rationale

ProGMLP does not derive any predicted quantity from an input definition or from a self-citation. The progressive training structure (Eqs. (3)-(4)), progressive KD loss (Eq. (6)), mixup loss (Eqs. (7)-(8)), and inference stopping/weighting rules (Eqs. (11)-(12)) are all specified within the paper and are not stated as consequences of a prior theorem. The method is evaluated against external benchmarks (Cora, Pubmed, Amazon, Coauthor, OGB) and compared with GLNN, AdaGMLP, NOSMOG, KRD, and HGMD. The self-citation to AdaGMLP [13] functions only as a related-work baseline; no load-bearing uniqueness or correctness claim is imported from prior work by the same authors. The skeptical concern that cost is measured by number of executed students rather than FLOPs is an experimental-design/fairness issue, not a circularity: it does not make any claimed prediction equal to its input. Therefore no circular step is identified.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on several hyperparameters and domain assumptions. The main free parameters are the loss weights alpha, beta, the mixup controller gamma/tau/sigma, the confidence threshold tau_conf, and the number of students K; each is set by hand or search in Section IV. The key assumptions are that a pretrained GNN teacher supplies reliable soft labels, that student capacity is sufficient, that mixup in hidden space is meaningful, and that the batch-level confidence threshold is a valid early-exit criterion. No invented entities are introduced.

free parameters (8)
  • alpha = 0.5 (set in main experiments; best 0.4 on CS)
    Balances cross-entropy and KD loss in Eq. (6); not tuned per dataset.
  • beta = 0.8
    Scales loss weight for later students in Eq. (6); sensitivity low.
  • gamma = 0.9
    Adaptation rate for mixup ratio in Eq. (9).
  • tau (mixup threshold) = 0.1
    Reference loss for lambda update in Eq. (9).
  • sigma = 0.1
    EMA smoothing factor in Eq. (10).
  • tau_conf = 0.9
    Confidence threshold for early exit in Sec. III-F.2.
  • K (number of students) = 10 for trade-off experiments; 2-6 in parameter search
    Number of student MLPs; directly controls inference cost.
  • hidden dimensionality d' = 128 in trade-off experiments; searched {16,32,64,128,256,512}
    Student width; affects cost and capacity.
assumptions (5)
  • domain assumption Teacher GNN is pretrained and provides good soft labels for distillation.
    Used throughout Sec. III-D; if teacher is weak, distillation cannot improve.
  • domain assumption Each MLP student has sufficient capacity (layers and hidden size) to approximate the teacher.
    Student architecture in Sec. III-C; no capacity analysis.
  • ad hoc to paper The confidence threshold over the batch of unlabeled nodes is a valid early-exit criterion.
    Sec. III-F.2, Eq. (11); this is the core mechanism for dynamic inference but is not validated against per-sample latency.
  • domain assumption Mixup in the concatenated feature-hidden space generates semantically meaningful training examples.
    Sec. III-E; standard mixup assumption extended to hidden representations.
  • domain assumption Progressive initialization and hidden-state propagation improve students monotonically.
    Sec. III-C; ablation supports empirically but no formal guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs." pith.science (2026). https://pith.science/paper/P7UB6TVH

@misc{pith2026250719031,
  author       = {Pith},
  title        = {Pith review of: ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P7UB6TVH}},
  note         = {Machine review of arXiv:2507.19031}
}
read the original abstract

GNN-to-MLP (G2M) methods have emerged as a promising approach to accelerate Graph Neural Networks (GNNs) by distilling their knowledge into simpler Multi-Layer Perceptrons (MLPs). These methods bridge the gap between the expressive power of GNNs and the computational efficiency of MLPs, making them well-suited for resource-constrained environments. However, existing G2M methods are limited by their inability to flexibly adjust inference cost and accuracy dynamically, a critical requirement for real-world applications where computational resources and time constraints can vary significantly. To address this, we introduce a Progressive framework designed to offer flexible and on-demand trade-offs between inference cost and accuracy for GNN-to-MLP knowledge distillation (ProGMLP). ProGMLP employs a Progressive Training Structure (PTS), where multiple MLP students are trained in sequence, each building on the previous one. Furthermore, ProGMLP incorporates Progressive Knowledge Distillation (PKD) to iteratively refine the distillation process from GNNs to MLPs, and Progressive Mixup Augmentation (PMA) to enhance generalization by progressively generating harder mixed samples. Our approach is validated through comprehensive experiments on eight real-world graph datasets, demonstrating that ProGMLP maintains high accuracy while dynamically adapting to varying runtime scenarios, making it highly effective for deployment in diverse application settings.

Figures

Figures reproduced from arXiv: 2507.19031 by the authors.

Figure 1
Figure 1. Illustration of the core motivation for ProGMLP. (a) Existing methods [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The training architecture of ProGMLP. for real-world applications where computational resources are limited, and the ability to control inference time is critical. B. Methodology 1) Notations: Consider a graph G = {V, E, Y } where V and E represent the set of nodes and edges, respectively. Let N denote the total number of nodes in the graph. The label matrix Y ∈ R N×C consists of one-hot vectors, where C denotes the… view at source ↗
Figure 3
Figure 3. Accuracy vs. Inference Cost (# Number of Executed Students) for ProGMLP, AdaGMLP, and GLNN. Each curve represents the progression of accuracy [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Hyperparameter Sensitivity Analysis. We have conducted a hyperparameter study to analyze the sensitivity of ProGMLP to key hyperparameters on the CS dataset using the GCN as the teacher. The results are shown in [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transferable Graph Condensation from the Causal Perspective

    cs.LG 2026-01 conditional novelty 5.0 of 10

    A graph-condensation framework that aligns condensed graphs with low-frequency 'causal' structure via spectral contrastive learning transfers better across tasks and datasets than existing condensation methods, with u...

Reference graph

Works this paper leans on

50 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2016

    arXiv 2016

  2. [2]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y . Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903, 2017

    arXiv 2017

  3. [3]

    Inductive representation learning on large graphs,

    W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in neural information processing systems , 2017, pp. 1024–1034

    2017

  4. [4]

    Simplifying graph convolutional networks,

    F. Wu, A. Souza, T. Zhang, C. Fifty, T. Yu, and K. Weinberger, “Simplifying graph convolutional networks,” in International conference on machine learning . PMLR, 2019, pp. 6861–6871

    2019

  5. [5]

    How powerful are graph neural networks?

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How powerful are graph neural networks?” arXiv preprint arXiv:1810.00826 , 2018

    arXiv 2018

  6. [6]

    Predict then propagate: Graph neural networks meet personalized pagerank,

    J. Klicpera, A. Bojchevski, and S. G ¨unnemann, “Predict then propagate: Graph neural networks meet personalized pagerank,” arXiv preprint arXiv:1810.05997, 2018

    arXiv 2018

  7. [8]

    Skipnode: On alleviating performance degradation for deep graph convolutional networks,

    W. Lu, Y . Zhan, B. Lin, Z. Guan, L. Liu, B. Yu, W. Zhao, Y . Yang, and D. Tao, “Skipnode: On alleviating performance degradation for deep graph convolutional networks,” IEEE Transactions on Knowledge and Data Engineering, pp. 1–14, 2024

    work page 2024

  8. [9]

    Graph-less neural networks: Teaching old mlps new tricks via distillation,

    S. Zhang, Y . Liu, Y . Sun, and N. Shah, “Graph-less neural networks: Teaching old mlps new tricks via distillation,” in International Conference on Learning Representations , 2021

    work page 2021

Show all 50 references
  1. [10]

    Learning mlps on graphs: A unified view of effectiveness, robustness, and efficiency,

    Y . Tian, C. Zhang, Z. Guo, X. Zhang, and N. Chawla, “Learning mlps on graphs: A unified view of effectiveness, robustness, and efficiency,” in The Eleventh International Conference on Learning Representations , 2022

    2022

  2. [11]

    Quantifying the knowledge in gnns for reliable distillation into mlps,

    L. Wu, H. Lin, Y . Huang, and S. Z. Li, “Quantifying the knowledge in gnns for reliable distillation into mlps,” arXiv preprint arXiv:2306.05628, 2023

    2023 arXiv

  3. [12]

    Extracting low-/high- frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework,

    L. Wu, H. Lin, Y . Huang, T. Fan, and S. Z. Li, “Extracting low-/high- frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework,” arXiv preprint arXiv:2305.10758, 2023

    2023 arXiv

  4. [13]

    Adagmlp: Adaboosting gnn-to- mlp knowledge distillation,

    W. Lu, Z. Guan, W. Zhao, and Y . Yang, “Adagmlp: Adaboosting gnn-to- mlp knowledge distillation,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , 2024, pp. 2060– 2071

    2024

  5. [14]

    An overview on edge computing research,

    K. Cao, Y . Liu, G. Meng, and Q. Sun, “An overview on edge computing research,” IEEE access, vol. 8, pp. 85 714–85 728, 2020

    2020

  6. [15]

    A survey on mobile edge computing: The communication perspective,

    Y . Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE communications surveys & tutorials , vol. 19, no. 4, pp. 2322–2358, 2017

    2017

  7. [16]

    Deep learning with edge computing: A review,

    J. Chen and X. Ran, “Deep learning with edge computing: A review,” Proceedings of the IEEE , vol. 107, no. 8, pp. 1655–1674, 2019

    2019

  8. [17]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE internet of things journal , vol. 3, no. 5, pp. 637–646, 2016

    2016

  9. [18]

    Mobile application usability,

    H. Hoehle and V . Venkatesh, “Mobile application usability,” MIS quarterly, vol. 39, no. 2, pp. 435–472, 2015

    2015

  10. [19]

    Mobile application and its global impact,

    R. Islam, R. Islam, and T. Mazumder, “Mobile application and its global impact,” International Journal of Engineering & Technology , vol. 10, no. 6, pp. 72–78, 2010

    2010

  11. [20]

    Adaptive neural networks for efficient inference,

    T. Bolukbasi, J. Wang, O. Dekel, and V . Saligrama, “Adaptive neural networks for efficient inference,” in International Conference on Machine Learning. PMLR, 2017, pp. 527–536

    2017

  12. [21]

    Multiple instance learning for efficient sequential data classification on resource-constrained devices,

    D. Dennis, C. Pabbaraju, H. V . Simhadri, and P. Jain, “Multiple instance learning for efficient sequential data classification on resource-constrained devices,” Advances in Neural Information Processing Systems , vol. 31, 2018

    2018

  13. [22]

    Anytime inference with distilled hierarchical neural ensembles,

    A. Ruiz and J. Verbeek, “Anytime inference with distilled hierarchical neural ensembles,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 11, 2021, pp. 9463–9471

    2021

  14. [23]

    Multi-scale dense networks for resource efficient image classification,

    G. Huang, D. Chen, T. Li, F. Wu, L. Van Der Maaten, and K. Q. Weinberger, “Multi-scale dense networks for resource efficient image classification,” arXiv preprint arXiv:1703.09844 , 2017

    2017 arXiv

  15. [24]

    Progressive ensemble distillation: building ensembles for efficient inference,

    D. Dennis, A. Shetty, A. P. Sevekari, K. Koishida, and V . Smith, “Progressive ensemble distillation: building ensembles for efficient inference,” Advances in Neural Information Processing Systems , vol. 36, pp. 43 525–43 543, 2023

    2023

  16. [25]

    Simple and deep graph convolutional networks,

    M. Chen, Z. Wei, Z. Huang, B. Ding, and Y . Li, “Simple and deep graph convolutional networks,” in International conference on machine learning. PMLR, 2020, pp. 1725–1735

    2020

  17. [26]

    Representation learning on graphs with jumping knowledge networks,

    K. Xu, C. Li, Y . Tian, T. Sonobe, K.-i. Kawarabayashi, and S. Jegelka, “Representation learning on graphs with jumping knowledge networks,” in International conference on machine learning . PMLR, 2018, pp. 5453–5462

    2018

  18. [27]

    Pseudo contrastive learning for graph-based semi-supervised learning,

    W. Lu, Z. Guan, W. Zhao, Y . Yang, Y . Lv, L. Xing, B. Yu, and D. Tao, “Pseudo contrastive learning for graph-based semi-supervised learning,” arXiv preprint arXiv:2302.09532 , 2023

    2023 arXiv

  19. [28]

    Nodemixup: Tackling under-reaching for graph neural networks,

    W. Lu, Z. Guan, W. Zhao, Y . Yang, and L. Jin, “Nodemixup: Tackling under-reaching for graph neural networks,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 13, 2024, pp. 14 175– 14 183

    2024

  20. [29]

    Lpformer: An adaptive graph transformer for link prediction,

    H. Shomer, Y . Ma, H. Mao, J. Li, B. Wu, and J. Tang, “Lpformer: An adaptive graph transformer for link prediction,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 2686–2698

    2024

  21. [30]

    Graph substructure assembling network with soft sequence and context attention,

    Y . Yang, Z. Guan, W. Zhao, W. Lu, and B. Zong, “Graph substructure assembling network with soft sequence and context attention,” IEEE Transactions on Knowledge and Data Engineering , vol. 35, no. 5, pp. 4894–4907, 2022

    2022

  22. [31]

    Deep geometric knowledge distillation with graphs,

    C. Lassance, M. Bontonou, G. B. Hacene, V . Gripon, J. Tang, and A. Ortega, “Deep geometric knowledge distillation with graphs,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2020, pp. 8484–8488

    2020

  23. [32]

    Iterative graph self-distillation,

    H. Zhang, S. Lin, W. Liu, P. Zhou, J. Tang, X. Liang, and E. P. Xing, “Iterative graph self-distillation,” IEEE Transactions on Knowledge and Data Engineering, 2023

    2023

  24. [33]

    Multi-task self-distillation for graph- based semi-supervised learning,

    Y . Ren, J. Ji, L. Niu, and M. Lei, “Multi-task self-distillation for graph- based semi-supervised learning,” arXiv preprint arXiv:2112.01174 , 2021

    2021 arXiv

  25. [34]

    On representation knowledge distillation for graph neural networks,

    C. K. Joshi, F. Liu, X. Xun, J. Lin, and C. S. Foo, “On representation knowledge distillation for graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems , 2022

    2022

  26. [35]

    Knowledge distillation improves graph structure augmentation for graph neural networks,

    L. Wu, H. Lin, Y . Huang, and S. Z. Li, “Knowledge distillation improves graph structure augmentation for graph neural networks,” Advances in Neural Information Processing Systems, vol. 35, pp. 11 815–11 827, 2022

    2022

  27. [36]

    Be your own teacher: Improve the performance of convolutional neural networks via self distillation,

    L. Zhang, J. Song, A. Gao, J. Chen, C. Bao, and K. Ma, “Be your own teacher: Improve the performance of convolutional neural networks via self distillation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 3713–3722

    2019

  28. [37]

    On self- distilling graph neural network,

    Y . Chen, Y . Bian, X. Xiao, Y . Rong, T. Xu, and J. Huang, “On self- distilling graph neural network,” arXiv preprint arXiv:2011.02255 , 2020

    2011 arXiv

  29. [38]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  30. [39]

    Do deep nets really need to be deep?

    J. Ba and R. Caruana, “Do deep nets really need to be deep?” Advances in neural information processing systems , vol. 27, 2014

    2014

  31. [40]

    Distilling knowledge from graph convolutional networks,

    Y . Yang, J. Qiu, M. Song, D. Tao, and X. Wang, “Distilling knowledge from graph convolutional networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 7074–7083

    2020

  32. [41]

    Tinygnn: Learning efficient graph neural networks,

    B. Yan, C. Wang, G. Guo, and Y . Lou, “Tinygnn: Learning efficient graph neural networks,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2020, pp. 1848–1856

    2020

  33. [42]

    Reliable data distillation on graph convolutional network,

    W. Zhang, X. Miao, Y . Shao, J. Jiang, L. Chen, O. Ruas, and B. Cui, “Reliable data distillation on graph convolutional network,” in Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data , ser. SIGMOD ’20. New York, NY , USA: Association for Computi...

    2020

  34. [43]

    Teaching yourself: Graph self-distillation on neighborhood for node classification,

    L. Wu, J. Xia, H. Lin, Z. Gao, Z. Liu, G. Zhao, and S. Z. Li, “Teaching yourself: Graph self-distillation on neighborhood for node classification,” arXiv preprint arXiv:2210.02097 , 2022

    2022 arXiv

  35. [44]

    Vqgraph: Rethinking graph representation space for bridging gnns and mlps,

    L. Yang, Y . Tian, M. Xu, Z. Liu, S. Hong, W. Qu, W. Zhang, C. Bin, M. Zhang, and J. Leskovec, “Vqgraph: Rethinking graph representation space for bridging gnns and mlps,” in The Twelfth International Conference on Learning Representations , 2024

    2024

  36. [45]

    Adaptive inference through early-exit networks: Design, challenges and directions,

    S. Laskaridis, A. Kouris, and N. D. Lane, “Adaptive inference through early-exit networks: Design, challenges and directions,” in Proceedings of the 5th International Workshop on Embedded and Mobile Deep Learning , 2021, pp. 1–6

    2021

  37. [46]

    Branchynet: Fast inference via early exiting from deep neural networks,

    S. Teerapittayanon, B. McDanel, and H.-T. Kung, “Branchynet: Fast inference via early exiting from deep neural networks,” in 2016 23rd international conference on pattern recognition (ICPR) . IEEE, 2016, pp. 2464–2469

    2016

  38. [47]

    Fast graph representation learning with PyTorch Geometric,

    M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in ICLR Workshop on Representation Learning on Graphs and Manifolds , 2019

    2019

  39. [48]

    Collective classification in network data,

    P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad, “Collective classification in network data,” AI magazine, vol. 29, no. 3, pp. 93–93, 2008

    2008

  40. [49]

    Pitfalls of graph neural network evaluation,

    O. Shchur, M. Mumme, A. Bojchevski, and S. G ¨unnemann, “Pitfalls of graph neural network evaluation,” arXiv preprint arXiv:1811.05868 , 2018

    2018 arXiv

  41. [50]

    Open graph benchmark: Datasets for machine learning on graphs,

    W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs,” arXiv preprint arXiv:2005.00687 , 2020

    2005 arXiv

  42. [51]

    Teach harder, learn poorer: Rethinking hard sample distillation for gnn-to-mlp knowledge distillation,

    L. Wu, Y . Liu, H. Lin, Y . Huang, and S. Z. Li, “Teach harder, learn poorer: Rethinking hard sample distillation for gnn-to-mlp knowledge distillation,” in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management , ser. CIKM ’24. New York, ...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.